Novel coronavirus infection ct image classification method and device based on deep learning and gaussian modeling

Abstract

The application discloses a novel coronavirus infection CT image classification method and device based on deep learning and Gaussian modeling, uses a pre-trained neural network to extract feature mapping, solves the overfitting problem of a deep learning network model caused by too small data volume of a training set, and proposes a subspace decomposition method, decomposes a feature vector into two subspaces, and reveals the role of the two subspaces in a CT image diagnosis task. A transformation matrix is constructed by using a subspace containing key features to reduce the dimension of the feature mapping, which not only removes redundant features, reduces the calculation cost of the algorithm, but also improves the classification performance of the model. Finally, a scientific threshold setting method is used to reduce the misdiagnosis probability, and an evaluation index is introduced to provide a reference for auxiliary diagnosis.

Description

TECHNICAL FIELD

[0001] The application relates to the field of medical image processing, in particular to a CT image classification method and device for novel coronavirus infection based on deep learning and Gaussian modeling. BACKGROUND

[0002] The main methods for detecting novel coronavirus infection are nucleic acid detection and lung CT. In recent years, with the development of deep learning in the field of computer vision, image-based anomaly detection methods have been widely used in medical image processing. From the current situation, the deep learning-based medical image diagnosis method can achieve higher accuracy than the traditional method under the support of data-driven. However, the deep learning-based method also has its problems: the network reaching the clinical standard is often very complex, and the deep features extracted by the network are abnormally large, and feature redundancy is inevitable. These redundant features not only increase the computational cost but also reduce the accuracy of the network, so how to reduce the redundancy of deep features and improve the computing performance has become a problem to be solved. SUMMARY

[0003] In view of the deficiencies of the existing deep learning-based medical image diagnosis method, the purpose of the embodiment of the present application is to provide a CT image classification method and device for novel coronavirus infection based on deep learning and Gaussian modeling, to solve the technical problems mentioned in the background part.

[0004] In the first aspect, the application provides a CT image classification method for novel coronavirus infection based on deep learning and Gaussian modeling, comprising the following steps:

[0005] S1, constructing a normal class training set {x N} and a novel coronavirus patient training set {x A}, using the normal class training set {x N} and the novel coronavirus patient training set {x A} to construct a Gaussian model group G N for the normal class and a Gaussian model group G A for the novel coronavirus patient through a deep learning network model respectively;

[0006] S2, obtaining a CT image set {y} to be classified, respectively calculating the Mahalanobis distance between the feature mapping after dimension reduction of the CT image set {y} to be classified through the deep learning network model and each layer of the Gaussian model of the normal class Gaussian model group G N and each layer of the Gaussian model of the novel coronavirus patient Gaussian model group G A , to obtain a first Mahalanobis distance and a second Mahalanobis distance

[0007] S3, according to the normal class training set {xN} and the training set of COVID-19 patients {x A} are respectively used to construct the Gaussian model G N of the normal class and the Gaussian model G A of the COVID-19 patients. N A ;

[0008] S4, according to the determination threshold T N of the normal class and the determination threshold T A of the COVID-19 patients and the first Mahalanobis distance D and the second Mahalanobis distance D , the classification index of the normal class and the classification index of the COVID-19 patients are obtained, and the classification result is determined according to the classification index of the normal class and the classification index of the COVID-19 patients.

[0009] As preferred, the deep learning network model has several layers of networks, each of which uses a classifier Ψ θ (·) with parameters θ to perform deep feature extraction, and maps the CT image x i to a deep feature representation domain f i = Ψ θ (x i ), N F is the feature dimension.

[0010] As preferred, the deep learning network model includes ResNet or EfficientNet.

[0011] As preferred, in step S1, the training set of the normal class {x N} and the training set of COVID-19 patients {x A} are used to construct the Gaussian model G N of the normal class and the Gaussian model G A of the COVID-19 patients by the deep learning network model, specifically including:

[0012] S11, based on the feature mapping {f i} of the training set, the corresponding mean vector and the covariance matrix are calculated. The principal component analysis is performed on the covariance matrix i ,i∈[1,N F ] are diagonal elements of a diagonal matrix;

[0013] S12, by setting a cut-off point, the eigenvalues λ i are divided into two groups: and​ The corresponding feature vector space is divided into two subspaces: a significant feature subspace Φ1 and a non-significant feature subspace Φ2; the estimation method of the subspace boundary point is:

[0014] m+1=argmax i {λ i |λ i <[2λ med -λ mr ]};

[0015] wherein m r is the rank of the covariance matrix, λ med is the median of the eigenvalues, the first m eigenvectors of the eigenvector matrix V belong to the significant feature subspace Φ1, and the remaining eigenvectors belong to the non-significant feature subspace Φ2;

[0016] S13, the non-significant feature subspace Φ2 is constructed into a transformation matrix, and for each feature mapping, the left multiplication of the transformation matrix obtains the dimension-reduced feature mapping The mean vector and the covariance matrix of the dimension-reduced feature mapping are expressed as:

[0017]

[0018]

[0019] wherein Ns represents the number of training set samples;

[0020] S14, steps S11-S13 are performed using the normal class training set {x N} and the new crown patient training set {x A}, respectively, to extract feature mappings through each layer of the deep learning network model to construct Gaussian models, and to obtain a normal class Gaussian model group and a new crown patient Gaussian model group wherein s represents the number of network layers of the deep learning network model.

[0021] As a preferred, step S2 specifically comprises:

[0022] The CT images in the CT image set to be classified are sequentially input into the deep learning network model, and through steps S11-S13, the dimension-reduced feature mapping is obtained. The Mahalanobis distance matrix dd=[d 1 ,d 2 ,…,d s ] between

[0023]

[0024] After normalizing the Mahalanobis distance matrix d by column and then averaging by row, the Mahalanobis distance between the CT image set to be classified and the normal class training set {x N} or the new crown patient training set {x A} is represented as:

[0025]

[0026] wherein, and respectively represent the mean and standard deviation of the Mahalanobis distance calculated by the jth layer feature of all samples of the normal class training set {x N} or the new crown patient training set {x A}, and the first Mahalanobis distance and the second Mahalanobis distance

[0027] As a preferred, the step S3 specifically comprises:

[0028] According to the maximum Mahalanobis distance of the normal class training set {x N} on the Gaussian model group G N of the normal class, the lower limit of the threshold value is determined According to the minimum Mahalanobis distance of the new crown patient training set {x A} on the Gaussian model group G N of the normal class, the upper limit of the threshold value is determined Then the determination threshold T N of the normal class is set as:

[0029]

[0030] According to the maximum Mahalanobis distance of the new crown patient training set {x A} on the Gaussian model group G A of the new crown patient, the lower limit of the threshold value is determined According to the minimum Mahalanobis distance of the normal class training set {x N} on the Gaussian model group G A of the new crown patient, the upper limit of the threshold value is determined Then the determination threshold T A of the new crown patient is set as:

[0031]

[0032] As a preferred, the step S4 specifically comprises:

[0033] According to the determination threshold T N of the normal class and the first Mahalanobis distance , the classification index w N:

[0034]

[0035] According to the determination threshold T of the new crown patient A And the second Mahalanobis distance Define the classification index w of the new crown patient A :

[0036]

[0037] When w k >t indicates that the CT image in the CT image set {y} to be classified does not belong to the k class, k is N or A, t is the threshold, and when w k ≤t indicates that the CT image in the CT image set {y} to be classified belongs to the k class, when classifying the CT image set {y} to be classified, there are the following four possible situations:

[0038] a) w N ≤t and w A >t;

[0039] b) w N >t and w A ≤t;

[0040] c) w N ≤t and w A ≤t;

[0041] d) w N >t and w A >t;

[0042] In response to determining that the classification index of the CT image in the CT image set {y} to be classified satisfies situation a), the CT image in the CT image set {y} to be classified is determined to be the normal class;

[0043] In response to determining that the classification index of the CT image in the CT image set {y} to be classified satisfies situation b), the CT image in the CT image set {y} to be classified is determined to be the new crown patient;

[0044] In response to determining that the classification index of the CT image in the CT image set {y} to be classified satisfies situations c) and d), the comprehensive class evaluation index w is calculated:

[0045] w = αw N +(1-α)(1-w A );

[0046] Wherein, represents the number of samples of the normal class training set {x N} and the new crown patient training set {xA} and the sample number ratio of the new crown patient class {x

[0047] In a second aspect, the present application provides a new coronavirus infection CT image classification device based on deep learning and Gaussian modeling, comprising:

[0048] a Gaussian model construction module configured to construct a normal class training set {x N} and a new crown patient training set {x A}; N} and the new crown patient training set {x A} are used to construct a Gaussian model group G N of the normal class and a Gaussian model group G A of the new crown patient through a deep learning network model respectively.

[0049] a Mahalanobis distance calculation module configured to obtain a CT image set {y} to be classified, calculate the Mahalanobis distance between the feature mapping after dimension reduction of the CT image set {y} to be classified obtained through the deep learning network model and each layer of Gaussian model in the Gaussian model group G N of the normal class and each layer of Gaussian model in the Gaussian model group G A of the new crown patient, to obtain a first Mahalanobis distance and a second Mahalanobis distance

[0050] a determination threshold calculation module configured to obtain a determination threshold T N of the normal class and a determination threshold T A of the new crown patient according to the Mahalanobis distance of the normal class training set {x N} and the new crown patient training set {x A} on the Gaussian model group G N of the normal class and the Gaussian model group G A of the new crown patient.

[0051] a classification module configured to obtain a classification index of the normal class and a classification index of the new crown patient according to the determination threshold T N of the normal class and the determination threshold T A of the new crown patient and the first Mahalanobis distance and the second Mahalanobis distance , and determine a classification result according to the classification index of the normal class and the classification index of the new crown patient.

[0052] In a third aspect, the present application provides an electronic device comprising one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementations of the first aspect.

[0053] In a fourth aspect, the present application provides a computer readable storage medium, having stored thereon a computer program which, when executed by a processor, implements the method according to any of the implementations of the first aspect.

[0054] Compared with the prior art, the present application has the following beneficial effects:

[0055] The new coronavirus infection CT image classification method based on deep learning and Gaussian modeling provided by the present application uses a pre-trained neural network to extract feature mapping of a training set, solving the overfitting problem of a deep learning network model caused by too little training set data. A subspace decomposition method is proposed, which not only removes redundant features and reduces the computational cost of the algorithm, but also improves the classification performance of the model. The feature dimension reduction method based on subspace decomposition is used to solve the problem of redundant network features and improve the computational efficiency. Finally, a scientific threshold setting method is used to reduce the false positive and false negative misdiagnosis probability, and an evaluation index is introduced to provide a reference for auxiliary diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0057] Figure 1 is an exemplary device architecture diagram to which an embodiment of the present application can be applied;

[0058] Figure 2 is a flowchart of the new coronavirus infection CT image classification method based on deep learning and Gaussian modeling of the embodiment of the present application;

[0059] Figure 3 is a lung CT image display, Figure 3 (a) is a normal lung CT image; Figure 3 (b) is a lung CT image of a new coronavirus infection patient, and the suspected lesion part is marked by a circle in the figure;

[0060] Figure 4 is an algorithm flowchart of the new coronavirus infection CT image classification method based on deep learning and Gaussian modeling of the embodiment of the present application;

[0061] Figure 5 is a test result graph of the new coronavirus infection CT image classification method based on deep learning and Gaussian modeling of the embodiment of the present application, wherein Figure 5(a) is the Mahalanobis distance {d N} of the test set on the normal class Gaussian model G N}, Figure 5 (b) is the Mahalanobis distance {d A} of the test set on the new crown patient Gaussian model G A} ; the abscissa is the test sample, the ordinate is the Mahalanobis distance, and the dashed line in the figure is the threshold line;

[0062] Figure 6 is a schematic diagram of a deep learning and Gaussian modeling based new coronavirus infection CT image classification device of an embodiment of the present application;

[0063] Figure 7 is a structural schematic diagram of a computer device suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0065] Figure 1 An exemplary device architecture 100 of a deep learning and Gaussian modeling based new coronavirus infection CT image classification method or a deep learning and Gaussian modeling based new coronavirus infection CT image classification device to which embodiments of the present application can be applied is shown.

[0066] As shown in Figure 1 , the device architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0067] A user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various applications can be installed on the terminal devices 101, 102, 103, such as data processing applications, file processing applications, etc.

[0068] The terminal devices 101, 102, and 103 can be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they can be various electronic devices including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like. When the terminal devices 101, 102, and 103 are software, they can be installed in the above-mentioned electronic devices. They can be implemented as multiple software or software modules (for example, software or software modules for providing distributed services) or as a single software or software module. No specific limitation is made herein.

[0069] The server 105 can be a server providing various services, for example, a background data processing server for processing files or data uploaded by the terminal devices 101, 102, and 103. The background data processing server can process the obtained files or data to generate processing results.

[0070] It should be noted that the new coronavirus infection CT image classification method based on deep learning and Gaussian modeling provided by the embodiments of the present application can be executed by the server 105 or the terminal devices 101, 102, and 103, and accordingly, the new coronavirus infection CT image classification device based on deep learning and Gaussian modeling can be arranged in the server 105 or the terminal devices 101, 102, and 103.

[0071] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above-mentioned device architecture is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks, and servers. In the case where the data to be processed does not need to be obtained remotely, the above-mentioned device architecture can not include a network, but only a server or a terminal device.

[0072] Figure 2 A new coronavirus infection CT image classification method based on deep learning and Gaussian modeling provided by the embodiments of the present application is shown, which includes the following steps:

[0073] S1, a normal class training set {x N} and a new crown patient training set {x A} are constructed, and a Gaussian model group G N of the normal class and a Gaussian model group G A of the new crown patient are respectively constructed by using the normal class training set {x N} and the new crown patient training set {x A} through a deep learning network model.

[0074] Specifically, in step 1), the image data used is derived from an open source data set or actual medical diagnosis and treatment data. The normal class training set {x N} are all normal lung CT images, and the training set x A} of new crown patients. In order to simplify the description, the training set x N} of normal class and the training set x A} of new crown patients are referred to as training set x i} in the following. Take the open source dataset COVID-CT-Dataset as an example. 200 normal lung CT images are randomly selected from the open source data as the training set x N} of normal class, and 200 lung CT images of new crown patients are selected as the training set x A} of new crown patients. Then, 100 normal lung CT images and 100 lung CT images of new crown patients are randomly selected as the CT image set to be classified to verify the effectiveness of the algorithm. The normal lung CT image is shown in Figure 3 (a), and the lung CT image of the new crown patient is shown in Figure 3 (b). Compared with the normal lung CT image, the lung CT image of the new crown patient has a "ground glass" lesion in the local, as shown in the mark in Figure 3 (b).

[0075] In specific embodiments, the deep learning network model includes a plurality of layers of networks, each layer of network uses a classifier Ψ θ (·) with parameters θ to perform deep feature extraction, and maps the CT image x i to a deep feature representation domain f i = Ψ θ (x i ), N F is the feature dimension.

[0076] In specific embodiments, the deep learning network model includes ResNet or EfficientNet.

[0077] In specific embodiments, the step S1 uses the training set x N} of normal class and the training set x A} of new crown patients to construct the Gaussian model G N} of normal class and the Gaussian model G A} of new crown patients through the deep learning network model, specifically including:

[0078] S11, based on the feature mapping {f i} of the training set, the corresponding mean vector and the covariance matrix are calculated. The principal component analysis is performed on the covariance matrix to obtain the eigenvector matrix V and the eigenvalue matrix D, wherein the eigenvalue matrix D is a matrix composed of eigenvalues λi i∈[1,N F ] is a diagonal matrix with diagonal elements

[0079] S12, the feature values λ i are divided into two groups by setting a boundary point: and The corresponding eigenvector space is divided into two subspaces: the significant feature subspace Φ1 and the insignificant feature subspace Φ2; the estimation method of the subspace boundary point is:

[0080]

[0081] where m r is the rank of the covariance matrix, λ med is the median of the eigenvalues, and the first m eigenvectors of the eigenvector matrix V belong to the significant feature subspace Φ1, and the remaining eigenvectors belong to the insignificant feature subspace Φ2;

[0082] S13, the insignificant feature subspace Φ2 is constructed into a transformation matrix, and for each feature mapping, the left multiplication of the transformation matrix obtains the reduced feature mapping The mean vector and the covariance matrix of the reduced feature mapping are expressed as:

[0083]

[0084]

[0085] where Ns represents the number of training set samples;

[0086] S14, steps S11-S13 are performed using the normal class training set {x N} and the new crown patient training set {x A}, respectively, to extract feature mappings through each layer of the deep learning network model to construct a Gaussian model, and to obtain the normal class Gaussian model group and the new crown patient Gaussian model group where l represents the number of network layers of the deep learning network model.

[0087] Specifically, the classifier Ψ θ (·) with parameters θ pre-trained on a large dataset (such as ImageNet) is used as a deep feature extraction device of the deep learning network model, and the input image x i is mapped to the deep feature representation domain f i = Ψ θ (x i ), N F is the feature dimension.

[0088] Deep learning network models such as ResNet and EfficientNet can be used; in this specific example, EfficientNet-B4 is adopted. The EfficientNet-B4 network has 9 layers, each containing a number of convolutional kernels for extracting deep features. For example, the first layer has 48 kernels, the second layer has 24, and so on. The specific implementation of this example is described in detail below using the seventh layer as an example. The seventh layer has 272 convolutional kernels, meaning the feature dimension N of the seventh layer is... F =272. Feature mapping {f} based on the training set i}, the corresponding mean vector can be calculated. Covariance Matrix For covariance matrix Principal component analysis is performed to obtain the eigenvector matrix V and the eigenvalue matrix D, where the eigenvalue matrix D is defined by the eigenvalues ​​λ. i ,i∈[1,272] is a diagonal matrix with diagonal elements. By setting a dividing point, the eigenvalues ​​are divided into two groups, and the corresponding eigenvector space is divided into two subspaces: the first group It contains the largest eigenvalue and its corresponding eigenvector, representing the eigenvector with the strongest commonality among all samples. Named as the salient feature subspace; second group It contains components with relatively weak commonalities, and their corresponding subspaces. This is called the insignificant feature subspace. The method for estimating the subspace boundary points in the embodiments of this application is as follows:

[0089]

[0090] Where, m r Let λ be the rank of the covariance matrix. med The median of the eigenvalues ​​is given. The estimated value is m = 54, meaning the first 54 eigenvectors of the eigenvector matrix V belong to Φ1, and the remaining eigenvectors belong to Φ2.

[0091] Select a subspace containing key features to construct a transformation matrix, and map the features to {f}. iDimensionality reduction. For the new coronavirus infection CT image diagnosis task, the lung CT images of the new coronavirus patients and the normal lung CT images only differ in local or details, which belongs to the pixel-level micro-difference. In such an application background, the information in the two subspaces will play different roles. The significant feature subspace Φ1 contains the dominant information of the features, which has an important contribution to the image reconstruction, however, Φ1 pays no attention to the details in the image when dealing with the new coronavirus CT image diagnosis task, which belongs to the redundant features. The insignificant feature subspace Φ2 represents the micro-detail information and the orthogonal feature components in the features, which contains the features different from the normal mode, and is a strong indicator of the existence of abnormalities. Based on the above considerations, the insignificant feature subspace Φ2 is constructed into a transformation matrix. For each feature mapping, left multiplication of the transformation matrix obtains the dimensionality-reduced feature mapping The mean vector of the dimensionality-reduced feature mapping And the covariance matrix Can be represented as:

[0092]

[0093]

[0094] Different types of Gaussian models can be constructed using different training data. EfficientNet-B4 has a total of 9 layers, and each layer can extract feature mappings to construct Gaussian models. In the embodiments of the present application, Gaussian model groups G are used to describe the normal class, and Gaussian model groups G are used to describe the new coronavirus patients. The modeling process is shown in the training process of Figure 4 . For a classification model, the AUROC value is usually used to describe the classification performance of the model, a random classifier has an AUROC value of 50%, and an ideal classifier is 100%. In the embodiments of the present application, the AUROC value of the Gaussian model group G N of the normal class reaches 94.04%, and the AUROC value of the Gaussian model group G A reaches 98.44%.

[0095] S2, obtaining a CT image set {y} to be classified, respectively calculating the Mahalanobis distance between the dimensionality-reduced feature mapping of the CT image set {y} to be classified obtained by the deep learning network model and each layer of the Gaussian model group G N of the normal class and each layer of the Gaussian model group G A of the new coronavirus patients, obtaining the first Mahalanobis distance and the second Mahalanobis distance

[0096] In specific embodiments, step S2 specifically comprises:

[0097] The CT images in the CT image set to be classified are sequentially input into the deep learning network model, and through steps S11-S13, the dimension-reduced feature mapping is obtained Calculate The Mahalanobis distance matrix d = [d 1 ,d 2 ,…,d l ] between each layer of Gaussian model, wherein:

[0098]

[0099] After the Mahalanobis distance matrix d is normalized by column and then averaged by row, the Mahalanobis distance between the CT image set to be classified and the normal class training set {x N} or the new crown patient training set {x A} is represented as:

[0100]

[0101] Wherein, And Respectively represent the mean and standard deviation of the Mahalanobis distance calculated by the jth layer feature of all samples of the normal class training set {x N} or the new crown patient training set {x A}, corresponding to the first Mahalanobis distance And the second Mahalanobis distance

[0102] Specifically, 200 CT images in the CT image set to be classified are sequentially input into the deep learning network model, and the dimension-reduced feature mapping is obtained through feature engineering Calculate The Mahalanobis distance d = [d 1 ,d 2 ,…,d 9 ] between each layer of Gaussian model, wherein:

[0103]

[0104] In this embodiment, the CT image set to be classified has a total of 200 CT images, which can be used as a test set, and the deep learning network model has a total of 9 layers, so the size of the Mahalanobis distance matrix d is [200, 9]. After the d is normalized by column and then averaged by row, the Mahalanobis distance between the CT image set to be classified and the training set is represented as:

[0105]

[0106] Wherein, And Respectively represent all training set samples (i.e., all normal samples of G N , or GA The Mahalanobis distance matrix is ​​obtained by calculating the mean and standard deviation of the Mahalanobis distance from the j-th layer features of the outlier samples. The size is [200, 1]. The Mahalanobis distance between the CT image set {y} to be classified and the Gaussian model of the normal class is... The Mahalanobis distance between the set of CT images to be classified {y} and the Gaussian model of COVID-19 patients is

[0107] S3, based on the normal class training set {x N} and the COVID-19 patient training set {x A In the normal class of Gaussian model group G N Gaussian model group G of COVID-19 patients A The Mahalanobis distance on the normal class is used to obtain the threshold T for class determination. N And the threshold for determining COVID-19 patients T A .

[0108] In a specific embodiment, step S3 specifically includes:

[0109] Based on the normal class training set {x N In the normal class of Gaussian model group G N The maximum Mahalanobis distance on the threshold determines the lower limit of the threshold. According to the training set of COVID-19 patients {x A In the normal class of Gaussian model group G N The minimum Mahalanobis distance on the threshold determines the upper limit of the threshold. The threshold T for determining the normal class N Set to:

[0110]

[0111] According to the training set of COVID-19 patients {x A In the Gaussian model group G of COVID-19 patients A The maximum Mahalanobis distance on the threshold determines the lower limit of the threshold. Based on the normal class training set {x N In the Gaussian model group G of COVID-19 patients A The minimum Mahalanobis distance on the threshold determines the upper limit of the threshold. The threshold T for identifying COVID-19 patients A Set to:

[0112]

[0113] Specifically, the threshold values ​​in the embodiments of this application are as follows: Figure 5 As shown in the middle threshold line.

[0114] S4, based on the normal class determination threshold T N And the threshold for determining COVID-19 patients TA and the first Mahalanobis distance and the second Mahalanobis distance obtaining a classification index of the normal class and a classification index of the COVID-19 patient, and determining the classification result according to the classification index of the normal class and the classification index of the COVID-19 patient.

[0115] In a specific embodiment, the step S4 specifically comprises:

[0116] according to the determination threshold T N and the first Mahalanobis distance defining the classification index w N of the normal class:

[0117]

[0118] according to the determination threshold T A and the second Mahalanobis distance defining the classification index w A of the COVID-19 patient:

[0119]

[0120] when w k >t indicates that the CT image in the CT image set {y} to be classified does not belong to the k class, k is N or A, t is a threshold, and when w k ≤t indicates that the CT image in the CT image set {y} to be classified belongs to the k class, when classifying the CT image set {y} to be classified, there are the following four possible situations:

[0121] a) w N ≤t and w A >t;

[0122] b) w N >t and w A ≤t;

[0123] c) w N ≤t and w A ≤t;

[0124] d) w N >t and w A >t;

[0125] In response to determining that the classification index of the CT image in the CT image set {y} to be classified satisfies situation a), the CT image in the CT image set {y} to be classified is determined as the normal class;

[0126] In response to determining that the classification indicator of the CT image in the CT image set {y} to be classified meets case b), the CT image in the CT image set {y} to be classified is determined to be a COVID-19 patient;

[0127] In response to determining that the classification indicator of the CT image in the CT image set {y} to be classified meets cases c) and d), a comprehensive class evaluation indicator w is calculated:

[0128] w = αw N + (1-α)(1-w A );

[0129] wherein, represents the ratio of the number of samples of the normal class training set {x N} and the number of samples of the COVID-19 patient training set {x A}, and the comprehensive class evaluation indicator w is used as a reference for auxiliary diagnosis.

[0130] In one embodiment, it is necessary to clarify that in cases c) and d), the present application has failed to accurately classify, and the comprehensive class evaluation indicator w is only used as a reference for auxiliary diagnosis, and accurate classification requires manual intervention. The following two cases can be used as a reference:

[0131] In response to determining that the comprehensive class evaluation indicator w≤t, it is determined that the CT image in the CT image set {y} to be classified is closer to the normal class;

[0132] In response to determining that the comprehensive class evaluation indicator w>t, it is determined that the CT image in the CT image set {y} to be classified is closer to the COVID-19 patient.

[0133] In this embodiment, the threshold value t is 0.5, and the definitions are and as the class evaluation indicators. When w k >0.5, it indicates that the sample does not belong to the k class (k is N or A), and when w k ≤0.5, it indicates that the sample belongs to the k class (k is N or A). When classifying the CT image set {y} to be classified, there are four possible cases:

[0134] a) w N ≤0.5 and w A >0.5;

[0135] b) w N >0.5 and w A ≤0.5;

[0136] c) w N ≤0.5 and w A ≤0.5;

[0137] d)w N >0.5and w A >0.5;

[0138] Of these, cases a) and b) have relatively high certainty. When case a) is met, it indicates that sample y is similar to the normal class and significantly different from COVID-19 patients, thus sample y can be classified as normal; conversely, when case b) is met, sample y is classified as a COVID-19 patient. Cases c) and d) have less clear classifications and require the calculation of an overall evaluation index. A comprehensive evaluation index can be further defined:

[0139] w = αw N +(1-α)(1-w A );

[0140] in, This represents the ratio of the number of normal samples to the number of abnormal samples in the training set. Since in this embodiment the normal class training set {x} N} and the training set of COVID-19 patients {x A Since the sample size is the same for both groups, the value of α is 1. If w ≤ 0.5, it means that sample y is relatively closer to the normal class; otherwise, it means that it is more similar to COVID-19 patients. Figure 5 As shown in the results, the vast majority of the CT images to be classified in this embodiment satisfy both a) and b), and can be accurately classified. Only a small portion of the samples meet both c) and d), requiring the calculation of a comprehensive evaluation index and manual intervention. As shown in step S2, the COVID-19 Gaussian model G in this embodiment... A Its classification performance is slightly better than the normal class Gaussian model G. N .like Figure 5 As shown in the middle circle, the proportion of COVID-19 samples requiring human intervention was slightly lower than that of normal samples, and the experimental results met theoretical expectations.

[0141] Further reference Figure 6 As an implementation of the methods shown in the above figures, this application provides an embodiment of a novel coronavirus infection CT image classification device based on deep learning and Gaussian modeling. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0142] This application provides a novel coronavirus infection CT image classification device based on deep learning and Gaussian modeling, including:

[0143] Gaussian model building module 1 is configured to build the normal class training set {x} N} and the COVID-19 patient training set {x A}, using a deep learning network model trained on the normal class training set {x} N} and the training set of the new crown patients {x A} respectively to construct the Gaussian model group G N of the normal class A ;

[0144] The Mahalanobis distance calculation module 2 is configured to obtain the CT image set {y} to be classified, respectively calculate the Mahalanobis distance between the feature mapping after dimension reduction of the CT image set {y} obtained by the deep learning network model and each layer of the Gaussian model of the normal class G N and each layer of the Gaussian model of the new crown patients G A , and obtain the first Mahalanobis distance and the second Mahalanobis distance

[0145] The determination threshold calculation module 3 is configured to obtain the determination threshold T N of the normal class and the determination threshold T A of the new crown patients according to the Mahalanobis distance of the training set of the normal class {x N} and the training set of the new crown patients {x A} on the Gaussian model of the normal class G N and the Gaussian model of the new crown patients G A .

[0146] The classification module 4 is configured to obtain the classification index of the normal class and the classification index of the new crown patients according to the determination threshold T N of the normal class and the determination threshold T A of the new crown patients and the first Mahalanobis distance and the second Mahalanobis distance , and determine the classification result according to the classification index of the normal class and the classification index of the new crown patients.

[0147] Reference will be made to Figure 7 which shows the structural schematic diagram of a computer device 700 of an electronic device (for example, a server or a terminal device shown in Figure 1 ) suitable for implementing the embodiments of the present application. Figure 7 The electronic device shown is only an example, and should not bring any limitation to the function and use range of the embodiments of the present application.

[0148] As Figure 7As shown, the computer device 700 includes a central processing unit (CPU) 701 and a graphics processor (GPU) 702, which can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 703 or programs loaded into a random access memory (RAM) 704 from the storage section 709. In the RAM 704, various programs and data required for the operation of the device 700 are also stored. The CPU 701, the GPU 702, the ROM 703, and the RAM 704 are connected to each other through a bus 705. An input / output (I / O) interface 706 is also connected to the bus 705.

[0149] Connected to the I / O interface 706 are an input section 707 including a keyboard, a mouse, etc.; an output section 708 including a display such as a cathode ray tube (CRT) display, a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 709 including a hard disk, etc.; and a communication section 710 including a network interface card such as a LAN card, a modem, etc. The communication section 710 performs communication processing via a network such as the Internet. A drive 711 can also be connected to the I / O interface 706 as needed. A removable recording medium 712 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 711 as needed, so that a computer program read therefrom is installed into the storage section 709 as needed.

[0150] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 710, and / or installed from the removable recording medium 712. When the computer program is executed by the central processing unit (CPU) 701 and the graphics processor (GPU) 702, the above-described functions defined in the methods of the present application are performed.

[0151] Note that the computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a computer-readable medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present context, a computer-readable medium can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present context, a computer-readable signal medium can include a computer-readable program code in a baseband or propagated as carrier waves in a propagated data signal associating with a carrier wave. Such a propagated signal can take a wide variety of forms including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium that can be used to carry or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The program contained in the computer-readable medium can be transmitted in any suitable format including, but not limited to, wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0152] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0153] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of apparatuses, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or can sometimes be executed in reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowcharts, and combinations thereof, can be implemented by dedicated hardware-based devices or by a combination of dedicated hardware-based devices and computer instructions.

[0154] The modules involved in the embodiments of the present application can be implemented in software or in hardware. The described modules can also be arranged in a processor.

[0155] As another aspect, the present application also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device. The computer readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: construct a normal class training set {x N} and a new crown patient training set {x A}, construct a Gaussian model group G N of the normal class and a Gaussian model group G A of the new crown patient through a deep learning network model using the normal class training set {x N} and the new crown patient training set {x A}, obtain a to-be-classified CT image set {y}, respectively calculate the Mahalanobis distance between the feature mapping after dimension reduction of the to-be-classified CT image set {y} obtained through the deep learning network model and each layer of the Gaussian model of the normal class G N and each layer of the Gaussian model of the new crown patient G A , and obtain a first Mahalanobis distance and a second Mahalanobis distance According to the Mahalanobis distance of the normal class training set {x N} and the new crown patient training set {x A} on the Gaussian model group G N of the normal class and the Gaussian model group G A of the new crown patient, a judgment threshold T of the normal class is obtained.N and the determination threshold T of the new crown patient A ; the determination threshold T of the normal class N and the determination threshold T of the new crown patient A and the first Mahalanobis distance and the second Mahalanobis distance obtain the classification index of the normal class and the classification index of the new crown patient, and determine the classification result according to the classification index of the normal class and the classification index of the new crown patient.

[0156] The above description is merely preferred embodiments of the present application and a description of the technical principles of the application. It should be understood by those skilled in the art that the scope of the application involved in the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with the technical features disclosed in the present application (but not limited to) having similar functions to form technical solutions.

Claims

1. A novel coronavirus infection CT image classification method based on deep learning and Gaussian modeling, characterized in that, The method comprises the following steps: S1, construct normal class training set {x N} and new crown patient training set {x A}, use the normal class training set {x N} and new crown patient training set {x A} to construct Gaussian model group G N of normal class and Gaussian model group G A of new crown patient through deep learning network model, specifically comprising: S11, Feature mapping {f} based on the training set i }, calculate the corresponding mean vector Covariance Matrix For covariance matrix Principal component analysis is performed to obtain the eigenvector matrix V and the eigenvalue matrix D, where the eigenvalue matrix D is defined by the eigenvalues ​​λ. i ,i∈[1,N F [A] is a diagonal matrix with diagonal elements; S12, divide the feature value λ i into two groups: and The corresponding feature vector space is divided into two subspaces: a significant feature subspace Φ1 and an insignificant feature subspace Φ2; the estimation method of the subspace boundary point is: wherein m r is the rank of the covariance matrix, λ med is the median of the eigenvalues, the first m eigenvectors of the eigenvector matrix V belong to the significant eigensubspace Φ1, and the remaining eigenvectors belong to the insignificant eigensubspace Φ2. S13, constructing the insignificant feature subspace Φ2 into a transformation matrix, left multiplying the transformation matrix for each feature mapping to obtain the reduced dimension feature mapping the mean vector of the reduced dimension feature mapping and the covariance matrix is expressed as: Wherein, Ns represents the number of training set samples; S14, using the normal class training set {x N} and the new crown patient training set {x A} respectively, constructing a Gaussian model through the feature map extracted by each layer network of the deep learning network model, and obtaining a Gaussian model group of the normal class and a Gaussian model group of the new crown patient respectively, wherein s represents the number of network layers of the deep learning network model; S2, obtaining a CT image set {y} to be classified, respectively calculating the Mahalanobis distance between each layer of Gaussian model in the Gaussian model group G N of the normal class and the reduced dimension feature mapping of the CT image set {y} to be classified obtained by the deep learning network model, and the Mahalanobis distance between each layer of Gaussian model in the Gaussian model group G A of the new crown patient, obtaining a first Mahalanobis distance and a second Mahalanobis distance S3, obtaining a decision threshold T N} and a new crown patient training set {x A} on the Gaussian model group G N} of the normal class and the Gaussian model group G A of the new crown patient, obtaining a decision threshold T N of the normal class and a decision threshold T A of the new crown patient. S4, based on the normal class determination threshold T N And the threshold for determining COVID-19 patients T A and the first Mahalanobis distance Second Mahalanobis distance The classification indicators for normal classes and COVID-19 patients are obtained, and the classification results are determined based on the classification indicators for normal classes and COVID-19 patients.

2. The deep learning and Gaussian modeling-based novel coronavirus infection CT image classification method according to claim 1, characterized in that, The deep learning network model has several layers of networks, each layer of network adopting a classifier Ψ with parameters θ θ (·) performs deep feature extraction to map the CT image x i to a deep feature representation domain f i = Ψ θ (x i ), N F is the feature dimension. 3.The deep learning and Gaussian modeling based COVID-19 CT image classification method of claim 1, wherein, The deep learning network model comprises ResNet or EfficientNet. 4.The deep learning and Gaussian modeling based COVID-19 CT image classification method of claim 1, wherein, The step S2 specifically comprises: The CT images in the CT image set to be classified are sequentially input into the deep learning network model, and through steps S11-S13, the feature mapping after dimension reduction is obtained Computing The Mahalanobis distance matrix dd = [d 1 ,d 2 ,…,d s ] between each layer of Gaussian models, wherein: After the Mahalanobis distance matrix dd is normalized by column and then averaged by row, the Mahalanobis distance between the CT image set to be classified and the normal class training set {x N} or the new crown patient training set {x A} is represented as: wherein, with respectively represent the mean and standard deviation of the Mahalanobis distance calculated for the jth layer feature of all samples of the normal class training set {x N} or the new crown patient training set {x A}, and the first Mahalanobis distance and the second Mahalanobis distance 5. The deep learning and Gaussian modeling-based novel coronavirus infection CT image classification method according to claim 1, characterized in that, The step S3 specifically comprises: The lower limit of the threshold value is determined by the maximum value of the Mahalanobis distance of the normal class training set {x N} on the Gaussian model G N of the normal class The upper limit of the threshold value is determined by the minimum value of the Mahalanobis distance of the new crown patient training set {x A} on the Gaussian model G N of the normal class The determination threshold T N of the normal class is set as: According to the training set of COVID-19 patients {x A } In the Gaussian model G of the COVID-19 patient A The maximum Mahalanobis distance on the threshold determines the lower limit of the threshold. According to the normal class training set {x N } In the Gaussian model G of the COVID-19 patient A The minimum Mahalanobis distance on the threshold determines the upper limit of the threshold. The threshold T for determining COVID-19 patients A Set to:

6. The deep learning and Gaussian modeling-based novel coronavirus infection CT image classification method according to claim 1, characterized in that, The step S4 specifically comprises: According to the decision threshold T of the normal class N and the first Mahalanobis distance defining a classification indicator w of the normal class N : According to the decision threshold T for new crown patients A and the second Mahalanobis distance Define the classification index w of new crown patients A : when w k t indicates that the CT image in the set of CT images to be classified {y} does not belong to the k class, k being N or A, t being a threshold, while when w k t indicates that the CT image in the set of CT images to be classified {y} belongs to the k class, there are four possible situations when classifying the set of CT images to be classified {y}: a) w N ≤ t and w A > t; b) w N t and w A ≤ t; c) w N ≤ t and w A ≤ t; d) w N t and w A t; In response to determining that the classification index of the CT image in the CT image set {y} to be classified meets condition a), the CT image in the CT image set {y} to be classified is determined as normal class; In response to determining that the classification index of the CT image in the CT image set {y} to be classified meets condition b), the CT image in the CT image set {y} to be classified is determined as a COVID-19 patient; In response to determining that the classification index of the CT image in the CT image set {y} to be classified meets conditions c) and d), a comprehensive class evaluation index w is calculated: w = aw N + (1 - a)(1 - w A ); wherein, the ratio of the number of samples of the normal class training set {x N} and the number of samples of the new crown patient training set {x A}; and the comprehensive class evaluation index w is taken as a reference for auxiliary diagnosis. 7.A CT image classification device for novel coronavirus infection based on deep learning and Gaussian modeling, characterized by, The method comprises the following steps: The Gaussian model construction module is configured to construct a normal class training set {x N} and a new crown patient training set {x A}, and construct a Gaussian model group G N of the normal class and a Gaussian model group G A of the new crown patient respectively by using the normal class training set {x N} and the new crown patient training set {x A} through a deep learning network model, and specifically includes: Feature mapping {f} based on the training set i }, calculate the corresponding mean vector Covariance Matrix For covariance matrix Principal component analysis is performed to obtain the eigenvector matrix V and the eigenvalue matrix D, where the eigenvalue matrix D is defined by the eigenvalues ​​λ. i ,i∈[1,N F [A] is a diagonal matrix with diagonal elements; By setting the demarcation point, the eigenvalue λ i is divided into two groups: and The corresponding eigenvector space is divided into two subspaces: the significant feature subspace Φ1 and the insignificant feature subspace Φ2; the estimation method of the subspace demarcation point is: where m r is the rank of the covariance matrix, λ med is the median of the eigenvalues, the first m eigenvectors of the eigenvector matrix V belong to the significant eigensubspace Φ1, and the remaining eigenvectors belong to the insignificant eigensubspace φ2. The insignificant feature subspace Φ2 is constructed as a transformation matrix, and left multiplication of the transformation matrix obtains the dimension-reduced feature mapping for each feature mapping The mean vector of the dimension-reduced feature mapping And the covariance matrix is expressed as: Wherein, Ns represents the number of training set samples; respectively using the normal class training set {x N} and the new crown patient training set {x A}, a Gaussian model is constructed by extracting a feature map from each layer network of the deep learning network model, and a Gaussian model group of the normal class and a Gaussian model group of the new crown patient are obtained respectively, wherein s represents the number of network layers of the deep learning network model; a Mahalanobis distance calculation module configured to obtain a CT image set {y} to be classified, respectively calculate the Mahalanobis distance between each layer of Gaussian model in the Gaussian model group G N of the normal class and the Gaussian model group G A of the new crown patient obtained by the reduced dimension feature mapping of the CT image set {y} to be classified through the deep learning network model, to obtain a first Mahalanobis distance and a second Mahalanobis distance A decision threshold calculation module is configured to obtain a normal class decision threshold T N} and a new crown patient training set {x A} Mahalanobis distance on the Gaussian model group G N of the normal class and the Gaussian model group G A of the new crown patient to obtain a normal class decision threshold T N and a new crown patient decision threshold T A ; a classification module configured to determine a classification result according to the classification index of the normal class and the classification index of the new crown patient N and the determination threshold T of the new crown patient A and the first Mahalanobis distance and the second Mahalanobis distance obtain the classification index of the normal class and the classification index of the new crown patient, and determine a classification result according to the classification index of the normal class and the classification index of the new crown patient. 8.An electronic device, comprising: one or more processors; a memory device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of any one of claims 1-6.

Images