Lung disease recognition method, device, medium and electronic device based on CT data

By blocking the lung CT data, block tensors are generated for classification identification, the problems of lesion information loss and noise in the existing technology are solved, accurate identification and lesion positioning of lung diseases are achieved, and labeling and characteristics are reduced. The dependence of design improves diagnostic efficiency.

CN113850796BActive Publication Date: 2024-11-12GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111187917.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-12
Publication Date
2024-11-12
Estimated Expiration
2041-10-12

AI Technical Summary

Technical Problem

In the existing lung CT image-assisted analysis methods, fixed-size 3D tensor sampling leads to the loss of lesion information or the introduction of repeated noise, feature classifiers require accurate labels to consume time and effort, and artificial design of feature space is likely to lead to model overfitting.

Method used

By blocking the pulmonary CT data to be identified, k block tensors are generated, and these block tensors are classified and identified for lung disease, output recognition results and determine lesion block tensors, avoiding sampling and labeling The defects of the model reduce the dependence of the feature quality.

Benefits of technology

Accurate identification of lung diseases and rough lesions are achieved, the absence of key information and noise introduction are avoided, and the need for precise labeling and artificial design of feature space is eliminated, which improves the generalization ability and diagnostic efficiency of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113850796B_ABST
    Figure CN113850796B_ABST
Patent Text Reader

Abstract

The present disclosure provides a lung disease recognition method, device, computer-readable medium and electronic device based on CT data, and relates to the field of image processing technology. The method includes: obtaining a 3D tensor corresponding to the lung CT data to be identified, and dividing the 3D tensor into blocks to obtain k block tensors; performing lung disease classification and recognition on the k block tensors to obtain k block classification results; outputting the recognition result corresponding to the lung CT data to be identified according to the k block classification results, and determining the lesion block tensor corresponding to the lung CT data to be identified in the k block tensors when the recognition result is an abnormal result. The present disclosure can not only avoid sampling of the 3D tensor by dividing the 3D tensor into blocks, but also avoid the missing of key lesion information or the introduction of repeated invalid noise; at the same time, the lesion block tensor can be directly determined in the k block tensors through the k block classification results, and then the coarse positioning of the lesion is achieved based on the lesion block tensor.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0002] CT (Computed Tomography) is an electronic computer tomography. Its imaging principle is to use X-ray beams, gamma rays, ultrasound, etc. to scan a certain thickness of a specific part of the human body, and obtain medical images after computer processing. Compared with conventional imaging examination methods, CT has the advantages of being able to obtain true cross-sectional images, high density resolution, quantitative analysis, and convenient subsequent image processing. Therefore, it has been increasingly widely used in medical imaging detection.

[0003] In recent years, the rapid development of computer-aided diagnosis technology has greatly facilitated the diagnosis and analysis of medical CT images. For lung CT images, there are usually three auxiliary analysis methods: one is to sample the 3D data of lung CT into a fixed-size 3D tensor through interpolation or sampling, and then input the fixed-size 3D tensor into a 3D convolution model for diagnosis; the second is to achieve diagnosis based on the feature classifier of the previous segmentation task, which requires the type, type and quantity of features such as color, texture and shape features of the lesions to be manually designed and extracted, and input into the classifier for diagnosis; the third is to achieve diagnosis based on the feature classifier of fuzzy patterns, which can construct feature space through features manually designed or automatically extracted by convolutional neural networks, and perform dimensionality reduction and weighting of features based on fuzzy patterns in the feature space.

[0004] However, among the three auxiliary analysis methods mentioned above, the first method involves the sampling of fixed-size 3D tensors, which may easily lead to the loss of key lesion information or the introduction of repeated invalid noise due to the lack of prior information; the second method involves the need to provide accurate lesion area labels when training the feature classifier, so labeling is time-consuming and labor-intensive; the third method requires manual design of feature space or feature space screening strategy, which is highly dependent on the feature quality of the input end and easily leads to model overfitting. Summary of the invention

[0005] The purpose of the present disclosure is to provide a lung disease identification method based on CT data, a lung disease identification device based on CT data, a computer-readable medium and an electronic device, which can provide spatial positioning information of lesions by determining lesion block tensors on the basis of identifying lung diseases according to complete original CT data.

[0006] According to a first aspect of the present disclosure, a method for lung disease recognition based on CT data is provided, comprising: obtaining a 3D tensor corresponding to lung CT data to be recognized, and dividing the 3D tensor into blocks to obtain k block tensors; k is a positive integer greater than or equal to 2; performing lung disease classification and recognition on the k block tensors to obtain k block classification results; outputting a recognition result corresponding to the lung CT data to be recognized based on the k block classification results, and determining a lesion block tensor corresponding to the lung CT data to be recognized in the k block tensors when the recognition result is an abnormal result.

[0007] According to a second aspect of the present disclosure, a lung disease identification device based on CT data is provided, comprising: a block acquisition module, used to acquire a 3D tensor corresponding to lung CT data to be identified, and block the 3D tensor to obtain k block tensors; k is a positive integer greater than or equal to 2; a classification and identification module, used to classify and identify lung diseases on the k block tensors to obtain k block classification results; a result output module, used to output the identification result corresponding to the lung CT data to be identified according to the k block classification results, and determine the lesion block tensor corresponding to the lung CT data to be identified in the k block tensors when the identification result is an abnormal result.

[0008] According to a third aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0009] According to a fourth aspect of the present disclosure, an electronic device is provided, characterized in that it includes: a processor; and a memory for storing one or more programs, which enables the one or more processors to implement the above method when the one or more programs are executed by the one or more processors.

[0010] An embodiment of the present disclosure provides a lung disease recognition method based on CT data. By acquiring a 3D tensor corresponding to the lung CT data to be identified, and dividing the 3D tensor into blocks, k block tensors can be obtained. Then, the k block tensors are respectively subjected to lung disease classification and recognition to obtain k block classification results. Then, the recognition results corresponding to the lung CT data to be identified can be output according to the k block classification results, and the lesion block tensor can be determined in the k block tensors. By dividing the 3D tensor into blocks, not only can the sampling of the 3D tensor be avoided, but also the missing of key lesion information or the introduction of repeated invalid noise can be avoided; at the same time, the lesion block tensor can be directly determined in the k block tensors through the k block classification results, and the coarse positioning of the lesion can be achieved based on the lesion block tensor.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0013] Figure 1 A schematic diagram showing an exemplary system architecture to which embodiments of the present disclosure may be applied;

[0014] Figure 2 A schematic diagram showing an electronic device to which the embodiments of the present disclosure can be applied;

[0015] Figure 3 A flowchart schematically illustrates a method for identifying lung diseases based on CT data in an exemplary embodiment of the present disclosure;

[0016] Figure 4 A schematic diagram of a process for determining a recognition result and a lesion segmentation tensor in an exemplary embodiment of the present disclosure is schematically shown;

[0017] Figure 5 A schematic diagram schematically illustrates a training process of a neural network for identifying lung diseases in an exemplary embodiment of the present disclosure;

[0018] Figure 6 A schematic diagram of another process for determining the recognition result and the lesion segmentation tensor in an exemplary embodiment of the present disclosure is schematically shown;

[0019] Figure 7 The following is a schematic diagram showing the composition of a lung disease identification device based on CT data in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0020] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0021] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0022] Figure 1 A schematic diagram of a system architecture of an exemplary application environment in which a method and apparatus for identifying lung diseases based on CT data according to an embodiment of the present disclosure can be applied is shown.

[0023] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with image recognition functions, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0024] The lung disease identification method based on CT data provided by the embodiment of the present disclosure can be executed by the terminal devices 101, 102, and 103, and accordingly, the lung disease identification device based on CT data is generally set in the terminal devices 101, 102, and 103; it can also be executed by the server 105, and accordingly, the lung disease identification device based on CT data can also be set in the server 105, and this is not particularly limited in this exemplary embodiment. For example, in an exemplary embodiment, the terminal device 101 can be a device that can directly obtain the 3D tensor corresponding to the lung CT data, such as a tomography device, and other terminal devices 102, 103 and the server 105 can collect the 3D tensor corresponding to the lung CT data collected by the terminal device 101 through the network, and then perform lung disease identification.

[0025] An exemplary embodiment of the present disclosure provides an electronic device for implementing a lung disease identification method based on CT data, which may be Figure 1The terminal device 101, 102, 103 or the server 105 in the embodiment of the present invention comprises at least a processor and a memory, wherein the memory is used to store executable instructions of the processor, and the processor is configured to execute the lung disease identification method based on CT data by executing the executable instructions.

[0026] Below Figure 2 Taking the mobile terminal 200 in FIG. 1 as an example, the structure of the electronic device is exemplarily described. It should be understood by those skilled in the art that, in addition to the components specifically used for mobile purposes, Figure 2 The structure in the figure can also be applied to fixed type devices. In other embodiments, the mobile terminal 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between the components is only shown schematically and does not constitute a structural limitation of the mobile terminal 200. In other embodiments, the mobile terminal 200 may also adopt the same Figure 2 Different interface connection methods, or a combination of multiple interface connection methods.

[0027] like Figure 2 As shown, the mobile terminal 200 may specifically include: a processor 210, an internal memory 221, an external memory interface 222, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 271, a receiver 272, a microphone 273, an earphone interface 274, a sensor module 280, a display screen 290, a camera module 291, an indicator 292, a motor 293, a button 294, and a subscriber identification module (SIM) card interface 295, etc. The sensor module 280 may include a depth sensor 2801, a pressure sensor 2802, a gyroscope sensor 2803, etc.

[0028] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor and / or a neural network processor (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0029] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the mobile terminal 200 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0030] In some embodiments, the NPU can be used to implement the steps of classifying and identifying lung diseases on k block tensors to obtain k block classification results, outputting the identification results corresponding to the lung CT data to be identified based on the k block classification results, determining the lesion block tensor corresponding to the lung CT data to be identified in the k block tensors, and adjusting and determining the parameters in the lung disease identification neural network.

[0031] The processor 210 is provided with a memory. The memory can store instructions for implementing six modular functions: detection instructions, connection instructions, information management instructions, analysis instructions, data transmission instructions, and notification instructions, and the execution is controlled by the processor 210.

[0032] The mobile terminal 200 implements the display function through a GPU, a display screen 290, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 290 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs that execute program instructions to generate or change display information. In some embodiments, steps such as the segmentation of 3D tensors can be implemented by the GPU.

[0033] In the related art, three auxiliary analysis methods are usually included for lung CT images.

[0034] One is to sample the 3D data of lung CT into a fixed-size 3D tensor through interpolation or sampling, and then input the fixed-size 3D tensor into a 3D convolutional model for diagnosis. For example, in the article titled "Joint learning of 3D lesion segmentation and classification for explainable COVID-19 diagnosis" written by Wang X, Jiang L, Li L, etc., a 3D convolutional neural network with three branches was proposed. The CT data was scaled and interpolated and downsampled, and the 3D convolutional neural network 1 was used to extract the underlying fusion features at multiple scales; then the 3D convolutional neural network branch 2 was used to process some of the features output by branch 1 and applied to the lesion segmentation task, and the lesion segmentation results were output; finally, the 3D convolutional neural network branch 3 was used to receive some of the features output by branch 1 and the segmentation results output by branch 2, and the final recognition results of new coronary lung disease and community-acquired lung disease were output.

[0035] However, this method requires sampling the lung CT data into a fixed-size 3D tensor through interpolation or sampling at the input end, and then inputting the 3D tensor into the 3D convolution model. Due to the lack of prior information, it is very easy to cause the loss of key lesion information or the introduction of repeated invalid noise.

[0036] The second is to achieve diagnosis based on the feature classifier of the previous segmentation task, which requires the type, category and quantity of artificially designed and extracted features such as color, texture and shape features of the lesions, and input them into the classifier to achieve diagnosis. For example, the patent application with publication number CN111724356A provides an image processing method and system for CT image lung disease recognition. The image processing method first extracts the lung disease area mask based on the lung disease segmentation model, and then adjusts the weight of the convolution feature at the spatial level by introducing the attention mechanism to make the lung disease area feature more prominent; the three-dimensional CT sequence is scaled according to the cross-sectional position, and each positive layer of the sequence (layer with lung disease) is classified layer by layer, and then the corresponding lung disease mask area area and the classification result are weighted and then the category with the highest probability is voted to obtain the lung disease classification result of the three-dimensional CT sequence.

[0037] However, the feature classifier based on the previous segmentation task used in this method requires accurate lesion area labels when training the model. Since labeling needs to be done manually, it is not only time-consuming and labor-intensive, but also limited by the different understanding of professional knowledge by labelers, resulting in uneven labeling quality, a small amount of corresponding training samples, and the generalization ability of the trained model needs to be verified.

[0038] The third is to achieve diagnosis based on the feature classifier of fuzzy patterns. The feature space can be constructed through features manually designed or automatically extracted by convolutional neural networks, and the feature space can be reduced in dimension and weighted based on the fuzzy patterns. For example, the patent application with publication number CN111414956A provides a multi-instance learning and recognition method for fuzzy patterns in lung CT images, which uses a pre-trained convolutional neural network to extract features from the acquired CT images, regards a single CT image as a package, and uses the features extracted from a single CT image as an example, and uses a multi-instance learning model (including the k-nearest neighbor model Citation-KNN, the multi-instance support vector machine model MI-SVM, and the expectation maximization-diversity density model EM-DD) to perform the classification task.

[0039] However, this method mainly uses features designed manually or automatically extracted by convolutional neural networks to construct feature space or feature space screening strategies, which is highly dependent on the feature quality at the input end and can easily lead to model overfitting.

[0040] Based on one or more of the above problems, this exemplary embodiment provides a lung disease recognition method based on CT data. The lung disease recognition method based on CT data can be applied to the above server 105, and can also be applied to one or more of the above terminal devices 101, 102, 103, which is not particularly limited in this exemplary embodiment. Figure 3 As shown, the lung disease identification method based on CT data may include the following steps S310 to S320:

[0041] In step S310, a 3D tensor corresponding to the lung CT data to be identified is obtained, and the 3D tensor is divided into blocks to obtain k block tensors.

[0042] Wherein, k is a positive integer greater than or equal to 2; the lung CT data to be identified may include lung CT data in different formats, such as DICOM format, NII format, and single-channel image sequence mode; correspondingly, the 3D tensor may also include a 3D tensor converted from lung CT data in different formats.

[0043] In an exemplary embodiment, before the 3D tensor is subsequently processed, the 3D tensor may be preprocessed. Specifically, the preprocessing process may include normalization, scaling, denoising, and the like.

[0044] Among them, since the contents stored in different formats of CT data are different. For example, HU values ​​are stored in DICOM and NII formats, and pixel grayscale values ​​are stored in single-channel image sequences. Therefore, it is necessary to normalize the 3D tensor and normalize it to a specific range, such as [0, 1]. It should be noted that normalization is processing of numerical values, so the lung CT data to be identified can also be directly normalized and then converted into a 3D tensor.

[0045] It should be noted that before normalization, the 3D tensor can also be windowed, that is, the HU values ​​or pixel grayscale values ​​outside the window range are filtered out, and the HU values ​​or pixel grayscale values ​​within the window range are retained. Through this processing method, some extremely large or extremely small HU values ​​or pixel grayscale values ​​can be filtered out to avoid the problem that the normalized data is too aggregated due to the existence of extremely large or extremely small values, which leads to the problem of unrecognizable. For example, assuming that the pixel grayscale is distributed between 0-100, if it is normalized to the range of [0, 1], it is only necessary to divide each value by 100; assuming that most of the pixel grayscale is distributed between 0-100, but there is a maximum value of 255, then each value needs to be divided by 255, which will cause the pixel grayscale values ​​distributed between 0-100 to be compressed and distributed between 0 and 0.39, and the part between 0.39 and 1 has only one pixel value. It is likely that the grayscale values ​​are too concentrated, and the difference between the grayscale values ​​is small, which leads to the problem of unrecognizable.

[0046] Among them, since the 3D tensors obtained after the conversion of the lung CT data to be identified collected by the CT device may be of different scales, such as different resolutions. Therefore, scale scaling can be performed on the basis of the 3D tensor, and the 3D tensors of different resolutions are interpolated by the nearest neighbor on the cross-sectional slices and scaled to a predetermined size, such as H (image height) × W (image width). It should be noted that when performing scale scaling, it is not necessary to scale the image data contained in the 3D tensor to avoid losing data at certain locations of the lungs.

[0047] Among them, noise may be introduced during the acquisition process, so the noise that may exist in the 3D tensor can be removed through denoising.

[0048] In order to convert the 3D tensor into a form more suitable for analysis and processing by a computer device such as a GPU, the 3D tensor may be subjected to contrast enhancement, for example, contrast enhancement based on histogram equalization.

[0049] In an exemplary embodiment, when a 3D tensor is divided into blocks, it can be divided into blocks according to different block rules. Among them, when the size of the 3D tensor is the number of images × image height × image width, the 3D tensor can be divided into blocks in sequence based on a pre-set block parameter to obtain k-1 block tensors with a size of block parameter × image height × image width, and then the remaining undivided part is used as 1 block tensor to obtain k tensors. It should be noted that when dividing into blocks, the size of the remaining part is the remaining number × image height × image width, and the remaining number is less than or equal to the block parameter.

[0050] For example, suppose the size of a 3D tensor is D×H×W, and the preset block parameter is N. At this time, if D / N is divisible, then D / N=k, and a total of k block tensors of size N×H×W are obtained, that is, the remaining number is equal to the block parameter; if D / N is not divisible, then D / N is rounded up to be equal to k, and a total of k-1 block tensors of size N×H×W and 1 block tensor of size (D-(k-1)N)×H×W are obtained. Among them, (D-(k-1)N) is the remaining number, and the remaining number is less than the block parameter.

[0051] In step S320, lung disease classification and identification are performed on the k block tensors to obtain k block classification results.

[0052] In an exemplary embodiment, after obtaining k block tensors, lung disease classification and identification can be performed for each block tensor respectively, and k block classification results can be obtained accordingly. Specifically, when performing lung disease classification and identification, machine learning, deep learning, etc. can be used. For example, the k block tensors can be classified and identified by a trained 3D convolutional neural network (such as 3DResNet-18 Model, 3D DenseNet, etc.).

[0053] In step S330, the recognition result corresponding to the lung CT data to be recognized is output according to the k block classification results, and when the recognition result is an abnormal result, the lesion block tensor corresponding to the lung CT data to be recognized is determined in the k block tensors.

[0054] Among them, the recognition results corresponding to the lung CT data to be identified can have multiple preset categories. For example, for pneumonia identification, the recognition results corresponding to a certain lung CT data to be identified can include three types: normal state, common pneumonia, and new coronary pneumonia. Correspondingly, the preset categories can be set to normal, common pneumonia, and new coronary pneumonia. On this basis, each block classification result includes the probability that the block tensor belongs to each preset category. For example, after a certain block is identified, the recognition results obtained may include: the probability of belonging to the normal state is 80%, the probability of belonging to common pneumonia is 11%, and the probability of belonging to new coronary pneumonia is 9%.

[0055] Among them, an abnormal result means that the recognition result corresponding to the lung CT data to be recognized is a result containing lung diseases. For example, for pneumonia, the recognition results include normal, common pneumonia and new coronary pneumonia, among which common pneumonia and new coronary pneumonia are both abnormal results.

[0056] In an exemplary embodiment, after obtaining k block classification results, the recognition result and lesion block tensor corresponding to the lung CT data to be identified can be determined based on the k block classification results. Specifically, machine learning, deep learning, etc. can be used. For example, the k block classification results can be integrated through a Bayesian Noisy-Or model to output the recognition result and lesion block tensor corresponding to the lung CT data to be identified.

[0057] In an exemplary embodiment, when the recognition results include multiple preset categories and the block classification results include the probability that the block tensor belongs to each preset category, when the recognition results corresponding to the lung CT data to be identified are output based on the k block classification results, for each preset category, the total probability corresponding to the preset category can be calculated based on the k block classification results, and then the preset category with the largest total probability is determined as the recognition result corresponding to the lung CT data to be identified.

[0058] In an exemplary embodiment, when calculating the total probability corresponding to a preset category, the calculation can be performed based on Formula 1. For example, assuming k=2, the block classification results corresponding to the two blocks are: the probability of belonging to the normal state is 80%, the probability of belonging to common pneumonia is 10%, and the probability of belonging to new coronary pneumonia is 10%; the probability of belonging to the normal state is 60%, the probability of belonging to common pneumonia is 30%, and the probability of belonging to new coronary pneumonia is 10%. At this time, based on Formula 1, it can be calculated that the total probability of belonging to the normal state is 92%, the total probability of belonging to common pneumonia is 37%, and the total probability of belonging to new coronary pneumonia is 19%. Formula (1) is as follows:

[0059]

[0060] Among them, P is the total probability corresponding to a preset category, and pt is the probability that the tth block belongs to the preset category.

[0061] In an exemplary embodiment, when the recognition result is an abnormal result, that is, when a lung disease exists, the lesion block tensor corresponding to the lung CT data to be identified can be determined in the k block tensors according to the k block classification results. Specifically, when the block classification result includes the probability that the block tensor belongs to each preset category, the target probability that the block tensor in the block classification result belongs to the recognition result can be obtained for each block classification result, and then the block tensor corresponding to the maximum target probability among the obtained k target probabilities is determined as the lesion block tensor corresponding to the lung CT data to be identified. By determining the lesion block tensor in the 3D tensor, the location where the abnormal result is most likely to occur can be located to assist medical staff and other personnel in making a diagnosis based on the determined spatial position.

[0062] In addition, in an exemplary embodiment, when the above steps S320 and S330 are performed based on artificial intelligence such as machine learning or deep learning, reference is made to Figure 4 As shown, k block tensors can be first taken as input, and the block tensors can be classified based on the lung disease classification model to obtain k block classification results. After obtaining the k block classification results, the k block classification results are taken as input, and the k block classification results are integrated based on the integration model to output the recognition result corresponding to the lung CT data to be identified, and when the recognition result is an abnormal result, the lesion block tensor corresponding to the lung CT data to be identified is determined in the k block tensors.

[0063] Among them, the lung disease classification model and the integrated model can be obtained by training the constructed lung disease recognition neural network. Specifically, during training, multiple groups of sample data can be obtained first, each group of sample data includes a block tensor corresponding to the lung CT data, and the recognition result label corresponding to the lung CT data; then, the multiple groups of sample data are divided into training sets, validation sets and test sets in groups, and then a lung disease recognition neural network is constructed, and the parameters in the lung disease recognition neural network are adjusted and determined according to the lung CT data included in the training set, validation set and test set, and the lung disease classification model and the integrated model are obtained. Among them, the constructed lung disease recognition neural network can include a lung disease classification network and an integrated network connected in series after the lung disease classification network.

[0064] The following is based on the 3D ResNet-18 Model, using the model weights pre-trained on the ImageNet dataset to initialize the model, and modifying the output dimension of the last fully connected layer of the model to 3. The model is a lung disease classification network, with the Bayesian Noisy-Or model as the integrated network, k=4, and the preset classifications including normal, common pneumonia, and new coronary pneumonia as examples. Figure 5 and Figure 6 The training process of the above model is explained in detail:

[0065] Reference Figure 5 As shown, during model training, the sample data can be divided into a training set, a validation set, and a test set, and then training is performed based on the training set, validation is performed based on the validation set, and testing is performed based on the test set to obtain a trained lung disease classification model and integration model.

[0066] Before training based on the training set, the data in the training set can be enhanced. For example, the 3D tensor can be randomly flipped horizontally and vertically, randomly rotated (in the range of 0-359°), randomly sheared (at least 81% of the original image area is retained), and random Gaussian noise is added (the mean of Gaussian noise is 0, the variance range is 0-0.1, and the noise coefficient range is 0-4).

[0067] After obtaining the enhanced training set, the lung disease recognition network can be trained based on the enhanced training set to update the parameters and weights in the network. The lung disease recognition neural network can be obtained by connecting the above integration network in series with the above lung disease classification network.

[0068] Specifically, each set of sample data contains 4 block tensors that are input into the lung disease classification network, and 4 normalized block prediction information vectors (block classification results) of size 3×1 are derived. The vector elements represent the probability that the block belongs to normal, common pneumonia, and new coronary pneumonia. After obtaining the block classification results, the block classification results are input into the integration network, and the integration network outputs the final recognition result, which is compared with the recognition result label corresponding to the sample data, and the loss function is calculated. The stochastic gradient descent method is then used to minimize the loss function, and the parameters and weights in the lung disease recognition neural network are updated by back propagation.

[0069] Among them, the cross entropy with L2 regularization can be used as the loss function for the COVID-19 and common pneumonia identification tasks, as shown in the following formula (2):

[0070]

[0071] Among them, yi is a Boolean value. When the sample data belongs to the i-th class, yi=1; otherwise, yi=0. pi represents the final predicted probability of the i-th class output by the integration layer; ||w|| 2 Represents the L2 norm of the model parameters. The larger the value, the more complex the model is and the more likely it is to overfit. λ is the L2 regularization coefficient, which is used to constrain the model complexity.

[0072] In this embodiment, λ=1×10 -4 , initial learning rate lr = 2 × 10 -3 , a better recognition result can be achieved by multiplying the learning rate by a decay coefficient of 0.8 after every 20 iterations.

[0073] After training, the lung disease classification network with updated parameters of the training set can be verified based on the validation set. Specifically, the lung disease classification network is debugged based on the validation set data to decide whether to save the parameters and weights of the lung disease classification network. Specifically, the Kappa coefficient κ can be used as an evaluation indicator to evaluate the model performance. For the three-classification problem of normal, common pneumonia, and new coronary pneumonia, the Kappa coefficient κ can be determined by the following formula (3):

[0074]

[0075] Among them, p o It represents the accuracy of overall classification, that is, the ratio of the number of correctly classified sample data to the total number of sample data, p e The hypothetical probability of chance consistency is calculated as follows:

[0076]

[0077] Among them, M represents the total number of sample data, xi represents the total number of identification result labels of the i-th category, and zi represents the total number of sample data predicted as the i-th category identification result. Among them, the closer the Kappa coefficient κ is to 1, the better the recognition effect of the lung disease classification network for COVID-19 and common pneumonia after updating the parameters based on the training set.

[0078] When the Kappa coefficient calculated on the validation set is close to 1 to the extent that it meets the preset requirements, the parameters and weights can be retained, and then the retained parameters can be tested based on the test set. Specifically, the test set data is input so that the lung disease classification network with retained parameters and weights can output the recognition result and the lesion block tensor. Among them, for the recognition result, among the block classification results corresponding to the block tensor, the block tensor with the highest probability corresponding to the recognition result is the lesion block tensor, which is ultimately the most likely location of the lesion.

[0079] After the test, the lung disease classification network in the lung disease classification network is the trained lung disease classification model, and after the test, the integrated network in the lung disease classification network is the trained integrated model.

[0080] After obtaining the trained lung disease classification model and integrated model, lung disease identification can be completed based on the lung disease classification model and integrated model. Figure 6 As shown, for the 3D tensor corresponding to the lung CT data to be identified, the 3D tensor can be divided into 4 block tensors, and then the 4 block tensors are input into the lung disease classification model, and 4 block classification results corresponding to the 4 block tensors can be output. Then, the 4 block classification results are input into the value integration model, and the identification result and lesion block tensor corresponding to the lung CT data to be identified can be output.

[0081] It should be noted that in the above-mentioned 3D ResNet-18 Model as the basis, the model weights pre-trained on the ImageNet dataset are used to initialize the model, and the output dimension of the last fully connected layer of the model is modified to 3. The model is a lung disease classification network, and in the embodiment with the Bayesian Noisy-Or model as the integrated network, it is found through experiments that when the 3D tensor is scaled to H=51, W=512, and N=2 or 4, better recognition effect can be obtained.

[0082] In addition, no data augmentation is required for the data input in the validation set and the test set.

[0083] In summary, this exemplary embodiment provides a lung disease recognition method based on multi-instance learning, which is based on lung CT data and can accurately recognize lung diseases. On the one hand, this method does not rely on specific interpolation or sampling techniques, retains detailed information of the original lung CT data, does not impose any a priori restrictions on the size and resolution of the lung CT data at the input end, and can be applied to lung CT data with different resolutions and different numbers of cross-sectional slices; on the other hand, when training the model, there is no need to perform additional annotation of the blocks, and the rough spatial position of the lesion can be located without segmented area annotation.

[0084] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0085] For further reference, Figure 7As shown, in the embodiment of this example, a lung disease identification device 700 based on CT data is also provided, including a block acquisition module 710, a classification identification module 720 and a result output module 730. Among them:

[0086] The block acquisition module 710 can be used to obtain a 3D tensor corresponding to the lung CT data to be identified, and block the 3D tensor to obtain k block tensors; k is a positive integer greater than or equal to 2.

[0087] The classification and identification module 720 can be used to perform lung disease classification and identification on the k block tensors to obtain k block classification results.

[0088] The result output module 730 can be used to output the recognition result corresponding to the lung CT data to be recognized based on the k block classification results, and determine the lesion block tensor corresponding to the lung CT data to be recognized in the k block tensors when the recognition result is an abnormal result.

[0089] In an exemplary embodiment, when the size of the 3D tensor is the number of images × image height × image width, the block acquisition module 710 can be used to block the 3D tensor in sequence based on the blocking parameter to obtain k-1 block tensors with a size of blocking parameter × image height × image width; the remaining part of the 3D tensor that has not been blocked is taken as 1 block tensor; wherein the size of 1 block tensor is the remaining number × image height × image width, and the remaining number is less than or equal to the blocking parameter.

[0090] In an exemplary embodiment, when the block classification result includes the probability that the block tensor belongs to each preset category, the result output module 730 can be used to obtain the target probability that the block tensor in the block classification result belongs to the identification result for each block classification result; and determine the block tensor corresponding to the maximum target probability among the k target probabilities as the lesion block tensor corresponding to the lung CT data to be identified.

[0091] In an exemplary embodiment, when the block classification result includes the probability that the block tensor belongs to each preset category, the result output module 730 can be used to calculate the total probability corresponding to the preset category based on k block classification results for each preset category; and determine the preset category with the largest total probability as the recognition result corresponding to the lung CT data to be identified.

[0092] In an exemplary embodiment, the block acquisition module 710 can be used to preprocess the 3D tensor to obtain a 3D tensor that meets preset conditions; the preprocessing includes at least one of the following processes: normalization, scaling, denoising, windowing, and contrast enhancement.

[0093] In an exemplary embodiment, the classification identification module 720 may be used to take k block tensors as input, classify the block tensors based on the lung disease classification model, and obtain k block classification results.

[0094] In an exemplary embodiment, the result output module 730 can be used to take k block classification results as input, integrate the k block classification results based on the integration model, so as to output the recognition result corresponding to the lung CT data to be identified, and determine the lesion block tensor corresponding to the lung CT data to be identified in the k block tensors when the recognition result is an abnormal result.

[0095] In an exemplary embodiment, the lung disease identification device based on CT data may also include a model training module, which is used to obtain multiple groups of sample data and divide the multiple groups of sample data into training sets, validation sets and test sets in units of groups; each group of sample data includes a group of block tensors corresponding to a lung CT data and an identification result label corresponding to the lung CT data; a lung disease identification neural network is constructed, and the parameters in the lung disease identification neural network are adjusted and determined based on the lung CT data included in the training set, validation set and test set to obtain a lung disease classification model and an integrated model; wherein the lung disease identification neural network includes a lung disease classification network and an integrated network connected in series after the lung disease classification network.

[0096] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0097] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0098] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification, for example, it can execute Figure 3 Any one or more steps in .

[0099] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0100] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0101] In addition, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0102] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0103] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A lung disease identification method based on CT data, characterized in that: include: Obtaining a 3D tensor corresponding to the lung CT data to be identified, and dividing the 3D tensor into blocks to obtain k block tensors; wherein k is a positive integer greater than or equal to 2; Performing lung disease classification and identification on the k block tensors to obtain k block classification results; Outputting a recognition result corresponding to the to-be-recognized lung CT data according to the k block classification results, and determining a lesion block tensor corresponding to the to-be-recognized lung CT data from the k block tensors when the recognition result is an abnormal result; Wherein, when the size of the 3D tensor is the number of images×image height×image width, dividing the 3D tensor into blocks to obtain k block tensors includes: Sequentially divide the 3D tensor into blocks based on the block parameter to obtain k-1 block tensors with a size of block parameter×image height×image width; The remaining part of the 3D tensor that has not been divided into blocks is taken as a block tensor; wherein the size of the block tensor is the remaining number×image height×image width, and the remaining number is less than or equal to the block parameter.

2. The method according to claim 1, characterized in that The block classification result includes the probability that the block tensor belongs to each preset category; The determining of the lesion block tensor corresponding to the to-be-identified lung CT data from the k block tensors when the identification result is an abnormal result includes: For each of the block classification results, obtaining a target probability that the block tensor in the block classification result belongs to the recognition result; The block tensor corresponding to the maximum target probability among the k target probabilities is determined as the lesion block tensor corresponding to the lung CT data to be identified.

3. The method according to claim 1, characterized in that The block classification result includes the probability that the block tensor belongs to each preset category; Outputting the recognition result corresponding to the lung CT data to be recognized according to the k block classification results includes: For each of the preset categories, calculating the total probability corresponding to the preset category based on the k block classification results; The preset category with the largest total probability is determined as the recognition result corresponding to the lung CT data to be recognized.

4. The method according to claim 1, characterized in that: Before dividing the 3D tensor into blocks to obtain k block tensors, the method further includes: Preprocessing the 3D tensor to obtain a 3D tensor that meets a preset condition; The preprocessing includes at least one of the following processes: normalization, scaling, denoising, windowing, and contrast enhancement.

5. The method according to claim 1, characterized in that The step of performing lung disease classification and identification on the k block tensors to obtain k block classification results includes: Taking the k block tensors as input, classifying the block tensors based on the lung disease classification model to obtain k block classification results; Outputting the recognition result corresponding to the to-be-recognized lung CT data according to the k block classification results, and determining the lesion block tensor corresponding to the to-be-recognized lung CT data in the k block tensors when the recognition result is an abnormal result, includes: Taking the k block classification results as input, the k block classification results are integrated based on the integration model to output the recognition result corresponding to the lung CT data to be identified, and when the recognition result is an abnormal result, the lesion block tensor corresponding to the lung CT data to be identified is determined in the k block tensors.

6. The method according to claim 5, characterized in that The method further comprises: Acquire multiple groups of sample data, and divide the multiple groups of sample data into a training set, a validation set, and a test set in units of groups; each group of sample data includes a group of block tensors corresponding to a lung CT data and a recognition result label corresponding to the lung CT data; Constructing a pulmonary disease recognition neural network, adjusting and determining parameters in the pulmonary disease recognition neural network based on the pulmonary CT data included in the training set, the validation set, and the test set, to obtain the pulmonary disease classification model and the integrated model; The lung disease recognition neural network includes a lung disease classification network and an integration network connected in series after the lung disease classification network.

7. A lung disease identification device based on CT data, characterized in that: include: A block acquisition module is used to acquire a 3D tensor corresponding to the lung CT data to be identified, and to block the 3D tensor to obtain k block tensors; k is a positive integer greater than or equal to 2; A classification and recognition module, used for performing lung disease classification and recognition on the k block tensors to obtain k block classification results; a result output module, configured to output a recognition result corresponding to the to-be-recognized lung CT data according to the k block classification results, and to determine a lesion block tensor corresponding to the to-be-recognized lung CT data from the k block tensors when the recognition result is an abnormal result; Wherein, when the size of the 3D tensor is the number of images×image height×image width, dividing the 3D tensor into blocks to obtain k block tensors includes: Sequentially divide the 3D tensor into blocks based on the block parameter to obtain k-1 block tensors with a size of block parameter×image height×image width; The remaining part of the 3D tensor that has not been divided into blocks is taken as a block tensor; wherein the size of the block tensor is the remaining number×image height×image width, and the remaining number is less than or equal to the block parameter.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 6 by executing the executable instructions.