Lung cancer early screening system and lung cancer screening method based on image processing

Abstract

The invention provides a lung cancer early screening system and a lung cancer screening method for predicting the probability that a lung nodule is a cancer by using a CT (Computed Tomography) image. Performing mask processing, feature extraction and feature quantity calculation on the continuous CT images; calculating the doubling time of the nodule and the volume percentage change of the pulmonary nodule; collecting feature information, applying weights, and filtering the feature information to extract and analyze effective information; generating analysis data and final analysis data; and comparing the final analysis data with the learned cancer diagnosis data set, and calculating the cancer probability of the nodule. According to the device and the method, the image data is analyzed by artificial intelligence, the data processing efficiency is improved, and the accuracy of image interpretation is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a diagnostic device and method for analyzing CT images using artificial intelligence to predict the probability that a lung nodule is cancer. Background Art

[0002] Computed tomography (CT) is different from X-ray photography in that it can obtain cross-sectional images of the human body. Compared with simple X-ray photography, CT scans can observe structures and lesions more clearly because there is less structural overlap. In addition, CT scans are cheaper and take less time than MRI, so they are a basic examination method for most organs and diseases when lesions are suspected and require detailed examinations.

[0003] Recently, with the popularization of multi-detector CT (MDCT), images can be reconstructed after shooting, and desired cross-sectional and three-dimensional (3D) images can be freely acquired like MRI, and CT scan images can be collected in medical institutions.

[0004] However, the lack of systems that can improve the confidence of image interpretation results, which requires accurate results, such as the diagnosis of cancer in nodules, is a problem. Summary of the invention

[0005] The present invention provides an early screening system for lung cancer, a lung cancer diagnosis method and a lung cancer diagnosis system for predicting the probability of a lung nodule being cancer by using CT images.

[0006] According to one embodiment, an early screening system for lung cancer using image processing includes: a primary layer masking module that divides continuous CT (Computed Tomography) images containing pre-nodule images and subsequent nodule images or three-dimensional pre-nodule cubes and subsequent nodule cubes into multiple unit images; extracts features of each unit image contained in the primary layer and calculates the number of features extracted for each unit image; an image processing module that calculates the doubling time (DT) and the percentage change in lung nodule volume (PVC) of the nodules contained in the pre-nodule images and subsequent nodule images; an extraction module that collects feature information extracted from each continuous CT image, applies learned weights, and filters feature information with weights below a certain level to extract analysis effective information; a generation module that adds the doubling time and lung nodule volume change calculated from the analysis results of the continuous CT images to the extracted effective information to generate analysis data, and extracts final analysis effective information from the analysis data to generate final analysis data; and a calculation module that compares the generated final analysis data with the learned cancer diagnosis data set to calculate the cancer probability of the nodule.

[0007] According to another embodiment, an online lung cancer diagnosis system using artificial intelligence includes: a user terminal that collects personal health information including X-rays, lung nodule CT images and health examination information; a medical institution server that collects lung nodule CT images including pre-nodule images and subsequent nodule images; and a lung cancer screening CT reading server that receives lung nodule CT images transmitted by the user terminal or the medical institution server, calculates the probability that the lung nodule is cancer, and sends the calculation results to the medical institution server and the user terminal.

[0008] According to another embodiment, the lung cancer diagnosis process of the early screening system for lung cancer using image processing includes: step 1: dividing the continuous CT images containing pre-nodule images and subsequent nodule images or three-dimensional pre-nodule cubes and subsequent nodule cubes into a primary layer mask of multiple unit images, and extracting the features of each unit image contained in the primary layer, and calculating the number of features extracted for each unit image; step 2: calculating the doubling time (DT) and the percentage change (PVC) of the nodules contained in the pre-nodule images and subsequent nodule images; step 3: collecting the feature information extracted from each continuous CT image, applying the learned weights, and filtering the feature information with weights below a certain level to extract analysis effective information; step 4: adding the doubling time and lung nodule volume change calculated from the analysis results of the continuous CT images to the extracted effective information to generate analysis data, and extracting the final analysis effective information from the analysis data to generate the final analysis data; step 5: comparing the generated final analysis data with the learned cancer diagnosis data set to calculate the cancer probability of the nodule.

[0009] Individuals and hospitals can quickly obtain the interpretation results of lung cancer screening CT images, improving the accuracy of CT image interpretation results. In addition, users can entrust their own CT image interpretation online, query the results at any time, and compare with previous results at any time during follow-up examinations to manage their own medical imaging information.

[0010] The early lung cancer screening system uses artificial intelligence to analyze image data, improve data processing efficiency, and at the same time improve the accuracy of image interpretation.

[0011] By introducing the CT image reading system, individuals, hospitals and clinics can interpret medical images online, promoting income generation for individuals and medical institutions.

[0012] The effects of the present invention are not limited to the above-mentioned effects, and should be understood to include all effects that can be inferred from the detailed description of the present invention or the scope of patent application.

[0013] Figure 1An online lung cancer diagnosis system using artificial intelligence according to one embodiment is presented.

[0014] Figure 2 is a block diagram of a lung cancer early screening system (100), according to one embodiment.

[0015] Figure 3 A lung cancer probabilistic diagnosis process is presented, according to one embodiment.

[0016] Figure 4 Specific data processing blocks of a computing module according to one embodiment are shown.

[0017] Figure 5 An image feature extraction process is presented, according to one embodiment.

[0018] Figure 6 The data processing flow of a lung cancer diagnosis method of a lung cancer screening and early lung cancer screening system using artificial intelligence is shown, according to one embodiment.

[0019] Figure 7 The feature extraction process of continuous images is shown, corresponding to step S710.

[0020] Figure 8 Examples of sequential CT images (pre-nodule image, subsequent nodule image) are shown.

[0021] The advantages and features of the present invention, and methods of achieving them, will become clear through the embodiments described in detail with reference to the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and can be implemented in different various forms, but these embodiments provide a complete disclosure of the present invention, and in order to fully inform the ordinary technicians in this technical field of the scope of the invention, the present invention is only defined by the scope of the claims. The same figure symbols refer to the same components throughout the specification.

[0022] When describing the embodiments of the present invention, if it is considered that the detailed description of known functions or structures unnecessarily obscures the main points of the present invention, such detailed description will be omitted. Moreover, the terms mentioned below are defined in consideration of the functions of the embodiments of the present invention and may vary according to the intention or convention of the user or operator. Therefore, these definitions should be determined based on the contents of the entire specification.

[0023] Figure 1 is a diagram showing an online lung cancer diagnosis system using artificial intelligence according to an embodiment.

[0024] Reference Figure 1 According to the embodiment, the lung cancer diagnosis system may include a lung cancer early screening system 100 , a user terminal 200 and a medical institution server 600 .

[0025] The user terminal 200 can collect personal health information of patients undergoing CT photography, such as X-rays, CT images, and lung nodule CT images, and can send each person's health information to a server that requests the health information for CT image interpretation.

[0026] The medical institution server 600 may collect continuous pulmonary nodule CT images including a pre-nodule image and a subsequent nodule image of the patient, and may collect and update the patient's health information.

[0027] The early screening system 100 for lung cancer can receive CT images of lung nodules from the user terminal 200 or the medical institution server 600 , calculate the probability that the lung nodules are cancer, and send the calculation results to the medical institution server 600 and the user terminal 200 .

[0028] In one embodiment, the lung cancer diagnosis system may further include a payment system server 500, a database server 300, a medical staff terminal 400, and a security system server 700. The payment system server 500 may pay for the interpretation fee when performing CT image interpretation in the lung cancer diagnosis system. The database server 300 may store reference data, data sets, and medical information of patients required for lung cancer probability calculation. The reference data may include lung nodule image data extracted from continuous CT images, etc.

[0029] The medical staff terminal 400 is a user terminal of a doctor who joins the lung cancer diagnosis system, and can confirm the CT image interpretation result or directly perform CT image interpretation, or perform abnormal nodule diagnosis and opinion generation. The security system server 700 can perform the security of data and personal information of each server included in the lung cancer diagnosis system.

[0030] Network A of the reading server receives the pre-nodule image and the subsequent nodule image after image preprocessing as input. Then, features are extracted from the two consecutive CT images, and the doubling time of the nodule and the rate of change of the nodule volume are calculated. The feature information of each image extracted by network A is passed to network B of the reading server. In network B, the extracted features are integrated, and a 3D convolutional neural network is used to select the analysis effective information and analysis data. Then, the lung nodule volume and volume change rate information are added to the analysis data. Then, the final analysis data is extracted and compared with the learning data set to output the cancer probability of the lung nodule.

[0031] Figure 2 is a block diagram of a lung cancer early screening system 100 , according to an embodiment.

[0032] Reference Figure 2According to the embodiment, the early screening system 100 for lung cancer may be composed of a network A and a network B. Network A extracts features of continuous CT images and uses the analysis results of continuous CT images to calculate the doubling time and the rate of change of the volume of lung nodules. Network B collects the features extracted by network A, generates final analysis data, and calculates the cancer probability of the nodules through this. To this end, network A may include an image processing module 110, and network B may include an extraction module 130, a generation module 150, and a calculation module 170. The word "module" used in this document should be interpreted as including software, hardware, or a combination thereof according to the context in which the word is used. For example, software may be machine code, firmware, embedded code, and application software. Another example is that hardware may be a circuit, a processor, a computer, an integrated circuit, an integrated circuit core, a sensor, a micro-electro-mechanical system (MEMS; Micro-Electro-Mechanical System), a manual device, or a combination thereof.

[0033] To help understand the data processing block functions of the server, refer to Figure 3 , which shows the lung cancer probabilistic diagnosis process according to an embodiment.

[0034] The image processing module 110 receives the pre-processed pre-nodule image and the subsequent nodule image as input, divides each image into a first layer of multiple unit images, and extracts the features of each unit image contained in the first layer. In an embodiment, the unit image features may include detailed information of the object in the image, such as the edge, color, and size of the object in the image. More specifically, the unit image features extracted from the first layer are the edges of each image area, the unit image features extracted from the second layer may be lines or surfaces, and the features extracted from the third layer may be overall shapes or contours. In other words, from the first layer to higher levels, from simple one-dimensional features to more complex two-dimensional and three-dimensional image features. In addition, the image processing module 110 can calculate the number of features extracted for each unit image. In addition, based on the analysis results of the continuous CT images, the doubling time (DT) and the percentage change (PVC) of the lung nodule volume contained in the pre-nodule image and the subsequent nodule image are calculated.

[0035] In an embodiment, the image input to the image processing module 110 may not only be a continuous CT image, but may also include continuous voxels. A voxel is a volume element that represents the value of a regular grid unit in a three-dimensional space. The term voxel is a hybrid word that combines volume and pixel, similar to the representation of two-dimensional image data by pixels. Voxels are often used for visualization and analysis of medical and scientific data. Like pixels, voxels themselves do not have spatial coordinates, but can be inferred from their positional relationship with other voxel groups (i.e., each position in the data structure that constitutes a volume image). In other words, in an embodiment, the calculation module may receive a three-dimensional pre-nodule cube and a subsequent nodule cube in voxel form as input, extract each feature of the input cube, and calculate the doubling time type and nodule volume change rate through the continuously input cube information. In an embodiment, the CT reading server may also include a preprocessing module not shown in the figure to convert continuous CT images into three-dimensional cubes in voxel form.

[0036] The extraction module 130 of the network B collects each feature information extracted from the previous nodule image and the subsequent nodule image or the three-dimensional previous nodule cube and the subsequent nodule cube, and reflects the learned weight. Then, the feature information with a weight lower than a certain level is filtered out to extract the analysis effective information. Figure 4 As described above, the extraction module 130 can collect 512 bytes of feature information of the previous nodule and 512 bytes of feature information of the subsequent nodule from the image processing module 110, creating a total of 1024 bytes of feature information (a). Then, the extraction module 130 reflects the pre-learned weights to the 1024 bytes of feature information, and only extracts 64 bytes of analysis effective information (b) with weights above a certain level. In an embodiment, the analysis effective information (b) may be information that is unnecessary for the calculation of the probability of cancer filtered out from the feature information. In an embodiment, the weights can be reflected according to the order of the metadata reflecting the weights, or according to the degree of inclusion of the edge of the object, so as to extract information with weights above a certain level as analysis effective information (b), and artificial intelligence algorithms such as deep learning can be used in the weight reflection.

[0037] The generation module 150 extracts data with metadata above a certain level from the analysis effective information, or applies an artificial intelligence algorithm to generate analysis preparation data, and adds the pulmonary nodule volume change information of the previous nodule image and the subsequent nodule image received from the image processing module 110 to generate analysis data (c). Here, examples of pulmonary nodule volume change information may be doubling time and volume percentage change (PVC). The front stage data used to generate the final analysis data may be a total of 4, 2 of which are image features extracted and selected from the previous nodule image and the subsequent nodule image, and the remaining 2 data are doubling time and PVC used as pulmonary nodule volume change information.

[0038] The analysis preparation data may be data from which important information is re-extracted from the analysis effective information. Then, the generation module 150 selects the most important data from the analysis data (c) and generates the final analysis data (d) using an artificial intelligence algorithm such as a deep learning model.

[0039] The calculation module 170 compares the final analysis data (d) with the pre-learned cancer diagnosis data set to calculate the cancer probability of the nodule.

[0040] Reference Figure 3 , the 3D CNN model used in the embodiment consists of two networks (NetworkA, Network B). In an embodiment, Network A and Network B can be trained separately. Each network A has a nodule in the form of a three-dimensional cube as input, and uses all the nodules owned as training data to train Network A, and trains to find features related to lung cancer. For Network B, the input is the features and volume change information of two lung nodules, and the result is to provide the probability of cancer of the lung nodules. In an embodiment, data expansion techniques such as rotation and flipping can be used to train the network.

[0041] Reference Figure 3 ,Network A inputs the previous nodule image and the subsequent nodule image or the three-dimensional previous nodule cube and the subsequent nodule cube, and extracts features from the two consecutive images and cubes. Figure 3 The figure shows a three-layer feature extraction process from two consecutive images and a cube. In the first feature extraction process, two consecutive images and a cube are received and two three-dimensional features can be generated. As a result, in the first feature extraction process, feature data of each size (32x32x32)x2 can be generated.

[0042] In the second feature extraction process, from the data of size (32x32x32)x2 generated by the first feature extraction, the feature extraction process of only retaining the unit image with the largest feature is performed again. As a result, in the second extraction process, 4 feature values ​​can be generated, and data of size (16x16x16)x4 can be created. If the third feature extraction process is executed, in the third feature extraction process, the unit image with only the largest feature will be filtered again from the data generated by the second feature extraction. As a result, in the third feature extraction process, data of size (8x8x8)x8 can be created. In an embodiment, by repeating the maximum feature extraction process, feature data is extracted only from continuous images and passed to network B. Then, network B creates an analysis image (a) that merges the feature data of continuous images, and based on this, repeatedly extracts valid data required for machine learning results for cancer judgment. Then, analysis data (c) of volume change information of the previous nodule and the subsequent nodule is created, and the final analysis data (d) is extracted from the analysis data (c). In an embodiment, the extraction process can use pre-built deep learning and machine learning results to extract the final analysis data (d). Then, network B determines whether the lung nodule is cancer through the final analysis data.

[0043] Figure 4 is a diagram showing specific data processing blocks of a computing module according to an embodiment.

[0044] Reference Figure 4 , the image processing module 110 may include a creation unit 111, a calculation unit 113 and a judgment unit 115. In order to help understand the function of the image processing module 110, Figure 4 When combined Figure 5 ,This figure shows the image feature extraction process.

[0045] The creation unit 111 creates a second layer composed of a certain number of unit images, extracts a maximum eigenvalue image having the largest number of features from the unit images constituting the second layer, and collects the maximum eigenvalue images extracted from the plurality of second layers to create n analysis images. Figure 4 As shown, the creation unit 111 in the embodiment creates a second layer (L21, L22, L23, L24) containing 4 unit images, and extracts the maximum eigenvalue image with the largest number of features therefrom. In the embodiment, in the second layer 2,1,1,7, an image representing the maximum eigenvalue 7 is extracted, and in the second layer 2,1,6,1, an image representing the maximum eigenvalue 6 is extracted, and through a similar process, images representing the maximum eigenvalues ​​6 and 8 are extracted.

[0046] Then, the creation section 111 collects the maximum eigenvalue images extracted from the plurality of second layers to create n analysis images. Figure 5 , collect the unit images representing the maximum eigenvalues ​​7, 6, 6, 8 to create a 2-time analysis image (e2). Then, extract the maximum eigenvalue 8 from 7, 6, 6, 8 to create a 3-time analysis image (e3). The 3-time analysis embodiment is described in the embodiment, but the feature extraction by the maximum value extraction process can be repeated n times, and accordingly, n-time analysis images can be created.

[0047] The calculation unit 113 calculates the doubling time (DT) and the nodule volume change rate using the characteristic information of the inputted continuous CT images and continuous three-dimensional nodule cube data.

[0048] The doubling time can be generated by mathematical formula 1.

[0049] Mathematical formula 1

[0050] DT = (T2-T1) × (log2 / logV2-logV1) DT = doubling time

[0051] V1: Nodule volume of the previous nodule image

[0052] V2: Nodule volume of subsequent nodule images

[0053] T1: reference point, T2: scanning completion time

[0054] The percentage change of lung nodule volume can be generated by mathematical formula 2.

[0055] Mathematical formula 2

[0056] PVC=((V2-V1) / V1)×100

[0057] PVC: Pulmonary nodule volume percentage change

[0058] V2: The volume of the nodule in the subsequent nodule image

[0059] V1: The volume of the nodule in the front nodule image

[0060] When the n-th analysis image contains the maximum number of features, the judgment unit 115 transmits the maximum feature value image information of the previous nodule image and the subsequent nodule image to the extraction module. In addition, the judgment unit 115 also transmits the doubling time and the lung nodule volume percentage change data to the extraction module of network B.

[0061] Next, the online lung cancer diagnosis method according to the embodiment will be described in sequence. The role (function) of the online lung cancer diagnosis method is essentially the same as that of the lung cancer early screening system and the lung cancer screening system, and therefore Figures 1 to 5 Duplicate descriptions will be omitted.

[0062] Figure 6is a diagram showing a data processing flow of a lung cancer diagnosis method of an early lung cancer screening system using image processing according to an embodiment.

[0063] In step S710, respective features are extracted from the continuous CT images of the previous nodule image and the subsequent nodule image. In step S710, the continuous CT images are converted into voxel form, a three-dimensional previous nodule cube and a subsequent nodule cube are created, and the previous nodule cube and the subsequent nodule cube are analyzed to extract features contained in the three-dimensional cube.

[0064] In step S720, the doubling time of the nodules and the volume percentage of the lung nodules are calculated in the image processing module.

[0065] In step S730, the extraction module collects feature information extracted from each continuous CT image, reflects the learned weights to the collected feature information, and generates analysis-effective information by filtering out feature information with weights below a certain level.

[0066] In step S740, the generation module adds the doubling time and lung nodule volume change calculated from the above-mentioned continuous CT image analysis results to the extracted valid information to generate analysis data, and in step S750, the final analysis valid information is extracted from the analysis data to generate final analysis data.

[0067] In step S760, the calculation module compares the generated final analysis data with the learned cancer diagnosis data set to calculate the cancer probability of the nodule.

[0068] Figure 7 The continuous image feature extraction process in step S710 is demonstrated.

[0069] In step S711, the creating unit creates a second layer composed of a certain number of unit images.

[0070] In step S713, a maximum eigenvalue image having the largest number of features is extracted from unit images constituting the second layer, and the maximum eigenvalue images extracted from a plurality of second layers are collected to create n analysis images.

[0071] In step S715, when the n-th analysis image contains the maximum eigenvalue, the determination unit transmits the maximum eigenvalue image information of the continuous CT images to the generation module.

[0072] Figure 8 Examples of sequential CT images (pre-nodule image, subsequent nodule image) are shown.

[0073] like Figure 8As shown, according to the embodiment, the lung cancer screening CT reading server using artificial intelligence continuously receives the previous nodule image (1-1, 2-1, 3-1) and the subsequent nodule image (1-2, 2-2, 3-2), generates analysis data and final analysis data using the two consecutive images, and performs cancer judgment through the final analysis data.

[0074] According to the embodiment, the lung cancer diagnosis system, lung cancer early screening system and lung cancer diagnosis method using artificial intelligence can quickly obtain the interpretation results of lung cancer screening CT images performed by individuals and hospitals, and improve the accuracy of CT image interpretation results. In addition, users can manage their medical imaging information by querying their CT image interpretation results online at any time and comparing them with previous results at any time during follow-up examinations. The lung cancer early screening system uses artificial intelligence to minimize the size of image data that needs to be analyzed, improve data processing efficiency and improve image interpretation accuracy. In addition, by introducing a CT image reading system, individual medical professionals and tertiary hospitals can perform medical image interpretation online, promoting income creation for individuals and medical institutions.

[0075] The disclosed contents are merely examples and can be implemented in many ways by a person skilled in the art without departing from the scope of the claims. Therefore, the protection scope of the disclosed contents is not limited to the above-mentioned specific embodiments.

Claims

1. An early screening system for lung cancer based on image processing, comprising: an image processing module, which divides continuous CT images, including pre-nodule images, subsequent nodule images, or three-dimensional pre-nodule cubes and subsequent nodule cubes, into a first layer of multiple unit images for mask processing, extracts features of each unit image, calculates the number of features of each unit image, and calculates the doubling time of nodules and the percentage change of lung nodule volume in the pre-nodule image and the subsequent nodule image; The extraction module collects feature information extracted from continuous images, applies learned weights, filters out feature information with weights below a certain level, and extracts valid information; A generation module, which adds the doubling time and the change in the volume of lung nodules calculated from the analysis results of continuous CT images to the extracted valid information to generate analysis data, and extracts the final analysis valid information from the analysis data to generate the final analysis data; The calculation module compares the generated final analysis data with the learned cancer diagnosis dataset and calculates the cancer probability of the nodule.

2. The early screening system for lung cancer according to claim 1, wherein: The image processing module includes: A creation part: generating a second layer composed of a certain number of unit images, extracting a maximum eigenvalue image having the largest number of features among the unit images constituting the second layer, and collecting a plurality of maximum eigenvalue images extracted in the second layer to create n analysis images; The judging part: if the n-th analysis image contains the maximum eigenvalue, the image information of the maximum eigenvalue of the continuous CT image is passed to the generating module; Calculation part: Calculate the doubling time of nodules and the percentage change of lung nodule volume in the previous nodule image and the subsequent nodule image; And when the n-th analysis image does not include the maximum eigenvalue, the determination unit re-inputs the n-th analysis image into the creation unit to generate an n+1-th analysis image; wherein n is a natural number.

3. The early screening system for lung cancer according to claim 1, wherein: The image processing module extracts features including edges, objects and colors and the number of features in a layer of the pre-nodule image and the subsequent nodule image.

4. The early screening system for lung cancer according to claim 1, wherein: A preprocessing module is also included for performing preprocessing of the pre-nodule image and the subsequent nodule image to create a three-dimensional pre-nodule cube and a three-dimensional subsequent nodule cube in voxel form.

5. A lung cancer screening method using an early lung cancer screening system based on image processing, comprising: Step 1: Mask processing, feature extraction and feature quantity calculation are performed on continuous CT images; Step 2: Calculate the doubling time of the nodule and the percentage change in the volume of the lung nodule; Step 3: Collect feature information, apply weights, and filter feature information to extract and analyze effective information; Step 4: Generate analysis data and final analysis data; Step 5: Compare the final analysis data with the learned cancer diagnosis dataset and calculate the cancer probability of the nodule.

6. The method of claim 5, wherein: The step of calculating the number of features of each unit image comprises: Creating a second layer, extracting the maximum eigenvalue image, and collecting the maximum eigenvalue images to create n analysis images; If the n-th analyzed image contains the maximum eigenvalue, the maximum eigenvalue image information of the continuous CT image is passed to the generation module; If the n-time analysis image does not contain the maximum eigenvalue, the n-time analysis image is re-input to generate an n+1-time analysis image; wherein n is a natural number.

7. The method of claim 5, wherein: The step of calculating the number of features of each unit image comprises: Features including edges, objects, and colors and the number of features are extracted in one layer of the pre-nodule image and the subsequent nodule image.

8. The method of claim 5, wherein: The step of calculating the number of features of each unit image comprises: Preprocessing of the pre-nodule image and the subsequent nodule image is performed to create a three-dimensional pre-nodule cube and a three-dimensional subsequent nodule cube in voxel form.

Images