Method and system for predicting pulmonary disease on basis of chest radiographic image and lung sound image

Abstract

This operation method of a pulmonary disease prediction system operated by at least one processor comprises the steps of: acquiring a chest radiographic image of a patient and a plurality of lung sound intensity images acquired from at least one chest region of the patient; extracting a lung region from the chest radiographic image so as to generate a lung region radiographic image; warping the plurality of lung sound intensity images to the extracted lung region so as to generate a plurality of lung sound intensity images warped to the lung region; and acquiring a pulmonary disease prediction result predicted from the lung region radiographic image and the plurality of warped lung sound intensity images by using a pulmonary disease prediction model trained to predict pulmonary disease from an input image.

Description

Method and system for predicting lung disease based on chest X-ray images and breath sound images

[0001] The present disclosure relates to a method and system for predicting diseases by analyzing medical images based on artificial intelligence.

[0002] As infectious diseases like COVID-19 spread, the importance of rapid and accurate diagnosis and prognosis for respiratory and pulmonary diseases is increasing. Typically, methods for diagnosing and predicting lung diseases are primarily based on chest X-ray images and breath sounds from chest auscultation.

[0003] X-ray-based chest radiography allows for noninvasive observation of the chest cavity, enabling quantitative examination of the lung area. Its relatively low cost and rapid examination time make it widely used for diagnosing lung disease. However, if characteristic findings of lung disease are not clearly visible on chest radiographs, clinical ambiguity can make interpretation difficult. Therefore, additional examinations such as computed tomography (CT) must be performed for an accurate diagnosis, and the inherent nature of chest radiography can also pose a risk of radiation exposure.

[0004] To address these issues, active research is underway to use AI models to diagnose and predict prognosis of lung disease from chest X-ray images. To enhance the performance of AI models, it is crucial to learn only the lung region's features. However, conventional models often input regions other than the lung region, resulting in poor lung disease prediction accuracy.

[0005] Meanwhile, a patient's breath sounds are also a key indicator for diagnosing and predicting lung disease. However, breath sounds are unstructured data, with characteristics specific to each respiratory disease unquantified. If medical professionals directly auscultate patients' breath sounds, diagnostic results may vary depending on the medical professional, or the medical professional may become infected. Non-contact breath sound measurement devices are used for this purpose. However, conventional breath sound measurement devices cannot measure breath sounds from multiple thoracic regions, and thus may be less accurate than traditional auscultation methods.

[0006] Therefore, a method is required to accurately predict lung disease based on a patient's chest X-ray image and breath sound data acquired from multiple thoracic regions.

[0007] The present disclosure provides a method and system for predicting lung disease based on chest X-ray images and breath sound images.

[0008] The present disclosure provides a method and system for extracting a lung region from a chest X-ray image, warping the extracted radiographic lung region with multi-channel breath sound intensity images acquired over time from multiple chest regions, and predicting lung disease based on artificial intelligence using the extracted lung region.

[0009] According to one embodiment, a method of operating a lung disease prediction system operated by at least one processor comprises the steps of: acquiring a chest X-ray image of a patient and a plurality of breath sound intensity images acquired from at least one chest region of the patient; extracting a lung region from the chest X-ray image to generate a lung region X-ray image; warping the plurality of breath sound intensity images in the extracted lung region to generate a plurality of breath sound intensity images warped to the lung region; and obtaining a lung disease prediction result predicted from the lung region X-ray image and the plurality of warped breath sound intensity images using a lung disease prediction model learned to predict lung disease from an input image.

[0010] The above breath sound intensity image may be an image generated based on breath sound signals measured over a certain period of time in multiple chest regions of the patient using a multi-channel breath sound measurement device.

[0011] The above-mentioned breath sound intensity image may be data that generates a breath sound intensity image at a specific point in time by expressing the distribution of breath sound intensity obtained based on the breath sound signal at each of a plurality of locations where the breath sound is measured, and generates the breath sound intensity image at the specific point in time by time for the entire measurement time.

[0012] The step of generating a plurality of warped breath sound intensity images in the lung region may include the steps of extracting feature points from the lung region radiographic image and the breath sound intensity image, respectively, and obtaining coordinate values ​​of each image based on the feature points, the step of displaying the coordinate values ​​of each image on the same point map of a two-dimensional plane, the step of calculating a transformation matrix between the lung region radiographic image and the breath sound intensity image based on a correspondence relationship obtained by registering the coordinate values ​​of each image, and the step of applying the transformation matrix to the breath sound intensity image to generate a plurality of warped breath sound intensity images in the lung region.

[0013] The step of obtaining the lung disease prediction result may include a step of generating learning data based on the lung region radiographic image and the plurality of warped breath sound intensity images, and a step of training the lung disease prediction model using the learning data.

[0014] The step of generating the learning data may generate the learning data using at least one of a method of concatenating at least one of the lung region radiographic image and the plurality of warped breath sound intensity images, a method of converting the at least one of the images into a color image and blending R, G, and B channels of the converted color images, and a method of performing a domain transform on the images and combining the at least one of the images with a plurality of domain transformed images obtained by performing a domain transform on the images.

[0015] The step of obtaining the lung disease prediction result may further include a step of providing an area related to the lung disease prediction result by displaying it in a manner differentiated according to importance in at least one of the lung region X-ray image or the plurality of warped breath sound intensity images.

[0016] According to one embodiment, a lung disease prediction system operated by at least one processor comprises: a radiation image processing unit for extracting a lung region from a chest radiographic image of a patient to generate a lung region radiographic image; a warping unit for warping a plurality of breath sound intensity images acquired from at least one chest region of the patient to the extracted lung region to generate a plurality of warped breath sound intensity images in the lung region; and a lung disease prediction unit for obtaining a lung disease prediction result predicted from the lung region radiographic image and the plurality of warped breath sound intensity images using a lung disease prediction model learned to predict lung disease from an input image.

[0017] The above breath sound intensity image may be an image generated based on breath sound signals measured over a certain period of time in multiple chest regions of the patient using a multi-channel breath sound measurement device.

[0018] The above-mentioned breath sound intensity image may be data that generates a breath sound intensity image at a specific point in time by expressing the distribution of breath sound intensity obtained based on the breath sound signal at each of a plurality of locations where the breath sound is measured, and generates the breath sound intensity image at the specific point in time by time for the entire measurement time.

[0019] The warping unit extracts feature points from the lung region radiographic image and the breath sound intensity image, respectively, obtains coordinate values ​​of each image based on the feature points, displays the coordinate values ​​of each image on the same point map of a two-dimensional plane, and calculates a transformation matrix between the lung region radiographic image and the breath sound intensity image based on a correspondence relationship obtained by registering the coordinate values ​​of each image, and then applies the transformation matrix to the breath sound intensity image to generate a plurality of breath sound intensity images warped to the lung region.

[0020] The lung disease prediction unit may further include a learning data generation unit that generates learning data based on the lung region X-ray image and the plurality of warped breath sound intensity images, and a learning unit that trains the lung disease prediction model using the learning data.

[0021] The learning data generation unit may generate the learning data using at least one of a method of concatenating at least one of the lung region radiographic images and the plurality of warped breath sound intensity images, a method of converting the at least one of the images into a color image and blending R, G, and B channels of the converted color images, and a method of performing a domain transform on the images and combining the at least one of the images with a plurality of domain transformed images obtained.

[0022] The method may further include a prediction information providing unit that provides an area related to the lung disease prediction result by displaying it in a manner differentiated according to importance in at least one of the lung region X-ray image or the warped plurality of breath sound intensity images.

[0023] A lung disease prediction system according to one embodiment comprises a memory for storing instructions, and at least one processor for executing the instructions, wherein the processor extracts a lung region from a chest X-ray image to generate a lung region X-ray image, warps a plurality of breath sound intensity images in the extracted lung region to generate a plurality of warped breath sound intensity images in the lung region, and obtains a lung disease prediction result predicted from the lung region X-ray image and the plurality of warped breath sound intensity images using a lung disease prediction model learned to predict lung disease from an input image.

[0024] The processor extracts feature points from the lung region X-ray image and the breath sound intensity image, respectively, obtains coordinate values ​​of each image based on the feature points, displays the coordinate values ​​of each image on the same point map of a two-dimensional plane, and applies a transformation matrix between the lung region X-ray image and the breath sound intensity image calculated based on a correspondence relationship obtained by registering the coordinate values ​​of each image to the breath sound intensity image, thereby generating a plurality of breath sound intensity images warped in the lung region.

[0025] The processor can generate learning data based on the lung region radiographic image and the warped plurality of breath sound intensity images, and train the lung disease prediction model using the learning data.

[0026] The processor may generate the learning data by using at least one of a method of concatenating the at least one image of the lung region radiographic image and the plurality of warped breath sound intensity images, a method of converting the at least one image into a color image and blending R, G, and B channels of the converted color images, and a method of performing a domain transform on the images and combining the at least one image with a plurality of domain transformed images obtained.

[0027] The processor may provide an area related to the lung disease prediction result by displaying it in order of importance in at least one of the lung region X-ray image or the warped plurality of breath sound intensity images.

[0028] According to the present disclosure, a patient's chest X-ray image and breath sound data can be comprehensively analyzed, thereby allowing more accurate prediction of lung disease compared to single data.

[0029] According to the present disclosure, by securing cohort data with standardized characteristics for lung diseases, it can be utilized in various medical fields.

[0030] According to the present disclosure, since a conventional chest X-ray device can be used, it can be expanded and applied to various medical fields.

[0031] According to the present disclosure, information for assisting medical staff in diagnosis can be provided by visualizing the cause of lung disease by location.

[0032] Figure 1 is a conceptual diagram of a respiratory disease prediction method using a lung disease prediction system according to one embodiment.

[0033] FIG. 2 is a diagram illustrating a method for generating a breath sound intensity image according to one embodiment.

[0034] Figure 3 is a configuration diagram of a lung disease prediction system according to one embodiment.

[0035] FIG. 4 is a diagram illustrating a method for generating a warped breath sound intensity image in a lung region according to one embodiment.

[0036] FIG. 5 is a diagram illustrating a lung disease prediction unit according to one embodiment.

[0037] FIG. 6 is a diagram illustrating a method for generating learning data of various combinations according to one embodiment.

[0038] Figure 7 is a flowchart of a lung disease prediction method according to one embodiment.

[0039] FIG. 8 is a flowchart of a method for generating a warped breath sound intensity image in a lung region according to one embodiment.

[0040] Figure 9 is a hardware configuration diagram of a lung disease prediction system according to one embodiment.

[0041] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted to clearly explain the present invention, and similar parts are designated with similar reference numerals throughout the specification.

[0042] When a part of this disclosure is said to "include" a component, this does not exclude other components, unless otherwise specifically stated, but rather means that other components may be included. Furthermore, terms such as "part," "unit," and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented using hardware, software, or a combination of hardware and software.

[0043] In the present disclosure, the devices are configured as hardware, including at least one processor, a memory device, a communication device, and the like, and a program is stored in a designated location, coupled with the hardware, to execute the method of the present invention. The hardware has a configuration and performance capable of executing the method of the present invention. The program includes instructions implementing the operating method of the present invention described with reference to the drawings, and executes the present invention when coupled with hardware such as a processor and a memory device.

[0044] In this disclosure, “transmitting or providing” may include not only direct transmission or providing, but also indirect transmission or providing via another device or by using a bypass route.

[0045] In this disclosure, expressions described in the singular may be interpreted as singular or plural, unless explicit expressions such as “one” or “single” are used.

[0046] In this disclosure, the same drawing numbers refer to the same components regardless of the drawings, and “and / or” includes each and every combination of one or more of the mentioned components.

[0047] In this disclosure, terms including ordinal numbers, such as "first" and "second," may be used to describe various components, but these components are not limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of this disclosure, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0048] In the flowcharts described with reference to the drawings in this disclosure, the order of operations may be changed, several operations may be merged, some operations may be split, and certain operations may not be performed.

[0049] In the present disclosure, an artificial intelligence model (AI model) is an AI model that learns at least one task and can be implemented as a computer program running on a computing device. The computer program is stored on a non-transitory storage medium and includes instructions that cause a processor to execute the operations of the present disclosure. The computer program can be downloaded over a network or sold in product form.

[0050] Figure 1 is a conceptual diagram of a respiratory disease prediction method using a lung disease prediction system according to one embodiment.

[0051] Referring to FIG. 1, a lung disease prediction system (simply, a lung disease prediction system) (10) based on chest X-ray images and breath sound images according to one embodiment is a computing device operated by at least one processor, and can predict lung diseases based on artificial intelligence. The artificial intelligence model included in the lung disease prediction system (10) is an artificial intelligence model capable of learning at least one task, and can be implemented in the form of software or a program that runs on a computing device.

[0052] The lung disease prediction system (10) can receive a chest X-ray image acquired using an X-ray-based chest X-ray imaging device and a breath sound intensity image acquired using a multi-channel breath sound measurement system. At this time, the breath sound intensity image can be data in the form of a time series that represents the breath sound intensity according to the measurement location as a change in brightness based on breath sound signals measured in multiple chest regions.

[0053] The lung disease prediction system (10) can extract a lung region from a chest X-ray image and warp a breath sound intensity image to the extracted lung region image.

[0054] The lung disease prediction system (10) can train an artificial intelligence model to predict lung disease based on lung region X-ray images and lung region warped breath sound intensity images. The lung disease prediction system (10) can predict lung disease using the trained artificial intelligence model. At this time, the artificial intelligence model can be trained and installed by a separate learning device, or the lung disease prediction system (10) can generate learning data and train the artificial intelligence model based on the learning data.

[0055] The lung disease prediction system (10) can provide predicted lung disease results to a hospital information system or an external terminal via a network. The external terminal may be, but is not necessarily limited to, a smartphone, a PC, etc., and may be any device capable of outputting lung disease results.

[0056] The lung disease prediction system (10) can comprehensively analyze a patient's chest X-ray images and breath sound signals, and thus can predict lung disease more accurately than lung disease prediction using a single data set. The method by which the lung disease prediction system (10) predicts lung disease is described in detail below.

[0057] FIG. 2 is a diagram illustrating a method for generating a breath sound intensity image according to one embodiment.

[0058] Referring to Fig. 2, the lung disease prediction system (10) can predict lung disease using multiple breath sound intensity images acquired based on breath sound signals measured in multiple chest regions of a patient. For example, the breath sound signals may be signals measured in the anterior and posterior chest regions of a patient using a multi-channel breath sound measurement system, and the number of breath sound measurement regions (the number of patches of the multi-channel breath sound measurement system) may vary depending on the judgment of the medical staff.

[0059] The breath sound intensity image conversion system converts the continuous form of the entire breath sound signal into a continuous form at a certain time interval (t1~t). n ) can generate a breath sound intensity image. That is, the breath sound intensity image conversion system can repeatedly generate a single breath sound intensity image acquired from multiple locations (patches) for a specific point in time at a certain time interval, and ultimately generate multiple breath sound intensity images in the form of a time series.

[0060] The lung disease prediction system (10) can obtain a breath sound intensity image generated from a breath sound intensity image conversion system through a network, or can obtain a breath sound intensity image stored in an external system such as a hospital information system.

[0061] Figure 3 is a configuration diagram of a lung disease prediction system according to one embodiment.

[0062] Referring to FIG. 3, the lung disease prediction system (10) may include a radiation image processing unit (100), a warping unit (200), a lung disease prediction unit (300), and a prediction information providing unit (400) that provides lung disease prediction results to an external terminal (20).

[0063] The radiographic image processing unit (100) can receive a chest radiographic image that captures the anterior and posterior aspects of the patient's chest. At this time, the radiographic image processing unit (100) can perform preprocessing to remove bones or soft tissues such as the clavicle and ribs included in the chest radiographic image. For example, the radiographic image processing unit (100) can remove bones or soft tissues from the chest radiographic image using a dual-energy based image processing method. In addition, the radiographic image processing unit (100) can also utilize a known image processing method.

[0064] The radiographic image processing unit (100) can extract a lung region from a chest radiographic image and obtain a lung region mask. For example, the radiographic image processing unit (100) can extract a lung region based on the probability of pixel values ​​being distributed within the image based on a random walk, or can extract the lung region by generating an outline of the lung region using an active contour model. In addition, the radiographic image processing unit (100) can extract the lung region using an artificial intelligence model trained to extract a specific region from the image.

[0065] The radiation image processing unit (100) can transmit the extracted lung region radiation image to the warping unit (200) and the lung disease prediction unit (300).

[0066] The method by which the warping unit (200) warps the breath sound intensity image to the lung area is described in detail with reference to FIG. 4.

[0067] FIG. 4 is a diagram illustrating a method for generating a warped breath sound intensity image in a lung region according to one embodiment.

[0068] Referring to FIG. 4, the warping unit (200) receives a radiographic image extracted from a lung region from the radiographic image processing unit (100) and can warp a breath sound intensity image to a lung region radiographic image.

[0069] The warping unit (200) can extract feature points from lung region X-ray images and breath sound intensity images, respectively, and obtain coordinates corresponding to the lung region or breath sound intensity images. Thereafter, the warping unit (200) can use the coordinates obtained from each image to express them on the same point map of a two-dimensional plane.

[0070] The warping unit (200) can calculate a transformation matrix by registering the coordinates of the lung region radiographic image expressed on the point map and the coordinates of the respiratory sound intensity image. For example, the warping unit (200) can find a correspondence between each image coordinate using the ICP (Iterative Closest Point) algorithm and calculate a transformation matrix using the SVD (Singular Value Decomposition) algorithm.

[0071] The warping unit (200) can calculate a transformation matrix including a translation matrix, a shear matrix, a rotation matrix, and a scale matrix based on the correspondence between the coordinates of two images found through alignment.

[0072] The warping unit (200) can obtain a warped breath sound intensity image in the lung area by applying at least one calculated transformation matrix to the breath sound intensity image.

[0073] Referring again to FIG. 3, the lung disease prediction unit (300) can receive a lung region radiation image extracted from the radiation image processing unit (100) and a lung region warped breath sound intensity image generated from the warping unit (200).

[0074] The lung disease prediction unit (300) can extract features related to a specific lung disease from the input images and predict the lung disease. At this time, the lung disease prediction unit (300) may include an artificial intelligence model trained to predict the lung disease. For example, the lung disease may be emphysema, chronic obstructive pulmonary disease (COPD), pneumothorax, etc.

[0075] The lung disease prediction unit (300) can visually overlay on each image which part of the input lung region X-ray image or the lung region warped breath sound intensity image is used to predict lung disease. For example, the lung disease prediction unit (300) can visualize the area related to the lung disease prediction result in the form of a heat map.

[0076] The prediction information provision unit (400) transmits a lung region X-ray image and a lung region warped breath sound intensity image, including the lung disease name predicted by the lung disease prediction unit (300), to an external terminal (20), and the external terminal (20) can display the received data on a user interface screen.

[0077] The predictive information providing unit (400) can provide stored data to an external terminal (20) at the request of a user (medical staff). For example, the predictive information providing unit (400) can provide chest radiography information and breath sound measurement information, such as examination time / cycle, conditions set during examination (tube voltage, tube current, etc.), etc., to the external terminal (20).

[0078] The prediction information provision unit (400) can provide medical data obtained by linking with an external system such as a hospital information system (HIS) through a network to an external terminal (40).

[0079] FIG. 5 is a diagram illustrating a lung disease prediction unit according to one embodiment.

[0080] FIG. 6 is a diagram illustrating a method for generating learning data of various combinations according to one embodiment.

[0081] Referring to FIG. 5, the lung disease prediction unit (300) may include a learning data generation unit (310) that generates learning data from a lung region X-ray image and a lung region warped breath sound intensity image, and a learning unit (320) that trains a lung disease prediction model (330) using the generated learning data. At this time, the learning unit (320) may not necessarily be included in the lung disease prediction unit (300), but for the convenience of explanation, it is assumed that the lung disease prediction unit (300) includes the learning unit (320).

[0082] Referring to Fig. 6, the learning data generation unit (310) can generate a learning dataset by combining lung region X-ray images and breath sound intensity images in various forms. In this case, the breath sound intensity images may be time-series data generated over a certain period of time.

[0083] The learning data generation unit (310) can generate learning data by concatenating lung region X-ray images and breath sound intensity images. At this time, the learning data generation unit (310) can generate learning data by concatenating only specific images from multiple breath sound intensity images.

[0084] The learning data generation unit (310) can generate learning data by blending a lung region X-ray image and at least one breath sound intensity image, or can convert a grayscale image into a color image having RGB values ​​using a digital image processing (DIP) method, and generate learning data in the form of blending the R, G, and B channels of the two images. In this case, the generated learning data can be three-dimensional data having R, G, and B values.

[0085] The learning data generation unit (310) can generate domain transform images for the lung region X-ray image and at least one breath sound intensity image. For example, the learning data generation unit (310) can use a method such as wavelet transform for domain transform. The learning data generation unit (310) can generate learning data by combining the lung region X-ray image, at least one breath sound intensity image, and at least one domain transform image.

[0086] In addition, the learning data generation unit (310) can generate learning data by combining an image generated by converting to a color image and then blending it with at least one domain-converted image among the methods described above.

[0087] According to another embodiment, the learning data generation unit (310) can generate four-dimensional learning data using the learning data generated using the aforementioned method. For example, the learning data generation unit (310) can generate four-dimensional learning data by inputting together an image generated by blending R, G, and B channels after conversion to a color image and time information acquired from a time-series breath sound intensity image, among the aforementioned methods.

[0088] The lung disease prediction model (330) can improve lung disease prediction performance through various types of learning data sets generated according to an embodiment in the learning data generation unit (310).

[0089] Referring again to FIG. 5, the learning unit (320) can receive a learning data set generated by the learning data generation unit (310) and train a lung disease prediction model (330) using various combinations of learning data sets.

[0090] The lung disease prediction model (330) can learn the relationship between lung region X-ray images and breath sound intensity images and lung diseases using a learning dataset. That is, the lung disease prediction model (330) may be a deep learning model trained to predict lung diseases from input images. The lung disease prediction model (330) can predict lung diseases such as emphysema, chronic obstructive pulmonary disease (COPD), pneumothorax, etc., based on the input lung region X-ray images and breath sound intensity images.

[0091] The lung disease prediction model (330) can visually express on a lung region X-ray image or a breath sound intensity image which part of the input image is used as the basis for predicting lung disease. In other words, the lung disease prediction model (330) can be an explainable artificial intelligence (XAI), and for this purpose, a method such as a class activation map (CAM) can be applied.

[0092] Figure 7 is a flowchart of a lung disease prediction method according to one embodiment.

[0093] Referring to Fig. 7, the lung disease prediction system (10) receives a chest X-ray image and a breath sound intensity image (S110). At this time, the chest X-ray image may be a X-ray image taken of the front or back of the patient's chest using an X-ray-based chest X-ray imaging device, and the lung disease prediction system (10) may perform preprocessing to remove bones or soft tissues included in the chest X-ray image. In addition, the breath sound intensity image may be an image that shows the breath sound intensity by measurement time and measurement location based on breath sound signals measured in multiple chest regions using a multi-channel breath sound measurement system.

[0094] The lung disease prediction system (10) extracts a lung region from a chest X-ray image to generate a lung region X-ray image, and warps a breath sound intensity image to the extracted lung region (S120). The lung disease prediction system (10) can extract the lung region using a random walk, an active contour model, or an artificial intelligence model trained to extract a specific region from an image.

[0095] The lung disease prediction system (10) inputs a lung region radiographic image and a lung region warped breath sound intensity image into a learned lung disease prediction model (330) to predict lung disease (S130). The lung disease prediction model (330) may be a deep learning model trained to predict lung disease from the input images. The lung disease prediction system (10) can train the lung disease prediction model (330) by generating a learning dataset with various combinations.

[0096] The lung disease prediction system (10) can predict lung diseases such as emphysema, chronic obstructive pulmonary disease (COPD), pneumothorax, etc., based on input lung region X-ray images and breath sound intensity images. The lung disease prediction system (10) can provide the predicted lung disease results to a hospital information system or an external terminal via a network.

[0097] FIG. 8 is a flowchart of a method for generating a warped breath sound intensity image in a lung region according to one embodiment.

[0098] Referring to Fig. 8, the lung disease prediction system (10) extracts feature points from the lung region X-ray image and the breath sound intensity image, respectively, and obtains coordinates (S210).

[0099] The lung disease prediction system (10) expresses coordinates obtained from each image on the same point map, aligns the coordinates of the two images, and calculates a transformation matrix (S220). For example, the ICP algorithm can be used to find the correspondence between the coordinates of each image, and the SVD algorithm can be used to calculate the transformation matrix.

[0100] The lung disease prediction system (10) applies a transformation matrix to a breath sound intensity image to obtain a breath sound intensity image warped to the lung region (S230). The lung disease prediction system (10) can calculate a transformation matrix including a translation matrix, a shear matrix, a rotation matrix, and a scale matrix based on the correspondence between the coordinates of the two images obtained by performing registration.

[0101] Figure 9 is a hardware configuration diagram of a lung disease prediction system according to one embodiment.

[0102] Referring to FIG. 9, the lung disease prediction system (10) is a computing device operated by one or more processors (11), and may include a processor (11), a memory (13) for loading a computer program executed by the processor (11), a storage (15) for storing the computer program and various data, a communication interface (17), and a bus (19) connecting them. In addition, the lung disease prediction system (10) may further include various components.

[0103] A computer program may include instructions that, when loaded into memory (13), cause the processor (11) to perform methods / operations according to various embodiments of the present disclosure. That is, the processor (11) may perform methods / operations according to various embodiments of the present disclosure by executing the instructions. A computer program is composed of a series of computer-readable instructions grouped based on function, and refers to instructions that are executed by a processor. A computer program may include instructions that perform the operations described in the present disclosure.

[0104] The processor (11) controls the overall operation of each component of the lung disease prediction system (10). The processor (11) may be configured to include at least one of a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), or any other type of processor well known in the art of the present disclosure. In addition, the processor (11) may perform operations for at least one application or computer program for executing methods / operations according to various embodiments of the present disclosure.

[0105] Memory (13) stores various data, commands, and / or information. Memory (13) can load one or more computer programs from storage (15) to execute methods / operations according to various embodiments of the present disclosure. Memory (13) may be implemented as volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0106] Storage (15) can non-temporarily store a computer program. Storage (15) can be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0107] The communication interface (17) supports wired and wireless Internet communication of the lung disease prediction system (10). Furthermore, the communication interface (17) may support various communication methods other than Internet communication. To this end, the communication interface (17) may be configured to include a communication module well known in the technical field of the present disclosure.

[0108] The bus (19) provides a communication function between components of the lung disease prediction system (10). The bus (19) can be implemented as various types of buses such as an address bus, a data bus, and a control bus.

[0109] In this way, according to the present disclosure, a patient's chest X-ray image and breath sound data can be comprehensively analyzed, thereby allowing for more accurate prediction of lung disease compared to single data.

[0110] According to the present disclosure, by securing cohort data with standardized characteristics for lung diseases, it can be utilized in various medical fields.

[0111] According to the present disclosure, since a conventional chest X-ray device can be used, it can be expanded and applied to various medical fields.

[0112] According to the present disclosure, information for assisting medical staff in diagnosis can be provided by visualizing the cause of lung disease by location.

[0113] The embodiments of the present disclosure described above are not implemented only through devices and methods, but may also be implemented through a program that realizes a function corresponding to the configuration of the embodiments of the present disclosure or a recording medium on which the program is recorded.

[0114] Although the embodiments of the present disclosure have been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concepts of the present disclosure defined in the following claims also fall within the scope of the present disclosure.

Claims

1. A method for operating a lung disease prediction system operated by at least one processor, A step of acquiring a chest X-ray image of a patient and a plurality of breath sound intensity images acquired from at least one chest region of the patient, A step of extracting a lung region from the chest X-ray image to generate a lung region X-ray image, A step of warping the plurality of breath sound intensity images in the extracted lung area to generate a plurality of breath sound intensity images warped in the lung area, and A step of obtaining a lung disease prediction result predicted from the lung region radiographic image and the plurality of warped breath sound intensity images using a lung disease prediction model learned to predict lung disease from an input image. A method of operation including:

2. In paragraph 1, The above breath sound intensity image is An operating method, wherein the image is generated based on breath sound signals measured over a certain period of time in multiple chest regions of the patient using a multi-channel breath sound measurement device.

3. In paragraph 2, The above breath sound intensity image is An operating method in which the distribution of the intensity of the breathing sound obtained based on the above breathing sound signal is expressed for each of a plurality of locations where the breathing sound is measured to generate an intensity image of the breathing sound at a specific point in time, and the breathing sound intensity image at the specific point in time is data generated by time for the entire measurement time.

4. In paragraph 1, The step of generating multiple warped breath sound intensity images in the above lung region is as follows. A step of extracting feature points from the lung region X-ray image and the breath sound intensity image, and obtaining coordinate values of each image based on the feature points; A step of displaying the coordinate values of each image above on the same point map of a two-dimensional plane, A step of calculating a transformation matrix between the lung region radiographic image and the breath sound intensity image based on the corresponding relationship obtained by registering the coordinate values of each image, and A step of applying the above transformation matrix to the above breath sound intensity image to generate a plurality of warped breath sound intensity images in the lung area. A method of operation including:

5. In paragraph 1, Before the step of obtaining the above lung disease prediction results A step of generating learning data based on the lung region radiographic image and the warped plurality of breath sound intensity images, and A step of training the lung disease prediction model using the above learning data. A method of operation including:

6. In paragraph 5, The steps for generating the above learning data are An operating method for generating the learning data by using at least one of a method of concatenating at least one of the lung region radiographic images and the warped plurality of breath sound intensity images, a method of converting the at least one of the images into color images and blending R, G, and B channels of the converted color images, and a method of performing a domain transform on the images and combining the at least one of the images with a plurality of domain transformed images obtained.

7. In paragraph 1, The step of obtaining the above lung disease prediction results is An operating method further comprising the step of providing an area related to the lung disease prediction result by displaying it in a manner differentiated according to importance in at least one of the lung region X-ray image or the warped plurality of breath sound intensity images.

8. A lung disease prediction system operated by at least one processor, A radiographic image processing unit that extracts the lung region from the patient's chest radiographic image and generates a lung region radiographic image; A warping unit that warps a plurality of breath sound intensity images acquired from at least one chest region of the patient in the extracted lung region to generate a plurality of breath sound intensity images warped to the lung region, and A lung disease prediction unit that obtains predicted lung disease prediction results from the lung region radiographic images and the warped plurality of breath sound intensity images using a lung disease prediction model learned to predict lung disease from an input image. A lung disease prediction system including:

9. In paragraph 8, The above breath sound intensity image is A lung disease prediction system, which is an image generated based on breath sound signals measured over a certain period of time in multiple chest regions of the patient using a multi-channel breath sound measurement device.

10. In paragraph 9, The above breath sound intensity image is A lung disease prediction system, which generates a respiratory sound intensity image at a specific point in time by expressing the distribution of the respiratory sound intensity obtained based on the above respiratory sound signal for each of a plurality of locations where the respiratory sound was measured, and which is data generated by generating the respiratory sound intensity image at the specific point in time by time for the entire measurement time.

11. In paragraph 8, The above warping part Extracting feature points from the lung region X-ray image and the breath sound intensity image, respectively, and obtaining the coordinate values of each image based on the feature points, The coordinate values of each image above are displayed on the same point map of a two-dimensional plane, Based on the corresponding relationship obtained by registering the coordinate values of each image above, a transformation matrix between the lung region radiographic image and the breath sound intensity image is calculated, and A lung disease prediction system that applies the above transformation matrix to the above breath sound intensity image to generate a plurality of warped breath sound intensity images in the lung region.

12. In paragraph 8, The above lung disease prediction section A learning data generation unit that generates learning data based on the lung region X-ray image and the warped plurality of breath sound intensity images, and A lung disease prediction system further comprising a learning unit that learns the lung disease prediction model using the learning data.

13. In paragraph 12, The above learning data generation unit A lung disease prediction system that generates the learning data by using at least one of a method of concatenating at least one of the lung region radiographic images and the plurality of warped breath sound intensity images, a method of converting the at least one of the images into color images and blending R, G, and B channels of the converted color images, and a method of performing a domain transform on the images and combining the at least one of the images with a plurality of domain transformed images obtained.

14. In paragraph 8, A lung disease prediction system further comprising a prediction information providing unit that provides an area related to the lung disease prediction result by displaying it in a manner differentiated according to importance in at least one of the lung region X-ray image or the warped plurality of breath sound intensity images.

15. As a lung disease prediction system, Memory that stores instructions, and At least one processor for executing the above instructions, The above processor A lung disease prediction system, which extracts a lung region from a chest X-ray image to generate a lung region X-ray image, warps a plurality of breath sound intensity images in the extracted lung region to generate a plurality of warped breath sound intensity images in the lung region, and obtains a lung disease prediction result predicted from the lung region X-ray image and the plurality of warped breath sound intensity images using a lung disease prediction model learned to predict lung disease from an input image.

16. In paragraph 15, The above processor A lung disease prediction system, which extracts feature points from the lung region X-ray image and the breath sound intensity image, obtains coordinate values of each image based on the feature points, displays the coordinate values of each image on the same point map of a two-dimensional plane, and applies a transformation matrix between the lung region X-ray image and the breath sound intensity image, which is calculated based on a correspondence relationship obtained by registering the coordinate values of each image, to the breath sound intensity image to generate a plurality of warped breath sound intensity images of the lung region.

17. In paragraph 15, The above processor A lung disease prediction system that generates learning data based on the lung region radiographic image and the warped plurality of breath sound intensity images, and trains the lung disease prediction model using the learning data.

18. In paragraph 17, The above processor A lung disease prediction system that generates the learning data by using at least one of a method of concatenating at least one of the lung region radiographic images and the plurality of warped breath sound intensity images, a method of converting the at least one of the images into color images and blending R, G, and B channels of the converted color images, and a method of performing a domain transform on the images and combining the at least one of the images with a plurality of domain transformed images obtained.

19. In paragraph 15, The above processor A lung disease prediction system that provides an area related to the lung disease prediction result by displaying it according to importance in at least one of the lung region X-ray image or the warped plurality of breath sound intensity images.

Images