Auxiliary measurement method using chest x-ray images for small sample pre-training and auxiliary measurement system using the same

Abstract

The present invention relates to an auxiliary measurement method for small sample pre-training using chest X-ray images and an auxiliary measurement system using the same, which uses free text reports and corresponding chest X-ray images with structured labels of at least one target disease to perform pre-training. The trained multimodal encoder, combined with the linear model of the embedding extracted by the model, achieves the effectiveness of measuring the target disease with a small sample size, thereby obtaining a performance comparable to that of a deep convolutional neural network trained on the full dataset. accuracy.

Description

It has an auxiliary measurement method for small-sample pre-training using chest X-ray images and an auxiliary measurement system for applying it. This invention belongs to the field of auxiliary diagnostic measurement technology. Specifically, it is an auxiliary measurement method and system that uses chest X-ray images for small-sample pre-training, thereby effectively obtaining an accuracy similar to that of a deep convolutional neural network trained on a full-sample dataset. Note that chest X-rays (CXRs) are a commonly used imaging technique, used at least 2 billion times globally each year. In the current development of CXR analysis techniques, deep learning techniques, especially Convolutional Neural Networks (CNNs), have been found to achieve expert-level performance in supervised learning models. However, the success of supervised learning models largely depends on large amounts of data and high-quality annotations, sometimes requiring collaboration among multiple experts to generate high-quality labels, which consumes significant time, manpower, and costs. The current lack of large and high-quality labeled datasets is considered a major limiting factor for supervised deep learning in medical imaging tasks. Therefore, developing suitable methods to reduce the dependence of supervised learning models on large, structured labeled datasets is one of the most critical technical challenges. Applying pre-trained models for transfer learning is a primary method for improving the performance of models with limited samples, and it has been widely accepted in the field of medical image analysis. Currently, there are several unsupervised pre-training algorithms for image classification tasks. Recently, a multimodal model called Contrastive Language-Image Pre-Training (CLIP), trained using text and image pairs, has shown better accuracy than the aforementioned unsupervised learning models in downstream tasks. In other words, developing a target prediction model for multiple target diseases using a deep learning model with smaller sample data to assist clinicians in identification, enabling early, objective, and more accurate diagnosis, and effectively serving as a basis for rapid treatment, is an important topic in the industry and is what this invention aims to explore. In view of the aforementioned shortcomings and needs, the inventor believes that further development is necessary. Therefore, based on years of experience in related technologies and product design and manufacturing, the inventor has researched and created solutions to address the above-mentioned deficiencies and actively sought solutions. Through continuous research and trial production, the inventor has developed an auxiliary measurement method and system that uses chest X-ray images for small-sample pre-training, in order to solve the inconvenience and trouble caused by the inability of existing auxiliary diagnostic systems to effectively assess left ventricular function due to insufficient accuracy. Therefore, the main objective of this invention is to provide an auxiliary measurement method and system that uses chest X-ray images for small-sample pre-training, thereby establishing a pre-trained model, extracting high-order features from chest X-ray images, and then training an elastic network for predicting diseases through a small-sample learning search method, thereby effectively improving the accuracy of disease prediction and enabling early, objective, and accurate diagnosis of target diseases. Secondly, another major objective of this invention is to provide an auxiliary measurement method and system that uses chest X-ray images for small-sample pre-training, which can establish multiple disease or clinical symptom target prediction models. It can be applied in hospitals, primary healthcare institutions, or mobile chest X-ray screening vehicles, and can not only be used for clinical assessment, but also provide extensive community screening to identify potential patients and enable medical personnel to monitor and intervene in real time. Therefore, the present invention mainly achieves the above-mentioned items and performance through the following technical means, which includes the following steps: Multiple chest X-ray images are input using an image acquisition device, each of which is accompanied by a certified text report and a structured label of at least one target disease, for storing the multiple chest X-ray images in a data storage device. The aforementioned chest X-ray image is preprocessed by an image processing device to adjust the aforementioned chest X-ray image into a preprocessed chest X-ray image of the same specification. A comparative learning model of at least one target disease is established on the aforementioned preprocessed chest X-ray images using a processing device; The high-level features of the preprocessed chest X-ray image are extracted using the contrastive learning model of the aforementioned processing device. A linear detection operation is performed on the aforementioned processed chest X-ray image with extracted high-order features using a computing device; The processing device establishes a target prediction model for at least one target disease using the aforementioned pre-trained model; and A chest X-ray image is output through a measurement and output device, which is marked by one of the key markers of the target disease as indicated by the aforementioned target prediction model. And use the following system to execute, which contains; An image acquisition device for acquiring multiple chest X-ray images, wherein each chest X-ray image is accompanied by a certified text report and a structured label of at least one target disease. A data storage device connected to the image acquisition device for storing the aforementioned chest X-ray image acquired by the image acquisition device; An image processing device is connected to the data storage device. The image processing device can preprocess the chest X-ray images into preprocessed chest X-ray images of the same size and store the preprocessed chest X-ray images back into the data storage device. A processing device connected to the data storage device, the processing device being able to establish a contrastive learning model from the preprocessed chest X-ray images; A computing device connected to the processing device, the computing device being able to extract high-order features and perform linear probing to establish a target prediction model, and the computing device being able to obtain markers of the corresponding target disease from at least one chest X-ray image to be tested; and A measurement output device connected to the processing device, which can output the results as a relative measurement chest X-ray image to assist in interpretation. Through the specific implementation of the aforementioned technical means, the present invention can greatly enhance its practicality, increase its added value, and improve its economic benefits. To enable your review committee to further understand the structure, features and other objectives of the present invention, several preferred embodiments of the present invention are described below in detail with reference to the accompanying drawings, so that those skilled in the art can implement them. The accompanying drawings illustrate specific embodiments of the invention and their components. All references to front and back, left and right, top and bottom, upper and lower, and horizontal and vertical are for convenience of description only and are not intended to limit the invention or restrict its components to any position or spatial orientation. Dimensions specified in the drawings and specification may be varied according to the design and requirements of specific embodiments of the invention without departing from the scope of the claims. This invention relates to an auxiliary measurement method using chest X-ray images for small-sample pre-training. It utilizes deep convolutional neural networks (DCNNs) to compute chest X-ray images (CXRs) to quickly assist in predicting the probability of multiple chest-related diseases. Figures 1 and 2 show the method flowchart and system architecture diagram of an embodiment of this invention. As shown, the auxiliary measurement method using chest X-ray images for small-sample pre-training is applied to this auxiliary measurement system, which includes an image acquisition device (10), a data storage device (20), an image processing device (30), a processing device (40), a computing device (50), and a measurement output device. (60), wherein the image acquisition device (10), the data storage device (20), the image processing device (30), the processing device (40), the computing device (50), and the measurement output device (60) can be integrated into an integrated structure or a separate structure. If it is a separate structure, it can be interconnected using wired technology (such as Ethernet) or wireless technology (such as wireless medical systems, Wi-Fi, or 3G or higher mobile communications) for mutual data transmission. The auxiliary measurement method includes the following steps (S01) to (S07), such as: Step (S01): Input multiple chest X-ray images using an image acquisition device, each chest X-ray image having a certified text report and a structured label of at least one target disease, for storing the multiple chest X-ray images in a data storage device: First, input multiple anteroposterior (AP) or posteroanterior (PA) chest X-ray images through an image acquisition device (10) for storing these chest X-ray images in a data storage device (20), and each chest X-ray image contains a text report certified by a professional physician and a structured label of at least one target disease marked by a certified radiologist, while ensuring that each patient selects only one chest X-ray image to be stored in the data storage device (20) to reduce potential problems related to sample interdependence in statistical analysis. In this embodiment, the image acquisition device (10) includes an X-ray scanner, a computer device for image file processing, and chest X-ray images stored in a database of a medical institution or research unit.These target diseases include, but are not limited to, 31 common chest-related diseases such as aneurysm, atelectasis, atherosclerosis, cardiomegaly, consolidation change, pleural angle blunting, degenerative joint disease, emphysematous change, endotracheal tube, fracture, inflammatory response, malignancy, nasogastric tube, osteoarthritis, osteoporosis, osteophyte formation, pacemaker, perm catheter insertion, pigtail or drainage, and pleural effusion. The study included various conditions such as effusion, pneumonia, pneumothorax, Port A implantation, prominence of hilar shadow, pulmonary edema, scalloping of the diaphragm, spondylosis, sternotomy, tracheostomy, vertebroplasty, and widening of the mediastinum. Further, it selected five target diseases that the medical community considered potentially related to chest X-ray images, including but not limited to left ventricular dysfunction, aortic stenosis, pulmonary arterial hypertension, left atrial enlargement, and pericardial effusion. [effusion] etc., and annotate the structural reports of echocardiograms (cardiac ultrasound) of these diseases in these chest X-ray images. Step (S02): Perform preprocessing of the aforementioned chest X-ray images by an image processing device to adjust the aforementioned chest X-ray images into preprocessed chest X-ray images of the same specification: Since the pixel size of the aforementioned chest X-ray images may not be consistent, an image processing device (30) adjusts the size of the aforementioned chest X-ray images stored in the data storage device (20) while maintaining the aspect ratio to form new preprocessed chest X-ray images. For example, the short side is scaled to 256 pixels, and the numerical range of each pixel is standardized to 8 bits. The length of the text report for these chest X-ray images is set to no more than 256 tags, for example, gpt-3.5-turbo-0613 model, and a prompt similar to "Please summarize and simplify the following report: <Original Radiographic Report>" is used until the text report of each preprocessed chest X-ray image is less than 256 tags, and the preprocessed chest X-ray images of the same specification are stored in the data storage device (20). Step (S03): Establish a contrastive learning model for at least one target disease using the aforementioned preprocessed chest X-ray images through a processing device: Then, the preprocessed chest X-ray images are obtained from the data storage device (20) through a processing device (40), and a contrastive learning model is established using Contrastive Language-Image Pre-Training (CLIP). This model includes an image encoder and a text encoder. The image encoder uses an architecture such as ViT-B / 32, dividing the aforementioned 256x256 pixel preprocessed chest X-ray images into an 8x8 matrix through a 32x32 convolutional neural network. The matrix is ​​then adjusted to a size of 64x1, and the [CLS] (Classification) label is added to the first position of the matrix, making the matrix a size of 65x1. The matrix is ​​then input into a 12-layer deep and hidden layer. The first layer is a 768-byte self-attention encoder (Transformer), which outputs a 512-byte vector labeled with [CLS]. The text encoder is a standard language encoder that first tokenizes the text into vectors of maximum length 256, adding a [CLS] tag at the first position. This is followed by a standard encoder with 12 layers, consisting of a 512-byte embedding layer, a 768-byte hidden layer, and eight attention heads, also outputting a 512-byte vector labeled with [CLS]. Before inputting the preprocessed chest X-ray image and text report into their respective encoders, an additional [CLS] tag is added, and the output from the [CLS] tag is used as the embedding for the chest X-ray image and text report. Finally, during backpropagation, the inner product of the image's [CLS] tag and the text's [CLS] tag is calculated to obtain a similarity matrix, which is then optimized using softmax and cross-entropy loss to produce the final output. In this embodiment, all technical details strictly follow OpenAI's CLIP model, and the weights of that model are used as initialization parameters. During the pre-training phase, the training set is randomly divided into 90% for model fitting and 10% for validation. All network parameters are fine-tuned using the standard parameters of the SGD optimizer, with a batch size of 64, a learning rate of 0.0001, and a momentum of 0.9. Throughout the training process, images are randomly cropped to 256×256 pixels. Model training is performed for 50 epochs, and the validation loss is calculated at the end of each epoch to select the best-performing model. The model training in this embodiment is performed in Python environment version 3.10.10, using the "torch" package version 2.0.1. Step (S04): Extract high-order features from the preprocessed chest X-ray images using the contrastive learning model of the aforementioned processing device: The trained contrastive learning model will be further processed by the processing device (40) to extract high-order features from all the preprocessed chest X-ray images in the data storage device (20). Each high-order feature is a vector of size 512. In terms of sample selection, a small sample search will be performed. In this invention, a fixed ratio of 1 is used for the case group (chest X-ray images with the target disease label) and 3 is used for the control group (chest X-ray images without the target disease label). In this embodiment, 128 samples are used for the case group and 384 samples are used for the control group. The secondary samples are randomly selected from 1 according to the ratio of the number of control group to the number of case group. The determined secondary samples will be oversampled. Step (S05): Perform a linear probing operation on the processed chest X-ray image with extracted high-order features using a computing device: After extracting the high-order features of the pre-trained model, perform a linear probing operation on the pre-processed chest X-ray image using the computing device (50). In this embodiment, the "glmnet" package version 2.0-16 in R software 3.4.4 is used to perform linear probing on the CXR embedding. The "glmnet" package uses the Elastic Network algorithm, which includes L1 and L2 regularization terms in logistic regression. For each target disease, the samples are first oversampled to ensure that the number of cases and controls is equal. Then, the "cv.glmnet" function is used to divide the samples into four subsets for cross-validation. During cross-validation, we search for the optimal value of lambda (regularization hyperparameter) in the range of e-9 to e-1 (a sequence of 45 log intervals), and search for alpha (the ratio of L1 and L2 penalties) in the range of 0 to 1 (in increments of 0.1), which is used to maximize the area under the curve (AUC) of the cross-validation, and then directly apply the model fitted by the hyperparameter with the highest cross-validation AUC. In this embodiment, using the aforementioned pre-trained model with 128 case samples and 384 control samples for linear probing, the accuracy achieved was comparable to that of a convolutional neural network (CNN) trained on the full sample dataset. It is noteworthy that although the free text reports in the pre-trained dataset are unlikely to contain descriptions of echocardiographic findings, the chest X-ray image embeddings extracted in this invention still show a strong correlation with these echocardiographic labels, highlighting their potential to surpass existing human knowledge. Step (S06): Establish a target prediction model for at least one target disease through the processing device using the aforementioned pre-trained model: Following steps (S01) to (S05), at least one target disease is analyzed in the processing device (40) using a fixed ratio of 1 case group (chest X-ray images with "target disease" label) and 3 control group (chest X-ray images without "target disease" label). In this embodiment, the target diseases are represented by small samples of 128 case group and 384 control group. A contrastive learning model is established by contrastive language-image pre-training on the pre-processed chest X-ray images. After high-order feature extraction and linear probing, a target prediction model for the corresponding target disease is established in the processing device (40). The predictive model, in this embodiment, includes, but is not limited to, 31 chest-related diseases such as aneurysm, pulmonary insufficiency, atherosclerosis, cardiac hypertrophy, parenchymal lesions, pleural angle blunting, degenerative joint diseases, emphysematous changes, endotracheal intubation, fracture, inflammatory response, malignant tumor, nasogastric tube, osteoarthritis, osteoporosis, osteophytes, pacemaker, catheter placement, drainage tube, pleural effusion, pneumonia, pneumothorax, artificial vascular implants, increased pulmonary striae, pulmonary edema, diaphragmatic hernia, vertebral arthropathy, sternotomy, tracheotomy, vertebroplasty, and septal enlargement, as well as five target diseases related to echocardiography, including, but not limited to, left ventricular dysfunction, aortic stenosis, pulmonary hypertension, left atrial enlargement, and pericardial effusion. Step (S07): Output a chest X-ray image of a patient to be tested through a measurement output device, which is marked with one of the key markers of the target disease by the aforementioned target prediction model. The chest X-ray image of a patient to be tested is input through the image acquisition device (10). The chest X-ray image to be tested is preprocessed and adjusted to the same specifications as the pre-trained preprocessed chest X-ray image stored in the data storage device (20). For example, the chest X-ray image to be tested is scaled with the short side set to 256 pixels while maintaining the aspect ratio. The processing device (40) performs classification calculations on the aforementioned contrast learning model and the target prediction model, so that the chest X-ray image to be tested obtains the marker of the corresponding target disease. The measurement output device (60) outputs these results as a corresponding chest X-ray image. Different colors can be used to mark the markers of the possible corresponding target diseases during output. The chest X-ray image is used to assist medical personnel or interpreters in understanding the patient's condition and to provide suggestions for subsequent treatment or surgery. The measurement output device can be various displays, including but not limited to computer screens, monitors, or handheld device displays used by medical personnel in medical institutions. The detailed configuration of the system of the present invention is as shown in the second figure. The image acquisition device (10) is connected to the data storage device (20) for storing chest X-ray images after acquisition. Each chest X-ray image is accompanied by a certified text report and a structured label for at least one target disease. The data storage device (20) is connected to the image processing device (30) so that the image processing device (30) can preprocess the chest X-ray images obtained from the data storage device (20) to form corresponding preprocessed chest X-ray images of the same specifications. Furthermore, the processing device (40) is also connected to the data storage device (20). The processing device (40) can process the preprocessed chest X-ray images. A contrast learning model is established using chest X-ray images, and the processing device (40) is further connected to the computing device (50) to extract high-order features and linear detection through the computing device (50) to establish a target prediction model. After inputting the chest X-ray image to be tested, the target prediction model of the processing device (40) and the computing device (50) are processed to obtain the marker of the corresponding target disease in the chest X-ray image to be tested. The processing device (40) is connected to the measurement output device (60), and the measurement output device (60) outputs these results into a corresponding measured chest X-ray image to assist medical personnel in interpreting the patient's condition, allowing medical personnel to monitor and intervene in real time, and to provide suggestions for subsequent treatment or surgery. In conclusion, it can be understood that this invention is an invention with excellent creativity. In addition to effectively solving the problems faced by the inventors, it also greatly improves the efficiency. Moreover, no identical or similar products have been created or publicly used in the same technical field. Since it has the effect of improving efficiency, this invention has met the requirements of "novelty" and "inventiveness" for invention patents, and an invention patent application is filed in accordance with the law. 10: Image acquisition device; 20: Data storage device; 30: Image processing device; 40: Processing device; 50: Calculation device; 60: Measurement output device. S01: Input multiple chest X-ray images using an image acquisition device, each chest X-ray image having an authenticated text report and at least one structured label for a target disease, for storing the multiple chest X-ray images in a data storage device. S02: Perform preprocessing of the chest X-ray images using an image processing device to adjust the chest X-ray images into preprocessed chest X-ray images of the same specification. S03: Through a... The processing device establishes a contrastive learning model for at least one target disease on the aforementioned preprocessed chest X-ray image. S04: High-order features are extracted from the aforementioned preprocessed chest X-ray image using the contrastive learning model of the processing device. S05: A linear detection operation is performed on the aforementioned processed chest X-ray image with extracted high-order features using a computing device. S06: A target prediction model for at least one target disease is established through the processing device using the aforementioned pre-trained model. S07: A measurement output device outputs a measured chest X-ray image marked with one of the key markers corresponding to the target disease by the aforementioned target prediction model. Figure 1: A schematic diagram of the process architecture of the auxiliary measurement method of the present invention, which uses chest X-ray images for small-sample pre-training. Figure 2: A schematic diagram of the system architecture of the auxiliary measurement method using chest X-ray images for small-sample pre-training, which applies the present invention. S01: Inputting multiple chest X-ray images using an image acquisition device, each chest X-ray image having an authenticated text report and a structured tag for at least one target disease, for constructing a data storage device to store the multiple chest X-ray images. S02: Preprocessing of the aforementioned chest X-ray image is performed by an image processing device to adjust the aforementioned chest X-ray image into a preprocessed chest X-ray image of the same specification. S03: Establish a comparative learning model for at least one target disease based on the aforementioned preprocessed chest X-ray images using a processing device. S04: Extract high-level features from the preprocessed chest X-ray image using the contrastive learning model of the aforementioned processing device. S05: Perform a linear detection operation on the processed chest X-ray image with extracted high-order features using a computing device. S06: Using the processing device, establish a target prediction model for at least one target disease through the aforementioned pre-trained model. S07: A chest X-ray image, marked by one of the key markers of the target disease as indicated by the aforementioned target prediction model, is output via a measurement and output device.

Claims

1. An auxiliary measurement method using chest X-ray images for small-sample pre-training includes the following steps: inputting multiple chest X-ray images using an image acquisition device, each chest X-ray image having an authenticated text report and a structured label for at least one target disease, for storing the multiple chest X-ray images in a data storage device; performing preprocessing of the chest X-ray images using an image processing device to adjust the chest X-ray images to a uniform preprocessed chest X-ray image format; establishing a contrastive learning model for at least one target disease using the preprocessed chest X-ray images using a processing device; extracting high-order features from the preprocessed chest X-ray images using the contrastive learning model of the processing device; performing a linear probing operation on the preprocessed chest X-ray images with extracted high-order features using a computing device; establishing a target prediction model for at least one target disease using the pretrained model using the processing device; and outputting a measured chest X-ray image marked with a key marker corresponding to the target disease by the target prediction model using a measurement output device. As described in claim 1, there is an auxiliary measurement method for small-sample pre-training using chest X-ray images, wherein the chest X-ray images can be anteroposterior chest X-ray images or posteroanterior chest X-ray images. As described in claim 1, there is an auxiliary measurement method using chest X-ray images for small-sample pre-training, wherein the target disease includes, but is not limited to, aneurysm, pulmonary insufficiency, atherosclerosis, cardiac hypertrophy, solidified lesions, blunting of the pleural angle, degenerative joint disease, emphysematous changes, endotracheal intubation, fracture, inflammatory response, malignancy, nasogastric tube, osteoarthritis, osteoporosis, osteophytes, pacemaker, catheter placement, drainage tube, pleural effusion, pneumonia, pneumothorax, artificial vascular implant, increased pulmonary striae, pulmonary edema, diaphragmatic hernia, vertebral arthropathy, sternotomy, tracheotomy, vertebroplasty, septal enlargement, or a combination thereof. As described in claim 1, there is an auxiliary measurement method for small-sample pre-training using chest X-ray images, wherein such chest X-ray images are further annotated with structural reports of echocardiography of the target disease. As described in claim 4, there is an auxiliary measurement method for small-sample pre-training using chest X-ray images, wherein the target disease includes, but is not limited to, left ventricular dysfunction, aortic stenosis, pulmonary hypertension, left atrial enlargement, pericardial effusion, or a combination thereof. As described in claim 1, there is an auxiliary measurement method that uses chest X-ray images for small-sample pre-training, wherein the step of extracting high-order features of the pre-processed chest X-ray images by means of the contrastive learning model of the aforementioned processing device is a small-sample search in terms of sample selection, and the analysis is performed with a fixed ratio of 1 for the case group of the target disease (chest X-ray images with the target disease label) and 3 for the control group (chest X-ray images without the target disease label). As described in Request 6, there is an auxiliary measurement method for small-sample pre-training using chest X-ray images, wherein the case group uses 128 samples and the control group uses 384 samples, and the subsamples are randomly selected starting from 1 according to the ratio of the number of control group to the number of case group. The determined subsamples are then oversampled. An auxiliary measurement system for performing the method described in any of claims 1 to 7, the system comprising: an image acquisition device for acquiring a plurality of chest X-ray images, wherein each chest X-ray image is accompanied by an authenticated text report and a structured label of at least one target disease; a data storage device connected to the image acquisition device for storing the aforementioned chest X-ray images acquired via the image acquisition device; an image processing device connected to the data storage device for preprocessing the chest X-ray images into preprocessed chest X-ray images of the same specification and storing the preprocessed chest X-ray images back to the data storage device; and a processing device connected to the data storage device for building a comparative learning model from the preprocessed chest X-ray images. A computing device connected to the processing device, the computing device can extract high-order features and linear detection through computation to establish a target prediction model, and the computing device can obtain the marker of the corresponding target disease from at least one chest X-ray image to be tested; and a measurement output device connected to the processing device, the measurement output device can output these results as a corresponding chest X-ray image to assist in interpretation. The auxiliary measurement system as described in claim 8 may include an integrated structure comprising the image acquisition device, the data storage device, the image processing device, the processing device, the computing device, and the measurement output device. The auxiliary measurement system as described in claim 8 may have a separate structure for the image acquisition device, the data storage device, the image processing device, the processing device, the computing device, and the measurement output device, and may be interconnected using wired or wireless technology for data transmission.

Images