A chest digital X-ray image processing method and system for pneumoconiosis

Abstract

The application discloses a kind of chest digital X-ray image processing method and system for pneumoconiosis, extract the complete rectangular area image with lung information in digital X-ray image, the rectangular area image is divided;For training set, apply random enhancement strategy to each sample, generate two global views and six local views;Teacher and student feature extraction network are constructed, two global views and six local views are input into teacher and student feature extraction network, respectively, after obtaining the depth feature output, it is sent into feature mapping layer and linear classification layer, respectively, to obtain the mapping representation and classification result corresponding to each view;Student network is trained in combination with loss function, teacher network uses exponential moving average strategy to obtain;After each round of training ends, input verification set into teacher network, to evaluate the classification performance of verification set;The teacher model weight of the best classification performance of verification set is saved, and the final test result is obtained by inputting test set into teacher network.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a chest digital X-ray image processing method and system for pneumoconiosis. BACKGROUND

[0002] Pneumoconiosis is a disease caused by long-term inhalation of production dust and retention in the lungs, mainly characterized by diffuse fibrosis of lung tissue. Due to the irreversibility of pneumoconiosis and the possible induction of multiple high-risk complications, middle and late pneumoconiosis will seriously endanger the quality of life of patients and cause economic losses. Early diagnosis and timely intervention to reduce patients' further engagement in dust exposure work are crucial to managing the progress of pneumoconiosis and protecting the health of patients.

[0003] For early diagnosis of pneumoconiosis, the lesion features presented on the early chest digital X-ray image are mainly small shadows, which are weak and difficult to determine whether the distribution density reaches the diagnosable level of pneumoconiosis. At least three radiologists need to analyze and examine the patient's digital X-ray by comparing with the standard diagnosis sheet, and the evaluation accuracy completely depends on the experience of the individual doctor, often resulting in inconsistent evaluation results among different doctors. In order to maintain concentration, the number of digital X-rays diagnosed by each doctor per day is small and limited. This poses a major challenge to regular large-scale screening of pneumoconiosis.

[0004] Effective use of deep learning methods for digital X-ray image classification provides hope for relieving the huge pressure on radiologists and solving the clinical challenges brought by pneumoconiosis, which has important practical significance. However, the low incidence of pneumoconiosis and the limitation of patient data sharing result in a small number of chest images available for training deep networks. In addition, the early symptoms of pneumoconiosis are more subtle and difficult to identify than other common lung diseases. These two challenges make most existing deep learning classification models prone to overfitting to inaccurate classification features, resulting in unreliable pneumoconiosis classification. Integrating the clinical diagnostic knowledge of professional radiologists into the design and learning process of deep networks can induce the model to learn more scientific and robust features from a small amount of limited data, which is of great significance to further improve the performance and reliability of image classification based on deep learning. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a chest digital X-ray image processing method and system for pneumoconiosis to solve the technical problems of overfitting and low classification accuracy of existing deep learning technology in the face of limited data and weak lesion features in digital X-ray image classification.

[0006] The technical scheme adopted by the present application is as follows:

[0007] A chest digital X-ray image processing method for pneumoconiosis, comprising the following steps:

[0008] Segmenting the lung region of the digital X-ray image, solving the minimum circumscribed rectangle of the segmentation result, extracting the complete rectangular region image with lung information in the digital X-ray image, and dividing the rectangular region image into a training set, a validation set and a test set;

[0009] Two global views and six local views are generated by applying different random enhancement strategies to each sample in the training set

[0010] A deep backbone classification network model is selected to construct a teacher and a student feature extraction network, a feature mapping layer is designed to map the deep features to a specific feature space, and a linear classification layer is designed to map the deep features to the classification result;

[0011] The two global views and six local views in the training set are input into the constructed teacher and student feature extraction networks respectively, and the deep feature outputs are sent into the feature mapping layer and the linear classification layer respectively; the teacher output and student output of the global lung view in the mapping space, the feature teacher output and student output of the six local lung views; the teacher global classification logic value and the student global classification logic value ; the six local lung area teacher classification logic value and the student classification logic value ;

[0012] The student network is trained and learned by combining three types of loss functions: global lung supervision classification loss, local-global self-supervised feature alignment loss, and representative local lung area classification regularization loss; update the student network parameters , and update the teacher model parameters using the exponential moving average strategy;

[0013] After each round of training, the updated teacher model is used to evaluate the classification performance of the rectangular region image samples in the validation set, and the teacher model weight with the best classification performance in the validation set is saved until the classification performance of the validation set no longer improves, and the model training is completed; in the test process, the teacher model weight with the best classification performance in the validation set is loaded, and the rectangular region image samples in the test set are input for classification output, and the chest digital X-ray image processing is completed.

[0014] Preferably, the global views in the training set After generating more than 40% of the original image by random color jittering with a probability of 0.8, random occlusion of areas outside the lung, and random cropping, the resolution size is adjusted to 224x224 by bilinear interpolation as the global view Input image;

[0015] Global view in training set After generating more than 40% of the original image by random color jittering with a probability of 0.8, random pixel flipping with a probability of 0.2, and random cropping of areas outside the lung, the resolution size is adjusted to 224x224 by bilinear interpolation as the global view Input image.

[0016] Preferably, six local views in the training set First, the acquired rectangular region image is divided into six parts from top to bottom and left and right symmetrically, i.e., right upper, left upper, middle right, middle left, right lower, and left lower. Data augmentation measures such as random color jittering with a probability of 0.8 and random occlusion of areas outside the lung with a probability of 0.5 are used to adjust the resolution to a uniform size of 96x96 by bilinear interpolation as the local lung area image local view Input image.

[0017] Preferably, the teacher and student feature extraction network adopts a deep classification network ResNet, ConvNeXt based on CNN, a deep classification network Swin Transformer based on Transformer, or a deep classification network VMamba based on Mamba; the feature mapping layer includes a 3-layer multilayer linear perceptron with a hidden dimension of 2048, L2 normalization, and a fully connected layer; and the linear classification layer is a fully connected layer.

[0018] Preferably, for a digital X-ray sample with a disease class label, the feature similarity calculation is performed between the feature outputs corresponding to the six local lung areas in the student model mapping space and the feature output corresponding to the global view in the teacher model mapping space, and the top two local lung areas with the highest similarity are selected as representative local classification negative samples; for a digital X-ray sample with a normal label, the feature similarity calculation is performed between the feature outputs corresponding to the six local lung areas in the student model mapping space and the feature output corresponding to the global view in the teacher model mapping space, and two local lung areas are randomly selected from the top five local lung areas with the highest similarity as representative local classification positive samples.

[0019] Preferably, the multi-supervised classification loss function is:

[0020]

[0021] wherein, respectively represent the balance loss weight of the local-global feature alignment loss and the classification regularization loss of the representative local lung region, represents a global lung supervision classification loss function, represents a local-global feature alignment loss function, represents a classification regularization loss function of the representative local lung region.

[0022] Preferably, the global lung supervision classification loss function is:

[0023]

[0024] wherein, is the label information representing whether the current sample is diseased, is the classification logic value corresponding to the global sample output by the student classification network, is the global view.

[0025] Preferably, the local-global feature alignment loss function is:

[0026]

[0027] wherein, is the global view, is the local view, represents a cross-entropy loss function calculation, represents the global feature probability distribution output by the teacher feature mapping network under the input condition of the global view, represents the feature probability distribution output by the student feature mapping network under the input condition of all global and local views.

[0028] Preferably, the classification regularization loss function of the representative local lung region is:

[0029]

[0030] wherein, respectively indicate the representative local lung region selected in the normal sample and the diseased sample, is the label information representing whether the current sample is diseased, is the classification logic value of the student classification network when the local view is input.

[0031] In a second aspect, the embodiments of the present application provide a chest digital X-ray image processing system for pneumoconiosis, comprising:

[0032] The segmentation module segments the lung region of the digital X-ray image, and solves the minimum circumscribed rectangle of the segmentation result to extract a complete rectangular region image with lung information in the digital X-ray image.

[0033] The data set division module divides all the rectangular region images into three parts of training, verification and testing.

[0034] The enhancement module applies different random enhancement strategies to generate two global views and six local views for each rectangular region image sample in the training set. ;

[0035] The network module selects a deep backbone classification network model to construct teacher and student feature extraction networks, designs a feature mapping layer to map deep features to a specific feature space, and designs a linear classification layer to map deep features to classification results.

[0036] The training module inputs two global views and six local views of the training set into the constructed teacher and student feature extraction networks respectively, and sends the deep feature outputs into the feature mapping layer and the linear classification layer respectively; obtains the teacher output and student output of the global lung view in the mapping space, the feature teacher output and student output of the six local lung area views; the teacher global classification logic value and the student global classification logic value ; the six local lung area teacher classification logic value and the student classification logic value ; combine the global lung supervision classification loss, the local-global self-supervised feature alignment loss, and the classification regularization loss of the representative local lung area to train and learn the teacher and student feature extraction networks; update the student model parameters , and update the teacher model parameters using the exponential moving average strategy; after each round of training, the updated teacher model is used to evaluate the classification performance of the rectangular region image samples in the verification set, and the teacher model weight with the best classification performance in the verification set is saved until the classification performance of the verification set no longer improves, and the model training is completed.

[0037] The test module loads the teacher model weight with the best classification performance in the verification set, inputs the rectangular region image samples in the test set, and outputs the classification results to complete the chest digital X-ray image processing.

[0038] ​In a third aspect, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the pneumoconiosis-oriented chest digital X-ray image processing method when executing the computer program.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium including a computer program, and the computer program implements the steps of the pneumoconiosis-oriented chest digital X-ray image processing method when executed by a processor.

[0040] In a fifth aspect, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the pneumoconiosis-oriented chest digital X-ray image processing method when executing the computer program.

[0041] In a sixth aspect, an embodiment of the present application provides an electronic device including a computer program, and the computer program implements the steps of the pneumoconiosis-oriented chest digital X-ray image processing method when executed by the electronic device.

[0042] Compared with the prior art, the present application has at least the following beneficial effects:

[0043] A pneumoconiosis-oriented chest digital X-ray image processing method extracts a lung rectangular region, thereby reducing the influence of irrelevant lung regions on the classification process; data augmentation is performed on each sample in a training set to generate two global views and six local views, thereby providing data support for exploring sample local-global representation and diagnosis relationship; a teacher-student dual-branch training network is built to extract teacher and student mapping representation outputs and classification results corresponding to global views and local views, thereby providing network support for combining pneumoconiosis clinical diagnosis guideline standards; a multi-supervised classification loss function is designed to effectively maximize the utilization of label and sample information, which is beneficial to alleviating overfitting phenomenon in small-scale data set training process; the screening standard of representative local lung regions is closest to the definition standard of the pneumoconiosis clinical diagnosis guideline, thereby adding reliability to the network learning process.

[0044] It can be understood that the beneficial effects of the above-mentioned second aspect can be referred to the related description in the above-mentioned first aspect, which will not be described herein again.

[0045] In summary, the present application builds a local-global multi-supervised learning architecture based on a deep network to maximize the learning of limited small-scale data, thereby reducing the overfitting phenomenon of digital X-ray image classification tasks in the case of data scarcity and improving the classification accuracy.

[0046] The technical solutions of the present application will be further described in detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A diagram for a local-global multi-supervised learning architecture based on a deep network is provided;

[0048] Figure 2 A schematic diagram of a computer device provided by an embodiment of the present application is shown;

[0049] Figure 3 A block diagram of a chip according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of the present application.

[0051] In the description of the present application, it should be understood that the terms “include” and “contain” indicate the existence of described features, integers, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or sets thereof.

[0052] It should also be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, unless otherwise clearly indicated by the context, the singular forms “a”, “an” and “the” are intended to include the plural forms.

[0053] It should be further understood that the term “and / or” used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character “ / ” in the present application generally represents an “or” relationship between the front and rear associated objects.

[0054] It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe the preset ranges, etc., these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from each other. For example, the first preset range can also be referred to as the second preset range, and similarly, the second preset range can also be referred to as the first preset range without departing from the scope of the embodiments of the present application.

[0055] Depending on the context, the word "if" as used herein can be interpreted to mean "when" or "while" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be interpreted to mean "when it is determined" or "in response to determining" or "when [a stated condition or event] is detected" or "in response to detecting [a stated condition or event]."

[0056] Various structural diagrams according to the disclosed embodiments of the present application are shown in the accompanying drawings. These diagrams are not drawn to scale in that certain details are exaggerated for the purpose of clarity, and certain details can be omitted. The shapes of various regions, layers, and the relative sizes and positional relationships among them shown in the diagrams are merely exemplary, and in actuality can be deviated due to manufacturing tolerances or technical limitations, and regions / layers with different shapes, sizes, and relative positions can be additionally designed according to actual needs by those skilled in the art.

[0057] The present application provides a chest digital X-ray image processing method for pneumoconiosis, which integrates the relevant knowledge of professional radiologists in the deep network model training stage, and improves the performance of various types of deep network models in this task; a single digital X-ray image input is expanded into six local lung regions (right upper, left upper, middle right, middle left, right lower and left lower) and global lung input, and the same deep model is used to represent and diagnose the six local lung regions and the global lung respectively; the deep model is trained and learned by combining three types of loss functions of global lung supervision classification, local-global feature alignment and classification regularization of part of representative local lung regions, prompting the network to explore stronger local-global internal association relationships in the case of limited data amount, to reduce the overfitting phenomenon of the model classification task in the case of data scarcity, and to improve the classification performance of digital X-ray images.

[0058] Referring to Figure 1 The present application provides a chest digital X-ray image processing method for pneumoconiosis, which integrates the relevant knowledge of professional radiologists in the deep network model training stage, and improves the performance of various types of deep network models in this task; a single digital X-ray image input is expanded into six local lung regions (right upper, left upper, middle right, middle left, right lower and left lower) and global lung input, and the same deep model is used to represent and diagnose the six local lung regions and the global lung respectively; the deep model is trained and learned by combining three types of loss functions of global lung supervision classification, local-global feature alignment and classification regularization of part of representative local lung regions, prompting the network to explore stronger local-global internal association relationships in the case of limited data amount, to reduce the overfitting phenomenon of the model classification task in the case of data scarcity, and to improve the classification performance of digital X-ray images.

[0059] S1, collect and establish a digital X-ray classification data set

[0060] 500 cases of digital X-ray DICOM format data were collected, including 275 cases of diseased digital X-ray and 225 cases of normal digital X-ray. After converting the collected DICOM format digital X-ray to UInt8 PNG format, the existing lung deep segmentation network technology CheXmask was used to segment the lung region of the chest digital X-ray image, and the minimum circumscribed rectangle of the segmentation result was solved to extract the complete rectangular region image with lung information in the digital X-ray image.

[0061] The CheXmask method of the prior art refers to the lung deep segmentation network method adopted in Gaggion et al. (Gaggion N, Mosquera C, Mansilla L, et al. CheXmask: a large-scale dataset of anatomical segmentation masks for multi-center chest x-ray images[J]. Scientific Data, 2024, 11(1):511.)

[0062] Preferably, the present application randomly divides 70% of the processed rectangular region image data into training and verification data, and 30% into test data. Further, 80% of the training and verification data is divided into a training set, and 20% into a verification set, to complete the establishment of the digital X-ray classification training-verification-test data set. Finally, the network classification performance is evaluated based on the performance on the test set.

[0063] S2, training set data augmentation and generation of local-global views

[0064] For each sample in the training set obtained in step S1, a plurality of different random augmentation strategies are applied to generate two global views and six local views .

[0065] Preferably, the global view is generated by random color jittering with a probability of 0.8, random occlusion of the area outside the lung, and random cropping to cover more than 40% of the original image, and then adjusted to a resolution of 224x224 by bilinear interpolation as the global view input image.

[0066] Preferably, the global view is generated by random color jittering with a probability of 0.8, random pixel flipping with a probability of 0.2, and random cropping of the area outside the lung to cover more than 40% of the original image, and then adjusted to a resolution of 224x224 by bilinear interpolation as the global view input image.

[0067] Preferably, the six local views are first divided into right upper, left upper, middle right, middle left, right lower, and left lower six parts from top to bottom and left-right symmetrically, and then enhanced by random color jittering with a probability of 0.8 and random occlusion of the area outside the lung with a probability of 0.5, and then adjusted to a uniform resolution of 96x96 by bilinear interpolation as the local lung area image local view input image.

[0068] S3, constructing teacher and student networks

[0069] The existing deep backbone classification network model is selected to construct the teacher and student feature extraction networks. In addition, an additional feature mapping layer is used to map the deep features to a specific space for subsequent evaluation of the feature distribution similarity, and a linear classification layer is used to map the deep features to the classification result.

[0070] Preferably, the deep backbone feature extraction and classification network model can select the existing CNN-based deep classification network ResNet, ConvNeXt, the Transformer-based deep classification network Swin Transformer, the Mamba-based deep classification network VMamba, etc.

[0071] Preferably, the feature mapping layer is composed of a 3-layer multilayer linear perceptron (MLP) with a hidden dimension of 2048, L2 normalization, and a fully connected layer, and the linear classification layer is composed of a fully connected layer.

[0072] S4, local-global feature extraction and diagnosis of training set

[0073] All global lung views and local lung region views in the training set obtained in step S2 are input into the teacher and student feature extraction networks constructed in step S3 respectively, and the deep feature outputs are sent into the feature mapping layer and the linear classification layer respectively; the teacher output of the global lung view in the mapping space can be obtained by sending it into the feature mapping layer and the student output The feature teacher output and student output of the six local lung region views; the teacher global classification logic value and the student global classification logic value can be obtained by sending it into the linear classification layer; the teacher classification logic value and the student classification logic value of the six local lung regions

[0074] S5, multi-supervised training of network model

[0075] The deep model is trained and learned in combination with three types of loss functions, including the global lung supervised classification loss, the local-global self-supervised feature alignment loss, and the classification regularization loss of the representative local lung region.

[0076] The global lung supervised classification loss function uses the health or disease label corresponding to each chest digital X-ray image to predict the global classification logic value of the student network ​The classification supervised training is preferably performed using a cross-entropy classification loss function.

[0077]

[0078] The local-global feature alignment loss function aims to measure the distribution difference between the local and global feature outputs of the student model and the global deep feature teacher output of the teacher model, and is preferably measured using a cross-entropy loss function.

[0079]

[0080] wherein, .

[0081] The representative local lung area classification regularization loss function aims to select two representative local lung areas from each global lung sample, and assign the same pseudo-label as the global lung label to participate in network training, and the training loss function is preferably selected as a cross-entropy loss function.

[0082]

[0083] wherein, , respectively, to indicate the selected representative local lung areas in normal samples and diseased samples.

[0084] The representative local lung area selection principle mainly draws on the description of the first stage of pneumoconiosis in the Occupational Pneumoconiosis Diagnosis (GBZ 70-2015): "There is a small shadow with total intensity of 1 level, and the distribution range reaches at least 2 lung areas."

[0085] The screening process is as follows:

[0086] For digital X-ray samples with a disease category label, two corresponding local lung areas most similar to the global deep feature teacher output are selected as representative local classification negative samples according to the distribution difference between the student model local feature output and the global deep feature teacher output of the teacher model corresponding to the six local lung areas; for digital X-ray samples with a normal label, since there may also be a small shadow intensity of 1 level in one of the local lung areas, two local lung areas are randomly selected from the remaining five local lung areas as representative local classification positive samples, excluding the corresponding local lung area that is least similar to the global deep feature teacher output.

[0087] The final multi-supervised classification loss function is:

[0088]

[0089] wherein, These represent the balance loss weights of the local-global feature alignment loss and the classification regularization loss of the representative local lung region, respectively.

[0090] Preferably, .

[0091] S6. Model Parameter Update and Saving

[0092] The student model parameters are analyzed using the three multi-supervised loss functions mentioned above. Perform gradient optimization updates for the teacher model parameters. In contrast, without gradient backpropagation optimization, the exponential moving average (EMA) strategy is used to iteratively update the parameters from the current student model and the historical teacher model. , The hyperparameters for dynamically updating the teacher model can be set to 0.996. After a complete iteration of the training set and parameter updates, a trial training round is performed. After each training round, the updated teacher model is used to evaluate the classification performance of the validation set samples. The weights of the teacher model with the best classification performance on the validation set are saved until the classification performance on the validation set no longer improves, thus completing the model parameter update and saving.

[0093] S7, Model Testing and Application

[0094] Load the teacher model weights with the best classification performance on the validation set saved in the above steps, input the test set samples into the teacher network model, obtain the corresponding classification output, and complete the chest digital X-ray image processing.

[0095] In one embodiment of the present invention, a chest digital X-ray image processing system for pneumoconiosis is provided. This system can be used to implement the above-mentioned chest digital X-ray image processing method for pneumoconiosis. Specifically, the chest digital X-ray image processing system for pneumoconiosis includes a segmentation module, a dataset partitioning module, an enhancement module, a network module, a training module, and a testing module.

[0096] The segmentation module segments the lung region of the digital X-ray image, solves the minimum bounding rectangle problem on the segmentation result, and extracts the complete rectangular region image containing lung information from the digital X-ray image.

[0097] The dataset partitioning module divides all rectangular region images into three main parts: training, validation, and testing.

[0098] The augmentation module applies multiple different random augmentation strategies to each rectangular region image sample in the training set to generate two global views. and six partial views ;

[0099] The network module selects a deep backbone classification network model to construct a teacher and a student feature extraction network, designs a feature mapping layer to map the deep features to a specific feature space, and designs a linear classification layer to map the deep features to a classification result.

[0100] The training module inputs two global views and six local views in the training set into the constructed teacher and student feature extraction networks respectively, obtains deep feature outputs, and then sends the deep feature outputs into the feature mapping layer and the linear classification layer respectively. The teacher output and the student output of the global lung view in the mapping space are obtained. The teacher output and the student output of the six local lung area views are obtained. The teacher global classification logic value and the student global classification logic value are obtained. The teacher classification logic value and the student classification logic value of the six local lung areas are obtained. The teacher and the student feature extraction networks are trained and learned by combining three loss functions, i.e., a global lung supervision classification loss, a local-global self-supervised feature alignment loss, and a classification regularization loss of a representative local lung area. The student model parameters are updated, and the teacher model parameters are updated using an exponential moving average strategy.

[0101] The test module loads the teacher model weight with the best classification performance in the validation set, inputs the rectangular region image samples in the test set, and performs classification output to complete the chest digital X-ray image processing.

[0102] ​​​​​​In another embodiment of the present application, a terminal device is provided, which comprises a processor and a memory, the memory being configured to store a computer program, the computer program comprising program instructions, and the processor being configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are particularly suitable for loading and executing one or more instructions to implement a corresponding method flow or a corresponding function; the processor in the embodiments of the present application can be used for the operation of the chest digital X-ray image processing method for pneumoconiosis, comprising:

[0103] Segmenting the lung region of the digital X-ray image, solving the minimum circumscribed rectangle of the segmentation result, extracting the complete rectangular region image with lung information in the digital X-ray image, and dividing the rectangular region image into a training set, a validation set and a test set;

[0104] Applying different random enhancement strategies to each sample in the training set to generate two global views and six local views ;

[0105] Selecting a deep backbone classification network model to construct a teacher and a student feature extraction network, designing a feature mapping layer to map the deep features to a specific representation space, and designing a linear classification layer to map the deep features to a classification result;

[0106] Inputting the two global views and the six local views in the training set into the constructed teacher and student feature extraction networks respectively, obtaining the deep feature outputs, and then inputting the deep feature outputs into the feature mapping layer and the linear classification layer respectively; obtaining the teacher output and the student output of the global lung view in the mapping space, the teacher output and the student output of the six local lung region views; the teacher global classification logic value and the student global classification logic value ; the six local lung region teacher classification logic values Student classification logic value ;

[0107] The student network is trained and learned in combination of three loss functions, global lung supervision classification loss, local-global self-supervised feature alignment loss and representative local lung area classification regularization loss; the student network parameters are updated The teacher model parameters are updated using the exponential moving average strategy ;

[0108] After each round of training, the updated teacher model is used to evaluate the classification performance of the rectangular region image samples in the validation set, and the teacher model weight with the best classification performance in the validation set is saved until the classification performance in the validation set no longer improves, and the model training is completed; in the testing process, the teacher model weight with the best classification performance in the validation set is loaded, and the rectangular region image samples in the test set are input for classification output, and the chest digital X-ray image processing is completed.

[0109] Please refer to Figure 2 , the terminal device is a computer device, the computer device 60 of the embodiment includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61, and the computer program 63 is executed by the processor 61 to realize the dust lung-oriented chest digital X-ray image processing method in the embodiment, to avoid repetition, which will not be repeated here. Alternatively, the computer program 63 is executed by the processor 61 to realize the functions of each model / unit in the dust lung-oriented chest digital X-ray image processing system of the embodiment, to avoid repetition, which will not be repeated here.

[0110] The computer device 60 can be a desktop computer, a notebook computer, a palm computer, and a cloud server, etc. The computer device 60 can include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art can understand that Figure 2 The computer device 60 is only an example and does not constitute a limitation on the computer device 60, and can include more or fewer components than the illustration, or combine certain components, or different components, for example, the computer device can also include an input / output device, a network access device, a bus, etc.

[0111] The processor 61 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0112] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or a memory of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0113] Further, the memory 62 can include both an internal storage unit and an external storage device of the computer device 60. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0114] Please refer to Figure 3 , the terminal device is a chip, the chip 600 of the embodiment includes a processor 622, the number of which can be one or more, and a memory 632 for storing computer programs executable by the processor 622. The computer programs stored in the memory 632 can include one or more than one module corresponding to a set of instructions. In addition, the processor 622 can be configured to execute the computer programs to perform the above-mentioned pneumoconiosis-oriented chest digital X-ray image processing method.

[0115] In addition, the chip 600 can also include a power supply component 626 and a communication component 650, the power supply component 626 can be configured to perform power management of the chip 600, and the communication component 650 can be configured to realize communication of the chip 600, such as wired or wireless communication. In addition, the chip 600 can also include an input / output interface 658. The chip 600 can operate based on an operating system stored in the memory 632.

[0116] In still another embodiment of the present application, the present application further provides a storage medium, specifically a computer readable storage medium, which is a memory device in the terminal device, for storing programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the terminal device, and of course can also include the expansion storage medium supported by the terminal device. The computer readable storage medium provides a storage space, which stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs. It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory.

[0117] The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the method for processing a chest digital X-ray image for pneumoconiosis in the above embodiments; the one or more instructions stored in the computer readable storage medium are loaded and executed by the processor to perform the following steps:

[0118] Segmenting the lung region of the digital X-ray image, solving the minimum circumscribed rectangle of the segmentation result, extracting the complete rectangular region image with lung information in the digital X-ray image, and dividing the rectangular region image into a training set, a validation set and a test set;

[0119] Applying different random enhancement strategies to each sample in the training set to generate two global views and six local views ;

[0120] Selecting a deep backbone classification network model to construct a teacher and a student feature extraction network, designing a feature mapping layer to map the deep features to a specific feature space, and designing a linear classification layer to map the deep features to a classification result;

[0121] Inputting the two global views and the six local views in the training set into the constructed teacher and student feature extraction networks respectively, obtaining the deep feature outputs, and then inputting the deep feature outputs into the feature mapping layer and the linear classification layer respectively; obtaining the teacher output and the student output of the global lung view in the mapping space, the feature teacher output and the student output of the six local lung region views; the teacher global classification logic value and the student global classification logic value ; and the six local lung region teacher classification logic values student classification logic value ;

[0122] The student network is trained and learned in combination of three types of loss functions, including a global lung supervision classification loss, a local-global self-supervised feature alignment loss and a classification regularization loss of representative local lung regions; and the student network parameters are updated The teacher model parameters are updated using an exponential moving average strategy ;

[0123] After each round of training, the updated teacher model is used to evaluate the classification performance of the rectangular region image samples in the validation set, and the teacher model weight with the best classification performance in the validation set is saved until the classification performance in the validation set no longer improves, and the model training is completed; in the testing process, the teacher model weight with the best classification performance in the validation set is loaded, and the rectangular region image samples in the test set are input for classification output, and the chest digital X-ray image processing is completed.

[0124] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0125] The effects of the present application will be further described below in combination with simulation experiments.

[0126] 1. Simulation experiment conditions:

[0127] The hardware platform of the simulation experiment of the present application: the processor is AMD EPYC 7T83, the main frequency is 2.5GHz, and the graphics card is 4090.

[0128] Software platform: Linux operating system, Pytorch 1.13.0.

[0129] 2. Simulation content and result analysis:

[0130] In order to verify the simulation experiment effect of the present application, the simulation experiment of the present application compares the experimental performance of four existing deep classification network technologies (ResNet method, ConvNeXt method, Swin Transformer, and VMamba method) under the network architecture of the present application, and uses four evaluation indexes of accuracy (Accuracy, Acc.), sensitivity (Sensitivity, Sens.), specificity (Specificity, Spec.), and area under the curve (Area Under the Curve AUC) to evaluate the performance of digital X-ray images for pneumoconiosis, and the higher the evaluation index, the better the classification performance.

[0131] In the simulation experiment, the present application and four methods in the prior art are used respectively:

[0132] The ResNet method of the prior art refers to the deep residual classification network method adopted in (K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.).

[0133] The ConvNeXt method of the prior art refers to the deep convolution classification network method adopted in (Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 11976-11986, 2022.).

[0134] The prior art Swin Transformer method refers to the Transformer-based deep classification network method adopted in Liu et al. (Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE / CVF International Conference on Computer Vision, pp. 10012-10022, 2021.).

[0135] The prior art VMamba method refers to the Mamba-based deep classification network method adopted in Liu et al. (Y. Liu, Y. Tian, Y. Zhao, H. Yu, L. Xie, Y. Wang, Q. Ye, and Y. Liu, “Vmamba: Visual state space model,” arXiv preprint arXiv:2401.10166, 2024.).

[0136] The classification performance of the above four methods under the architecture proposed in the present application is shown in Table 1.

[0137] Table 1 Classification performance table of the above four methods under the architecture proposed in the present application

[0138]

[0139] As can be seen from Table 1, the classification performance of each type of deep backbone classification network is improved under the data classification processing architecture proposed in the present application, indicating that the method of the present application is helpful to improve the classification performance under limited sample data.

[0140] In summary, the present application provides a pneumoconiosis-oriented chest digital X-ray image processing method and system, which can achieve excellent classification performance improvement in the classification task of pneumoconiosis-oriented digital X-ray images, and is not limited to a specific feature extraction network selection, facilitating migration to various deep classification model application scenarios. Without increasing the test complexity, only by changing the training method, the classification performance of the model under limited data can be greatly improved.

[0141] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0142] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0143] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0144] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / terminal and method can be implemented by other ways. For example, the above-mentioned apparatus / terminal embodiments are only schematic, and the division of the modules or units is only a logical function division, and there can be another division way in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0145] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0146] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0147] The integrated module / unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer-readable storage medium. The computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the computer-readable medium can include or exclude contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0148] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices, and computer program products of embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks

[0149] These computer program instructions can also be stored in a computer-readable storage medium that can guide the computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable storage medium produce a manufactured product including instruction devices that implement the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0150] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate a computer-implemented process, so that the instructions executed on the computer or other programmable data processing devices provide a process for implementing the flow Figure 1 one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0151] The above is only to illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical scheme falls within the protection scope of the claims of the present application.

Claims

1. A chest digital X-ray image processing method for pneumoconiosis, characterized by, The method comprises the following steps: Segmenting the lung region of the digital X-ray image, solving the minimum circumscribed rectangle of the segmentation result, extracting the complete rectangular region image with lung information in the digital X-ray image, and dividing the rectangular region image into a training set, a validation set and a test set; applying a plurality of different random augmentation strategies to each sample in the training set to generate two global views and six local views ; Selecting a deep backbone classification network model to construct a teacher and a student feature extraction network, designing a feature mapping layer to map the deep features to a specific representation space, and designing a linear classification layer to map the deep features to a classification result; two global views and six local views in the training set are respectively input into the constructed teacher and student feature extraction networks, and the deep feature outputs are respectively sent into the feature mapping layer and the linear classification layer; the teacher output and the student output of the global lung view in the mapping space are obtained; the feature teacher output and the student output of the six local lung area views are obtained; the teacher global classification logic value and the student global classification logic value are obtained; the teacher classification logic value and the student classification logic value of the six local lung areas are obtained; The student network is trained and learned in combination of three loss functions of global lung supervision classification loss, local-global self-supervised feature alignment loss and classification regularization loss of representative local lung area; and the student network parameters are updated The teacher model parameters are updated using an exponential moving average strategy ​ After each round of training, the updated teacher model is used to evaluate the classification performance of the rectangular region image samples in the validation set, and the teacher model weight with the best classification performance in the validation set is saved until the classification performance in the validation set no longer improves, and the model training is completed; In the testing process, the teacher model weight with the best classification performance in the validation set is loaded, and the rectangular region image samples in the test set are input for classification output to complete the chest digital X-ray image processing.

2. The method of claim 1, wherein, Global view in training set After generating more than 40% coverage of the original image by random color jittering with a probability of 0.8 and random occlusion and random cropping of areas other than the lung, adjust to 224x224 resolution size as a global view by bilinear interpolation Input image; Global view in training set After generating more than 40% coverage of the original image by random color jittering with a probability of 0.8 and random occlusion and random cropping of areas other than the lung, adjust to 224x224 resolution size as a global view using bilinear interpolation Input image.

3. The method of claim 1, wherein, Six local views in the training set First, the acquired rectangular region image is divided into six parts, right upper, left upper, middle right, middle left, right lower and left lower, from top to bottom and left-right symmetrically. Random color jitter with a probability of 0.8, random occlusion data augmentation measures for regions other than the lung with a probability of 0.5, and the resolution is adjusted to a uniform size of 96x96 by bilinear interpolation as a local lung area image local view Input image.

4. The method of claim 1, wherein, The teacher and student feature extraction network adopts a deep classification network ResNet, ConvNeXt based on CNN, a deep classification network Swin Transformer based on Transformer, or a deep classification network VMamba based on Mamba; the feature mapping layer includes a 3-layer multilayer linear perceptron with a hidden dimension of 2048, L2 normalization and a fully connected layer; and the linear classification layer is a fully connected layer.

5. The method of claim 1, wherein, For the digital X-ray sample with a disease class label, the feature similarity is calculated between the feature outputs of the six local lung regions in the student model mapping space and the feature output of the global view in the teacher model mapping space, and the top two local lung regions with the highest similarity are selected as representative local classification negative samples; for the digital X-ray sample with a normal class label, the feature similarity is calculated between the feature outputs of the six local lung regions in the student model mapping space and the feature output of the global view in the teacher model mapping space, and two local lung regions are randomly selected from the top five local lung regions with the highest similarity as representative local classification positive samples.

6. The method of claim 1, wherein, Multi-supervised classification loss function is: wherein, respectively represent the balanced loss weights for the local-global feature alignment loss and the classification regularization loss of the representative local lung regions, represents the global lung supervision classification loss function, represents the local-global feature alignment loss function, represents the classification regularization loss function of the representative local lung regions.

7. The method of processing chest digital X-ray images for pneumoconiosis according to claim 6, characterized in that, Global lung supervision classification loss function is: wherein, is label information representing whether the current sample is diseased, is a classification logic value corresponding to the global sample output by the student classification network, is a global view.

8. The method of claim 6, wherein the method is a pneumo- oriented chest digital X-ray image processing method, characterized by, Local-global self-supervised feature alignment loss function is: wherein, is a global view, is a local view, represents a cross-entropy loss function calculation, represents a global feature probability distribution output by the teacher feature mapping network in the case of global view input, represents a feature probability distribution output by the student feature mapping network in the case of global and local view input.

9. The method of claim 6, wherein the method is a method of processing a chest digital X-ray image for pneumoconiosis, characterized by, Classification regularized loss function for representative local lung zones is: wherein, respectively, to indicate normal and diseased samples, is label information representing whether the current sample is diseased or not, is the classification logic value of the student classification network for the local view at the input time.

10. A chest digital X-ray image processing system for pneumoconiosis, characterized by, It comprises: A segmentation module for segmenting the lung region of the digital X-ray image, solving the minimum circumscribed rectangle of the segmentation result, and extracting the complete rectangular region image with lung information in the digital X-ray image; A data set division module for dividing all rectangular region images into three parts of training-validation-testing; An augmentation module, for each rectangular region image sample within the training set, applies multiple sets of different random augmentation strategies to generate two global views and six local views ; A network module for selecting a deep backbone classification network model to construct a teacher and a student feature extraction network, designing a feature mapping layer to map the deep features to a specific representation space, and designing a linear classification layer to map the deep features to a classification result; The training module combines two global views from the training set. and six partial views The deep feature outputs are then fed into the constructed teacher and student feature extraction networks, respectively, and then fed into the feature mapping layer and the linear classification layer to obtain the teacher's output, which represents the global lung view in the mapping space. With student output Teacher output of features from six local lung region views With student output ; teacher global classification logic value student global classification logic value six local lung area teacher classification logic values student classification logic value combined global lung supervision classification loss, local-global self-supervised feature alignment loss, and representative local lung area classification regularization loss three types of loss functions are used to train and learn the teacher and student feature extraction networks; update the student model parameters the teacher model parameters are updated using the exponential moving average strategy after each round of training, the updated teacher model is used to evaluate the classification performance of the rectangular region image samples in the validation set, and the teacher model weight with the best classification performance in the validation set is saved until the classification performance of the validation set no longer improves, and the model training is completed; A test module for loading the teacher model weight with the best classification performance in the validation set, inputting the rectangular region image samples in the test set for classification output, and completing the chest digital X-ray image processing.

Images