Intelligent pneumoconiosis diagnosis system based on weighted information flow data increment network

By introducing a weighted information flow data incremental network in medical image processing, using a multi-grained attention module with random attribute masking and prior knowledge, the problems of insufficient data and insufficient feature extraction capabilities in traditional diagnostic methods are solved, and high accuracy is achieved Intelligent diagnosis of pneumoconiosis.

CN118967614APending Publication Date: 2024-11-15TAIYUAN UNIVERSITY OF TECHNOLOGY
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411027874.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Traditional medical chest radiograph diagnosis depends on experience. Deep learning models cannot effectively mine subtle information when there is insufficient data, and the generalization ability of feature extraction and classification models is insufficient, resulting in poor diagnostic performance.

Method used

An intelligent diagnostic system based on weighted information flow data incremental network (WIDINet) was designed, and a multi-grained attention module using random attribute masking technology and prior knowledge was used to improve data quality and reliability of feature screening networks through KL weighted judgment algorithm. to achieve staged diagnosis of pneumoconiosis.

Benefits of technology

Improve the quality of incremental data and the reliability of feature extraction, enhance the robustness and stability of the model, and achieve more accurate diagnosis of pneumoconiosis. The area under the ROC curve (AUC) is better than the comparison model and the test set model. The accuracy rate reached 93.4%.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967614A_ABST
    Figure CN118967614A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent pneumoconiosis diagnosis system based on a weighted information flow data incremental network. The intelligent pneumoconiosis diagnosis system comprises an incremental data module based on random attribute masking, a multi-granularity feature extraction module based on priori knowledge and a KL weighted judgment module. And the ARMGan incremental data module obtains incremental data with higher data quality by using a random attribute masking technology. The MKTransform module uses a disease relationship as priori knowledge under the guidance of professional doctors, supplements and improves an existing Transform deep extraction framework, and solves the problem of low reliability of a feature screening network based on incremental data by adopting a coarse-grained and fine-grained double-attention mechanism. And the KL weighting judgment module is used for comparing the difference between the test chest X-ray film and the national standard film to realize stage diagnosis of pneumoconiosis. Training of the whole network is restrained through combination of resistance loss, feature screening loss and KL divergence loss, and high efficiency and stability of the model in practical application are ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to an intelligent pneumoconiosis diagnosis system based on a weighted information flow data incremental network (WIDINet). Background Art

[0002] X-ray chest radiographs are one of the most commonly used medical imaging examinations in clinical diagnosis. The results of traditional medical chest radiographs rely heavily on the operator's experience and expertise. With the increase in new cases of occupational coal workers' pneumoconiosis, traditional diagnostic methods can no longer meet the actual needs of patients. Although the powerful advantages of deep learning can indeed make up for the shortcomings of manual methods in actual work, due to the small amount of data in medical data sets, the actual application effect of deep models is not satisfactory. Previous researchers have done a lot of pioneering work in solving the problem of insufficient data. For example, a two-stage transfer learning method TSTL was designed based on the idea of ​​model generalization to solve the intermittent problem. Although this method can deal with the problem of insufficient data for qualitative lung diseases, the model cannot mine and obtain subtle information scattered in specific areas. Therefore, the researchers designed a modified feedback polynomial neural network for epidemic lung diseases, which achieved a relatively small prediction error through feedback correction during training, but due to the limitations of network optimization, this design could not avoid the problem of model instability caused by a small data set.

[0003] Different from the model training problems caused by small data sets, cutting-edge researchers have found that better feature extraction networks and algorithms can often achieve better experimental results. Therefore, how to design feature extraction learning modules for models is an important research topic. Some researchers have integrated multiple detection models through feature vectors based on the summary of previous work. However, the feature extraction method of integrated detection not only expands the advantages of different models, but also exposes their disadvantages, and the performance is poor from a global perspective. Other researchers have made breakthroughs in extraction algorithms, using extracted features as variables in classification models to verify whether the target gene is a potential drug target for lung adenocarcinoma, but the new algorithm's generalization ability is insufficient, resulting in poor performance when applied in different research fields. Summary of the invention

[0004] Based on the above problems, this paper proposes an intelligent diagnosis system for pneumoconiosis based on the weighted information flow data incremental network (WIDINet). The system first uses random attribute masking technology to obtain higher quality incremental data, and then uses prior knowledge and multi-granularity attention modules to solve the low reliability problem of feature screening networks based on incremental data. Finally, the KL weighted judgment algorithm is used to measure the difference between the test chest radiograph and the current national gold standard to achieve the staged diagnosis of the target task.

[0005] The technical solution of the present invention is as follows:

[0006] An intelligent pneumoconiosis diagnosis system based on a weighted information flow data incremental network (WIDINet) is characterized by comprising a data preprocessing module, an incremental data module based on random attribute masking (ARMGan), a multi-granularity feature extraction module based on prior knowledge (MKTransformer) and a KL weighted judgment module:

[0007] Data preprocessing module: By mixing the collected pneumoconiosis X-ray chest films with normal chest films and using histogram equalization technology to enhance the image contrast, a quality-balanced and diverse dataset is provided for the subsequent ARMGan data incremental network training;

[0008] Incremental data module based on random attribute masking (ARMGan): This module uses random attribute masking technology to improve the data quality of incremental data. It uses an asymmetric encoder-decoder network model to focus on different areas of the input chest X-ray and maximize the retention of key features through lung interval decoupling;

[0009] Multi-granularity feature extraction module based on prior knowledge (MKTransformer): Under the guidance of professional doctors' prior knowledge, this module uses a dual attention mechanism of coarse-grained and fine-grained to assist the feature extraction process of pneumoconiosis X-ray chest films. In addition, the present invention innovatively adopts a multi-head attention module optimized by Kullback-Leibler to minimize information redundancy through the links between filters of different granularity;

[0010] KL weighted judgment module: This module is used to measure the difference between the test chest X-ray and the national gold standard, and to achieve the staging diagnosis of pneumoconiosis by comparing the statistical differences between the generated chest X-ray and the national standard.

[0011] The intelligent diagnosis system and the ARMGan data increment module use random attribute masking technology to achieve data expansion and improve the data quality of incremental data. First, the input chest film is divided into six parts according to the doctor's diagnostic habits and the national standard film. The LoT coding layer is used to map the image from a high-dimensional image to a low-dimensional matrix to achieve decoupling of the lung intervals to expand the difference between the lung intervals. Subsequently, an asymmetric coding model is designed, using fewer coding layers on the lung frame decoupling encoder, and more computing power is concentrated on the coding of the lesion texture in the lung. The bias factor μ is introduced in each encoding to control the layer correlation function, and a difference contrast effect is produced in the decoupling learning of the lung region frame and the lesion texture in the lung. Next, the decoupling coding process of the target area sampling image is defined: one lung frame of the chest film is defined as I1, and two lung masses of the chest film are defined as I2. Therefore, its coding can be expressed as (z1s, z1t) and (z2s, z2t), based on the use of G to represent the synthesis process, Dec to represent the dot product, and D to represent the generation process, and the difference between the two is compared by the following formula during actual operation:

[0012]

[0013] The target reconstruction loss is obtained by calculating the difference between the information before and after generation. The calculation formula is as follows:

[0014]

[0015] Furthermore, the entropy difference is used to calculate the network generation loss:

[0016]

[0017] Where X represents the dataset used for training. In the decoupled learning process, in order to control the intra-lung texture supervision and lung frame overflow supervision through different loss functions, the original loss function is rewritten to better achieve the target task in practice, unlike the traditional model. The result is calculated as follows:

[0018]

[0019] L T&S =L Texture +L Structure

[0020] where τ is a random distribution factor that represents the randomness of the sampling operation.

[0021] Not only limited to the decoupled learning of lung frame and lung mass, ARMGan also focuses on the intra-pulmonary areas where the lesion features are more deeply coupled, enabling it to generate training data that is more consistent with the characteristics of pneumoconiosis. Since the Laplace transform can well achieve dimensional transformation in n-dimensional Euclidean space, the feature decoupling task that is difficult to achieve in high-dimensional space is easier to perform in low-dimensional space. In order to make the coding map after asymmetric coding exchange more focused on a single lung area, the Laplace operator is introduced, and the LoG domain transform is applied to enhance the Laplace operator's ability to resist weak interference.

[0022] For different lung regions of the input image, define Where R xy Represents the selected target area. Since the Laplace transform is very sensitive to image noise, a Gaussian operation is performed before the Laplace operation. The whole process can be expressed as a LoG operator, as shown in the following formula:

[0023] LoG*f(x,y)=Δ[G σ (x,y)]*f(x,y)

[0024] Among them, the standard deviation of the Gaussian function is set to σ = 1. In addition, in order to achieve the decoupling of the lung intervals, after applying the above operation, another LoG region operation is performed on the encoding map, and a partition bias factor λ is introduced to deform it as follows:

[0025]

[0026] In the decoupling learning process between different lung regions, the low-dimensional space matrix processed by Laplace LoG domain transform (LoT) is used to achieve the decoupling of each sub-interval of the lung lobe. Through two decoupling operations, one for lung frame and lung mass decoupling and the other for lung interval decoupling, different regional encodings of chest radiographs can be combined more flexibly to generate more competitive data images.

[0027] In addition, a random attribute masking layer is added to the ARMGan module. By actively adding perceptual masking, the training process of the subsequent model is guided. This not only eliminates the trouble caused by attribute masking on model training, but also makes an important contribution to enriching the diversity of training data sets. The important basic theory of this innovation is:

[0028]

[0029] At the same time, the distribution of the output space Ω of the deep generative network is proportional to the square root divisor of the product of the non-zero singular decomposition matrices when mapping this space to the low-dimensional matrix domain. The sample concentration of each output space region depends on whether the slope matrix of this region shrinks or expands relative to other regions. In practice, by controlling the size of the slope matrix, it is possible to control the concentration of samples in the output space, thereby randomly controlling the masking or non-masking of attributes in the generated image. The random space parameter ρ∈R allows for better random attribute selection in the target domain, as shown in the following equation:

[0030]

[0031] The ARMGan training loss supervision function, to which the synthesis part belongs, can be expressed as follows:

[0032]

[0033] This loss function will serve as a loss supervision layer to constrain the training and development of this module.

[0034] The intelligent diagnosis system, the MKTransformer feature extraction module uses prior knowledge and multi-granularity attention modules to specifically solve the problem of low reliability of feature screening networks based on incremental data. In the actual training process, the acquired expertise will be used as prior training parameters. After a certain number of training cycles (5 epochs), the model network will freeze relative to the training parameters, compare the performance difference between the network using prior parameters and the network trained using the model, and incorporate this difference into subsequent network training. Through network-level training guidance, the feature extraction network strikes a balance between the superior feature extraction requirements of deep neural networks and actual diagnostic work.

[0035] In addition, in order to reduce the redundancy of feature information, a coarse-grained and fine-grained supervision mechanism is introduced, which is used in the entire feature extraction process to distinguish hidden features and reduce information redundancy by comparing the difference in granularity. The construction of the MKTransformer framework relies on the alternation of encoders and decoders. The coarse-grained and fine-grained attention mechanism operations are expressed by the following formula,

[0036]

[0037] Where Q is the query matrix, K is the content of the coarse-grained or fine-grained information, V is the target domain, QK T , that is, the purpose of the dot product is to calculate the attention weight of Q on V. Finally, in order to prevent the gradient from disappearing, the dimension of the output layer (dk) is squared.

[0038] From a macro perspective, the feature improvement extraction model is a priori network-multi-granularity feature extraction process, which is continuously interactive and iteratively advanced through the process. From a micro perspective, facing the input data, MKTransformer first maps it from high-dimensional space to low-dimensional space, and then uses a coarse-grained attention adjustment mechanism during the encoding process to perform preliminary feature screening on the ROI area in the low-dimensional space. The preliminary feature extraction data will be input into the MKTransformer decoding module in two steps. The first step is to pass the data as input to the backbone network, and use a fine-grained attention module in the decoder to explore deep hidden information based on preliminary screening. The second step is to interactively link the preliminary screened data with the further screened data in the multi-head attention module. Fine-grained information guides the feature extraction of coarse-grained information, and coarse-grained information can also supplement the knowledge gap of fine-grained information, thereby minimizing information redundancy in bidirectional supervision and bidirectional information flow. The process can be expressed as follows:

[0039] MultiHead(Q,K,V)=Concat(head1,head2,....head n )W O ,

[0040] Head i =Attention Coarse&Fine (QW Q i ,KW k i ,VW v i ),i∈(1,N)

[0041] This module optimizes the operation of the multi-attention mechanism to adapt to the diffuse cotton-like characteristics of pneumoconiosis. In addition, KL divergence is used in the probability and statistics of data to replace the complex multivariate distribution of feature information extracted from the model by approximate distribution, and uses probability entropy to measure the degree of information loss when selecting effective information, which can be expressed as follows:

[0042]

[0043] Among them, p and q represent the probability distribution of the target domain, and {x1,x2,...x n} represents the feature domain to be selected in the low-dimensional space. It is worth noting that KL divergence is not used as a method to measure the distance between different information differences, but the asymmetry of KL divergence is used to measure the consistency between two distributions. The entropy of KL information distribution is used in the novel multi-head attention module to quantify the information loss in coarse-grained and fine-grained knowledge distillation and constrain the training of the model to make it orderly.

[0044] Therefore, in MKTransformer, a limiter is used to distinguish the training state and the frozen state of the model. When the limiter is turned on and the model is in the training state, both the encoder and decoder of MKTransformer will be in working state, processing the data input to the model. For networks using prior parameters, the limiter will be in a restricted state, and the feature mixing and superposition modules in the decoder will be turned off, so that the performance difference and consistency between the prior network and the original model are compared only from a low-dimensional perspective. In order to improve the robustness and stability of the model, when the model encounters new features in the recognition judgment, a report will be provided, and then the model parameters will be adjusted or weighed according to the specific suggestions of the experts.

[0045] The intelligent diagnosis system, the KL weighted judgment module: In the actual pneumoconiosis staging process, experts usually compare the chest X-ray to be judged with the national standard X-ray in detail and make a final diagnosis based on their field experience. This module uses KL divergence to measure the similarity difference between the image to be staged and the standard slice. First, a probability distribution pstd(x i ), based on the national standard slice library and combined with the measurement results of three doctors, this method is feasible in practice because when the feature reaches a certain approximation, we can assign it to a specific feature category. The deep features extracted by MKTransformer are compared with the standard slice library to obtain the probability distribution qpro(x i ).

[0046] According to the current national diagnostic standards for pneumoconiosis, standard chest X-ray films are widely used as the gold standard. This module uses KL weighting to introduce entropy measurement into model judgment. In the actual operation of the network, the KL weighting algorithm will focus on measuring the KL entropy difference between the chest X-ray to be staged and the standard film of each lung area, and obtain the similarity weight p(x i ), as shown in the following formula:

[0047]

[0048] Using Q KL (x i ) to calculate the similarity and complete the staging. The higher the value, the higher the similarity between the image to be staged and the staging standard. KL (x i )The calculation process is as follows:

[0049]

[0050] Finally, the loss function during the network iteration is as follows:

[0051]

[0052] Where W is the network weight; Y is the pairwise label, m is the hyperparameter, and when the sample pair X1 and X2 belong to the same category, Y = 0; when they belong to different categories, Y = 1. w is the Euclidean distance between X1 and X2 in the latent variable space. When Y = 0, adjusting the parameters will minimize the distance between X1 and X2. When Y = 1, if the distance between X1 and X2 is greater than m, no optimization is performed; if the distance is less than m, the distance between them is increased to m.

[0053] A method according to any of the above systems, comprising the steps of:

[0054] Data preprocessing step: By mixing the collected pneumoconiosis X-ray chest films with normal chest films and using histogram equalization technology to enhance the image contrast, a quality-balanced and diverse dataset is provided for the subsequent ARMGan data incremental network training;

[0055] Incremental data based on random attribute masking (ARMGan) step: This step uses random attribute masking technology to improve the data quality of incremental data. It uses an asymmetric encoder-decoder network model to focus on different areas of the input chest X-ray and maximize the retention of key features through lung interval decoupling;

[0056] Multi-granularity feature extraction (MKTransformer) based on prior knowledge: Under the guidance of professional doctors’ prior knowledge, this step uses a dual attention mechanism of coarse-grained and fine-grained to assist the feature extraction process of pneumoconiosis chest X-rays. In addition, this step innovatively uses the Kullback-Leibler optimized multi-head attention module to minimize information redundancy through links between filters of different granularities;

[0057] KL weighted judgment step: This step is used to measure the difference between the test chest X-ray and the national gold standard. By comparing the statistical differences between the generated chest X-ray and the national standard film, the staging diagnosis of pneumoconiosis can be achieved.

[0058] In order to avoid the problem of attribute masking in the subsequent extraction model due to excessive attention to a specific area during the feature extraction process, this paper actively introduces random attribute masking into the data set incremental network, designs an asymmetric encoding and decoding structure, and uses the feature screening advantages of mathematical models to overcome the problem of poor data quality generated by the traditional data incremental network. Secondly, under the guidance of professional doctors, the disease relationship between the substages of pneumoconiosis is used as prior knowledge to supplement and improve the existing Transformer deep extraction framework, and a coarse-grained and fine-grained dual attention mechanism is used to assist the feature extraction process of pneumoconiosis X-ray chest radiographs. At the same time, in order to realize the information flow between multi-granularity attention modules, this paper innovatively adopts a multi-head attention module based on Kullback-Leibler optimization to minimize information redundancy through the association between filters of different granularities. Finally, in order to meet the actual needs of pneumoconiosis staging, this paper designs a weighting function based on KL variants, and achieves the final staging by comparing the chest radiographs generated by the WIDINet network with the national standard chest radiographs. This study not only shows reasonable performance in medical applications, but also has the potential to be developed for pneumoconiosis staging diagnosis in clinical practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a schematic diagram of the overall process of the intelligent diagnosis system for pneumoconiosis based on the weighted information flow data increment network (WIDINet) of the present invention;

[0060] Figure 2 This is a schematic diagram of the ARMGan module structure of the present invention;

[0061] Figure 3 This is a schematic diagram of the structure of the MKTransformer module of the present invention;

[0062] Figure 4 Comparison of image generation results when using different data increment modules in the present invention;

[0063] Figure 5 is a ROC curve diagram of the present invention in the pneumoconiosis staging;

[0064] Figure 6 Comparison of histograms when different modules are integrated in the present invention; DETAILED DESCRIPTION

[0065] The present invention is described in detail below in conjunction with specific embodiments.

[0066] A pneumoconiosis intelligent diagnosis system based on a weighted information flow data incremental network (WIDINet), characterized by comprising data preprocessing (A1), an incremental data module based on random attribute masking (ARMGan) (A2), a multi-granularity feature extraction module based on prior knowledge (MKTransformer) (A3) and a KL weighted judgment module (A4).

[0067] The overall process diagram is as follows Figure 1 As shown;

[0068] A1 mixes the collected pneumoconiosis X-ray chest films with normal chest films and uses histogram equalization technology to enhance the image contrast, providing a quality-balanced and diverse dataset for subsequent ARMGan data incremental network training;

[0069] The A2 module uses random attribute masking technology to achieve data expansion and improve the data quality of incremental data. The detailed architecture of the ARMGan data increment is Figure 2 . First, the input chest X-ray is divided into six parts according to the doctor's diagnostic habits and the national standard film. The LoT coding layer is used to decouple the lung intervals after mapping the image from a high-dimensional image to a low-dimensional matrix to expand the difference between the lung intervals. Subsequently, an asymmetric coding model is designed, using fewer coding layers on the lung frame decoupling encoder, and more computing power is concentrated on the coding of the lesion texture in the lung. The bias factor μ is introduced in each encoding to control the layer correlation function, which produces a difference contrast effect in the decoupling learning of the lung region frame and the lesion texture in the lung. Next, the decoupling coding process of the target area sampling image is defined: one lung frame of the chest X-ray is defined as I1, and two lung masses of the chest X-ray are defined as I2. Therefore, its encoding can be expressed as (z1s, z1t) and (z2s, z2t), based on the use of G to represent the synthesis process, Dec to represent the dot product, and D to represent the generation process, and the difference between the two is compared by the following formula during actual operation:

[0070]

[0071] The target reconstruction loss is obtained by calculating the difference between the information before and after generation. The calculation formula is as follows:

[0072]

[0073] Furthermore, the entropy difference is used to calculate the network generation loss:

[0074]

[0075] Where X represents the dataset used for training. In the decoupled learning process, in order to control the intra-lung texture supervision and lung frame overflow supervision through different loss functions, the original loss function is rewritten to better achieve the target task in practice, unlike the traditional model. The result is calculated as follows:

[0076]

[0077] L T&S =L Texture +L Structure

[0078] where τ is a random distribution factor that represents the randomness of the sampling operation.

[0079] Not only limited to the decoupled learning of lung frame and lung mass, ARMGan also focuses on the intra-pulmonary areas where the lesion features are more deeply coupled, enabling it to generate training data that is more consistent with the characteristics of pneumoconiosis. Since the Laplace transform can well achieve dimensional transformation in n-dimensional Euclidean space, the feature decoupling task that is difficult to achieve in high-dimensional space is easier to perform in low-dimensional space. In order to make the coding map after asymmetric coding exchange more focused on a single lung area, the Laplace operator is introduced, and the LoG domain transform is applied to enhance the Laplace operator's ability to resist weak interference.

[0080] For different lung regions of the input image, define Where R xy Represents the selected target area. Since the Laplace transform is very sensitive to image noise, a Gaussian operation is performed before the Laplace operation. The whole process can be expressed as a LoG operator, as shown in the following formula:

[0081] LoG*f(x,y)=Δ[G σ (x,y)]*f(x,y)

[0082] Among them, the standard deviation of the Gaussian function is set to σ = 1. In addition, in order to achieve the decoupling of the lung intervals, after applying the above operation, another LoG region operation is performed on the encoding map, and a partition bias factor λ is introduced to deform it as follows:

[0083]

[0084] In the decoupling learning process between different lung regions, the low-dimensional space matrix processed by Laplace LoG domain transform (LoT) is used to achieve the decoupling of each sub-interval of the lung lobe. Through two decoupling operations, one for lung frame and lung mass decoupling and the other for lung interval decoupling, different regional encodings of chest radiographs can be combined more flexibly to generate more competitive data images.

[0085] In addition, a random attribute masking layer is added to the ARMGan module. By actively adding perceptual masking, the training process of the subsequent model is guided. This not only eliminates the trouble caused by attribute masking on model training, but also makes an important contribution to enriching the diversity of training data sets. The important basic theory of this innovation is:

[0086]

[0087] At the same time, the distribution of the output space Ω of the deep generative network is proportional to the square root divisor of the product of the non-zero singular decomposition matrices when mapping this space to the low-dimensional matrix domain. The sample concentration of each output space region depends on whether the slope matrix of this region shrinks or expands relative to other regions. In practice, by controlling the size of the slope matrix, it is possible to control the concentration of samples in the output space, thereby randomly controlling the masking or non-masking of attributes in the generated image. The random space parameter ρ∈R allows for better random attribute selection in the target domain, as shown in the following equation:

[0088]

[0089] The ARMGan training loss supervision function, to which the synthesis part belongs, can be expressed as follows:

[0090]

[0091] This loss function will serve as a loss supervision layer to constrain the training and development of this module;

[0092] A3 uses a multi-granularity feature extraction module (MKTransformer) based on prior knowledge and multi-granularity attention mechanism to specifically solve the problem of low reliability of feature screening networks based on incremental data. In the actual training process, the acquired expertise will be used as prior training parameters. After a certain number of training cycles (5 epochs), the model network will freeze relative to the training parameters, compare the performance difference between the network using prior parameters and the network trained using the model, and incorporate this difference into subsequent network training. Through network-level training guidance, the feature extraction network strikes a balance between the superior feature extraction requirements of deep neural networks and actual diagnostic work. The MKTransformer feature extraction module is detailed in Figure 2 Displayed in.

[0093] In addition, in order to reduce the redundancy of feature information, a coarse-grained and fine-grained dual attention mechanism is introduced. This mechanism is used throughout the feature extraction process to distinguish hidden features and reduce information redundancy by comparing the difference in granularity. The construction of the MKTransformer framework relies on the alternation of encoders and decoders. The coarse-grained and fine-grained attention mechanism operations are expressed by the following formula,

[0094]

[0095] Where Q is the query matrix, K is the content of the coarse-grained or fine-grained information, V is the target domain, QK T , that is, the purpose of the dot product is to calculate the attention weight of Q on V. Finally, in order to prevent the gradient from disappearing, the dimension of the output layer (dk) is squared.

[0096] From a macro perspective, the feature improvement extraction model is a priori network-multi-granularity feature extraction process, which is continuously interactive and iteratively advanced through the process. From a micro perspective, facing the input data, MKTransformer first maps it from high-dimensional space to low-dimensional space, and then uses a coarse-grained attention adjustment mechanism during the encoding process to perform preliminary feature screening on the ROI area in the low-dimensional space. The preliminary feature extraction data will be input into the MKTransformer decoding module in two steps. The first step is to pass the data as input to the backbone network, and use a fine-grained attention module in the decoder to explore deep hidden information based on preliminary screening. The second step is to interactively link the preliminary screened data with the further screened data in the multi-head attention module. Fine-grained information guides the feature extraction of coarse-grained information, and coarse-grained information can also supplement the knowledge gap of fine-grained information, thereby minimizing information redundancy in bidirectional supervision and bidirectional information flow. The process can be expressed as follows:

[0097] MultiHead(Q,K,V)=Concat(head1,head2,....head n )W O ,

[0098] Head i =Attention Coarse&Fine (QW Q i ,KW k i ,VW v i ),i∈(1,N)

[0099] This module optimizes the operation of the multi-attention mechanism to adapt to the diffuse cotton-like characteristics of pneumoconiosis. In addition, KL divergence is used in the probability and statistics of data to replace the complex multivariate distribution of feature information extracted from the model by approximate distribution, and uses probability entropy to measure the degree of information loss when selecting effective information, which can be expressed as follows:

[0100]

[0101] Among them, p and q represent the probability distribution of the target domain, and {x1,x2,...x n} represents the feature domain to be selected in the low-dimensional space. It is worth noting that KL divergence is not used as a method to measure the distance between different information differences, but the asymmetry of KL divergence is used to measure the consistency between two distributions. The entropy of KL information distribution is used in the novel multi-head attention module to quantify the information loss in coarse-grained and fine-grained knowledge distillation and constrain the training of the model to make it orderly.

[0102] Therefore, in MKTransformer, a limiter is used to distinguish the training state and the frozen state of the model. When the limiter is turned on and the model is in the training state, both the encoder and the decoder of MKTransformer will be in working state, processing the data input to the model. For networks using prior parameters, the limiter will be in a restricted state, and the feature mixing and superposition modules in the decoder will be turned off, so that the performance difference and consistency between the prior network and the original model are compared only from a low-dimensional perspective. In order to improve the robustness and stability of the model, when the model encounters new features in the recognition judgment, a report will be provided, and then the model parameters will be adjusted or weighed according to the specific suggestions of the experts;

[0103] A4 In the actual pneumoconiosis staging process, experts usually compare the chest X-ray to be judged with the national standard X-ray in detail and make a final diagnosis based on their field experience. Figure 1 This module uses KL divergence to measure the similarity difference between the image to be staged and the standard slice. First, a probability distribution pstd(x i ), based on the national standard slice library and combined with the measurement results of three doctors, this method is feasible in practice because when the feature reaches a certain approximation, we can assign it to a specific feature category. The deep features extracted by MKTransformer are compared with the standard slice library to obtain the probability distribution qpro(x i ).

[0104] According to the current national diagnostic standards for pneumoconiosis, standard chest X-ray films are widely used as the gold standard. This module uses KL weighting to introduce entropy measurement into model judgment. In the actual operation of the network, the KL weighting algorithm will focus on measuring the KL entropy difference between the chest X-ray to be staged and the standard film of each lung area, and obtain the similarity weight p(x i ), as shown in the following formula:

[0105]

[0106] Using Q KL (xi ) to calculate the similarity and complete the staging. The higher the value, the higher the similarity between the image to be staged and the staging standard. KL (x i )The calculation process is as follows:

[0107]

[0108] Finally, the loss function during the network iteration is as follows:

[0109]

[0110] Where W is the network weight; Y is the pairwise label, m is the hyperparameter, and when the sample pair X1 and X2 belong to the same category, Y = 0; when they belong to different categories, Y = 1. w is the Euclidean distance between X1 and X2 in the latent variable space. When Y = 0, adjusting the parameters will minimize the distance between X1 and X2. When Y = 1, if the distance between X1 and X2 is greater than m, no optimization is performed; if the distance is less than m, the distance between them is increased to m;

[0111] Figure 4 The following is a comparison of the image generation results when using different data increment modules in the present invention. Figure 5 The ROC curve diagrams of the present invention in the staging of pneumoconiosis are as follows: (a) a training result diagram using the LSAE model, (b) a training result diagram using the SAE model, and (c) a training result diagram using the ARMGan model. Figure 6 The histogram comparison when different modules are integrated in the present invention.

[0112] Existing lung dataset expansion methods, such as LSAE or its prototype method SAE, are only designed for general lung diseases and natural image sets. They perform poorly for diffuse, cotton-like disease characteristics such as pneumoconiosis. It is obviously unreasonable to directly apply them to the pneumoconiosis dataset. When these methods are directly applied to enhance the data quality of the pneumoconiosis dataset, the problem of model collapse is bound to occur. Although the general lung dataset expansion method, LSAE, performs well on common lung disease datasets, when it is directly applied to the expansion of the pneumoconiosis dataset, it will encounter the problem of missing key features or incomplete image representation when generating pneumoconiosis chest radiographs. This problem is effectively alleviated by applying the improved data enhancement method for pneumoconiosis proposed in the present invention;

[0113] It can be observed from the experimental results that T is the initial model network, which remains unchanged in this study and is used as a comparison item for subsequent experiments; F is prior knowledge; KL1 represents the first application of the KL divergence idea to the improved network, that is, the coarse-grained and fine-grained KL functions in the attention fusion module; KL2 is a weighted judgment algorithm used to measure the difference between the test chest radiograph and the national standard gold standard. According to the performance results in the figure, the improved model of this study has a smoother convergence performance and better meets the application requirements of deep models. And its area under the ROC curve (AUC) is better than the comparison model, which proves that the present invention has a stronger competitive advantage;

[0114] From the experimental results, it can be concluded that: by using different parameters for comparison, it is found that the multi-granularity interactive information screening mechanism proposed by the invention can reduce the feature redundancy in the experimental results, meet the requirements of prior knowledge application in deep extraction, and the test effect accuracy of the test set model is as high as 93.4%. However, the network still has the problem of information redundancy. Therefore, the KL divergence control function is added to perform module suppression testing. Among them, KL1 performs the granularity information screening work well in the multi-granularity feature fusion module, while the KL2 loss control function guides the output. The final experiment shows that adding KL1 and KL2 at the same time can effectively reduce information redundancy, retain key features to the maximum extent, and make the model achieve higher sensitivity.

[0115] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. An intelligent pneumoconiosis diagnosis system based on weighted information flow data increment network, characterized in that: It includes a data preprocessing module, an incremental data module based on random attribute masking (ARMGan), a multi-granularity feature extraction module based on prior knowledge (MKTransformer), and a KL weighted judgment module: Data preprocessing module: By mixing the collected pneumoconiosis X-ray chest films with normal chest films and using histogram equalization technology to enhance the image contrast, a quality-balanced and diverse dataset is provided for the subsequent ARMGan data incremental network training; Incremental Data Module Based on Random Attribute Masking (ARMGan): This module uses random attribute masking technology to obtain incremental data with higher data quality; it uses an asymmetric encoder-decoder network model to focus on different areas of the input chest X-ray and maximizes the retention of key features through lung interval decoupling; Multi-granularity feature extraction module based on prior knowledge (MKTransformer): Under the guidance of professional doctors’ prior knowledge, this module uses a dual attention mechanism of coarse-grained and fine-grained to assist in the feature extraction process of pneumoconiosis X-ray chest films. At the same time, Kullback-Leibler (KL divergence) is used to optimize the multi-head attention module to minimize information redundancy through the links between filters of different granularities. KL weighted judgment module: This module is used to measure the difference between the test chest X-ray and the national standard chest X-ray. By comparing the statistical differences between the generated chest X-ray and the national standard X-ray, the staging diagnosis of pneumoconiosis can be achieved.

2. According to claim 1, a pneumoconiosis intelligent diagnosis system based on weighted information flow data increment network (WIDINet) is characterized in that: The ARMGan data increment module uses random attribute masking technology to achieve data expansion and improve the data quality of incremental data; specifically includes the following steps: A21 divides the input chest X-ray into six parts according to the doctor's diagnostic habits and based on the national standard film, and uses the LoT coding layer to realize the decoupling of the lung interval after mapping the image from the high-dimensional image to the low-dimensional matrix to expand the difference between the lung intervals; A22 designs an asymmetric coding model, uses fewer coding layers on the lung frame decoupling encoder, and concentrates more computing power on the coding of the lesion texture in the lung; introduces a bias factor μ in each encoding to control the layer correlation function, and produces a difference contrast effect in the decoupling learning of the lung area frame and the lesion texture in the lung; A23 defines the decoupling coding process of the target area sampling image: a lung frame of the chest X-ray is defined as I1, and two lung masses of the chest X-ray are defined as I2; therefore, its coding can be expressed as (z1s, z1t) and (z2s, z2t), using G to represent the synthesis process, Dec to represent the dot product, and D to represent the generation process, and compares the difference between the two during actual operation through the following formula: The target reconstruction loss is obtained by calculating the difference between the information before and after generation. The calculation formula is as follows: Furthermore, the entropy difference is used to calculate the network generation loss: Where X represents the dataset used for training; in the decoupled learning process, in order to control the intra-lung texture supervision and lung frame overflow supervision through different loss functions, the original loss function is rewritten to better achieve the target task in practice, unlike the traditional model, and the result is calculated as follows: L T&S =L Texture +L Structure Where τ is the random distribution factor representing the randomness of the sampling operation; In addition, ARMGan is not limited to the decoupled learning of lung framework and lung mass. It also focuses on the intra-lung areas where the lesion characteristics are more deeply coupled, enabling it to generate training data that is more consistent with the characteristics of pneumoconiosis. In order to make the coding map after asymmetric coding exchange more focused on a single lung area, the Laplacian operator was introduced, and the LoG domain transformation was applied to enhance the weak interference resistance of the Laplacian operator; For different lung regions of the input image, define Where R xy Represents the selected target area; since the Laplace transform is very sensitive to image noise, a Gaussian operation is performed before the Laplace operation. The whole process can be expressed as a LoG operator, as shown in the following formula: LoG*f(x,y)=Δ[G σ (x,y)]*f(x,y) Among them, the standard deviation of the Gaussian function is set to σ = 1; in addition, in order to achieve the decoupling of the lung intervals, after applying the above operation, another LoG region operation is performed on the encoding map, and a partition bias factor λ is introduced to deform it as follows: In the decoupling learning process between different lung regions, the low-dimensional space matrix processed by Laplace LoG domain transform (LoT) is used to achieve the decoupling of each sub-region of the lung lobe; through two decoupling operations, one for lung frame and lung mass decoupling, and the other for lung region decoupling, different regional encodings of chest radiographs are more flexibly combined to generate more competitive data images; In addition, a random attribute masking layer is added to the ARMGan module; by actively adding perceptual masking, the training process of the subsequent model is guided; it not only eliminates the trouble caused by attribute masking on model training, but also makes an important contribution to enriching the diversity of training data sets; at the same time, the distribution of the output space Ω of the deep generative network is proportional to the square root divisor of the product of the non-zero singular decomposition matrices when mapping the space to the low-dimensional matrix domain; the sample concentration of each output space region depends on whether the slope matrix of the region shrinks or expands relative to other regions; in actual operation, by controlling the size of the slope matrix, the concentration of samples in the output space can be controlled, thereby randomly controlling the masking or non-masking of attributes in the generated image; the random space parameter ρ∈R allows better random attribute selection in the target domain, as shown in the following formula: The ARMGan training loss supervision function, to which the synthesis part belongs, can be expressed as follows: This loss function will serve as a loss supervision layer to constrain the training and development of this module.

3. According to claim 1, a pneumoconiosis intelligent diagnosis system based on weighted information flow data increment network (WIDINet) is characterized in that: The MKTransformer feature extraction module uses prior knowledge and multi-granularity attention modules to specifically address the problem of low reliability of feature screening networks based on incremental data; in the actual training process, the acquired expertise will be used as prior training parameters; after 5 epochs of training cycles, the model network will freeze relative to the training parameters, compare the performance difference between the network using the prior parameters and the network trained using the model, and incorporate this difference into subsequent network training. Through network-level training guidance, the feature extraction network strikes a balance between the superior feature extraction requirements of deep neural networks and actual diagnostic work; In addition, in order to reduce the redundancy of feature information, a coarse-grained and fine-grained supervision mechanism is introduced. This mechanism is used throughout the feature extraction process to distinguish hidden features and reduce information redundancy by comparing the differences in granularity. The construction of the MKTransformer framework relies on the alternation of encoders and decoders. The coarse-grained and fine-grained attention mechanism operations are expressed by the following formula: Where Q is the query matrix, K is the content of the coarse-grained or fine-grained information, V is the target domain, QK T , that is, the purpose of the dot product is to calculate the attention weight of Q on V; finally, in order to prevent the gradient from disappearing, the dimension of the output layer (dk) is squared; From a macro perspective, the feature improvement extraction model is a priori network - multi-granularity feature extraction process, which is continuously interactive and iteratively advanced through the process; from a micro perspective, facing the input data, MKTransformer first maps it from high-dimensional space to low-dimensional space, and then uses a coarse-grained attention adjustment mechanism in the encoding process to perform preliminary feature screening on the ROI area in the low-dimensional space; the preliminary feature extraction data will be input into the MKTransformer decoding module in two steps; the first step is to pass the data as input to the backbone network, and use the fine-grained attention module in the decoder to explore deep hidden information based on the preliminary screening; the second step is to interactively link the preliminary screened data with the further screened data in the multi-head attention module; Fine-grained information guides the feature extraction of coarse-grained information, and coarse-grained information can also supplement the knowledge gap of fine-grained information, thereby minimizing information redundancy in bidirectional supervision and bidirectional information flow; the process can be expressed as follows: MultiHead(Q,K,V)=Concat(head1,head2,....head n )W O , Head i =Attention Coarse&Fine (QW Q i ,KW k i ,VW v i ),i∈(1,N) This module optimizes the operation of the multi-attention mechanism to adapt to the diffuse cotton-like characteristics of pneumoconiosis; In addition, KL divergence is used in the probability and statistics of data. It replaces the complex multivariate distribution of feature information extracted in the model by approximate distribution, and uses probability entropy to measure the degree of information loss when selecting valid information. It can be expressed as follows: Among them, p and q represent the probability distribution of the target domain, and {x1,x2,...x n } represents the feature domain to be selected in the low-dimensional space; it is worth noting that KL divergence is not used as a method to measure the distance between different information differences, but the asymmetry of KL divergence is used to measure the consistency between two distributions; the entropy of KL information distribution is used in the novel multi-head attention module to quantify the information loss in coarse-grained and fine-grained knowledge distillation, and to restrict the training of the model to make it proceed in an orderly manner; Therefore, in MKTransformer, a limiter is used to distinguish the training state and the frozen state of the model; when the limiter is turned on and the model is in the training state, both the encoder and the decoder of MKTransformer will be in working state to process the data input into the model; for the network using prior parameters, the limiter will be in the restricted state, and the feature mixing and superposition modules in the decoder will be turned off, so that the performance difference and consistency between the prior network and the original model are compared only from a low-dimensional perspective; in order to improve the robustness and stability of the model, when the model encounters new features in the recognition judgment, a report will be provided, and then the model parameters will be adjusted or weighed according to the specific suggestions of the experts.

4. The intelligent pneumoconiosis diagnosis system based on weighted information flow data increment network (WIDINet) according to claim 1 is characterized in that: The KL weighted judgment module: Use KL weighting to introduce entropy value measurement into model judgment. In the actual operation of the network, the KL weighted algorithm will focus on measuring the KL entropy value difference between the chest X-ray film to be staged and the standard film of each lung area, and obtain the similarity weight p(x i ), as shown in the following formula: Using Q KL (x i ) to calculate the similarity and complete the staging. The higher the value, the higher the similarity between the image to be staged and the staging standard. KL (x i )The calculation process is as follows: Finally, the loss function during the network iteration is as follows: Where W is the network weight; Y is the pairwise label, m is the hyperparameter, when the sample pair X1 and X2 belong to the same category, Y = 0; when they belong to different categories, Y = 1; D w is the Euclidean distance between X1 and X2 in the latent variable space; when Y = 0, adjusting the parameters will minimize the distance between X1 and X2; when Y = 1, if the distance between X1 and X2 is greater than m, no optimization is performed; if the distance is less than m, the distance between them is increased to m.

Citation Information

Cited By

  • Pancreatic cancer and breast cancer early imaging diagnosis method based on multi-modal comparative learning

    CN121122651A

  • Pancreatic cancer and breast cancer early image diagnosis method based on multi-modal contrast learning

    CN121122651B