Construction method of IPA recognition model, IPA recognition method, terminal device and storage medium

Abstract

The application provides an IPA recognition model construction method, an IPA recognition method, a terminal device and a storage medium. The construction method comprises: constructing a training sample set from collected clinical metadata and chest CT image samples; and inputting the training sample set to a pre-constructed initial neural network model for iterative training; wherein a feature extraction layer outputs a lesion area prediction result of the chest CT image sample and corresponding image features based on a pre-set segmentation network, and extracts clinical features; a feature fusion layer fuses the received features to output primary fusion features; and a diagnosis reasoning layer maps the weighted primary fusion features to an IPA disease prediction probability based on a fully connected layer network, so as to use the IPA disease prediction probability as the output of the initial neural network model, and the training is ended when the segmentation loss function and the pre-set constraint term both reach a pre-set value, thereby obtaining an IPA recognition model. The application can improve the timeliness and accuracy of the identification of invasive pulmonary aspergillosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and medical health technology, in particular to an IPA recognition model construction method, an IPA recognition method, a terminal device and a storage medium. BACKGROUND

[0002] Invasive Pulmonary Aspergillosis (IPA) is a serious infectious disease caused by Aspergillus invasion of the lungs. It mainly occurs in high-risk groups such as the elderly and immunocompromised individuals. The clinical features are insidious onset, and it is not easy to be detected in the early stage of the disease, but the disease progresses very rapidly. If IPA is not diagnosed and treated effectively in time, the mortality rate of patients can be as high as 60% to 90%, posing a serious threat to the health of patients.

[0003] Currently, the diagnosis of IPA mainly relies on imaging features, microbiological evidence and clinical symptom information. Among them, imaging features use CT examination to observe specific imaging manifestations such as "halo sign" and "air meniscus sign" to assist in diagnosis; microbiological evidence obtains microbiological diagnostic basis through sputum culture, blood fungus detection, etc.; clinical symptom information is a comprehensive judgment made by the physician in combination with the relevant clinical symptoms exhibited by the patient. However, the CT images of early IPA patients lack specificity, and their imaging features are highly similar to bacterial pneumonia, pulmonary hemorrhage and other diseases, which leads to a high misdiagnosis rate in the diagnosis process. Histological diagnosis requires the removal of lung tissue, which not only has high surgical risk, but also can cause a series of complications such as loss of lung function, which will cause additional adverse effects on the health of patients. Microbiology does not cause additional harm to patients, but the culture period of microbiological detection is relatively long, usually 3 to 7 days, which obviously cannot meet the clinical urgent need of "early diagnosis and early treatment" of IPA. How to realize the timely and accurate identification of IPA is a technical problem to be solved. SUMMARY

[0004] The present application aims to provide an IPA recognition model construction method, an IPA recognition method, a terminal device and a storage medium to solve the above technical problems and improve the timeliness and accuracy of IPA identification.

[0005] To solve the above technical problems, the present application provides an IPA recognition model construction method, comprising:

[0006] The clinical metadata and chest CT image samples of a patient within a preset time length are taken as a training sample, and patients with different immune subtypes are taken as data collection objects, and a training sample set is constructed based on the collected clinical metadata and chest CT image samples;

[0007] The training sample set is input to a pre-constructed initial neural network model for iterative training, wherein the initial neural network model includes a feature extraction layer, a feature fusion layer, and a diagnosis reasoning layer.

[0008] During the iterative training of the initial neural network, the chest CT image sample collected at the latest time in each training sample is taken as input to perform iterative training in the feature extraction layer, so that the feature extraction layer outputs the lesion area prediction result of the chest CT image sample based on a preset segmentation network and the corresponding image features, and extracts clinical features according to the corresponding clinical metadata of the chest CT image sample.

[0009] The image time sequence features and the clinical time sequence features of the training sample within the preset time length are calculated, and the image features, the clinical features, and the corresponding image time sequence features and clinical time sequence features are taken as input, so that the feature fusion layer fuses the received features to output primary fusion features.

[0010] The primary fusion features are taken as input, so that the diagnosis reasoning layer maps the weighted primary fusion features to the IPA disease prediction probability based on a fully connected layer network, and the IPA disease prediction probability is taken as the output of the initial neural network model.

[0011] The corresponding segmentation loss function is established according to the lesion features of different immune subtypes, the segmentation loss function is calculated according to the immune subtype to which the training sample belongs, the corresponding lesion area prediction result, and the IPA disease prediction probability, and the training is ended when the segmentation loss function and the preset constraint term both reach the preset value, and the IPA recognition model is obtained.

[0012] In the above scheme, the clinical metadata and chest CT image samples of patients with different immune subtypes are used to construct a training sample set, which enables the model to learn the features of different immune subtypes and enhance the recognition ability of different situations. Secondly, the feature extraction layer, the feature fusion layer, and the diagnosis reasoning layer of the initial neural network model work together, which can effectively extract and fuse image and clinical features, including time sequence features, to comprehensively and deeply mine data information. Thirdly, the chest CT image sample collected at the latest time is used for iterative training, which enables the model to follow the latest data features. Finally, the segmentation loss function is established according to the lesion features of different immune subtypes, and the training is combined with the preset constraint term, which can ensure the accuracy and stability of the model training, so as to obtain a model with excellent performance and accurate recognition of IPA, and improve the timeliness and accuracy of the identification of invasive pulmonary aspergillosis.

[0013] In an implementation manner, patients of different immune subtypes are taken as data collection objects, a training sample set is constructed based on collected clinical metadata and chest CT image samples, the training sample set is input into a pre-constructed initial neural network model for iterative training, and the iterative training specifically includes:

[0014] Clinical metadata of different patients within a preset time length are acquired, wherein the clinical metadata include basic disease history, immune suppressor type, and neutrophil deficiency duration;

[0015] An immune subtype to which each patient belongs is divided according to the clinical metadata, and a corresponding subtype label is established for the clinical metadata, wherein the types of the immune subtypes include a first subtype, a second subtype, and a third subtype;

[0016] A corresponding data collection list is determined according to the immune subtype to which each patient belongs, wherein when the patient is of the first subtype, chest CT image sample data of the patient within a first preset time length is collected, when the patient is of the second subtype, a clinical index associated with bleeding is added in the clinical metadata, and when the patient is of the third subtype, chest CT image sample data of the patient within a second preset time length is collected, the second preset time length being greater than the first preset time length;

[0017] The training sample set is constructed according to the clinical metadata and the data collection list.

[0018] In the above scheme, by acquiring clinical metadata of patients within a preset time, including basic disease history, and dividing immune subtypes and establishing subtype labels, patients in different immune states can be accurately distinguished. According to different immune subtypes, a corresponding data collection list is determined, such as different subtypes in the chest CT image sample collection time or the addition of clinical indexes, so that the collected data is more targeted and specific. Based on this, the training sample set is constructed, which enables the initial neural network model to learn the characteristics of different immune subtypes when inputting the initial neural network model for iterative training, thereby improving the recognition accuracy and generalization ability of the model for patients of different immune subtypes, and providing strong support for accurate diagnosis and treatment.

[0019] In an implementation manner, the feature extraction layer outputs a lesion region prediction result of the chest CT image sample and corresponding image features based on a preset segmentation network, and extracts clinical features according to the clinical metadata corresponding to the chest CT image sample, and the feature extraction layer specifically includes:

[0020] An image recognition target corresponding to the immune subtype to which the chest CT image sample belongs is called;

[0021] The chest image sample is recognized based on the U-Net++ network and the image recognition target, and a lesion prediction region mask and a lung parenchyma region are output;

[0022] the lesion prediction area mask and the lung parenchyma area as the lesion area prediction result;

[0023] extracting a lesion feature of the chest CT image sample according to the lesion area prediction result, and taking the lesion feature as the image feature; wherein the image feature includes a lesion shape feature, a lesion density feature, a lesion edge feature, and a lung parenchyma background feature;

[0024] classifying and cleaning the clinical metadata to obtain clinical features; wherein the clinical features include patient body temperature, neutrophil count indicators, immune state indicators, infection marker indicators, and bleeding risk indicators.

[0025] In the above scheme, by calling the corresponding image recognition target according to the immune subtype to which the chest CT image sample belongs, and then using the U-Net++ network to combine this target to recognize the chest image sample, the lesion prediction area mask and the lung parenchyma area are output as the lesion area prediction result, which realizes accurate image recognition for different immune subtypes, improves the accuracy and pertinence of lesion area prediction, and provides more reliable image basis for subsequent IPA diagnosis. On the one hand, according to the lesion area prediction result, the image features such as lesion shape, density, edge, and lung parenchyma background are accurately extracted from the chest CT image sample, which fully reflects the lesion situation and provides rich and valuable image information for disease diagnosis; on the other hand, the clinical metadata is classified and cleaned to obtain patient body temperature, neutrophil count, immune state, infection marker, and bleeding risk clinical features, effectively removing invalid data and highlighting key information. By combining image features and clinical features, more comprehensive and accurate basis can be provided for the training and diagnosis of subsequent IPA recognition models, thereby improving the accuracy and reliability of disease diagnosis.

[0026] In one implementation, after the feature extraction layer outputs the lesion area prediction result of the chest CT image sample based on the preset segmentation network, it further includes: determining whether to take the lesion area prediction result as the output result of the feature extraction layer based on a preset segmentation quality evaluation index, specifically:

[0027] calculating the segmentation quality score of the lesion area prediction result according to the segmentation quality expression; wherein the segmentation quality expression is:

[0028] ;

[0029] wherein, is the segmentation quality score; is the total number of non-zero pixels in the segmentation area, and the segmentation area is the lesion associated area in the lung parenchyma area; is the total number of pixels in the lesion area; a gradient value of the lung parenchyma region pixel; a gradient value of the lung parenchyma region pixel;

[0030] When the segmentation quality score is greater than the preset segmentation quality evaluation index, the lesion region prediction result is taken as the output result of the feature extraction layer;

[0031] Otherwise, the lesion region prediction result of the feature extraction layer is regenerated based on a preset multi-model recognition and voting mechanism.

[0032] In the above scheme, after the feature extraction layer outputs the lesion region prediction result of the chest CT image sample, a preset segmentation quality evaluation index is introduced for judgment, and the accuracy of the prediction result is measured by calculating the segmentation quality score. If the score is greater than the preset index, the prediction result is taken as the output, which ensures the reliability of the output result. If the score does not meet the standard, the prediction result is regenerated based on the multi-model recognition and voting mechanism, which further improves the accuracy and stability of the lesion region prediction, provides a better and more accurate data basis for subsequent IPA diagnosis and other processes, and helps to improve the quality and efficiency of the entire diagnosis process

[0033] In an implementation manner, the lesion region prediction result of the feature extraction layer is regenerated based on a preset multi-model recognition and voting mechanism, specifically including:

[0034] N different networks including the U-Net++ network are used to identify the lesion region of the chest CT image sample, and a lesion prediction region mask is output; wherein N is an odd number;

[0035] The lesion prediction region mask output by each network is binarized to obtain a binarization result; the binarization result is used to represent the lesion region judgment result of each pixel in the chest CT image sample;

[0036] The binarization results of each pixel in the chest CT image sample are counted, and when more than half of the binarization results determine that the current pixel is a lesion region, it is determined that the position of the current pixel is a lesion region;

[0037] According to the statistical results of each pixel, the lesion region prediction result of the feature extraction layer is regenerated.

[0038] In the scheme, an odd number of different networks including the U-Net++ network are used to identify the lesion area of the chest CT image sample, output the lesion prediction area mask and binarize it, then the binarization results of each pixel of the image sample are counted, and whether the pixel is a lesion area is determined according to the rule that more than half of the pixels are determined as a lesion area, and finally the lesion area prediction result of the feature extraction layer is regenerated. The multi-model identification and voting mechanism effectively reduces the limitations and errors of a single model, improves the accuracy and reliability of the lesion area prediction, and provides more accurate basis for subsequent disease diagnosis and analysis.

[0039] In an implementation manner, image time sequence features and clinical time sequence features of the training sample within a preset time length are calculated, image features and clinical features and corresponding image time sequence features and clinical time sequence features are taken as inputs, and the received features are fused by the feature fusion layer to output primary fusion features, and the method specifically comprises the following steps:

[0040] In the same training sample, image features of the chest CT image sample are extracted every preset interval, image features output by the feature extraction layer are combined, a change rate between the image features extracted at adjacent times is calculated, and image time sequence features are generated;

[0041] In the same training sample, data of each clinical index in the clinical metadata is extracted every preset time interval, clinical features output by the feature extraction layer are combined, a change rate between the clinical features extracted at adjacent times is calculated, and clinical time sequence features are generated;

[0042] The image time sequence features and the clinical time sequence features are spliced to obtain the primary fusion features.

[0043] In the scheme, image features of the chest CT image sample and data of each clinical index in the clinical metadata are extracted every preset interval, features output by the feature extraction layer are combined, a change rate between the features extracted at adjacent times is calculated, image time sequence features and clinical time sequence features are generated, and the two are spliced to obtain the primary fusion features. This process comprehensively considers the dynamic changes of image and clinical features over time, enriches the feature dimension, provides more comprehensive and time-effective information for subsequent model training, and helps to improve the accuracy and reliability of disease diagnosis and prediction of the model.

[0044] In an implementation scheme, before splicing the image time sequence features and the clinical time sequence features, the following steps are further included:

[0045] The image features corresponding to each chest CT image sample are weighted according to a time distance between a collection time of the chest CT image sample and a latest collection time;

[0046] Attention weights within the chest CT image modality are calculated based on the weighted image features; the expression for the attention weights is:

[0047] ;

[0048] In the formula, Attention weights within the chest CT image modality; For the first Image feature vectors at each time point; No. Image feature vectors at each time point, and ≠ ; This is the cosine similarity function, used to calculate the similarity between two image feature vectors; This is the key weight matrix for attention. This represents the total dimension of the image features.

[0049] In the above scheme, before stitching together the temporal features of the images and the clinical temporal features, the image features are first weighted according to the time distance between the acquisition time of the chest CT image sample and the latest acquisition time. Then, attention weights within the chest CT image modality are calculated based on the weighted image features. This approach can highlight the differences in the importance of image features at different time points, make reasonable adjustments to the features based on temporal correlation, and further explore the correlation between image features through the attention mechanism, thereby enhancing the representational ability of image features. This makes the primary fusion features obtained by subsequent stitching more accurately reflect the development trend of the disease, which helps to improve the model's diagnostic and analytical effects on chest diseases.

[0050] In one implementation, image temporal features and clinical temporal features are concatenated to obtain primary fusion features, which specifically include:

[0051] The image temporal features and the clinical temporal features are mapped to the same dimension using a cross-modal mapping matrix; wherein, the expression for the cross-modal attention weight is:

[0052] ;

[0053] In the formula, Cross-modal attention weights; The first of the image temporal features Dimension transpose; This is a cross-modal mapping matrix; This is the rule weight matrix, which retrieves the corresponding values ​​based on the immune subtype to which the training samples belong. For the first One clinical time-series feature vector; This is a bias term, initially set to 0.1, used to adjust the baseline value of the cross-modal mapping; The total dimension of clinical characteristics; For the first One clinical time-series feature vector; Confidence level of chest CT image samples; Confidence level of clinical data;

[0054] in, = ; ;

[0055] In the formula, Signal-to-noise ratio of chest CT images; and These represent the minimum and maximum signal-to-noise ratios of the image acquisition device. To segment quality scores; For data integrity; For measurement error; This represents the maximum permissible measurement error.

[0056] The primary fusion feature is obtained by concatenating the image time-series features and clinical time-series features mapped to the same dimension.

[0057] In the above scheme, after obtaining the temporal features of imaging and clinical data, they are first mapped to the same dimension using a cross-modal mapping matrix, and then concatenated to obtain the primary fusion feature. Its expression comprehensively considers the cross-modal attention weights of imaging and clinical features, introduces a regular weight matrix to adjust values ​​according to immune subtypes, and adds a bias term to adjust the mapping baseline. Simultaneously, the confidence scores of chest CT image samples and clinical data are incorporated into the calculation. The image confidence score combines the image signal-to-noise ratio and segmentation quality score, while the clinical confidence score considers data integrity and measurement error. This approach achieves effective fusion of imaging and clinical temporal features, enabling the fused features to more comprehensively and accurately reflect disease characteristics, fully utilize multi-source data information, provide a more reliable and high-quality data foundation for subsequent disease diagnosis and prediction tasks, and improve the model's performance and accuracy.

[0058] In one implementation, the diagnostic inference layer further includes weighting based on the immune subtype corresponding to the training samples containing the primary fusion features, specifically:

[0059] Multi-dimensional weighted cross-entropy is calculated based on the immune subtype corresponding to the training sample containing the primary fusion feature; the expression for the multi-dimensional weighted cross-entropy is:

[0060] ;

[0061] In the formula, Weighted cross-entropy; For training sample index; a weight corresponding to an immune subtype; a training sample category weight; an IPA true label of an i-th training sample; an IPA disease prediction probability of an i-th training sample. an IPA disease prediction probability of an i-th training sample. an IPA disease prediction probability of an i-th training sample.

[0062] In the above scheme, the immune subtype corresponding to the training sample where the primary fusion feature is located is subjected to multi-dimensional weighted cross-entropy processing. Through a specific weighted cross-entropy expression, the weight corresponding to the immune subtype, the training sample category weight, the IPA true label of the training sample, and the IPA disease prediction probability are comprehensively considered. In this way, the loss function can be adjusted according to different immune subtypes and sample categories, highlighting the importance of different samples, so that the model pays more attention to key samples and features during the training process, thereby improving the accuracy and reliability of the model in diagnosing diseases under different immune subtypes, enhancing the generalization ability and adaptability of the model, and better processing complex and diverse clinical data.

[0063] In one implementation, a corresponding segmentation loss function is established in advance according to the lesion features of different immune subtypes, the segmentation loss function is calculated according to the immune subtype to which the training sample belongs, the corresponding lesion area prediction result, and the IPA disease prediction probability, and the training is ended when the segmentation loss function and the preset constraint term both reach the preset value, and the IPA recognition model is obtained. Specifically, it includes:

[0064] A basic loss function is established in advance; wherein the expression of the basic loss function is:

[0065] ;

[0066] In the formula, is a true lesion area mask; is a lesion prediction area mask;

[0067] An annular sign edge gradient constraint term of the segmentation loss function is established, and the annular sign and inflammation are identified based on the annular sign edge gradient constraint term; wherein the expression of the annular sign edge gradient constraint term is:

[0068] ;

[0069] In the formula, is an annular sign edge gradient constraint value; is an annular sign area; is a gradient value of an annular sign edge pixel ; Iinflam(x, y) is a gradient value of an inflammation area edge pixel ;

[0070] The first segmentation loss function corresponding to the first immune subtype is established according to the basic loss function and the halo edge gradient constraint term; wherein the expression of the first segmentation loss function is:

[0071] ;

[0072] In the formula, the first segmentation loss function corresponding to the first immune subtype is established according to the basic loss function and the halo edge gradient constraint term; wherein the expression of the first segmentation loss function is: the basic loss function is established according to the basic loss function and the halo edge gradient constraint term; wherein the expression of the first segmentation loss function is:

[0073] The second segmentation loss function and the third segmentation loss function are established according to the basic loss function and the lesion characteristics of different immune subtypes; wherein the expression of the second segmentation loss function is:

[0074] ;

[0075] In the formula, the second segmentation loss function corresponding to the second immune subtype is established according to the basic loss function and the halo edge gradient constraint term; wherein the expression of the first segmentation loss function is: the area of the bleeding area; the bleeding area mask; the IPA disease prediction probability;

[0076] The expression of the third segmentation loss function is:

[0077] ;

[0078] In the formula, the third segmentation loss function corresponding to the third immune subtype is established according to the basic loss function and the halo edge gradient constraint term; wherein the expression of the first segmentation loss function is: the area of the chronic cavity region; the local binary pattern value of the cavity wall pixel; the texture weight is valued according to the local binary pattern value of the cavity wall pixel;

[0079] When the segmentation loss functions corresponding to different immune subtypes and the halo edge gradient constraint term all reach the preset value, the iteration training is ended to obtain the IPA recognition model.

[0080] In the above scheme, the corresponding segmentation loss function is established in advance according to the lesion characteristics of different immune subtypes to train the IPA recognition model. First, a basic loss function is constructed to measure the difference between the predicted lesion area and the real area, and then an halo edge gradient constraint term is established to identify halo and inflammation. Based on this, the first, second and third segmentation loss functions are established for different immune subtypes. On the basis of the basic loss function, different features such as halo edge gradient constraint, bleeding area, disease prediction probability, local binary pattern value of chronic cavity area and texture weight are combined. When the segmentation loss function and the halo edge gradient constraint term of each immune subtype reach the preset value, the training is ended. This approach fully considers the lesion characteristics of different immune subtypes, enabling the model to more accurately segment and identify lesions under different immune subtypes, improving the accuracy and adaptability of the IPA recognition model, and enhancing the performance of the model in complex clinical scenarios.

[0081] In a second aspect, the present application also provides an IPA recognition method, comprising:

[0082] Collecting clinical metadata and chest CT image sample data of a patient within a preset time length range from the current time;

[0083] Inputting the clinical metadata and chest CT image sample data into the IPA recognition model constructed by the IPA recognition model construction method as above, and outputting the IPA disease prediction probability of the patient;

[0084] Feeding back the IPA disease prediction probability to the user terminal to enable the user terminal to generate a corresponding treatment recommendation scheme according to the IPA disease prediction probability.

[0085] In the above scheme, through the automatic IPA recognition model, the clinical metadata and chest CT image sample data of the patient can be quickly processed, and the disease prediction probability can be output within a short time, greatly shortening the diagnosis time and improving the medical efficiency. The model is trained based on a large amount of data of patients with different immune subtypes, and has undergone parameter optimization and clinical verification, and can more accurately identify IPA, reduce misdiagnosis and missed diagnosis, and provide more reliable diagnosis results for patients.

[0086] In a third aspect, the present application also provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the IPA recognition model construction method as above when executing the computer program.

[0087] In a fourth aspect, the present application also provides a computer readable storage medium, comprising a stored computer program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to execute the IPA recognition model construction method as above when the computer program runs. Attached Figure Description

[0088] Figure 1 This is a flowchart illustrating a method for constructing an IPA identification model according to an embodiment of the present invention;

[0089] Figure 2 This is a flowchart illustrating an IPA identification method provided in one embodiment of the present invention. Detailed Implementation

[0090] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0091] The terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0092] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0093] Example 1

[0094] See Figure 1 , Figure 1 This is a flowchart illustrating a method for constructing an IPA identification model according to an embodiment of the present invention. The embodiment of the present invention provides a method for constructing an IPA identification model, including steps 101 to 102, each step of which is detailed below:

[0095] Step 101: Use a patient's clinical metadata and chest CT image samples within a preset time period as a training sample, and use patients with different immune subtypes as data collection objects to construct a training sample set based on the collected clinical metadata and chest CT image samples.

[0096] In the embodiment of the present application, patients with different immune subtypes are taken as data collection objects, the clinical metadata and chest CT image samples of each patient within a preset time length are combined into a training sample, and then a training sample set is constructed. Different immune subtype patients are selected for data collection, so that the training sample set covers patient information in multiple immune states, can reflect the characteristics of people of different immune types suffering from related diseases, helps the model to learn more comprehensive and representative feature patterns, improves the generalization ability of the model, and enables the model to have better performance when facing patients with various immune states. Further, the clinical metadata and chest CT image samples are combined as training samples, realizing the fusion of multi-modal data. The clinical metadata may include basic information, medical history, symptoms, etc. of the patient, while the chest CT image can directly present the morphological structure of the lung. The two kinds of data complement each other, and can provide the model with more rich and accurate information, improving the performance and accuracy of the model.

[0097] In an embodiment, different immune subtype patients are taken as data collection objects, and a training sample set is constructed based on the collected clinical metadata and chest CT image samples; the training sample set is input into a pre-constructed initial neural network model for iterative training, specifically including: obtaining the clinical metadata of different patients within a preset time length; wherein the clinical metadata includes basic disease history, immune suppressant type and neutrophil deficiency duration; dividing the immune subtype to which each patient belongs according to the clinical metadata and establishing a corresponding subtype label for the clinical metadata; wherein the type of immune subtype includes a first subtype, a second subtype and a third subtype; determining the corresponding data collection list according to the immune subtype to which each patient belongs; wherein when the patient is of the first subtype, the chest CT image sample data of the patient within a first preset time length is collected; when the patient is of the second subtype, a clinical index associated with bleeding is added in the clinical metadata; when the patient is of the third subtype, the chest CT image sample data of the patient within a second preset time length is collected; the second preset time length is greater than the first preset time length; and constructing the training sample set according to the clinical metadata and the data collection list.

[0098] In the embodiments of the present application, by collecting, classifying and processing the clinical metadata and chest CT image samples of patients of different immune subtypes, a training sample set is constructed and used for iterative training of the initial neural network model. The core is to divide the immune subtypes according to the clinical metadata of the patients, and to develop customized data collection lists for different subtypes to ensure that the input data meet the needs of subsequent model training. Specifically, the clinical metadata of different patients within a preset time length is collected, including the history of underlying diseases, the type of immunosuppressants, and the duration of neutropenia. These information is an important basis for judging the immune subtype of the patient. According to the obtained clinical metadata, the patients are divided into different immune subtypes, such as the first subtype (corresponding to subtype A), the second subtype (corresponding to subtype B), and the third subtype (corresponding to subtype C). The clinical metadata of each patient is labeled with the corresponding subtype label to facilitate subsequent data management and model training. For different immune subtypes, the corresponding data collection list is determined as follows:

[0099] First subtype (subtype A): This type of patient is mostly after acute myeloid leukemia chemotherapy, and the lesion is mainly "halo sign", which progresses quickly. Therefore, it is necessary to preferentially collect the chest CT image sample data of the patient within the first preset time length (such as within 48 hours) to timely capture the changes in the patient's condition;

[0100] Second subtype (subtype B): Patients after hematopoietic stem cell transplantation belong to this subtype, and the lesion is prone to be associated with bleeding. When collecting clinical metadata, clinical indicators associated with bleeding, such as platelet count and coagulation function indicators, are added to focus on distinguishing bleeding from lesions;

[0101] Third subtype (subtype C): Chronic immunodeficient patients, such as HIV patients, have chronic hollow lesions and slow progression. It is necessary to collect the chest CT image sample data of the patient within the second preset time length (such as within 3 months) to observe the long-term changes in the patient's condition.

[0102] According to the clinical metadata and the customized data collection list, the relevant information of each patient is combined into a training sample, and finally a training sample set is constructed.

[0103] For example, the clinical metadata of 100 patients is collected, including the history of underlying diseases, the type of immunosuppressants, and the duration of neutropenia. At the same time, the chest CT image data of these patients is collected. Through analysis of the clinical metadata, the patients are divided into different immune subtypes:

[0104] First subtype (subtype A): 30 patients are after acute myeloid leukemia chemotherapy, and the clinical metadata of these patients is labeled with the first subtype label;

[0105] Second subtype (subtype B): 20 patients are after hematopoietic stem cell transplantation, and the second subtype label is added;

[0106] Subtype 3 (Subtype C): 50 patients with chronic immunodeficiency (such as HIV patients) were added with the subtype 3 label.

[0107] The following data collection checklists were developed for different patient subtypes: For 30 patients in subtype A, chest CT image samples within 48 hours were prioritized for collection; for 20 patients in subtype B, clinical indicators related to bleeding, such as platelet count and prothrombin time, were added to their clinical metadata; and for 50 patients in subtype C, chest CT image samples within 3 months were collected. Each patient's clinical metadata and corresponding chest CT image sample data (collected according to different subtypes) were combined into a training sample, ultimately constructing a training sample set containing 100 samples. The constructed training sample set was input into a pre-built initial neural network model for iterative training to improve the model's accuracy in diagnosing diseases in patients with different immune subtypes.

[0108] Step 102: Input the training sample set into a pre-constructed initial neural network model for iterative training; wherein, the initial neural network model includes a feature extraction layer, a feature fusion layer, and a diagnostic inference layer; during the iterative training of the initial neural network, the chest CT image sample acquired at the latest time in each training sample is used as input to perform iterative training on the feature extraction layer, so that the feature extraction layer outputs the lesion region prediction result and corresponding image features of the chest CT image sample based on a preset segmentation network, and extracts clinical features according to the clinical metadata corresponding to the chest CT image sample; calculate the image temporal features and clinical temporal features of the training sample within the preset time length, and use the image features and clinical features... The features and corresponding image time-series features and clinical time-series features are used as inputs to enable the feature fusion layer to fuse the received features and output primary fused features. The primary fused features are used as inputs to enable the diagnostic inference layer to map the weighted primary fused features to IPA disease prediction probabilities based on a fully connected layer network. The IPA disease prediction probabilities are used as the output of the initial neural network model. A corresponding segmentation loss function is established in advance based on the lesion features of different immune subtypes. The segmentation loss function is calculated based on the immune subtype to which the training sample belongs, the corresponding lesion region prediction results, and the IPA disease prediction probabilities. Training ends when the segmentation loss function and the preset constraint terms both reach preset values, and the IPA recognition model is obtained.

[0109] The training sample set is input into a pre-built initial neural network model for iterative training; the initial neural network model includes a feature extraction layer, a feature fusion layer, and a diagnostic inference layer.

[0110] In this embodiment of the invention, the initial neural network model adopts a hierarchical structure consisting of a feature extraction layer, a feature fusion layer, and a diagnostic inference layer. The feature extraction layer extracts key features from the input data, the feature fusion layer effectively integrates features from different sources, and the diagnostic inference layer performs disease diagnosis inference based on the fused features. This hierarchical design allows the model to progressively deepen its training by iteratively inputting the training sample set. The model can continuously adjust its parameters and structure to adapt to the characteristics and patterns of the training data. As the number of iterations increases, the model can gradually learn more accurate diagnostic patterns, improving its diagnostic and generalization abilities, enabling it to make reliable diagnoses even when faced with new patient data. This process processes the data, mines the potential information within it, and improves the model's performance and diagnostic accuracy.

[0111] When iteratively training the initial neural network, the chest CT image sample acquired at the latest time in each training sample is used as input to perform iterative training in the feature extraction layer. This allows the feature extraction layer to output the lesion region prediction result and corresponding image features of the chest CT image sample based on the preset segmentation network, and extract clinical features based on the clinical metadata corresponding to the chest CT image sample.

[0112] In this embodiment of the invention, chest CT image samples acquired at the most recent time are used for training, which can reflect the patient's current condition. The feature extraction layer uses a preset segmentation network to output lesion region prediction results to locate the lesion position in the chest CT image. In addition to image features, clinical features are also extracted based on the clinical metadata corresponding to the chest CT image samples. Combining image features and clinical features achieves the fusion of multi-source data, providing the model with more comprehensive and richer information, enabling the model to comprehensively consider multiple factors, thereby improving the accuracy and reliability of diagnosis and making more scientific and reasonable diagnostic decisions.

[0113] In one embodiment, the feature extraction layer outputs the lesion region prediction result of the chest CT image sample based on a preset segmentation network, specifically including: calling the corresponding image recognition target according to the immune subtype to which the chest CT image sample belongs; recognizing the chest image sample based on the U-Net++ network and the image recognition target and outputting the lesion prediction region mask and the lung parenchyma region; and using the lesion prediction region mask and the lung parenchyma region as the lesion region prediction result.

[0114] In this embodiment of the invention, the aim is to predict lesion regions in chest CT images of patients with different immune subtypes. The core lies in the feature extraction layer, which uses a pre-defined segmentation network to output accurate lesion region prediction results. A pre-trained U-Net++ model is used as the basic segmentation model. Pre-trained on the LIDC-IDRI dataset, it outputs masks for lung parenchyma and lesions, providing a unified framework for image segmentation of different immune subtypes. The feature extraction layer, based on the immune subtype of the chest CT image sample, calls the corresponding image recognition target and uses the U-Net++ network to perform recognition, ultimately outputting a lesion prediction region mask and a lung parenchyma region, which are then used as the lesion region prediction results.

[0115] The core challenge of subtype A lies in the high incidence of chemotherapy-associated pneumonia (CIP). The inflammatory ground-glass opacity highly overlaps with the CT grayscale values ​​of the IPA "halo sign," leading traditional segmentation models to overestimate lesion area and affect diagnostic accuracy. Therefore, precise identification of the halo sign and inflammatory area is crucial; they should be used as image recognition targets. The core challenge of subtype B is the high risk of bleeding. The grayscale characteristics of the bleeding area highly overlap with IPA lesions, easily leading to missegmentation. Precise identification of the bleeding area is necessary to avoid misclassification as a lesion. Subtype C is characterized by chronic immunodeficiency, with lesions often presenting as "chronic cavitation," but it is prone to co-occurring with bullae, resulting in a high false-positive rate. Therefore, enhanced identification of the halo sign's edge is needed to reduce missegmentation of bullae.

[0116] Specifically: First, the immune subtype of the chest CT image sample is identified to prepare for subsequent image recognition targeting. Based on the identified immune subtype, the corresponding image recognition target is called, guiding the U-Net++ network's recognition process. Using the U-Net++ network, combined with the called image recognition target, the chest image sample is identified, outputting a lesion prediction region mask and the lung parenchyma region. The lesion prediction region mask and the lung parenchyma region are used as the final lesion region prediction result.

[0117] For example, the patient is subtype A, suffering from acute myeloid leukemia, and has a neutrophil count <0.5×10⁻⁶ after chemotherapy. 9 / L. Acquire chest CT image samples from the patient. For subtype A, use an image recognition target that accurately distinguishes between the halo sign and inflammatory areas. Input the chest CT image samples into a pre-trained U-Net++ network. During network training, optimize using a specific loss function, combining the halo sign and inflammatory area differentiation targets. After training and recognition, the U-Net++ network can accurately output a lesion prediction mask and lung parenchyma region. Use the obtained lesion prediction mask and lung parenchyma region as the lesion region prediction result for the patient's chest CT image sample.

[0118] It's important to note that the lung parenchyma mask is a binary image that marks the "left and right lung parenchyma regions" in CT images. The lung parenchyma is the core tissue for gas exchange in the lungs (mainly composed of alveoli and interstitium) and is the "only site of occurrence" for IPA lesions (lesions will not appear in areas outside the lung parenchyma such as the thoracic cage or mediastinum). The lung parenchyma mask is automatically generated by a pre-trained U-Net++ model. The marked "1" regions cover the entire area of ​​both lungs, while the "0" regions exclude non-lung tissues such as the thoracic skeleton, heart, major blood vessels, trachea, and background air. The lesion mask is a binary image specifically marking "suspected IPA lesions" in CT images. The marked areas must conform to the imaging characteristics of IPA (such as halo sign, air crescent sign, chronic cavitation, etc.) and are the "core basis" for subsequent extraction of lesion morphology, density, and edge features. Essentially, the lung parenchyma region and the lesion region have an inclusion relationship of "lung parenchyma (overall background) → lesion (local target)."

[0119] In one embodiment, a pre-defined segmentation network outputs the lesion region prediction results and corresponding image features of chest CT image samples, and extracts clinical features based on the clinical metadata corresponding to the chest CT image samples. Specifically, this includes: extracting lesion features of chest CT image samples based on the lesion region prediction results, and using the lesion features as image features; wherein, the image features include lesion morphology features, lesion density features, lesion edge features, and lung parenchyma background features; classifying and cleaning the clinical metadata to obtain clinical features; wherein, the clinical features include: patient body temperature, neutrophil count, immune status, infection markers, and bleeding risk indicators.

[0120] In this embodiment of the invention, the aim is to extract the standard deviation of the domain CT value, the difference in CT value between the halo sign and the lesion core, etc. from the chest CT image samples and the corresponding clinical metadata; (3) lesion edge features: including whether the edge is clear and smooth, whether there are spiculations, lobulation, etc., specific indicators such as the mean gradient of the halo sign edge (calculated based on the Sobel operator), the smoothness of the lesion edge (number of edge pixels / circumference of the lesion), these features have a certain indicative role in judging the benign or malignant nature of the lesion; (4) lung parenchyma background features: involving the overall state of the lung parenchyma, such as whether there are background changes such as emphysema and fibrosis, which are helpful in assessing the overall health status of the lungs and the background of the occurrence of lesions. Specific indicators include the total volume of lung parenchyma (mm³), the proportion of lesions to lung parenchyma volume (%), the volume of other abnormal areas in the lung (such as inflammation), etc. And process the clinical metadata corresponding to the chest CT image samples, mainly including two steps: classification and data cleaning, to obtain useful clinical features. Clinical features mainly include the following aspects: (1) Body temperature is an important indicator reflecting whether the patient has an inflammatory response such as infection. Different body temperature change patterns may be related to different disease states; (2) Neutrophil count indicators: Neutrophils are an important part of the human immune system. Changes in their count can reflect the patient's immune defense status and infection status; (3) Immune status indicators: such as immunoglobulin levels, T cell subsets, etc., are used to assess the patient's overall immune function and have important guiding significance for the diagnosis and treatment of immune-related diseases; (4) Infection marker indicators: such as C-reactive protein (CRP), procalcitonin (PCT), etc. These markers will show significant changes when infection occurs, which can help determine the presence and severity of infection. Bleeding risk indicators: such as platelet count, coagulation function indicators, etc., play an important role in assessing whether the patient has a bleeding tendency and the likelihood of bleeding.

[0121] For example, the patient is a 55-year-old male who presented with cough and fever. Clinically suspected lung infection was found, and a chest CT scan was performed, and relevant clinical metadata was collected. The image features extracted are as follows: (1) Lesion morphology features: The chest CT image was processed by a preset segmentation network to output the lesion area prediction results. The lesion boundary was accurately defined by the image segmentation algorithm. The lesion's long axis was measured to be 3.5 cm, the short axis to be 2.8 cm, and the perimeter to be approximately 10.2 cm. The lesion was approximately elliptical in shape. (2) Lesion density features: The density inside the lesion was measured. It was found that the density was uneven and there were some liquefied areas. By setting a density threshold, the lesion density was divided into high-density, isodense, and low-density areas, and the proportion of each area was recorded. (3) Lesion edge features: The lesion edge was observed. It was found that the edge had slight burrs. The edge roughness and other parameters were calculated by the edge detection algorithm to quantify the edge features. (4) Lung parenchyma background features: The overall state of the lung parenchyma was observed. It was found that there was mild emphysema. The relevant feature parameters were extracted by analyzing the texture and measuring the density of the lung parenchyma area. Based on the collected clinical metadata, the following clinical features were extracted: (1) Patient's body temperature: The patient's body temperature upon admission was 38.5℃, and the measurement time and trend of the body temperature were recorded; (2) Neutrophil count: The routine blood test showed that the neutrophil count was 12×10 9 / L, higher than the normal range, indicating possible infection; (3) Immune status indicators: the patient's immunoglobulin levels were measured, IgG was 12g / L, IgA was 2.5g / L, and IgM was 1.2g / L, all within the normal range, indicating that the patient's overall immune function is basically normal; (3) C-reactive protein (CRP) was 80mg / L and procalcitonin (PCT) was 0.5ng / ml, both significantly elevated, further supporting the diagnosis of infection; (4) Platelet count was 150×10 9 / L, with normal coagulation function indicators, suggests a low current risk of bleeding. The extracted imaging and clinical features will be comprehensively analyzed, combined with the patient's symptoms and other examination results, to determine subsequent diagnosis and treatment.

[0122] In one embodiment, after the feature extraction layer outputs the lesion region prediction result of the chest CT image sample based on a preset segmentation network, the method further includes: determining whether to use the lesion region prediction result as the output result of the feature extraction layer based on a preset segmentation quality evaluation index. Specifically:

[0123] The segmentation quality score of the predicted lesion region is calculated based on the segmentation quality expression; whereby the segmentation quality expression is:

[0124] ;

[0125] In the formula, To segment quality scores; The total number of non-zero pixels within the segmented region, where the segmented region is the lesion-associated region in the lung parenchyma. pixels of the lesion area The gradient value, the lesion area is the intersection or union of the actual lesion area and the predicted lesion area; The gradient value of the pixels in the lung parenchyma region is used. When the segmentation quality score is greater than the preset segmentation quality evaluation index, the lesion region prediction result is used as the output result of the feature extraction layer. Otherwise, the lesion region prediction result of the feature extraction layer is regenerated based on the preset multi-model recognition and voting mechanism.

[0126] In this embodiment of the invention, after the feature extraction layer obtains the lesion region prediction result of the chest CT image sample using a preset segmentation network, its segmentation quality needs to be evaluated to ensure the reliability and accuracy of the result. Based on the evaluation result, it is decided whether to directly use the prediction result as the output of the feature extraction layer or to regenerate the lesion region prediction result using a preset multi-model recognition and voting mechanism. Specifically: Automatic lung parenchyma segmentation is performed using a U-Net++ model pre-trained on the LIDC-IDRI dataset, outputting a lung parenchyma mask to exclude non-lung tissues such as the thoracic cage and mediastinum, thereby determining the lung parenchyma region. Within the lung parenchyma region, the actual lesion mask annotated by the doctor is denoted as... The predicted lesion mask output by the model is denoted as The intersection and union of these two areas constitute the lesion-related region. The segmented region refers to the "lesion-related region within the lung parenchyma," which includes the actual lesion, the predicted lesion, and the transitional zone between the two. This represents the total number of non-zero pixels within the region, ensuring that the segmentation quality assessment focuses on the core segmentation area. When the segmentation quality score is greater than the preset segmentation quality assessment index, it indicates that the current lesion region prediction result has good segmentation quality and can be used as the output result of the feature extraction layer. If the segmentation quality score is less than or equal to the preset segmentation quality assessment index, it indicates that the current prediction result has poor segmentation quality, and the lesion region prediction result of the feature extraction layer needs to be regenerated based on the preset multi-model recognition and voting mechanism.

[0127] For example, a 60-year-old female patient presented with cough and chest pain and underwent a chest CT scan. The patient's chest CT images were input into a U-Net++ model pre-trained on the LIDC-IDRI dataset. The model output a lung parenchyma mask, excluding non-lung tissues such as the thoracic cavity and mediastinum, to identify the lung parenchyma region. The doctor then labeled the lesions within the lung parenchyma region, obtaining... Simultaneously, the preset segmentation network outputs a predicted lesion mask. The intersection and union of the two lesions constitute the lesion-related region. Within the lung parenchyma, the region encompassing the actual lesion, the predicted lesion, and the transition zone between them is defined as the segmentation region, and the total number of non-zero pixels within this region is N=5000. The Sobel operator is used to calculate the gradient values ​​of pixels in the lesion region and pixels in the lung parenchyma region, and these values ​​are substituted into the segmentation quality expression, resulting in a segmentation quality score of 0.6. The preset segmentation quality evaluation index is 0.7, because... =0.6 < 0.7, indicating that the current lesion region prediction results are of poor segmentation quality. The preset multi-model identification and voting mechanism is activated to regenerate the lesion region prediction results.

[0128] In one embodiment, the lesion region prediction result of the feature extraction layer is regenerated based on a preset multi-model recognition and voting mechanism. Specifically, this includes: using N different networks, including the U-Net++ network, to identify lesion regions in chest CT image samples and outputting lesion prediction region masks; where N is an odd number; binarizing the lesion prediction region mask output by each network to obtain a binarization result; the binarization result is used to characterize the lesion region judgment result of each pixel in the chest CT image sample; statistically analyzing the binarization results of each pixel in the chest CT image sample, and determining the current pixel location as a lesion region when more than half of the binarization results determine that the current pixel is a lesion region; and regenerating the lesion region prediction result of the feature extraction layer based on the statistical results of each pixel.

[0129] In this embodiment of the invention, when the segmentation quality of the lesion region prediction result output by the preset segmentation network is substandard, a multi-model identification and voting mechanism is used to regenerate a more accurate lesion region prediction result, thereby improving the reliability of lesion region identification in chest CT images. N different networks, including the U-Net++ network, are used to identify lesion regions in chest CT image samples. Each network outputs a lesion prediction region mask, which reflects the prediction of lesion regions in the image by each model. The lesion prediction region mask output by each network is binarized, typically by setting a threshold (e.g., 0.5). Pixels with a probability greater than or equal to the threshold are recorded as 1 (representing a lesion), and pixels with a probability less than the threshold are recorded as 0 (representing a non-lesion). Simultaneously, pixels with an area smaller than 5 mm are removed. 2 To minimize noise interference and improve the accuracy of subsequent voting results, a small area is selected. Based on the statistical results of each pixel, the final lesion region prediction result is generated, which is the final lesion mask.

[0130] For example, a 55-year-old male patient presented with dyspnea and underwent a chest CT scan. A pre-defined segmentation network outputs the predicted lesion regions from the patient's chest CT images, and a segmentation quality score is calculated. =0.5, because ≤0.6, activate multi-model recognition and voting mechanism. Input the patient's chest CT images into three independent pre-trained models: U-Net++, Swin-UNET, and nnU-Net. Each model outputs a lesion prediction region mask. ,right Binarization is performed with a threshold of 0.5. Pixels with a probability greater than or equal to 0.5 are recorded as 1, and pixels with a probability less than 0.5 are recorded as 0. Pixels with an area less than 5mm in each binarized mask are removed. 2 A tiny region. Taking a single pixel (x, y) in an image as an example, statistically... The annotation result for this pixel. Assume... This pixel is marked as 1. If a pixel is labeled as 0, and two of the masks are labeled as 1, exceeding half (when N=3, half is 1.5, rounded up to 2), then that pixel is ultimately identified as a lesion. This statistical analysis and identification process is performed on every pixel in the image. Based on the identification results of each pixel, the final lesion region prediction result is generated and the final lesion mask is output. This mask can be used for subsequent tasks such as feature extraction and disease diagnosis.

[0131] For calculating the image time-series features and clinical time-series features of the training samples within a preset time length, the image features, clinical features, and corresponding image time-series features and clinical time-series features are used as inputs so that the feature fusion layer can fuse the received features and output the primary fused features.

[0132] In this embodiment of the invention, feature calculation and fusion operations are performed around the training samples. Imaging features and clinical features reflect the state of the research subjects from different perspectives. Imaging features can intuitively present information such as body structure and lesion morphology; clinical features include symptoms, examination indicators, and other content. Furthermore, imaging temporal features and clinical temporal features further supplement dynamic information that changes over time. By fusing these four types of features, the training samples can be comprehensively and holistically described, avoiding the limitations of single feature information and providing a more complete and accurate basis for subsequent analysis and decision-making.

[0133] In one embodiment, the image temporal features and clinical temporal features of the training sample within a preset time period are calculated. Using image features, clinical features, and their corresponding image temporal features and clinical temporal features as input, the feature fusion layer fuses the received features and outputs a primary fusion feature. Specifically, this includes: extracting image features from chest CT image samples at preset intervals within the same training sample; combining the image features output by the feature extraction layer; calculating the rate of change between two adjacent extracted image features and generating image temporal features; extracting data from various clinical indicators in clinical metadata at preset time intervals within the same training sample; combining the clinical features output by the feature extraction layer; calculating the rate of change between two adjacent extracted clinical features and generating clinical temporal features; and concatenating the image temporal features and clinical temporal features to obtain the primary fusion feature.

[0134] In this embodiment of the invention, image temporal features and clinical temporal features of training samples within a preset time period are calculated, and these features are fused with the original image features and clinical features to generate primary fused features. This captures the dynamic changes of the training samples over time, providing a more comprehensive and accurate basis for subsequent analysis and decision-making. The image temporal features and clinical temporal features are encoded. For example, for image temporal features, temporal features from multiple CT scans within 24-48 hours (such as lesion volume change rate and halo sign density change) can be extracted, outputting a fixed-dimensional (e.g., 256-dimensional) "temporal image feature"; for clinical temporal features, a Long Short-Term Memory (LSTM) network can be used to extract the changing trends of clinical indicators (body temperature, neutrophil count) within 72 hours, outputting a fixed-dimensional (e.g., 128-dimensional) "temporal clinical feature". Within the same training sample, image features of chest CT image samples are extracted at preset intervals. Combining the image features output by the feature extraction layer, the rate of change between two adjacent extracted image features is calculated, thereby generating the image temporal features. The core of this step is to capture the dynamic changes in chest CT images over time, such as changes in lesion volume and density. Similarly, within the same training sample, data on various clinical indicators are extracted from clinical metadata at preset time intervals. Combining the clinical features output from the feature extraction layer, the rate of change between two adjacent extracted clinical features is calculated, thereby generating clinical temporal features. This helps capture the changing trends of a patient's clinical state over time, such as fluctuations in body temperature and changes in blood parameters. The generated image temporal features and clinical temporal features are then concatenated to obtain primary fusion features. This fusion method integrates dynamic information from both imaging and clinical aspects, providing richer information for subsequent analysis.

[0135] For example, a 60-year-old male patient was admitted to the hospital due to cough and fever. The doctor suspected lung disease and arranged multiple chest CT scans and clinical indicator tests. Within 72 hours of admission, a chest CT scan was performed every 12 hours, and the patient's clinical indicators (such as body temperature, white blood cell count, neutrophil count, etc.) were recorded. Imaging features, such as lesion volume and density, were extracted from the chest CT image samples every 12 hours.

[0136] By combining the image features output by the feature extraction layer, the rate of change between two adjacent extracted image features is calculated. For example, the rate of change of lesion volume is calculated using the following formula:

[0137] ;

[0138] The temporal features of the images are encoded, outputting 256-dimensional temporal image features. Simultaneously, data for various clinical indicators are extracted from clinical metadata every 12 hours. Combining the clinical features output from the feature extraction layer, the rate of change between two extracted clinical features is calculated. For example, the rate of change in body temperature is calculated using the following formula:

[0139] ;

[0140] A Long Short-Term Memory (LSTM) network was used to encode the clinical temporal features, outputting 128-dimensional temporal clinical features. The 256-dimensional "temporal image features" and the 128-dimensional "temporal clinical features" were then concatenated to obtain a primary fusion feature. Doctors can use this complete information, including both dynamic and static data, to more accurately determine whether a patient has lung disease, as well as the severity and progression of the disease, avoiding misdiagnosis based solely on static information.

[0141] In one embodiment, before stitching together the image temporal features and clinical temporal features, the method further includes:

[0142] The image features corresponding to each chest CT image sample are weighted according to the time distance between the acquisition time and the latest acquisition time.

[0143] Attention weights within the chest CT image modality are calculated based on the weighted image features; the expression for the attention weights is:

[0144] ;

[0145] In the formula, Attention weights within the chest CT image modality; For the first Image feature vectors at each time point; No. Image feature vectors at each time point, and ≠ ; This is the cosine similarity function, used to calculate the similarity between two image feature vectors; This is the key weight matrix for attention. This represents the total dimension of the image features.

[0146] In this embodiment of the invention, before stitching together the temporal features of the images and the clinical temporal features, the image features are weighted and the attention weights within the chest CT image modality are calculated, thereby highlighting key information and reducing redundancy. The image features corresponding to each chest CT image sample are weighted according to the time distance between the acquisition time of the chest CT image sample and the latest acquisition time. This is based on the premise that recent image data may contain more key information about the current state and trend of the disease, so recent image features are given higher weights. For example, in the intramodal fusion, the weight of recent CT is assigned 0.7, and the weight of distant CT is assigned 0.3, thus highlighting "recent dynamics". The attention weights within the chest CT image modality are calculated based on the weighted image features. is the attention weight within the chest CT image modality, representing the correlation strength between the image features at time point i and at time point j (range [0,1]). A higher value indicates a stronger consistency between the two in the trend of lesion change. The cosine similarity function calculates the cosine of the angle between two feature vectors (with values ​​[-1, 1]) to measure feature similarity. The attention key weight matrix (256×256 dimensions) is learned through model training and is used to enhance the similarity calculation of the "lesion-related feature dimension". The initial weight of the core feature dimension is set to 1.5, and the weight of the non-core dimension is set to 0.5.

[0147] For example, a 55-year-old female patient was admitted to the hospital for treatment of a lung infection. Within 48 hours of admission, a chest CT scan was performed every 12 hours, and relevant clinical indicators were recorded. Assuming the latest data acquisition time is 48 hours, the four chest CT scans were performed at 0, 12, 24, 36, and 48 hours, respectively. The images were weighted according to the principle of 0.7 for recent CT scans and 0.3 for later CT scans. The weight of the imaging features at 48 hours was 0.7, at 36 hours was 0.6, at 24 hours was 0.4, at 12 hours was 0.3, and at 0 hours was 0.3. The image feature vector at hour 48. These are the image feature vectors for hours 0, 12, 24, and 36, respectively. The cosine similarity function is used. and The similarity between (j=0,12,24,36) is then compared with the attention key weight matrix. Multiply and perform exponentiation, then finally calculate based on the expression for attention weights. (j=0,12,24,36). For clinical indicators, such as body temperature and white blood cell count, determine their trend. If two or more consecutive measurements show a downward trend, the weight of that indicator is 1.2; if two consecutive measurements show a fluctuation of ≤10%, the weight of that indicator is 0.8. For example, a patient's white blood cell count at 24 hours is 12 × 10⁻⁶. 9 / L, 10×10 at the 36th hour 9 / L, 8×10 at the 48th hour 9 If the white blood cell count ( / L) shows a decreasing trend, then the weight of this indicator is 1.2. The weighted and attention-weighted image features and clinical features are then stitched together to obtain primary fusion features, which are used for subsequent disease diagnosis and analysis. This method highlights key information in the image and clinical data, reduces redundancy, and improves diagnostic accuracy.

[0148] In one embodiment, image temporal features and clinical temporal features are concatenated to obtain primary fusion features, specifically including:

[0149] The image temporal features and the clinical temporal features are mapped to the same dimension using a cross-modal mapping matrix; wherein, the expression for the cross-modal attention weight is:

[0150] ;

[0151] In the formula, Cross-modal attention weights; The first of the image temporal features Dimension transpose; This is a cross-modal mapping matrix; This is the rule weight matrix, which retrieves the corresponding values ​​based on the immune subtype to which the training samples belong. For the first One clinical time-series feature vector; This is a bias term, initially set to 0.1, used to adjust the baseline value of the cross-modal mapping; The total dimension of clinical characteristics; For the first One clinical time-series feature vector; Confidence level of chest CT image samples; Confidence level of clinical data;

[0152] in, = ; ;

[0153] In the formula, Signal-to-noise ratio of chest CT images; and These represent the minimum and maximum signal-to-noise ratios of the image acquisition device. To segment quality scores; For data integrity; For measurement error; This represents the maximum permissible measurement error.

[0154] The primary fusion feature is obtained by concatenating the image time-series features and clinical time-series features mapped to the same dimension.

[0155] In this embodiment of the invention, the aim is to effectively concatenate image temporal features and clinical temporal features to obtain primary fusion features, thereby achieving cross-modal information synergy. A cross-modal mapping matrix maps 256-dimensional image temporal features and 128-dimensional clinical temporal features to the same dimensional space, enabling cross-modal similarity calculation and resolving the problem of dimensional mismatch between different modal features. Specifically, the rule weight matrix... The possible values ​​are shown in the table below:

[0156]

[0157] By concatenating the image time-series features and clinical time-series features mapped to the same dimension, a primary fusion feature is obtained.

[0158] For example, a 60-year-old male patient has been diagnosed with a lung disease and requires fusion analysis of imaging and clinical data to aid in diagnosis. The patient underwent a chest CT scan, yielding imaging data. The signal-to-noise ratio of this CT image was calculated. =25, Segmentation Quality Rating =0.8. (Given) =10, =40. Multiple clinical indicators of the patient were collected. Data integrity Comp=0.9, measurement error Err=5, and maximum permissible measurement error. =10. The patient's immune subtype is type B. The confidence level of the CT images was calculated based on the above data. =0.62, confidence level of clinical data =0.45. Assume... =1, =1, The first-dimensional transpose of the image temporal features, and the regular weight matrix. The value of (1,1) is 1.5, and the cross-modal mapping moment is... For a trained 256×128 dimensional matrix, the bias term The value is 0.1. Based on the above data, the cross-modal attention weight is obtained, and the image temporal features and clinical temporal features are spliced ​​together to obtain the primary fusion features.

[0159] For inputting primary fusion features, the diagnostic inference layer maps the weighted primary fusion features to IPA disease prediction probabilities based on a fully connected layer network, and uses the IPA disease prediction probabilities as the output of the initial neural network model.

[0160] In this embodiment of the invention, the primary fusion features are obtained and then passed as input to the diagnostic inference layer. The diagnostic inference layer processes the weighted primary fusion features using a fully connected layer network, mapping them to a predicted probability of IPA (Idiopathic Pulmonary Fibrosis Acute Exacerbation). The fully connected layer network possesses powerful nonlinear mapping capabilities, enabling complex transformations and combinations of the weighted primary fusion features. It can learn the complex relationships between features and uncover potential patterns in the data, thereby more accurately mapping features to IPA predicted probabilities and improving the model's predictive performance.

[0161] Specifically, a structure of "two-layer fully connected network + Sigmoid activation" is adopted. The two-layer fully connected network can perform non-linear transformation and combination on the 256-dimensional fused features of the input, and explore the complex relationships between features. The Sigmoid activation function maps the network output to the range of 0-1, that is, the IPA probability range of 0-100%, which conforms to the probability value range.

[0162] As a preferred embodiment of the present invention, the method further includes: when generating the output of the IPA identification model, classifying the severity of the disease based on the predicted probability of IPA and invoking the corresponding clinical recommendations. For example, a probability ≥ 0.8 indicates a high suspicion level, and immediate treatment is recommended; a probability < 0.3 indicates a moderate suspicion level, and a follow-up CT scan within 48 hours is recommended; a probability ≤ 0.3 indicates a low suspicion level, and microbiological testing is recommended. This classification method provides doctors with clear decision-making references, helping to take appropriate medical measures in a timely manner.

[0163] As another embodiment of the present invention, it further includes: calculating the contribution degree of each feature according to the feature contribution formula when generating the output of the IPA recognition model; wherein, the expression of the feature contribution formula is:

[0164] ;

[0165] In the formula, Used to characterize the marginal contribution of a feature to the prediction result (positive values ​​increase the probability of IPA, negative values ​​decrease the probability of IPA, used to solve the problem of "contribution cannot be quantified", so that doctors can clearly understand the specific impact of each feature on the prediction result). The clinical importance coefficient is set based on the EORTC / MSG guidelines (e.g., GM positive = 1.2, body temperature > 38.5℃ = 0.8, to ensure that the contribution ranking is consistent with clinical understanding and makes it easier for doctors to understand and accept the diagnostic basis of the algorithm). The confidence level of the modality to which the feature belongs (if the image feature is taken as...) Clinical features This is used to avoid over-interpreting low-quality characteristics. It should be noted that "GM" in GM-positive refers to galactomannan, a polysaccharide antigen unique to the cell wall of Aspergillus. A positive GM result usually indicates the possible presence of Aspergillus infection in the body.

[0166] A CT heatmap is generated based on the contribution level of each feature; features whose contribution level exceeds a preset threshold are highlighted in the CT heatmap.

[0167] For example, a 55-year-old male patient was admitted to the hospital due to cough and fever. A chest CT scan and multiple clinical indicators were performed, yielding temporal features of the imaging and clinical data. These were fused to obtain a 256-dimensional primary fusion feature. This 256-dimensional fusion feature was then input into a diagnostic inference layer consisting of a two-layer fully connected network and Sigmoid activation. After network calculation and mapping using the Sigmoid activation function, the patient's predicted probability of IPA was 0.85. Since the probability is ≥0.8, it is considered a high-suspected case, and immediate treatment is recommended. Assuming the existence of several key features, such as pulmonary consolidation in the CT images (feature f1), positive GM in the clinical indicators (feature f2), and body temperature >38.5℃ (feature f3), the SHAP algorithm was used to calculate... SHAPf3= 0.1. Obtained according to the EORTC / MSG guidelines. Furthermore, obtain the confidence scores for each modality, given: ,but The contribution levels of each feature are calculated as follows:

[0168]

[0169] The feature contribution table and CT heatmap are output to the results output layer. In the CT heatmap, the area corresponding to lung consolidation features is due to... Regions with a feature Contrib(f) value of 0.21 < 0.6 are not highlighted, while other regions with a Contrib(f) value > 0.6 are highlighted. Doctors can intuitively understand, through the feature contribution table and CT heatmap, that a positive GM result plays an important positive role in the diagnosis of IPA, while a body temperature > 38.5℃ reduces the probability of IPA to some extent, thus increasing their trust in and acceptance of the algorithm's diagnostic results.

[0170] In one embodiment, the diagnostic inference layer further includes weighting based on the immune subtype corresponding to the training sample containing the primary fusion feature, specifically:

[0171] Multi-dimensional weighted cross-entropy is calculated based on the immune subtype corresponding to the training sample containing the primary fusion feature; the expression for the multi-dimensional weighted cross-entropy is:

[0172] ;

[0173] In the formula, Weighted cross-entropy; For training sample index; Weights corresponding to immune subtypes; For training sample class weights; For the first The true IPA labels of each training sample; For the first IPA disease prediction probability for each training sample.

[0174] This invention utilizes the 256-dimensional high-quality fusion features output from the fusion layer to output the probability of IPA based on these features, while simultaneously addressing the issues of sample imbalance and uninterpretable diagnosis, providing accurate and reliable diagnostic results for clinical decision-making. Specifically, in the diagnostic inference layer, a multi-dimensional weighted cross-entropy loss function is used to optimize model training, improving diagnostic accuracy for different immune subtypes and sample categories. Traditional cross-entropy loss functions assign the same weight to all samples, which has significant drawbacks in handling sample imbalance. In IPA diagnosis, there are subtype sample imbalances (e.g., fewer rare subtype samples, easily influenced by dominant subtype features) and positive / negative sample imbalances (low percentage of IPA-positive samples). The multi-dimensional weighted cross-entropy loss function addresses these issues by introducing weights corresponding to immune subtypes. and training sample class weights Differential weighting of different samples can solve these problems. Weighted cross-entropy is used to measure the difference between the model's prediction and the true label, and it is the objective function that needs to be minimized during model training. The weights for different immune subtypes are used to address the subtype sample imbalance problem. Different immune subtypes (such as subtypes A, B, and C) have different weights; for example, subtype A = 1.2, subtype B = 1.1, and subtype C = 1.5. Rare subtypes have relatively higher weights to increase the model's attention to rare subtype samples. For the first The true IPA label for each training sample is a binary variable, where 1 indicates a confirmed IPA and 0 indicates an excluded IPA, determined by the clinical gold standard.

[0175] For example, suppose there are 1000 training samples (K=1000), containing samples of different immune subtypes (A, B, C), and the proportion of IPA-positive samples is 30%. The immune subtype distribution of the samples is as follows: subtype A has 500 samples, subtype B has 300 samples, and subtype C has 200 samples. Immune subtype weights: (Subtype A) = 1.2, Subtype B) = 1.1, (Subtype C) = 1.5; Training sample class weights: (positive sample, =1) =1.3, (negative sample) =0) =0.8. Assuming the first sample belongs to immune subtype A, its true label is... =1 (Confirmed IPA), the model-predicted probability of IPA. If the value is 0.8, then the contribution of this sample to the weighted cross-entropy is:

[0176] .

[0177] Assuming the second sample belongs to immune subtype C, its true label =0 (excluding IPA), the model predicts the probability of IPA prevalence. If the value is 0.2, then the contribution of this sample to the weighted cross-entropy is: .

[0178] Similarly, calculate the contribution of all 1000 samples to the weighted cross-entropy, then sum them and divide by the total number of samples K=1000 to obtain the weighted cross-entropy. During model training, the model parameters are continuously adjusted to minimize... This improves the model's diagnostic accuracy for different immune subtypes and sample categories.

[0179] The segmentation loss function is pre-established based on the lesion characteristics of different immune subtypes. The segmentation loss function is calculated based on the immune subtype to which the training sample belongs, the corresponding lesion region prediction results, and the IPA disease prediction probability. Training ends when both the segmentation loss function and the preset constraints reach the preset values, thus obtaining the IPA recognition model.

[0180] In this embodiment of the invention, a corresponding segmentation loss function is constructed based on the lesion characteristics of different immune subtypes. Then, during training, the segmentation loss function is calculated based on the immune subtype of the training sample, the predicted lesion region, and the predicted probability of IPA disease. The lesion characteristics of different immune subtypes often differ. By customizing a segmentation loss function for each immune subtype, the model can more accurately learn the unique characteristics of each subtype, improving the ability to identify and distinguish IPA lesions under different immune subtypes and avoiding identification errors caused by general models ignoring subtype differences. Pre-defined constraints are introduced to constrain model training, ensuring that the model optimizes in a more expected direction during training, preventing overfitting or getting trapped in local optima, thereby improving the model's generalization ability and stability.

[0181] In one embodiment, a segmentation loss function is pre-established based on the lesion characteristics of different immune subtypes. The segmentation loss function is calculated based on the immune subtype to which the training sample belongs, the corresponding lesion region prediction results, and the IPA disease prediction probability. Training ends when both the segmentation loss function and the preset constraints reach preset values, resulting in an IPA identification model, specifically including:

[0182] A base loss function is pre-defined; the expression for the base loss function is:

[0183] ;

[0184] In the formula, Masking the actual lesion area; For lesion prediction region mask;

[0185] A halo sign edge gradient constraint term is established for the segmentation loss function, and the halo sign and inflammation are identified based on this term; the expression for the halo sign edge gradient constraint term is:

[0186] ;

[0187] In the formula, The gradient constraint value at the edge of the halo sign; The area of ​​the halo sign; For the edge pixels of the halo feature The gradient value; Iinflam(x,y) is the edge pixel of the inflammatory region. The gradient value;

[0188] The first segmentation loss function corresponding to the first immune subtype is established based on the base loss function and the halo sign edge gradient constraint term; the expression of the first segmentation loss function is as follows:

[0189] ;

[0190] In the formula, The first segmentation loss function corresponds to the first immune subtype; The basic loss function;

[0191] Based on the basic loss function and the lesion characteristics of different immune subtypes, a second segmentation loss function and a third segmentation loss function are established; wherein, the expression of the second segmentation loss function is:

[0192] ;

[0193] In the formula, This is the second segmentation loss function corresponding to the second immune subtype; The area of ​​the bleeding region; Mask the bleeding area; To predict the probability of IPA disease;

[0194] The expression for the third segmentation loss function is:

[0195] ;

[0196] In the formula: The third segmentation loss function corresponds to the third immune subtype; Area of ​​the chronically cavitary region; The local binary mode value of the hole wall pixel; The texture weight is determined based on the local binary mode value of the hole wall pixel;

[0197] When the segmentation loss function and the halo sign edge gradient constraint term corresponding to different immune subtypes all reach the preset value, the iterative training ends and the IPA recognition model is obtained.

[0198] In this embodiment of the invention, the basic loss function is used and lesion preset area mask The degree of overlap between them. Manual annotation based on CT images by two or more senior radiologists, and passed Dice consistency verification (consistency ≥ 0.85 is considered valid annotation; lesion pre-set area mask). The feature extraction layer outputs directly from the pre-trained U-Net++ model. The halo sign edge gradient constraint term identifies the halo sign and inflammation through "edge gradient difference," rather than relying solely on grayscale values.

[0199] CT image data of 100 cases with different immune subtypes were collected. Among them, there were 30 cases of subtype A, 30 cases of subtype B, and 40 cases of subtype C. The CT images were manually annotated by at least two senior radiologists to obtain masks of the actual lesion areas. Bleeding area mask And a mask for chronic cavity regions. A pre-trained U-Net++ model is used in the feature extraction layer to perform preliminary segmentation of the CT images, obtaining the mask for the predicted lesion region. Calculate the basic loss function DiceLoss. For subtype A, calculate the halo sign edge gradient constraint term. Then, the segmentation loss function corresponding to several immune subtypes is calculated separately. The model parameters are continuously adjusted to achieve the desired effect. , , and All reached the preset values.

[0200] In a preferred embodiment of the present invention, for subtype A patients with two CT scans, the volume change rate ΔV is calculated. If ΔV > 30%, a multi-scale segmentation retry is triggered, and the downsampling scale is changed from 4x to 2x. By reducing the downsampling scale, the model can capture more subtle lesion edge details, and then the lesion volume is recalculated. If the volume change rate ΔV ≤ 30% after the retry, the segmentation result after the retry is adopted; if it is still > 30%, further verification is performed in conjunction with clinical symptoms (such as body temperature, inflammatory markers, etc.).

[0201] As another optimization scheme of this invention embodiment, after obtaining the IPA recognition model, the method further includes parameter optimization of the IPA recognition model, specifically:

[0202] Each time the IPA recognition model completes a preset number of recognition tasks, the current loss gradient of the IPA recognition model is calculated.

[0203] The parameters of the IPA recognition model are optimized based on the current loss gradient; the expression for parameter optimization is:

[0204] ;

[0205] In the formula, These are the updated model parameters; These are the current model parameters; The learning rate; This is the current loss gradient; =0.9, which is the momentum coefficient used to retain historical update directions; These are the parameters for the historical model.

[0206] In this embodiment of the invention, after obtaining the IPA recognition model, parameter optimization is required to ensure the model can adapt to various situations and maintain good performance. The parameter optimization expression combines the current model parameters, learning rate, current loss gradient, momentum coefficient, and historical model parameters, with the momentum coefficient retaining the historical update direction. This allows for dynamic adjustment of parameters based on the model's performance in actual recognition tasks, enabling the model to continuously learn and improve, avoid getting trapped in local optima, accelerate model convergence, and improve model stability and generalization ability, thereby enhancing the recognition accuracy and performance of the IPA recognition model. Specifically, whenever the model completes a preset number of recognition tasks, the current loss gradient is calculated, and the model parameters are updated based on this gradient. The parameter optimization formula combines the current loss gradient and historical update direction, enabling more effective adjustment of model parameters, avoiding the model from getting trapped in local optima, and making model training more stable. Since performance degradation due to the emergence of new subtypes and changes in data distribution occurs when the model does not iterate, continuous iterative improvement of the model is achieved through clinical validation and incremental learning. The model is connected to the hospital's LIS and EMR systems to collect three types of key data. Gold standard data (bronchoscopy biopsy results) provides the most reliable basis for diagnostic accuracy; treatment response data (CT lesion changes 72 hours after drug treatment) can verify diagnostic accuracy; misdiagnosis annotation data (causes of misdiagnosis annotated by clinicians) can guide module optimization. Traditional full retraining methods are time-consuming and resource-intensive, while incremental learning only updates key module parameters while retaining historical knowledge, avoiding "catastrophic forgetting." Furthermore, model parameters are updated using specific parameter optimization formulas, and weighted F1 scores are used to evaluate model performance. The expression for the weighted F1 score is:

[0207] ;

[0208] In the formula, The subtype weights (e.g., A=0.4, B=0.3, C=0.3) can be set according to the clinical incidence rate; For subtype s, the accuracy rate; Let be the recall rate for subtype s.

[0209] Traditional overall accuracy ignores subtype differences, while weighted F1 scores take into account both precision and recall for each subtype. Whether to save updated parameters is determined by the improvement in the weighted F1 score; if there is no improvement, the parameters are rolled back and the learning rate is adjusted for re-optimization. Parameters for different modules can also be updated specifically based on the cause of misdiagnosis.

[0210] For example, the system interfaces with the hospital's LIS and EMR systems. Data is collected weekly, covering information on all suspected IPA patients within the hospital. Within one month, data from 200 patients was collected, including bronchoscopic biopsy results, CT changes in lesions 72 hours after drug treatment, and reasons for misdiagnosis annotated by clinicians. The collected data was reviewed and cleaned to remove duplicates, errors, or incomplete data, ultimately yielding 180 valid data points. The IPA identification model calculates the current loss gradient every 50 identifications completed. Assume the current model parameters are... The historical model parameters are Learning rate = momentum coefficient =0.9, and the updated model parameters are calculated based on the current loss gradient and the parameter optimization expression. Further, incremental learning is performed using 180 collected clinical validation cases. The data is divided into three batches of 60 cases each. After each batch is completed, the weighted F1 score is calculated. Assuming that the weighted F1 score improves by 2% after the first batch, which does not meet the 3% improvement standard, the model is rolled back to the current parameters. And adjust the learning rate to = Then, optimize again. After the second batch of learning, if the weighted F1 score improves by 4% and meets the standard, save and update the parameters.

[0211] In this embodiment of the invention, an IPA identification model construction device is also provided, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the above-described IPA identification model construction method.

[0212] In this embodiment of the invention, a computer-readable storage medium is also provided, which includes a stored computer program, wherein the computer program controls the device where the computer-readable storage medium is located to execute the above-described method for constructing the IPA identification model when it is running.

[0213] For example, a computer program can be divided into one or more modules, one or more of which are stored in memory and executed by a processor to carry out the present invention. The one or more modules can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in an IPA identification model building device.

[0214] The device for building the IPA identification model can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The device for building the IPA identification model may include, but is not limited to, processors, memory, and displays. Those skilled in the art will understand that the above components are merely examples of devices for building the IPA identification model and do not constitute a limitation on the device. The device may include more or fewer components, or a combination of certain components, or different components. For example, the device for building the IPA identification model may also include input / output devices, network access devices, buses, etc.

[0215] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the IPA identification model building device, connecting various parts of the device through various interfaces and lines.

[0216] The memory can be used to store computer programs and / or modules. The processor, by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory, realizes various functions of the device for building the IPA recognition model. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function (such as sound playback function, text conversion function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, text message data, etc.). In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0217] If the module based on the IPA identification model is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. Those skilled in the art can understand and implement this without any inventive effort.

[0218] This invention provides a method for constructing an IPA (Invasive Pertussis Awareness) identification model. A training sample set is constructed using clinical metadata and chest CT image samples from patients with different immune subtypes. This allows the model to learn the characteristics of different immune subtypes, enhancing its ability to identify various conditions. Secondly, the feature extraction layer, feature fusion layer, and diagnostic inference layer of the initial neural network model work collaboratively to effectively extract and fuse image and clinical features, including temporal features, comprehensively and deeply mining data information. Furthermore, iterative training using chest CT image samples acquired at the latest time allows the model to keep up with the latest data features. Finally, a segmentation loss function is established based on the lesion characteristics of different immune subtypes, and training is performed using preset constraints to ensure the accuracy and stability of model training. This results in a high-performance model capable of accurately identifying IPA, improving the timeliness and accuracy of identifying invasive pulmonary aspergillosis.

[0219] Example 2

[0220] See Figure 2 , Figure 2 This is a flowchart illustrating an IPA identification method provided in one embodiment of the present invention. The embodiment of the present invention provides an IPA identification method, including:

[0221] Step 201: Collect clinical metadata and chest CT image sample data of the patient within a preset time range from the current time.

[0222] In this embodiment of the invention, by collecting clinical metadata (such as basic patient information, symptoms, and test results) and chest CT image sample data of patients within a specific time range, necessary data support is provided for subsequent IPA identification. Furthermore, a predetermined time length is defined, for example, it can be set to the past three days, ensuring the timeliness of the collected data and better reflecting the patient's current condition. Standards and specifications for data collection are also established to ensure that the collected clinical metadata is accurate and complete, and that the chest CT image sample data is clear and meets diagnostic requirements. For example, for clinical metadata, the accuracy of all indicators must be ensured; for chest CT image sample data, the consistency of scanning parameters must be guaranteed.

[0223] Step 202: Input clinical metadata and chest CT image sample data into the IPA recognition model constructed using the IPA recognition model construction method described in Example 1, and output the patient's IPA disease prediction probability.

[0224] In this embodiment of the invention, clinical metadata and chest CT image sample data are processed according to the data preprocessing method used during model training in Example 1, and then input into the IPA recognition model constructed using the construction method described in Example 1. Using the previously constructed IPA recognition model, the collected patient data is analyzed and processed. The model comprehensively considers the characteristics of clinical metadata and chest CT image sample data, and outputs the predicted probability of the patient having IPA.

[0225] Step 203: Feed back the predicted probability of IPA to the user terminal so that the user terminal can generate a corresponding treatment recommendation plan based on the predicted probability of IPA.

[0226] In this embodiment of the invention, the IPA disease prediction probability output by the model is fed back to the user terminal (such as the doctor's computer terminal, the patient's mobile APP, etc.). Based on this probability value, the user terminal generates the corresponding IPA identification result (such as confirmed, suspected, excluded, etc.) and treatment recommendation plan (such as medication suggestions, further examination suggestions, etc.).

[0227] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the identification method described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0228] This invention provides an IPA (immune leukemia) identification method. Through an automated IPA identification model, it can quickly process patient clinical metadata and chest CT image sample data, outputting a disease prediction probability in a short time, significantly shortening diagnosis time and improving medical efficiency. The model is trained based on data from a large number of patients with different immune subtypes and has undergone parameter optimization and clinical validation, enabling more accurate IPA identification, reducing misdiagnosis and missed diagnosis, and providing patients with more reliable diagnostic results.

[0229] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several improvements and substitutions without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A method for constructing an IPA identification model, characterized in that, include: The clinical metadata and chest CT image samples of a patient within a preset time period are used as a training sample. Patients with different immune subtypes are used as data collection objects, and a training sample set is constructed based on the collected clinical metadata and chest CT image samples. The training sample set is input into a pre-constructed initial neural network model for iterative training; wherein the initial neural network model includes a feature extraction layer, a feature fusion layer, and a diagnostic inference layer; When iteratively training the initial neural network, the chest CT image sample acquired at the latest time in each training sample is used as input to iteratively train the feature extraction layer, so that the feature extraction layer outputs the lesion area prediction result and corresponding image features of the chest CT image sample based on the preset segmentation network, and extracts clinical features according to the clinical metadata corresponding to the chest CT image sample. The image temporal features and clinical temporal features of the training samples within the preset time length are calculated. The image features, clinical features and corresponding image temporal features and clinical temporal features are used as inputs so that the feature fusion layer fuses the received features and outputs the primary fused features. Using the primary fusion features as input, the diagnostic inference layer maps the weighted primary fusion features to IPA disease prediction probability based on a fully connected layer network, and uses the IPA disease prediction probability as the output of the initial neural network model. A segmentation loss function is pre-established based on the lesion characteristics of different immune subtypes. The segmentation loss function is calculated based on the immune subtype to which the training sample belongs, the corresponding lesion region prediction results, and the IPA disease prediction probability. Training ends when the segmentation loss function and the preset constraint terms both reach preset values, and an IPA recognition model is obtained. The process involves pre-establishing corresponding segmentation loss functions based on the lesion characteristics of different immune subtypes, calculating the segmentation loss function based on the immune subtype to which the training sample belongs, the corresponding lesion region prediction results, and the IPA disease prediction probability, and ending training when both the segmentation loss function and the preset constraints reach preset values, thus obtaining the IPA identification model. Specifically, this includes: A basic loss function is pre-established; wherein the expression of the basic loss function is: ； In the formula, Masking the actual lesion area; For lesion prediction region mask; A halo sign edge gradient constraint term is established for the segmentation loss function, and the halo sign and inflammation are identified based on the halo sign edge gradient constraint term; wherein, the expression of the halo sign edge gradient constraint term is: ； In the formula, The gradient constraint value at the edge of the halo sign; The area of ​​the halo sign; For the edge pixels of the halo feature The gradient value; I inflam (x, y) represents the pixel edge of the inflammatory region. The gradient value; A first segmentation loss function corresponding to the first immune subtype is established based on the basic loss function and the halo sign edge gradient constraint term; wherein, the expression of the first segmentation loss function is: ； In the formula, The first segmentation loss function corresponds to the first immune subtype; The basic loss function; Based on the aforementioned basic loss function and the lesion characteristics of different immune subtypes, a second segmentation loss function and a third segmentation loss function are established; wherein, the expression for the second segmentation loss function is: ； In the formula, This is the second segmentation loss function corresponding to the second immune subtype; The area of ​​the bleeding region; Mask the bleeding area; To predict the probability of IPA disease; The expression for the third segmentation loss function is: ； In the formula: This is the third segmentation loss function corresponding to the third immune subtype; Area of ​​the chronically cavitary region; The local binary mode value of the hole wall pixel; The texture weight is determined based on the local binary mode value of the hole wall pixel; When the segmentation loss function corresponding to different immune subtypes and the halo sign edge gradient constraint term both reach the preset value, the iterative training ends and the IPA recognition model is obtained.

2. The method for constructing the IPA identification model as described in claim 1, characterized in that, The data collection targets patients with different immune subtypes, and a training sample set is constructed based on the collected clinical metadata and chest CT image samples. The training sample set is input into a pre-built initial neural network model for iterative training, specifically including: Acquire clinical metadata for different patients within a preset time period; wherein, the clinical metadata includes underlying disease history, type of immunosuppressant, and duration of neutropenia; Each patient is classified into an immune subtype based on the clinical metadata, and a corresponding subtype label is established for the clinical metadata; wherein, the types of immune subtypes include the first subtype, the second subtype, and the third subtype; A corresponding data collection list is determined based on the immune subtype to which each patient belongs; wherein, when the patient belongs to the first subtype, chest CT image sample data of the patient is collected within a first preset time length; when the patient belongs to the second subtype, clinical indicators associated with bleeding are added to the clinical metadata; when the patient belongs to the third subtype, chest CT image sample data of the patient is collected within a second preset time length; the second preset time length is longer than the first preset time length. The training sample set is constructed based on the clinical metadata and the data collection list.

3. The method for constructing the IPA identification model as described in claim 1, characterized in that, The feature extraction layer outputs the lesion region prediction results and corresponding image features of the chest CT image sample based on a preset segmentation network, and extracts clinical features based on the clinical metadata corresponding to the chest CT image sample, specifically including: The corresponding image recognition target is invoked based on the immune subtype to which the chest CT image sample belongs; Based on the U-Net++ network and the image recognition target, chest image samples are identified and lesion prediction region masks and lung parenchyma regions are output. The lesion prediction region mask and the lung parenchyma region are used as the lesion region prediction results; Based on the lesion region prediction results, the lesion features of the chest CT image samples are extracted, and the lesion features are used as the image features; wherein, the image features include lesion morphology features, lesion density features, lesion edge features, and lung parenchyma background features; The clinical metadata is classified and cleaned to obtain clinical features, which include: patient body temperature, neutrophil count, immune status, infection markers, and bleeding risk indicators.

4. The method for constructing the IPA identification model as described in claim 1, characterized in that, After the feature extraction layer outputs the lesion region prediction result of the chest CT image sample based on the preset segmentation network, the method further includes: determining whether to use the lesion region prediction result as the output result of the feature extraction layer based on the preset segmentation quality evaluation index, specifically: The segmentation quality score of the predicted lesion region is calculated based on the segmentation quality expression; wherein, the segmentation quality expression is: ； In the formula, To segment quality scores; The total number of non-zero pixels within the segmented region, where the segmented region is the lesion-associated region in the lung parenchyma. pixels of the lesion area The gradient value, the lesion area is the intersection or union of the actual lesion area and the predicted lesion area; The gradient value of the pixels in the lung parenchyma region; When the segmentation quality score is greater than the preset segmentation quality evaluation index, the prediction result of the lesion region is used as the output result of the feature extraction layer. Otherwise, the lesion region prediction results of the feature extraction layer are regenerated based on the preset multi-model identification and voting mechanism.

5. The method for constructing the IPA identification model as described in claim 4, characterized in that, The regeneration of the lesion region prediction result of the feature extraction layer based on the preset multi-model recognition and voting mechanism specifically includes: Employing networks including U-Net++ Different networks are used to identify lesion regions in chest CT image samples and output lesion prediction region masks; among them... It is an odd number; The lesion prediction region mask output by each network is binarized to obtain the binarization result; the binarization result is used to characterize the lesion region judgment result of each pixel in the chest CT image sample; The binarization results of each pixel in the chest CT image sample are statistically analyzed. When more than half of the binarization results determine that the current pixel is a lesion area, the location of the current pixel is determined to be a lesion area. Based on the statistical results of each pixel, the lesion area prediction result of the feature extraction layer is regenerated.

6. The method for constructing the IPA identification model as described in claim 1, characterized in that, The calculation of the training samples within the preset time length involves image temporal features and clinical temporal features, using image features, clinical features, and their corresponding image temporal features and clinical temporal features as input. This allows the feature fusion layer to fuse the received features and output primary fused features, specifically including: In the same training sample, image features of chest CT image samples are extracted at preset intervals. Combined with the image features output by the feature extraction layer, the rate of change between two adjacent extracted image features is calculated and image temporal features are generated. In the same training sample, at preset time intervals, data of each clinical indicator in the clinical metadata are extracted, and combined with the clinical features output by the feature extraction layer, the rate of change between two adjacent extracted clinical features is calculated and clinical time-series features are generated. The image temporal features and the clinical temporal features are spliced ​​together to obtain the primary fusion features.

7. The method for constructing the IPA identification model as described in claim 6, characterized in that, Before stitching together the image temporal features and the clinical temporal features, the process also includes: The image features corresponding to each chest CT image sample are weighted according to the time distance between the acquisition time and the latest acquisition time. And the attention weights within the chest CT image modality are calculated based on the weighted image features; wherein the expression for the attention weights is: ； In the formula, Attention weights within the chest CT image modality; For the first Image feature vectors at each time point; No. Image feature vectors at each time point, and ≠ ; This is the cosine similarity function, used to calculate the similarity between two image feature vectors; This is the key weight matrix for attention. This represents the total dimension of the image features.

8. The method for constructing the IPA identification model as described in claim 6, characterized in that, The step of concatenating the image temporal features and the clinical temporal features to obtain the primary fusion feature specifically includes: The image temporal features and the clinical temporal features are mapped to the same dimension using a cross-modal mapping matrix; wherein, the expression for the cross-modal attention weight is: ； In the formula, Cross-modal attention weights; The first of the image temporal features Dimension transpose; This is the cross-modal mapping matrix; This is the rule weight matrix, which retrieves the corresponding values ​​based on the immune subtype to which the training samples belong. For the first One clinical time-series feature vector; This is a bias term, initially set to 0.1, used to adjust the baseline value of the cross-modal mapping; The total dimension of clinical characteristics; For the first One clinical time-series feature vector; Confidence level of chest CT image samples; Confidence level of clinical data; in, = ; ; In the formula, Signal-to-noise ratio of chest CT images; and These represent the minimum and maximum signal-to-noise ratios of the image acquisition device. To segment quality scores; For data integrity; For measurement error; This represents the maximum permissible measurement error. The primary fusion feature is obtained by concatenating the image time-series features and clinical time-series features mapped to the same dimension.

9. The method for constructing the IPA identification model as described in claim 1, characterized in that, The diagnostic inference layer further includes weighting based on the immune subtype corresponding to the training sample containing the primary fusion feature. Specifically: A multi-dimensional weighted cross-entropy is calculated based on the immune subtype corresponding to the training sample containing the primary fusion feature; wherein, the expression for the multi-dimensional weighted cross-entropy is: ； In the formula, Weighted cross-entropy; For training sample index; Weights corresponding to immune subtypes; For training sample class weights; For the first The true IPA labels of each training sample; For the first IPA disease prediction probability for each training sample.

10. An IPA identification method, characterized in that, include: Collect clinical metadata and chest CT image sample data of the patient within a preset time range from the current time; The clinical metadata and the chest CT image sample data are input into the IPA recognition model constructed using the construction method of the IPA recognition model according to any one of claims 1 to 9, and the predicted probability of the patient's IPA disease is output. The predicted probability of IPA disease is fed back to the user terminal so that the user terminal can generate a corresponding treatment recommendation plan based on the predicted probability of IPA disease.

11. A terminal device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the method for constructing the IPA identification model as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method for constructing the IPA identification model as described in any one of claims 1 to 9.

Images