Pulmonary nodule early warning method and system based on multi-model fusion

Abstract

The invention relates to a pulmonary nodule early warning method and system based on multi-model fusion, and belongs to the field of health management. The method comprises the steps that a pulmonary nodule CT image is collected and standardized, and a pulmonary nodule analysis image is generated; executing feature recognition of the pulmonary nodule analysis image to generate a pulmonary nodule annotation image carrying a multi-dimensional pathological feature tag, and constructing a pulmonary nodule time sequence data set; constructing a pulmonary nodule fusion prediction model based on the pulmonary nodule annotation image and the pulmonary nodule time sequence data set, and outputting a pulmonary nodule re-tagging pathological image, wherein the pulmonary nodule fusion prediction model integrates a time sequence probability prediction model, a learning supervision model and a re-tagging model; a three-dimensional convolutional neural network early warning model is constructed based on the pulmonary nodule retagging pathological image, a deterioration risk coefficient is quantified, and a clinical early warning mechanism is triggered when the deterioration risk coefficient breaks through a preset warning threshold, retagging of the potential state of the pulmonary nodule of the patient is achieved through the fusion model, and pulmonary nodule early warning is completed.

Description

Technical Field

[0001] The present invention belongs to the field of health management technology, and specifically relates to a lung nodule early warning method and system based on multi-model fusion. Background Art

[0002] Pulmonary nodules are focal, round, dense shadows of varying sizes, with clear or blurred margins and a diameter of 3 cm or less on lung images. They are a key indicator of many lung diseases and are of great significance. Pulmonary nodules can also be categorized by density as solid or non-solid. In clinical medicine, nodules larger than 8 mm in diameter are associated with a higher probability of developing malignancy and require careful evaluation.

[0003] In recent years, with the development of artificial intelligence and its application in the medical field, a large number of methods for identifying lung nodules in CT images have been proposed, supplemented by models for distinguishing benign from malignant nodules. Existing disease prediction methods rely on supervised learning or predictions based on generated images. Supervised learning-based methods are limited by the cost of labeling CT image datasets. Most images are only labeled when the condition is severe, lacking intermediate labeling during the disease progression. This makes medical intervention impossible during the disease progression, and the lack of an effective dynamic control mechanism leads to reduced prediction accuracy. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a lung nodule early warning method and system based on multi-model fusion.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A pulmonary nodule early warning method based on multi-model fusion, the implementation of the pulmonary nodule early warning method comprising the following steps:

[0007] Acquiring CT images of lung nodules and performing standardization processing to generate pulmonary nodule analysis images, wherein the standardization processing includes MHD analysis, CT value matrix reading, and CT value normalization;

[0008] Performing feature recognition on the lung nodule analysis image to generate a lung nodule labeled image with multi-dimensional pathological feature labels, and constructing a lung nodule time series dataset;

[0009] Building a lung nodule fusion prediction model based on the lung nodule annotated image and the lung nodule time series dataset and outputting a lung nodule re-labeled pathological image, wherein the lung nodule fusion prediction model integrates a time series probability prediction model, a learning supervision model, and a re-labeling model;

[0010] A three-dimensional convolutional neural network early warning model is constructed based on the relabeled pathological images of the lung nodules to quantify the deterioration risk coefficient. When the deterioration risk coefficient exceeds the preset warning threshold, a clinical early warning mechanism is triggered.

[0011] Preferably, the standardization process includes:

[0012] Acquiring the CT image of the lung nodule and attaching a benign or malignant status label and a timestamp;

[0013] Extracting the MHD parameters of the pulmonary nodule CT image to complete the MHD analysis and obtain an MHD file, and locate the three-dimensional coordinates of the pathological area;

[0014] Read the MHD file to obtain a flipped three-dimensional CT value matrix, and perform truncation and zeroing processing on the CT values exceeding [-1000, 400] to complete the reading of the CT value matrix;

[0015] The flipped three-dimensional CT value matrix is normalized to obtain the pulmonary nodule analytical image.

[0016] Preferably, the generation of the annotated lung nodule image includes:

[0017] Identify the lung nodule analysis image to obtain lung nodule target segmentation area and multidimensional recognition data, the multidimensional recognition data covers the major and minor axes of the lung nodule, the volume of the lung nodule, the density of the lung nodule, the composition of the lung nodule and the spicules of the lung nodule; form the multidimensional pathological feature label based on the multidimensional recognition data, the multidimensional pathological feature label includes the patient's name, timestamp, benign or malignant status label, lung nodule regional coordinates, the major and minor axes of the lung nodule, the volume of the lung nodule, the density of the lung nodule, the composition of the lung nodule and the spicules of the lung nodule; integrate the lung nodule annotated images carrying the multidimensional pathological feature label of the same patient according to group and timestamp to obtain the lung nodule time series data set and the lung nodule annotated image set;

[0018] The pulmonary nodule annotated image set is marked as benign images, warning images, and malignant images, and is divided into a model training image set and a model test and verification image set.

[0019] Preferably, the construction of the time series probability prediction model includes:

[0020] The time series probability prediction model λ=(A, B, Π) is defined, where A represents the hidden state transition probability matrix, B represents the observed state transition probability matrix, Π represents the initial state probability matrix, and the model states correspond to benign, warning, and malignant. At the same time, the state sequence S, the observation sequence O, the number of observed variables M, and the number of states N are defined;

[0021] Parameter optimization of the time series probability prediction model is implemented based on the model training image set and the model test verification image set.

[0022] Preferably, the construction of the learning supervision model includes:

[0023] Construct a learning supervision model, input the pulmonary nodule time series data set, and learn the state s at time t. i The probability γ t (i), the status includes benign, warning and malignant.

[0024] Preferably, the construction of the re-labeling model includes:

[0025] The relabeling model is constructed to obtain the disease state prediction result and the relabeled pathological image of the lung nodule.

[0026] Preferably, the acquisition of the relabeled pathological image of the pulmonary nodule includes:

[0027] Preset a dynamic correction prior knowledge matrix and calculate the forward probability α at time t based on the dynamic correction prior knowledge matrix t (i) and the backward probability β t (i) Derivation of the state of the pulmonary nodule temporal data set at time t i The first-order probability γ t (i) and the second-order joint probability γ t (i, j);

[0028] Get the first-order prediction probability γ t (i) pre , based on the first-order probability and the first-order predicted probability, the probability of the pulmonary nodule time series data sequence P{0|λ} is calculated. When the first-order predicted probability is less than the probability of the pulmonary nodule time series data sequence, the first-order probability γ is updated. t The value of (i) is the first-order prediction probability γ t (i) pre ;

[0029] Circularly calculate and update the time series probability prediction model λ=(A, B, Π) until all parameters converge;

[0030] Obtaining a pathological end-state probability based on the updated time series probability prediction model, wherein the pathological end-state probability includes a benign end-state probability, a warning end-state probability, and a malignant end-state probability, and capturing the maximum pathological end-state probability value as the disease state prediction result;

[0031] The disease state prediction result is used as the disease state prediction label, and the pulmonary nodule annotated image is fused to construct the re-labeling model, and a pulmonary nodule re-labeled pathological image is obtained.

[0032] Preferably, the dynamic correction prior knowledge matrix introduces a time decay factor and a feedback correction mechanism based on the prior knowledge matrix. The prior knowledge matrix is Among them, s1 represents benign, s2 represents warning, and s3 represents malignant. The expression of the dynamic correction prior knowledge matrix is H t ′=H t-1 ′·κ t +H·(1-r)·(1-κ t ), where H t ′ is the dynamic modified prior knowledge matrix at time t, H t-1 ′ is the dynamic modified prior knowledge matrix at time t-1, κ t is the time attenuation factor, which takes the value of [0, 1], and r is the dynamic misdiagnosis rate feedback coefficient.

[0033] Preferably, the three-dimensional convolutional neural network warning model consists of 6 convolution blocks, the function of convolution block 5 is to obtain bottleneck features, the convolution block 6 is used as a predictor to obtain the deterioration risk coefficient, and each convolution block contains a downsampling layer.

[0034] A pulmonary nodule early warning system based on multi-model fusion, used to implement the pulmonary nodule early warning method described above, including a standardization module, a feature recognition module, a fusion prediction module and a risk warning module;

[0035] The standardization module is used to collect CT images of lung nodules and perform standardization processing to generate pulmonary nodule analysis images, wherein the standardization processing includes MHD analysis, CT value matrix reading and CT value normalization;

[0036] The feature recognition module is used to perform feature recognition of the pulmonary nodule analysis image to generate a pulmonary nodule annotated image carrying a multi-dimensional pathological feature label and construct a pulmonary nodule time series data set;

[0037] The fusion prediction module is used to build a lung nodule fusion prediction model based on the lung nodule annotated image and the lung nodule time series data set and output a lung nodule re-labeled pathological image, wherein the lung nodule fusion prediction model integrates a time series probability prediction model, a learning supervision model and a re-labeling model;

[0038] The risk warning module is used to construct a three-dimensional convolutional neural network warning model based on the re-labeled pathological images of the lung nodules, quantify the deterioration risk coefficient, and trigger a clinical warning mechanism when the deterioration risk coefficient exceeds a preset warning threshold.

[0039] The beneficial effects of the present invention are:

[0040] (1) The problem of lack of potential intermediate annotations in lung nodule images is solved through multi-model fusion. The lung nodule fusion prediction model can integrate time series information and lung nodule characteristics to predict the potential status of lung nodules, provide re-labeled lung nodule data for the early warning model, and provide support for medical intervention.

[0041] (2) By introducing the time decay factor and feedback correction mechanism, a dynamic correction prior knowledge matrix is obtained to achieve dynamic correction of the model and avoid the cumulative impact of erroneous labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0043] Figure 1 This is a flowchart of the steps of a lung nodule early warning method based on multi-model fusion of the present invention. DETAILED DESCRIPTION

[0044] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0045] The working principle and use process of the present invention:

[0046] See also Figure 1 , a lung nodule early warning method based on multi-model fusion, including:

[0047] S1: Acquire CT images of lung nodules and perform standardization processing to generate pulmonary nodule analysis images. The standardization processing includes MHD analysis, CT value matrix reading and CT value normalization;

[0048] S2: Execute feature recognition of the lung nodule analysis image to generate a lung nodule labeled image carrying multi-dimensional pathological feature labels, and construct a lung nodule time series dataset.

[0049] S3: constructing a lung nodule fusion prediction model based on the lung nodule annotated image and the lung nodule time series dataset and outputting a lung nodule re-labeled pathological image, wherein the lung nodule fusion prediction model integrates a time series probability prediction model, a learning supervision model, and a re-labeling model;

[0050] S4: A three-dimensional convolutional neural network early warning model is constructed based on the re-labeled pathological images of the lung nodules to quantify the deterioration risk coefficient. When the deterioration risk coefficient exceeds the preset warning threshold, a clinical early warning mechanism is triggered.

[0051] In this embodiment, CT images of lung nodules are collected and standardized, which can be specifically implemented through the following steps:

[0052] S101: Acquire the CT image of the lung nodule and attach a benign or malignant status label and a timestamp, wherein the benign or malignant status label is verified and annotated by a radiology expert;

[0053] S102: Extracting MHD parameters of the pulmonary nodule CT image to complete the MHD analysis and obtain an MHD file, and locating the three-dimensional coordinates of the pathological area. The MHD parameters include raw data dimensions, a flip flag, centroid coordinates, dimensions in three-dimensional directions, and a step size. The MHD analysis can determine the location information of the pulmonary nodule on the pulmonary nodule CT image, i.e., the coordinates of the pulmonary nodule area.

[0054] S103: reading the MHD file to obtain a flipped three-dimensional CT value matrix, and performing truncation and zeroing processing on the CT values exceeding [-1000, 400] to complete the reading of the CT value matrix;

[0055] S104: Normalizing the flipped three-dimensional CT value matrix and obtaining the pulmonary nodule analytical image.

[0056] In this embodiment, the feature recognition of the lung nodule analysis image is performed to generate a lung nodule labeled image carrying a multi-dimensional pathological feature label, which can be specifically implemented by the following steps:

[0057] S201: Identify the lung nodule analysis image through the uAI fully automatic lung nodule recognition platform to obtain lung nodule target segmentation area and multidimensional recognition data, the uAI fully automatic lung nodule recognition platform supports the detection of lung nodules of 3 mm and above, the multidimensional recognition data covers the long and short axes of lung nodules, lung nodule volume, lung nodule density, lung nodule component composition and lung nodule burrs, form the multidimensional pathological feature label based on the multidimensional recognition data, the multidimensional pathological feature label includes the patient's name, timestamp, benign or malignant status label, lung nodule region coordinates, lung nodule long and short axes, lung nodule volume, lung nodule density, lung nodule component and lung nodule burrs, integrate the lung nodule annotated images carrying the multidimensional pathological feature label of the same patient according to group and timestamp, and obtain the lung nodule time series data set and lung nodule annotated image set;

[0058] S202: The pulmonary nodule annotated image set is marked by medical experts as benign images, warning images, and malignant images. CT images with serious conditions are marked as malignant images by experts, and the remaining images are temporarily recorded as benign images. On this basis, benign images with a malignant trend are recorded as warning images and divided into a model training image set and a model test and verification image set.

[0059] In this embodiment, a lung nodule fusion prediction model is constructed based on the lung nodule annotated image and the lung nodule time-series dataset, and a lung nodule re-labeled pathological image is output. This can be specifically implemented by the following steps:

[0060] S301: Define the time series probability prediction model λ = (A, B, п), where A represents the hidden state transition probability matrix, B represents the observed state transition probability matrix, п represents the initial state probability matrix, and the model states correspond to benign, warning, and malignant. At the same time, define the state sequence S, the observation sequence O, the number of observed variables M, and the number of states N. The hidden state refers to the true state of the model, which is invisible but can be inferred through the observed state. The initial state probability matrix describes the initial probability of the model in each hidden state. The hidden state transition probability matrix describes the probability of transitioning from one hidden state to another. The observed state transition probability matrix describes the probability of generating an observed state under a certain hidden state. For example: in the time series probability prediction model of patient A, the initial state probability matrix describes that under the initial condition, patient A's lung nodules may be 99% benign and 1% malignant. The hidden state transition probability matrix describes that patient A's lung nodules may have a 50% probability of transitioning from benign to malignant. The observed state transition probability matrix establishes a probability mapping relationship between pathological features and states. The time series probability prediction model can preliminarily predict the most likely subsequent state sequence based on the current observation sequence.

[0061] Parameter optimization of the time series probability prediction model is implemented based on the model training image set and the model test verification image set.

[0062] S302: Construct a learning supervision model, which is used to provide supervision for the training process of the time series probability prediction model, input the time series data set of pulmonary nodules, and learn the state s at time t. i The probability γ t (i), the status includes benign, warning and malignant.

[0063] The specific construction process of the supervised learning model is as follows: a random sample permutation strategy is implemented, whereby only the numerical encoding of the categorical features of each sample is calculated using the samples ranked earlier. When constructing the model, four random sample permutations are designed, and one of these permutations is randomly selected each time the tree structure is constructed. The 'max_ctrcomplexity' parameter controls the maximum number of features for feature crossover, which is set to 4 here. Next, the model constructed after randomly permuting the samples again is used to estimate the gradient of the prediction results, obtain the first- and second-order gradients of the model, and construct the tree structure based on the gradient calculation results. Finally, all samples are used to calculate the value of each leaf node in the tree, and the optimal solution of the objective function is obtained through weighted accumulation.

[0064] S303: Construct the relabeling model to obtain the disease state prediction result and obtain the relabeled pathological image of the lung nodule. The relabeling model is used to predict the possibility of each hidden state corresponding to the input lung nodule time series data set.

[0065] In this embodiment, the re-labeling model is constructed to obtain the disease state prediction result and the re-labeled pathological image of the pulmonary nodule, which can be specifically implemented by the following steps:

[0066] S303-1: Preset a dynamically modified priori knowledge matrix and calculate the forward probability α at time t based on the dynamically modified priori knowledge matrix t (i) and the backward probability β t (i) Derivation of the state of the pulmonary nodule temporal data set at time t i The first-order probability γ t (i) and the second-order joint probability γ t (i, j);

[0067] The dynamic correction prior knowledge matrix introduces a time decay factor and a feedback correction mechanism based on the prior knowledge matrix. The prior knowledge matrix is Among them, s1 represents benign, s2 represents warning, and s3 represents malignant. The prior knowledge matrix expresses the initial label of each state, indicating that CT images marked as s1 or s2 have a 1% chance of being marked as s3, and CT images marked as s3 have a 5% chance of being marked as s1. The expression of the dynamic correction prior knowledge matrix is H t ′=H t-1 ′·κ t +H·(1-r)·(1-K t ), where H t ′ is the dynamic modified prior knowledge matrix at time t, H t-1 ′ is the dynamic modified prior knowledge matrix at time t-1, κ t is the time decay factor, with a value of [0, 1], and is usually designed to be a function that increases or decreases over time. r is the dynamic misdiagnosis rate feedback coefficient, that is, the error rate of the recent model prediction. For example: in the initial prior knowledge matrix, H(S3, S3) = 0.95, that is, the probability of the malignant state being correctly marked is 95%, but the dynamic misdiagnosis rate feedback coefficient of malignant cases in the recent prediction is higher (such as 20%), then it is corrected by r = 0.20, combined with κ t =0.8, then dynamically modify the prior knowledge matrix H t ′(3,3)=0.95×0.8+0.95×0.8×0.2=0.912. The cumulative effect of incorrect labeling can be avoided by dynamically correcting the prior knowledge matrix.

[0068] The first-order probability γ t The calculation formula for (i) is The second-order joint probability γ t The calculation formula for (i, j) is in,

[0069] S303-2: Introducing the CatBoost model to obtain the first-order prediction probability γ t (i) pre , based on the first-order probability and the first-order predicted probability, the probability of the pulmonary nodule time series data sequence P{O|λ} is calculated. When the first-order predicted probability is less than the probability of the pulmonary nodule time series data sequence, the value of the first-order probability γt(i) is updated to the first-order predicted probability γ t (i) pre ;

[0070] S303-3: cyclically calculating and updating the time series probability prediction model λ=(A, B, Π) until all parameters converge;

[0071] S303-4: Obtaining a pathological end-state probability based on the updated time series probability prediction model, wherein the pathological end-state probability includes a benign end-state probability, a warning end-state probability, and a malignant end-state probability, and capturing the maximum pathological end-state probability value as the disease state prediction result;

[0072] S303-5: Using the disease state prediction result as the disease state prediction label, integrating the pulmonary nodule annotated image to construct the re-labeling model, and obtaining the pulmonary nodule re-labeled pathological image.

[0073] In this embodiment, a three-dimensional convolutional neural network early warning model is constructed based on the relabeled pathological images of lung nodules, which can be implemented by the following steps:

[0074] The three-dimensional convolutional neural network early warning model takes the relabeled lung nodule pathological image as input and is trained. It consists of 6 convolution blocks, and the input layer size is 64×64×64 lung nodule relabeled pathological image. Among them, the function of convolution block 5 is to obtain the bottleneck feature, and the convolution block 6 is used as a predictor to obtain the deterioration risk coefficient. If the threshold is exceeded, an early warning is issued. Each convolution block of the three-dimensional convolutional neural network model contains a downsampling layer, and the activation function adopts the ReLu function. The structure table of the three-dimensional neural network model is shown below:

[0075] Input layer 64×64×64，1 <![CDATA[Average Pooling > 32×64×64，1 Convolutional Block 1 32×64×64，64 <![CDATA[Maximum Pooling > 16×16×16，64 Convolutional Block 2 16×16×16，128 <![CDATA[Maximum Pooling > 8×8×8，128 Convolutional Block 3 8×8×8，256 <![CDATA[Maximum Pooling > 4×4×4，256 Convolutional Block 4 4×4×4，512 <![CDATA[Maximum Pooling > 2×2×2，512 Convolutional Block 5 1×1×1，64 Convolutional Block 6 1×1×1，1

[0076] A pulmonary nodule early warning system based on multi-model fusion, including a standardization module, a feature recognition module, a fusion prediction module, and a risk warning module;

[0077] The standardization module is used to collect CT images of lung nodules and perform standardization processing to generate pulmonary nodule analysis images, wherein the standardization processing includes MHD analysis, CT value matrix reading and CT value normalization;

[0078] The feature recognition module is used to perform feature recognition of the pulmonary nodule analysis image to generate a pulmonary nodule annotated image carrying a multi-dimensional pathological feature label and construct a pulmonary nodule time series data set;

[0079] The fusion prediction module is used to build a lung nodule fusion prediction model based on the lung nodule annotated image and the lung nodule time series data set and output a lung nodule re-labeled pathological image, wherein the lung nodule fusion prediction model integrates a time series probability prediction model, a learning supervision model and a re-labeling model;

[0080] The risk warning module is used to construct a three-dimensional convolutional neural network warning model based on the re-labeled pathological images of the lung nodules, quantify the deterioration risk coefficient, and trigger a clinical warning mechanism when the deterioration risk coefficient exceeds a preset warning threshold.

[0081] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0082] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0083] The program code included in the computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF or the like, or any suitable combination thereof. The computer program code for performing the operation of the present invention can be written in one or more programming languages or a combination thereof, and the programming language includes an object-oriented programming language such as Java, Smalltalk, C++, and also includes a conventional procedural programming language such as "C" language or similar programming language. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, utilizing an Internet service provider to connect through the Internet).

[0084] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A pulmonary nodule early warning method based on multi-model fusion, characterized in that: The implementation of the pulmonary nodule early warning method includes the following steps: Acquiring CT images of lung nodules and performing standardization processing to generate pulmonary nodule analysis images, wherein the standardization processing includes MHD analysis, CT value matrix reading, and CT value normalization; Performing feature recognition on the lung nodule analysis image to generate a lung nodule labeled image with multi-dimensional pathological feature labels, and constructing a lung nodule time series dataset; Building a lung nodule fusion prediction model based on the lung nodule annotated image and the lung nodule time series dataset and outputting a lung nodule re-labeled pathological image, wherein the lung nodule fusion prediction model integrates a time series probability prediction model, a learning supervision model, and a re-labeling model; A three-dimensional convolutional neural network early warning model is constructed based on the relabeled pathological images of the lung nodules to quantify the deterioration risk coefficient. When the deterioration risk coefficient exceeds the preset warning threshold, a clinical early warning mechanism is triggered.

2. The pulmonary nodule early warning method according to claim 1, characterized in that: The standardization process includes: Acquiring the CT image of the lung nodule and attaching a benign or malignant status label and a timestamp; Extracting the MHD parameters of the pulmonary nodule CT image to complete the MHD analysis and obtain an MHD file, and locate the three-dimensional coordinates of the pathological area; Read the MHD file to obtain a flipped three-dimensional CT value matrix, and perform truncation and zeroing processing on the CT values exceeding [-1000, 400] to complete the reading of the CT value matrix; Normalize the flipped three-dimensional CT value matrix and obtain the pulmonary nodule analytical image.

3. The pulmonary nodule early warning method according to claim 1, characterized in that: The generation of the pulmonary nodule annotated image includes: Identify the lung nodule analysis image to obtain lung nodule target segmentation area and multidimensional recognition data, the multidimensional recognition data covers the major and minor axes of the lung nodule, the volume of the lung nodule, the density of the lung nodule, the composition of the lung nodule and the spicules of the lung nodule; form the multidimensional pathological feature label based on the multidimensional recognition data, the multidimensional pathological feature label includes the patient's name, timestamp, benign or malignant status label, lung nodule regional coordinates, the major and minor axes of the lung nodule, the volume of the lung nodule, the density of the lung nodule, the composition of the lung nodule and the spicules of the lung nodule; integrate the lung nodule annotated images carrying the multidimensional pathological feature label of the same patient according to group and timestamp to obtain the lung nodule time series data set and the lung nodule annotated image set; The pulmonary nodule annotated image set is marked as benign images, warning images, and malignant images, and is divided into a model training image set and a model test and verification image set.

4. The pulmonary nodule early warning method according to claim 1, characterized in that: The construction of the time series probability prediction model includes: The time series probability prediction model λ=(A, B, Π) is defined, where A represents the hidden state transition probability matrix, B represents the observed state transition probability matrix, Π represents the initial state probability matrix, and the model states correspond to benign, warning, and malignant. At the same time, the state sequence S, the observation sequence O, the number of observed variables M, and the number of states N are defined; Parameter optimization of the time series probability prediction model is implemented based on the model training image set and the model test verification image set.

5. The pulmonary nodule early warning method according to claim 1, characterized in that: The construction of the learning supervision model includes: Construct a learning supervision model, input the pulmonary nodule time series data set, and learn the state s at time t. i The probability γ t (i), the status includes benign, warning and malignant.

6. The pulmonary nodule early warning method according to claim 1, characterized in that: The construction of the re-labeling model includes: The relabeling model is constructed to obtain the disease state prediction result and the relabeled pathological image of the lung nodule.

7. The pulmonary nodule early warning method according to claim 6, characterized in that: The acquisition of the relabeled pathological image of the pulmonary nodule includes: Preset a dynamic correction prior knowledge matrix and calculate the forward probability α at time t based on the dynamic correction prior knowledge matrix t (i) and the backward probability β t (i) Derivation of the state of the pulmonary nodule temporal data set at time t i The first-order probability γ t (i) and the second-order joint probability γ t (i, j); Get the first-order prediction probability γ t (i) pre , based on the first-order probability and the first-order predicted probability, the probability of the pulmonary nodule time series data sequence P{0|λ} is calculated. When the first-order predicted probability is less than the probability of the pulmonary nodule time series data sequence, the first-order probability γ is updated. t The value of (i) is the first-order prediction probability γ t (i) pre ; Circularly calculate and update the time series probability prediction model λ=(A, B, Π) until all parameters converge; Obtaining a pathological end-state probability based on the updated time series probability prediction model, wherein the pathological end-state probability includes a benign end-state probability, a warning end-state probability, and a malignant end-state probability, and capturing the maximum pathological end-state probability value as the disease state prediction result; The disease state prediction result is used as the disease state prediction label, and the pulmonary nodule annotated image is fused to construct the re-labeling model, and a pulmonary nodule re-labeled pathological image is obtained.

8. The pulmonary nodule early warning method according to claim 7, characterized in that: The dynamic correction prior knowledge matrix introduces a time decay factor and a feedback correction mechanism based on the prior knowledge matrix. The prior knowledge matrix is Among them, s1 represents benign, s2 represents warning, and s3 represents malignant. The expression of the dynamic correction prior knowledge matrix is H t ′=H t-1 ′·κ t +H·(1-r)·(1-κ t ), where H t ′ is the dynamic modified prior knowledge matrix at time t, H t-1 ′ is the dynamic modified prior knowledge matrix at time t-1, κ t is the time attenuation factor, which takes the value of [0, 1], and r is the dynamic misdiagnosis rate feedback coefficient.

9. The pulmonary nodule early warning method according to claim 1, characterized in that: The three-dimensional convolutional neural network early warning model consists of 6 convolution blocks, where convolution block 5 is used to obtain bottleneck features, and convolution block 6 is used as a predictor to obtain the deterioration risk coefficient. A downsampling layer is included between each convolution block.

10. A pulmonary nodule early warning system based on multi-model fusion, characterized in that: The device is applied to the pulmonary nodule early warning method according to any one of claims 1 to 9, comprising a standardization module, a feature recognition module, a fusion prediction module, and a risk warning module; The standardization module is used to collect CT images of lung nodules and perform standardization processing to generate pulmonary nodule analysis images, wherein the standardization processing includes MHD analysis, CT value matrix reading and CT value normalization; The feature recognition module is used to perform feature recognition of the pulmonary nodule analysis image to generate a pulmonary nodule annotated image carrying a multi-dimensional pathological feature label and construct a pulmonary nodule time series data set; The fusion prediction module is used to build a lung nodule fusion prediction model based on the lung nodule annotated image and the lung nodule time series data set and output a lung nodule re-labeled pathological image, wherein the lung nodule fusion prediction model integrates a time series probability prediction model, a learning supervision model and a re-labeling model; The risk warning module is used to construct a three-dimensional convolutional neural network warning model based on the re-labeled pathological images of the lung nodules, quantify the deterioration risk coefficient, and trigger a clinical warning mechanism when the deterioration risk coefficient exceeds a preset warning threshold.

Images