Disease Monitoring Method and System Based on Medical Image Data Recognition

Abstract

The present disclosure provides a disease monitoring method and system based on medical image data recognition, which relates to the field of image processing technology. The method includes: obtaining multi-level prior information related to a target disease from a medical knowledge graph, encoding the lesion attributes and temporal attributes in the multi-level prior information as knowledge prior vectors and inputting them into a generator, so that the generator outputs lesion image data based on the knowledge prior vectors; while evaluating the difference between the lesion image data output by the generator and the knowledge prior vectors, the discriminator also calls the lesion boundary and staging restriction rules in the medical knowledge graph to determine the compliance of the morphology and staging status of the lesion image data; based on the results of the difference evaluation and compliance determination output by the discriminator, updating the network parameters of the generator to obtain target lesion image data; after combining the lesion image data with the original medical image, inputting them into a preset disease monitoring model to generate potential lesion detection results.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and more particularly, to a disease monitoring method and system based on the recognition of medical image data. Background Art

[0002] Currently, medical images play a crucial role in clinical diagnosis, but there are still several deficiencies: on the one hand, manual reading of large-scale image data is time-consuming and laborious, and it is difficult to guarantee the integrity and consistency of diagnostic reports due to subjective factors; on the other hand, image-assisted diagnosis often lacks the full utilization of multi-dimensional prior information about diseases (such as staging rules, anatomical structure dependence relationships, temporal evolution, etc.), resulting in limited accuracy and interpretability in the recognition of rare lesions and complex stages. In addition, traditional methods or systems often lack sufficient support for the automatic detection and structured description of image lesions, and it is difficult to meet the accurate recognition requirements of intelligent diagnosis for multiple disease types and multiple stages.

[0003] To solve the above problems, for example, Chinese Patent Publication No. CN115062165B discloses a medical image diagnosis method and device based on a reading knowledge graph. It performs lesion recognition on medical images through multi-algorithm fusion (such as image classification, object detection, image segmentation, etc.), and associates with a pre-constructed reading knowledge graph to generate "structured examination findings" containing abnormal image features and lesion location information, and then uses a deep learning model and configuration rules to obtain "examination impressions" and "medical image diagnostic reports", which improves the diagnostic automation and report generation efficiency to a certain extent.

[0004] However, the above solution mainly focuses on using the image features and location information of the knowledge graph when generating diagnostic reports, and does not fully utilize the comprehensive utilization of disease staging evolution rules, lesion temporal attributes, GAN (Generative Adversarial Network) and other methods; it also lacks in-depth research on aspects such as lesion image synthesis, multi-round adversarial training, and deep constraints, and it is difficult to balance the fine coupling of morphology and staging priors, and it is impossible to more accurately simulate and detect multi-stage lesions or rare sample lesions. Based on this, it is necessary to further expand and improve on the basis of the existing technology, and comprehensively apply knowledge graphs, GANs, and lesion morphology staging prior information to more comprehensively improve the intelligent recognition and diagnosis level of medical images in multi-stage scenarios. Summary of the Invention

[0005] In view of the deficiencies of the prior art, embodiments of the present disclosure provide a disease monitoring method and system based on the recognition of medical image data.

[0006] In a first aspect, embodiments of the present disclosure provide a disease monitoring method based on the recognition of medical image data, including:

[0007] Obtain multi-level prior information related to the target disease from the medical knowledge graph, where the multi-level prior information includes:

[0008] Lesion attributes, where the lesion attributes include lesion size, shape, location, and anatomical structure dependency;

[0009] At least one temporal attribute describing the progression law of the target disease; and lesion boundary and staging restriction rules for restricting the lesion morphology and staging of the target disease;

[0010] Use a generative adversarial network as the lesion generation model, where the generative adversarial network includes a generator and a discriminator;

[0011] Encode the lesion attributes and temporal attributes in the multi-level prior information into a knowledge prior vector and input it into the generator, so that the generator outputs lesion image data based on the knowledge prior vector to match the anatomical structure and progression law of the target disease;

[0012] While the discriminator evaluates the difference between the lesion image data output by the generator and the knowledge prior vector, the discriminator also calls the lesion boundary and staging restriction rules in the medical knowledge graph to determine the compliance of the morphology and staging status of the lesion image data;

[0013] Based on the results of the difference evaluation and compliance determination output by the discriminator, update the network parameters of the generator through multiple rounds of adversarial training until the lesion image data that meets the requirements of the multi-level prior information is obtained;

[0014] After combining the lesion image data that meets the requirements of the multi-level prior information with the original medical image, input it into a preset disease monitoring model to generate potential lesion detection results.

[0015] As an optional implementation, the step of making the generator output lesion image data based on the knowledge prior vector includes:

[0016] Numerically process the size, shape, location, and anatomical structure dependency in the lesion attributes to generate a lesion attribute vector, where the lesion attribute vector is used to represent the spatial characteristics and anatomical prior of the target disease lesion;

[0017] Perform time series feature extraction on the temporal attributes describing the progression law of the target disease, and fuse the obtained temporal features with the lesion attribute vector to form a temporal fusion vector, where the temporal fusion vector is used to represent the dynamic changes of the lesion at different stages and progression phases;

[0018] In the temporal fusion vector, based on the critical threshold and stage interval defined by the lesion boundary and staging restriction rules obtained from the medical knowledge graph, prior restriction processing is performed to annotate and restrict the temporal fusion vector in terms of lesion morphology and staging dimensions;

[0019] Input the temporally fused vector after prior restriction processing into the generator, and use the lesion spatial features and progression stage information contained in the temporally fused vector to generate lesion image data during both the training and inference phases.

[0020] As an alternative implementation, the difference evaluation of the lesion image data output by the generator and the knowledge prior vector includes:

[0021] Extract lesion image features from the lesion image data output by the generator through a feature extraction algorithm to form an image feature vector;

[0022] Compare the image feature vector with the knowledge prior vector one by one in the spatial attribute dimension, temporal progression dimension, and staging restriction dimension, and calculate the difference metric respectively;

[0023] Determine the difference evaluation result based on the difference metric results of each dimension, and use the difference evaluation result to indicate the deviation degree of the lesion image data from the knowledge prior vector.

[0024] As an alternative implementation, the compliance determination of the morphology and staging status of the lesion image data includes:

[0025] Based on the boundary restriction rules obtained from the medical knowledge graph, match the edge contour in the lesion image data with a preset lesion morphology threshold, and output a morphology non-compliance indication when the lesion morphology exceeds the lesion morphology threshold;

[0026] Based on the staging interval in the temporal attributes, compare the staging status of the lesion image data with the staging restrictions in the medical knowledge graph, and output a staging non-compliance indication when the staging status exceeds the staging interval;

[0027] Integrate the results of morphology compliance and staging compliance determination to form a compliance determination result, and use the compliance determination result to guide the continuation or correction of the adversarial training process.

[0028] As an alternative implementation, based on the results of the difference evaluation and compliance determination output by the discriminator, updating the network parameters of the generator through multiple rounds of adversarial training includes:

[0029] Based on the difference evaluation result, a difference loss is obtained. Based on the compliance determination result, a compliance loss is obtained. The difference loss and the compliance loss are fused at a preset ratio to generate a total loss;

[0030] During the training process, gradient backpropagation optimization is performed on the generator;

[0031] When the total loss is lower than a preset threshold or the number of training epochs reaches the upper limit, the network parameters of the generator are stopped from being updated, and the lesion image data that meets the requirements of the multi-level prior information is obtained.

[0032] As an optional implementation manner, the combining the lesion image data that meets the requirements of the multi-level prior information with the original medical image includes:

[0033] Based on the position attribute in the lesion image data, the lesion pixel coordinates are converted and aligned to the corresponding coordinate positions of the original medical image;

[0034] After the coordinate alignment, a preset pixel fusion process is performed on the boundary area where the lesion area overlaps with the original medical image. The pixel fusion process includes gradient correction or seamless cloning operations at the fusion boundary;

[0035] After the pixel fusion process of the boundary area is completed, the merged image data is output as the synthetic image data.

[0036] As an optional implementation manner, after the combining the lesion image data that meets the requirements of the multi-level prior information with the original medical image, inputting it into a preset disease monitoring model to generate potential lesion detection results includes:

[0037] Load the preset disease monitoring model and perform image preprocessing operations on the synthetic image data;

[0038] In the model inference stage, feature extraction and segmentation calculations are performed on the input synthetic image data to generate the position information, size, and shape of the potential lesion area;

[0039] Based on the potential lesion information obtained from the model inference, potential lesion detection results are generated; the potential lesion detection results include lesion spatial coordinates, morphological parameters, and staging information.

[0040] As an optional implementation manner, it further includes:

[0041] In response to the staging information, lesion size, and morphological characteristics of the potential lesion detection results not matching the multi-level prior information, the non-matching information is recorded as update data;

[0042] Send the updated data to the update interface of the medical knowledge graph to adjust the information of the target disease in the medical knowledge graph.

[0043] Second, the embodiments of the present disclosure also provide a disease monitoring system based on the recognition of medical image data, including: an acquisition module, a first processing module, a second processing module, and a generation module;

[0044] The acquisition module is used to obtain multi-level prior information related to the target disease from the medical knowledge graph. The multi-level prior information includes:

[0045] Lesion attributes, which include lesion size, shape, location, and anatomical structure dependency;

[0046] At least one temporal attribute describing the progression law of the target disease; and lesion boundary and staging restriction rules for restricting the lesion morphology and staging of the target disease;

[0047] The first processing module is used to use a generative adversarial network as the lesion generation model. The generative adversarial network includes a generator and a discriminator;

[0048] Encode the lesion attributes and temporal attributes in the multi-level prior information into a knowledge prior vector and input it into the generator, so that the generator outputs lesion image data based on the knowledge prior vector to match the anatomical structure and progression law of the target disease;

[0049] While the discriminator evaluates the difference between the lesion image data output by the generator and the knowledge prior vector, the discriminator also calls the lesion boundary and staging restriction rules in the medical knowledge graph to determine the compliance of the morphology and staging status of the lesion image data;

[0050] The second processing module is used to update the network parameters of the generator through multiple rounds of adversarial training based on the results of the difference evaluation and compliance determination output by the discriminator until the lesion image data that meets the requirements of the multi-level prior information is obtained;

[0051] The generation module is used to combine the lesion image data that meets the requirements of the multi-level prior information with the original medical image and input it into a preset disease monitoring model to generate potential lesion detection results.

[0052] Compared with the prior art, the present application organically combines the generative adversarial network with multi-level prior information (including lesion morphology, staging limitations, and temporal attributes), uses means such as multi-round adversarial training and boundary fusion to achieve precise simulation and detection of rare staging and multi-morphology lesions, and forms a closed loop through the dynamic feedback between the monitoring results and the knowledge graph, significantly improving the accuracy and interpretability of lesion synthesis and recognition, so as to better meet the clinical needs of medical imaging in multi-staging diagnosis scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flowchart of a disease monitoring method based on medical image data identification provided by an embodiment of the present disclosure;

[0054] Figure 2 is a flowchart of a method for enabling a generator to output lesion image data based on a knowledge prior vector provided by an embodiment of the present disclosure;

[0055] Figure 3 is a schematic diagram of a disease monitoring system based on medical image data identification provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0057] See Figure 1 As shown, it is a flowchart of a disease monitoring method based on medical image data identification provided by an embodiment of the present disclosure. The method includes steps S101 to S106, where:

[0058] S101: Obtain multi-level prior information related to the target disease from the medical knowledge graph. The multi-level prior information includes: lesion attributes, where the lesion attributes include lesion size, morphology, location, and anatomical structure dependency; at least one temporal attribute describing the progression law of the target disease; and lesion boundary and staging limitation rules for restricting the lesion morphology and staging of the target disease;

[0059] S102: Use the generative adversarial network as the lesion generation model, and the generative adversarial network includes a generator and a discriminator;

[0060] S103: Encode the lesion attributes and temporal attributes in the multi-level prior information into a knowledge prior vector and input it into the generator, so that the generator outputs lesion image data based on the knowledge prior vector to match the anatomical structure and progression law of the target disease;

[0061] S104: While the discriminator evaluates the difference between the lesion image data output by the generator and the knowledge prior vector, the discriminator also calls the lesion boundary and staging restriction rules in the medical knowledge graph to determine the compliance of the morphology and staging status of the lesion image data;

[0062] S105: Based on the results of the difference evaluation and compliance determination output by the discriminator, update the network parameters of the generator through multiple rounds of adversarial training until the lesion image data that meets the requirements of the multi-level prior information is obtained;

[0063] S106: After combining the lesion image data that meets the requirements of the multi-level prior information with the original medical images, input them into a preset disease monitoring model to generate potential lesion detection results.

[0064] This disclosure obtains multi-level prior information related to the target disease from the medical knowledge graph; uses a generative adversarial network (GAN) as a lesion generation model; encodes the multi-level prior information (lesion attributes and temporal attributes) as a knowledge prior vector and inputs it into the generator; while the discriminator evaluates the difference between the lesion image data output by the generator and the knowledge prior vector, it calls the lesion boundary and staging restriction rules in the medical knowledge graph for compliance determination; based on the results of the difference evaluation and compliance determination, update the network parameters of the generator through multiple rounds of adversarial training until the lesion image data that meets the requirements of the multi-level prior information is obtained; after combining the lesion image data that meets the requirements of the multi-level prior information with the original medical images, input them into the disease monitoring model to generate potential lesion detection results.

[0065] Regarding the above S101:

[0066] In the present invention, the medical knowledge graph stores prior knowledge of the target disease in multiple dimensions such as anatomical structure, lesion morphology, temporal progression, and staging restrictions; by extracting this prior information, the subsequent lesion generation and detection processes can be closer to the clinical real scenario and improve the reliability of lesion image synthesis and detection results.

[0067] In a specific implementation, the medical knowledge graph can be constructed from multiple sources, such as clinical medical records, literature materials, and medical image databases. Through natural language processing and relationship extraction techniques, information such as disease names, lesion locations, anatomical structure dependencies, and staging criteria are stored in a graph structure. For the target disease (such as lung tumors, liver tumors, etc.), lesion attributes (size, morphology, location, anatomical structure dependencies), temporal attributes (reflecting the progression law of the disease over time, such as staging, clinical course, etc.), and lesion boundary and staging restriction rules (used to define the morphological boundaries of lesions, the morphological change ranges corresponding to different stages, etc.) can be queried in the medical knowledge graph.

[0068] After obtaining the relevant nodes and their associated information of the target disease, the lesion attributes, temporal attributes, and boundary / staging restriction rules are read and preprocessed respectively. For example:

[0069] Lesion attributes: Record the maximum / minimum size of the lesion, common shape types (nodular, mass-like, etc.), specific anatomical locations, etc.;

[0070] Temporal attributes: Include the typical manifestations of the disease at different stages or progression phases and time intervals, etc.;

[0071] Lesion boundary and staging restriction rules: Include the numerical threshold regulations for the morphological boundaries of the lesion, and the constraints on the size or shape characteristics of the lesions at different stages, etc.

[0072] Exemplarily, by collecting clinical medical records, medical imaging databases, and literature materials, the lesion characteristics (size, shape, location, staging criteria, etc.) related to the target disease (such as lung cancer, liver cancer) can be stored in a graph database such as Neo4j in the form of nodes - relationships.

[0073] Exemplarily, if the target disease is "lung cancer", the system can retrieve the typical size range (such as from a few millimeters to a few centimeters), common shape types (nodular, patchy, etc.) of the disease at stages I - IV, as well as the corresponding lesion boundary restriction rules (such as edge smoothness, spiculation, etc.).

[0074] Regarding the above S102:

[0075] The present disclosure selects the generative adversarial network (GAN) as the lesion generation model. Utilizing the adversarial training mechanism of GAN, it learns the lesion morphology and staging characteristics from the prior information and the distribution of a small number of real samples, and then synthesizes lesion image data that is closer to clinical reality; it provides a network framework basis for the compliance determination and adversarial training of the subsequent discriminator. GAN consists of two parts: a generator and a discriminator, where:

[0076] The generator is used to receive inputs such as a prior vector and output lesion image data;

[0077] The discriminator is used to evaluate the lesion images output by the generator and make authenticity or compliance determinations based on prior knowledge.

[0078] In specific implementations, the generator and the discriminator can be implemented using a convolutional neural network (CNN) or other deep network structures, and the initial parameters can be randomly initialized or migrated from a pre - trained model.

[0079] In addition, if more complex temporal features are involved, a temporal modeling module (such as LSTM, Transformer, etc.) can be additionally introduced in the generator or discriminator to adapt to the progression rules described in the multi-level prior information.

[0080] Exemplarily, mainstream GAN structures such as DCGAN and WGAN can be selected, and both the generator and the discriminator are based on the convolutional neural network (CNN).

[0081] Regarding the above S103:

[0082] The present disclosure converts discrete or continuous information such as lesion size, shape, location, and temporal progression into a vector format readable by the network; introduces prior rules such as lesion boundary and staging restrictions, and performs necessary upper and lower bound constraints on the vector, so that the generator complies with anatomical and staging priors when synthesizing lesions.

[0083] In a specific implementation, first, the lesion attributes (size, shape, location, anatomical structure dependencies, etc.) are characterized. For example, the shape is represented as a shape vector in several dimensions, and the location is represented as coordinate or mask information to form a lesion attribute vector. In temporal attributes, the information in different stages or phases is serialized. For example, the features corresponding to stage I, stage II, and stage III are quantified as discrete or continuous variables; if there is time series data (such as follow-up time intervals, disease course evolution rules, etc.), a time series feature extraction method (such as multi-scale temporal embedding) can be used to generate a temporal vector.

[0084] Furthermore, after fusing the lesion attribute vector and the temporal vector, a preliminary temporal fusion vector is obtained.

[0085] The present disclosure also obtains the threshold or rule information corresponding to the lesion boundary and staging restriction rules from the medical knowledge graph, and performs prior restriction processing on the temporal fusion vector.

[0086] Exemplarily, if the staging restriction stipulates the maximum shape range, minimum aspect ratio, etc. of the lesion, corresponding upper and lower bound annotations are set in the fusion vector.

[0087] Finally, the fusion vector after prior restriction processing is input into the generator, so that the generator can use this prior vector for lesion image synthesis during the training and inference stages, thereby ensuring that the generated lesion images better meet the clinical real needs and progression rules.

[0088] Exemplarily, when vectorizing the lesion attributes, the lesion size can be quantified as a one-dimensional or two-dimensional value, the lesion shape is encoded by shape (such as spherical, elliptical, lobulated, etc.), the lesion location is represented by coordinate or mask information, and the dependency relationship is appended in the form of a One-Hot or embedding vector.

[0089] Exemplarily, when performing temporal property fusion, if there are multiple CT scan time points for the target disease, the growth rate information of the lesion volume or the morphological change rate in each stage can be extracted; the typical morphological differences in different stages are encoded.

[0090] Exemplarily, when performing prior restriction processing, the temporal fusion vector can be calibrated according to the "stage restriction rule" recorded in the medical knowledge graph. For example, when the lesion is in stage III, it is required that its diameter ≥ X mm; if not, the dimension is corrected to the compliance interval.

[0091] As an alternative implementation, please refer to Figure 2 , Figure 2 is a flowchart of a method for enabling a generator to output lesion image data based on a knowledge prior vector provided by an embodiment of the present disclosure, including steps S201 to S204, where:

[0092] S201: Numerically process the size, shape, position, and anatomical structure dependency in the lesion attributes to generate a lesion attribute vector, where the lesion attribute vector is used to characterize the spatial features and anatomical prior of the target disease lesion;

[0093] S202: Perform time series feature extraction on the temporal attributes describing the progression law of the target disease, and fuse the obtained temporal features with the lesion attribute vector to form a temporal fusion vector, where the temporal fusion vector is used to characterize the dynamic changes of the lesion in different stages and progression phases;

[0094] S203: In the temporal fusion vector, perform prior restriction processing based on the critical threshold and stage interval defined by the lesion boundary and stage restriction rule obtained from the medical knowledge graph, so as to label and restrict the temporal fusion vector in the lesion shape and stage dimensions;

[0095] S204: Input the temporally fused vector after prior restriction processing into the generator, and use the lesion spatial features and progression stage information contained in the temporally fused vector to generate lesion image data during the training and inference phases.

[0096] In order to enable the generator to more accurately output lesion image data that meets clinical requirements, it is necessary to numerically process and fuse the lesion attributes and temporal attributes, and perform prior restriction processing based on the lesion boundary and stage restriction rule in the medical knowledge graph.

[0097] Regarding the above S201:

[0098] Convert discrete or continuous information such as the size, shape, location of the lesion, and its anatomical structure dependence relationship into a numerical vector that can be read by the generator, endowing the lesion with the ability to be expressed in terms of "spatial features" and "anatomical priors".

[0099] In specific implementation, one-dimensional or multi-dimensional numerical quantities can be defined for the maximum diameter, minimum diameter, or major axis and minor axis of the lesion. For example, 10 mm - 15 mm represents a size range; if the common shapes of the lesion include spherical, oval, lobulated, etc., the shape type can be recorded using the One-Hot or embedding vector method; indexed by three-dimensional (x, y, z) coordinates or based on anatomical sections (such as lung lobes, liver segments) to record the location of the lesion in the original image coordinate system or anatomical structure.

[0100] In addition, for the anatomical structure dependence relationship, if the medical knowledge graph indicates that "lesions often appear near the vascular structures in a certain lung segment", a specific identifier or numerical quantity can be assigned to this relationship to represent the degree of closeness of its relationship with the surrounding tissues.

[0101] Exemplarily, if the target disease is lung cancer: the size quantization dimension (14.0, 9.5) represents a major axis of 14 mm and a minor axis of 9.5 mm; the shape encoding (0, 1, 0) represents an oval shape (One-Hot vector); the position coordinates (45, 60, 32) represent the three-dimensional coordinates of the center point of the lesion within a certain lung lobe; the anatomical structure dependence can be additionally represented by 1 for "adjacent to the hilar vessels". Finally, these numericalized information are combined into a lesion attribute vector, which can be further fused with the temporal information in subsequent steps.

[0102] Regarding the above S202:

[0103] The extraction and fusion of the time series features of the temporal attributes enable the generator to reflect the dynamic evolution law of the disease over time or with the staging stage (such as how the lesion grows larger or changes in shape from stage I to stage II), so that the synthesized lesion images have multi-stage and a series of time-varying features.

[0104] In specific implementation, the typical manifestations (such as the rate of shape change, the growth rate of the tumor, etc.) of the target disease at different stages (stages I, II, III, IV) or clinical progression stages can be queried from the medical knowledge graph. If there are multi-stage imaging data or the evolution rate of the lesion recorded in the clinical literature, multi-scale temporal embedding (Multi-Scale Temporal Embedding), LSTM or other temporal models can be used to extract the feature vectors reflecting the change trend of the lesion.

[0105] Perform dimension-by-dimension concatenation or weighted combination of the temporal features and the aforementioned lesion attribute vectors to form a temporal fusion vector, representing the comprehensive attributes (size, shape, speed, stage, etc.) that the lesion should have at the current moment (or stage).

[0106] Exemplarily, for the typical growth rates (0.5, 0.7, 1.2) (unit: mm / week) during stage I to stage III of lung cancer, they can be used as temporal features and concatenated with the lesion attribute vector (14.0, 9.5) (size) to form three-dimensional features such as (14.0, 9.5, 0.7);

[0107] If the morphology of the disease shows multiple lobulations in stage III, the morphology type code (lobulated) can be additionally marked in the temporal fusion vector to make the generator generate more complex contours during synthesis.

[0108] Regarding the above S203:

[0109] The present disclosure uses the critical thresholds, stage intervals, etc. defined in the medical atlas to perform range detection and correction on the temporal fusion vector to ensure the "rationality" of the lesion in terms of morphology and stage, and to avoid generating unrealistic lesion images.

[0110] In specific implementation, for the boundary restriction, if the smoothness of the lesion edge is specified in the atlas as the interval of 0.1, 0.3 (indicating the degree from smooth to lobulated), when the corresponding dimension in the fusion vector exceeds 0.3, it is truncated or rolled back to 0.3; for the stage restriction, if the stage is stage II and the temporal features indicate that the tumor diameter should be between 8 mm and 20 mm, then when the fusion vector shows that the diameter < 8 mm or > 20 mm, correction is performed.

[0111] Exemplarily, when the stage is stage III, but the dimension value of "morphological concavity and convexity" in the fusion vector is too small (too smooth), the value of this dimension is automatically increased according to the restriction rule to reflect the lobulation or burr of the stage III lesion;

[0112] When the stage is stage I, but the fusion vector indicates that the major axis of the tumor is greater than 30 mm, it is then rolled back to less than 8 mm to meet the size restriction of stage I.

[0113] In this way, after the prior restriction processing is completed, the obtained temporal fusion vector is more in line with the prior of the medical knowledge atlas in terms of lesion morphology and stage, providing a reliable input for subsequent synthesis.

[0114] Regarding the above S204:

[0115] The present disclosure uses the above-mentioned vector that has been fused and processed by prior restrictions as the main input for the generator during the training and inference stages, so that the generated lesion images are closer to the clinical real manifestations.

[0116] In a specific implementation, for the training stage: In the adversarial training loop, one or a batch of temporal fusion vectors are extracted from the atlas prior in each round; the generator outputs a lesion image based on the fusion vector; the discriminator performs difference evaluation and compliance determination, and feeds back the difference loss and compliance loss; continuous iteration is performed until the generator learns to produce lesion images with "high compliance and low difference".

[0117] For the inference stage: Given the target stage or morphological requirements, the input vector is obtained through the same fusion and prior constraint processing; this vector is input into the trained generator, which outputs the corresponding lesion image; the output result can be combined with real images or subsequent detection models to achieve the simulation and analysis of rare morphological or multi-stage lesions.

[0118] Exemplarily, if it is clinically necessary to simulate the lesion morphology at stage III of liver cancer, the size threshold and lobulation degree information at stage III can be first obtained from the atlas and incorporated into the lesion attribute vector and temporal attribute vector, and then input into the generator after prior constraint.

[0119] The generator can output lesion images conforming to the typical manifestations at stage III (such as a diameter of 25 mm and a multi-lobulated margin) during the inference stage for physicians or AI detection models to test or compare.

[0120] In this way, during the training stage, the vectors after fusion and prior constraint processing can be used to continuously optimize the generator, making it approximate the multi-level prior information constraints; during the inference stage, lesion images conforming to the target stage or morphological characteristics can be generated by inputting the same vectors, thus achieving the simulation and detection requirements for rare lesions or multi-stage lesions in both stages, ensuring the internal consistency between morphology and stage, and better meeting the clinical accuracy and lesion diversity requirements.

[0121] Regarding the above S104:

[0122] The present disclosure measures in real time the deviation degree of the lesion image output by the generator from the prior vector in the spatial, temporal, and staging dimensions; by invoking the lesion boundary and staging rules in the atlas, a determination of "whether it violates the clinical prior" is made for the morphology and staging, forming a "compliant" or "non-compliant" feedback.

[0123] In a specific implementation, after the generator outputs the lesion image data, the discriminator extracts features from it, and calculates the difference metrics (such as L1 distance, cosine similarity, etc.) between the extracted image features and the knowledge prior vector in the spatial attribute, temporal progression, and staging constraint dimensions respectively, forming a difference evaluation result indicating the deviation degree of the lesion image from the prior vector.

[0124] In addition, while performing the difference evaluation, the discriminator further invokes the lesion boundary and staging constraint rules recorded in the atlas for compliance checks of the morphology and staging.

[0125] For example, if the morphology exceeds the allowable range of the lesion boundary, the discriminator outputs non-compliant morphology; if the staging status is not within the a priori specified staging interval, the discriminator outputs non-compliant staging.

[0126] Finally, the morphology compliance result and the staging compliance result are integrated into a compliance determination result.

[0127] Exemplarily, after the discriminator extracts features from the lesion image data through a feature extraction method such as CNN, an image feature vector is obtained, and it is compared one by one with the knowledge prior vector in the spatial dimension (such as the smoothness of the morphology edge) and the temporal dimension (such as the growth rate), and metrics such as cosine similarity or L1 distance are calculated.

[0128] Exemplarily, if the lesion edge is too sharp or the concavity and convexity exceed the staging limit threshold in the atlas, it is determined that the morphology is non-compliant; if the temporal staging interval does not match (for example, the staging information should be stage II but conforms to the morphology of stage IV), then it is determined that the staging is non-compliant. The discriminator outputs the comprehensive determination results of morphology and staging.

[0129] As an alternative implementation, the difference evaluation of the lesion image data output by the generator and the knowledge prior vector includes:

[0130] Extract lesion image features from the lesion image data output by the generator through a feature extraction algorithm to form an image feature vector;

[0131] Compare the image feature vector with the knowledge prior vector one by one in the spatial attribute dimension, the temporal progression dimension, and the staging limit dimension, and calculate the difference metrics respectively;

[0132] Determine the difference evaluation result according to the difference metric results of each dimension, and use the difference evaluation result to indicate the deviation degree of the lesion image data from the knowledge prior vector.

[0133] In the adversarial training framework of the present invention, the discriminator needs to simultaneously consider the matching degree of the lesion image data and the knowledge prior vector in multiple dimensions (such as spatial attributes, temporal progression, staging limit). Through the difference evaluation, the deviation degree of the lesion image from the prior vector can be quantified, and further a difference evaluation result can be generated, providing a basis for subsequent compliance determination or loss function fusion.

[0134] Convert the lesion image data output by the generator into a numerical feature representation, which is convenient for one-dimensional comparison with the prior vector, and then evaluate the differences between the lesion and the prior vector in terms of morphology, size, time series, etc.

[0135] In a specific implementation, a convolutional neural network (CNN) can be used to perform multi-layer convolution and pooling operations on the lesion image data to obtain one or more levels of feature maps. A fixed-length vector can be output at the last layer or the penultimate few layers, which is called the image feature vector. In addition, if richer morphological information is needed, local texture features can be extracted from the penultimate layer and then fully connected to obtain a vector, or methods such as global average pooling (GAP) can be used to ensure the controllability of the vector length.

[0136] In a specific implementation, if the generator output contains the complete lesion and background, the lesion area can also be initially segmented or cropped first to reduce the interference of the irrelevant background on the features; if full-image feature extraction is adopted, it is necessary to ensure that the CNN can focus on the main lesion area and reduce the interference of background noise through methods such as the attention mechanism.

[0137] Exemplarily, if the size of the lesion image is 128×128 pixels, a lightweight CNN structure (such as 4-layer convolution) can be used to extract a 64-dimensional lesion image feature vector F img =[ f 1 , f 2 ,…, f d ] ; where is a high-dimensional vector obtained after feature extraction algorithms such as CNN; d is the dimension of the feature vector. If the CNN outputs 64 dimensions, then d = 64; if a more complex network outputs 128 dimensions, then d = 128; represents the i-th eigenvalue, such as the morphological information or texture information learned by the deep network, etc.

[0138] If more spatial resolution information is needed in the implementation, a layer of SPP (Spatial Pyramid Pooling) can also be added after the CNN for multi-scale feature fusion.

[0139] The present disclosure calculates the difference metrics respectively in the spatial attribute dimension, the temporal progression dimension, and the staging restriction dimension, avoiding mixing all the information into an overall loss and ignoring the key details. Through dimension-by-dimension comparison, the deviation degree of the lesion in terms of morphological size, edge smoothness, temporal growth rate, staging symbol, etc. can be finely measured.

[0140] In a specific implementation, if the prior vector dimension contains lesion spatial attributes (such as size, morphological encoding), temporal attributes (such as growth rate or staging label), staging restrictions, etc., then the corresponding mapping or encoding should also be included in the image feature vector;

[0141] If the number of dimensions is inconsistent, an index or sub-vector correspondence can be established between the prior vector and the image feature vector, and expansion / dimensionality reduction can be performed at the output end of the CNN or the prior vector end.

[0142] In addition, for the calculation of the difference metric:

[0143] Morphological / spatial attribute dimension: For example, the L2 distance (L2 Norm) or cosine similarity is used to measure the similarity between lesion morphological vectors;

[0144] Temporal dimension: For example, by comparing the mapped values of "growth rate" or "stage", subtraction or similarity calculation is performed with the corresponding values learned from the image features;

[0145] Stage restriction dimension: For example, if the stage restriction stipulates that this lesion shall not exceed a certain morphological threshold, the corresponding stage coding of the image feature vector is compared with the prior stage value, and if it exceeds the interval, the difference is marked.

[0146] Furthermore, the difference metric results of each dimension are combined to generate a difference evaluation result, which is used to indicate the overall deviation degree, or as a part of the discriminator output to the generator as a reverse optimization signal in the adversarial training process.

[0147] In a specific implementation, different weights can be assigned to the three major dimensions of morphology, time series, and stage restriction, and the obtained difference metric values are multiplied by the corresponding weights and then summed or averaged; a total difference evaluation score is generated , the larger the value, the higher the deviation degree of the lesion, and the smaller the value, the more it conforms to the prior.

[0148] In adversarial training, can be combined with other losses (such as morphological / stage non-compliance items in compliance determination) for the generator to adjust network parameters during gradient backpropagation;

[0149] In the inference stage, can also be output to the doctor or subsequent detection model to prompt the compliance degree of the current synthesized lesion in the prior dimension.

[0150] Exemplarily, in adversarial training, if the value is large, it indicates that the lesion image does not conform to the prior seriously, and a higher difference loss will be generated in the training loss function for backpropagation to the generator; if is small, it indicates that the morphological and temporal features of this lesion have a high degree of agreement with the prior, and its penalty intensity can be reduced during training.

[0151] Exemplarily, in the inference scenario, when the system is only inferring or simulating lesions, by calculating It is possible to evaluate whether the currently synthesized lesion exceeds the prior range; if it deviates excessively, the user is reminded to adjust the staging settings or attribute parameters. It is also possible to compare with subsequent detection results to form an index of "lesion rationality" or "simulation case reliability".

[0152] Exemplarily, if the typical diameter range of stage II lung cancer is specified a priori as [10 mm, 20 mm] and the growth rate is less than or equal to 1.0 mm / week; the generator outputs the lesion image feature vector F img =[15mm,0.8,…] , which falls within the range both spatially and temporally; then the difference metric is small, and it is determined that the lesion conforms to the prior.

[0153] Exemplarily, if the lesion is specified as stage III a priori, lobulation or spiculation is allowed in the morphology, but the maximum diameter should not be less than 25 mm; the maximum diameter in the lesion image output by the generator is only 18 mm, then the difference metric in the staging restriction dimension is large, resulting in an increase; during the training process, the penalty on the generator will be increased to prompt it to generate a larger diameter or a lesion more in line with the stage III morphology in the next iteration.

[0154] Exemplarily, if the difference metric values for each dimension are: spatial difference = 0.15, temporal difference = 0.02, staging difference = 0.3, and the weights are 0.4, 0.3, 0.3, then:

[0155]

[0156] When < 0.10, it can be considered that the lesion image is in very good agreement with the prior; when > 0.30, then the deviation is large and the generator needs to be further corrected.

[0157] Exemplarily, the prior vector is also d-dimensional, or can be split into sub-vectors corresponding to , and the cosine similarity can be expressed as:

[0158]

[0159] Among them, " " represents the vector dot product operation, and represent the vector norm.

[0160] In this way, through the above processes of feature extraction, dimension-by-dimension comparison, and difference evaluation results, the present disclosure can meticulously measure the degree of fit between the lesion image data and the prior vector during the adversarial training or inference process. After combining this difference evaluation with the subsequent determination of morphology / stage compliance, more targeted feedback can be provided to the generator, enabling it to continuously approach the clinical real needs in the synthesis of lesion spatial features and chronological staging features, thereby achieving the effects of improving the credibility of lesion image synthesis and assisting in the training of disease monitoring models.

[0161] As an optional implementation manner, the determination of the compliance of the morphology and staging status of the lesion image data includes:

[0162] Based on the boundary limit rules obtained from the medical knowledge graph, the edge contour in the lesion image data is matched with a preset lesion morphology threshold, and a morphology non-compliance indication is output when the lesion morphology exceeds the lesion morphology threshold;

[0163] Based on the staging interval in the temporal attributes, the staging status of the lesion image data is compared with the staging restrictions in the medical knowledge graph, and a staging non-compliance indication is output when the staging status exceeds the staging interval;

[0164] The results of morphology compliance determination and staging compliance determination are integrated to form a compliance determination result, and the compliance determination result is used to guide the continuation or correction of the adversarial training process.

[0165] Among them, "staging restrictions" refers to the comprehensive restriction rules for lesion staging in the text of this application, which may include constraints on elements such as size, morphology, and growth rate; "staging interval" focuses more on numerical ranges (such as upper and lower limits of diameter and growth rate).

[0166] In the discriminator of the present invention, in addition to difference evaluation, it further determines whether the "lesion morphology" and "staging status" exceed the restricted ranges defined in the medical knowledge graph. If the morphology is non-compliant or the staging is non-compliant, the discriminator will output the corresponding non-compliance indication to help the adversarial training process make targeted corrections to the generator.

[0167] In specific implementation, the medical knowledge graph usually records the morphological characteristics and boundary restrictions of the target disease at different stages. For example, if there are threshold intervals for the contour smoothness, burr coefficient, concavity and convexity degree, etc. of a certain stage of lung cancer lesions, the present disclosure will extract this interval and perform matching in the discriminator.

[0168] In the medical knowledge graph, for different stages or different progression stages such as stage I, stage II, stage III, and stage IV of the target disease, corresponding intervals of size, morphology, growth rate, etc. are often defined. For example:

[0169] The diameter of stage I tumors is usually less than 20 mm;

[0170] In stage III, the morphology is more lobulated or invasive, and the size is greater than 25 mm, etc.

[0171] These staging intervals can be recorded in the medical knowledge graph through numerical upper and lower limits or specific annotations.

[0172] Furthermore, when the discriminator extracts features from the lesion image, it can additionally detect or estimate the edge contour of the lesion, such as morphological indicators like smoothness, the number of lobulated protrusions, and the degree of spiculation.

[0173] When the discriminator extracts features from the lesion image, it can also infer or estimate its corresponding "staging tendency". For example, through comprehensive analysis of information such as morphology, size, and texture, a staging judgment value is obtained;

[0174] If the inferred staging status exceeds the established interval, such as "should be in stage II but shows a size or degree of invasion equivalent to stage IV", then an "inconsistent staging" indication is output.

[0175] If the above-mentioned morphological indicators exceed or are lower than the upper and lower threshold values in the medical knowledge graph, it will be determined as "morphologically inconsistent", and the corresponding indication will be output;

[0176] For example, if stage II requires that the degree of spiculation does not exceed a certain value, once it exceeds this value, a morphological non-compliance prompt can be output.

[0177] The discriminator will combine the determination results of morphological compliance and staging compliance to form an overall "compliance determination result". If both are compliant, a "compliant" indication can be output; if non-compliance occurs in any dimension, it can be recorded as "non-compliant", along with the specific reasons for exceeding the limit in terms of morphology or staging.

[0178] This compliance determination result can act on the generator together with the "difference metric" generated by the difference evaluation during the adversarial training process: if the determination result shows morphological or staging non-compliance, the generator will correct the lesion sample by increasing the loss term in the next round of training; if it is compliant, it means that the lesion basically meets the atlas prior and no further correction is required in terms of morphology or staging.

[0179] Exemplarily, if a certain stage of lung cancer stipulates that the range of the degree of spiculation is between 0.1 and 0.3, and the discriminator detects that the spiculation coefficient of the lesion = 0.45, exceeding the upper limit of 0.3, then an "indication of morphological non-compliance" is output. A greater loss will be imposed on the generator during training, causing the generator to reduce the degree of spiculation in the next round of synthesis.

[0180] Exemplarily, if the specified tumor diameter range for stage II is 10 - 20 mm, and the inferred diameter in the lesion image is 25 mm, it is determined as "stage non - compliant". Subsequently, the adversarial training will increase the corresponding penalty term to reduce the diameter to within 10 - 20 mm when the generator synthesizes.

[0181] Exemplarily, if the lesion is compliant in the morphological dimension but non - compliant in the staging dimension, the final compliance determination result is recorded as "non - compliant", and the corresponding gradient update is performed on the generator during training; if both are compliant, a compliance indication is output, and the difference loss is relatively low.

[0182] In this way, compared with only performing simple difference evaluation, this embodiment makes a more fine - grained determination through "morphological compliance" and "staging compliance", which can significantly reduce the generation of lesion samples by the generator that do not conform to clinical common sense, thereby improving the credibility of the final generation result. Regardless of whether the lesion is in the initial stage, the mid - progress stage, or the late stage, its size and morphological limitations can be flexibly determined according to the staging interval, and it is applicable to various malignant tumors or other disease types.

[0183] It should be emphasized that the present disclosure is not limited to malignant tumors, and can also be extended to benign lesions or lesion scenarios of other organs; only the corresponding morphological rules or staging specifications need to be added to the medical knowledge graph.

[0184] Regarding the above S105:

[0185] The present disclosure combines the "difference evaluation result" and the "compliance determination result" into a total loss to achieve the common optimization of the lesion image in terms of spatial morphology and clinical staging during adversarial training; and improves the "clinical acceptability" of the generator output through multiple rounds of iteration, reducing the deviation from prior knowledge.

[0186] In a specific implementation, the difference loss is obtained according to the difference evaluation result, the compliance loss is obtained according to the compliance determination result, and they are fused with a preset ratio to form a total loss. The gradient backpropagation can be performed on the generator through the standard GAN training process; after each round of training, the discriminator then performs difference evaluation and compliance determination on the newly output lesion image. This cycle is repeated for multiple rounds until the total loss is lower than the threshold or the number of training epochs reaches the upper limit. Finally, the lesion image data that can accurately generate in line with the anatomical structure and staging prior requirements is obtained.

[0187] Among them, for the difference loss, during adversarial training, the discriminator first performs a difference evaluation on the lesion image data and the prior vector, and calculates the deviation degree of the lesion in terms of spatial morphology, temporal characteristics, and staging dimension. This deviation degree can be directly regarded as the "difference loss", or appropriate weighting / normalization processing can be performed on this basis to form the value finally used for gradient backpropagation.

[0188] For compliance losses, the present disclosure records each "non-compliance" matter as a corresponding penalty amount, referred to as "compliance loss". For example, if the morphological overstep degree is large, a higher weight can be assigned to the loss; if the staging and the prior are extremely discrepant, the loss can also be increased.

[0189] When the lesion meets the morphological and staging priors, the compliance loss can be recorded as 0 or a small value.

[0190] Exemplarily, the difference loss uses the L2 distance, and the compliance loss uses the "morphological overstep penalty term" or the "staging overstep penalty term", which are combined into the total loss at a ratio of 0.5:0.5; when the total loss is lower than a certain threshold (such as 0.01) or the number of training rounds reaches a preset upper limit (such as 200 epochs), the training can be stopped; at this time, the generator can better synthesize lesion images that conform to the prior rules.

[0191] As an alternative implementation manner, in practical applications, a phased strategy can also be adopted:

[0192] In the early stage of training, increase the proportion of the difference loss so that the generator first learns the basic morphology;

[0193] In the later stage, increase the proportion of the compliance loss so that the generator is more accurate in terms of detailed staging and morphological features.

[0194] Regarding the above S106:

[0195] The present disclosure fuses the synthesized lesion with the original medical image to form a "simulated case" or an "image with lesion features" and inputs it into the subsequent disease monitoring model to simulate scarce lesion data in a clinical scenario and improve the recognition ability of the detection model for multi-staged or rare morphological lesions.

[0196] In a specific implementation, after obtaining lesion image data that meets the requirements of multi-level prior information, it is combined with the original medical image (such as through fusion algorithms such as coordinate mapping, seamless cloning, and gradient correction) to generate synthetic image data. This synthetic image data not only retains the original image context but also adds lesion information that conforms to the temporal sequence and staging prior at the target position.

[0197] Input the synthetic image data into the disease monitoring model for inference. For example, use a deep learning detection / segmentation model to perform inference on the synthetic image to obtain potential lesion regions and their possible staging information, morphological parameters, etc.

[0198] Exemplarily, the output result form can be:

[0199] Lesion coordinates or masks;

[0200] Lesion morphological indicators (major axis, minor axis, density, etc.);

[0201] Stage recognition (such as stage I, stage II, etc.) or benign / malignant judgment (such as indication of benign nodules or malignant tumors).

[0202] In addition, the result can be compared or updated with the medical knowledge graph again to form a closed-loop knowledge and model iteration.

[0203] Exemplarily, networks such as UNet, Mask R-CNN, YOLO, etc. can be selected to automatically detect / segment the synthetic image; output potential lesion location, size, shape or staging information. If the identified stage is consistent with the stage recorded in the medical knowledge graph, it is determined that the model detection is correct; if the detection deviation is too large, it can be fed back to the medical knowledge graph or the GAN model for correction.

[0204] In this way, the present invention can make full use of the multi-level prior information (including chronological staging) recorded in the medical knowledge graph in the lesion synthesis link, and use a combination of differential evaluation and compliance determination in the adversarial training stage to make the generated lesions better meet the clinical real needs.

[0205] After the synthetic image is input into the disease monitoring model, in the case of rare lesions or scarce staging samples, it can supplement the shortage of real samples, thereby improving the accuracy of disease detection or staging recognition.

[0206] As an alternative implementation, the combining the lesion image data meeting the requirements of the multi-level prior information with the original medical image includes:[[]]END

[0207] Based on the position attribute in the lesion image data, convert and align the lesion pixel coordinates to the corresponding coordinate positions of the original medical image;

[0208] After coordinate alignment, perform a preset pixel fusion process on the boundary area where the lesion area overlaps with the original medical image, and the pixel fusion process includes gradient correction or seamless cloning operations at the fusion boundary;

[0209] After completing the pixel fusion process of the boundary area, output the merged image data as synthetic image data to reflect the position and morphological information of the lesion in the original medical image.

[0210] In a specific implementation, the lesion image data usually includes position attributes or spatial coordinate annotations, reflecting the reference positioning of the lesion before synthesis. If the positioning is inconsistent with the coordinate system of the original medical image, conversion is required first. The specific approach can be divided into the following steps:

[0211] In the generation stage, if the center point (or key point) and size information of the lesion image have been recorded, coordinate alignment can be achieved through simple translation, scaling or two-dimensional / three-dimensional rotation;

[0212] If the lesion morphology is relatively complex, a small-range interpolation algorithm (such as nearest neighbor or bilinear interpolation) can also be introduced during coordinate transformation to ensure the consistency of image resolution and coordinate system.

[0213] Through the above coordinate mapping operation, the lesion image can be seamlessly placed at the corresponding anatomical position of the original image, thus ensuring the rationality of the final synthesized image in terms of anatomical structure.

[0214] For example, in the lung scenario, the lesion is placed in the designated lung segment; in the liver scenario, the corresponding coordinate range of the designated liver segment is used.

[0215] Furthermore, when the coordinates of the lesion image are aligned with those of the original medical image, pixel-level differences will appear at the boundaries or overlapping areas. The present disclosure solves the possible obvious stitching marks or abrupt transitions in the following ways:

[0216] Method 1: Gradient correction. If there is a jump in grayscale or brightness in the boundary area, gradient smoothing can be performed to gradually transition the pixels at the lesion edge to the surrounding tissues; this can be achieved by calculating the gradient field in the boundary area and performing low-intensity interpolation or blending processing.

[0217] Method 2: Seamless cloning. If a higher-quality and better visually continuous fusion is required, seamless cloning techniques such as Poisson Blending can be used to clone the lesion area as the foreground into the background image, so as to naturally connect with the surrounding tissues in terms of color and brightness. This method is suitable for scenarios with large differences in color or grayscale gradients.

[0218] Method 3: Processed synthesized image data. After the above fusion steps are completed, the merged image data can be output to become "synthesized image data". This synthesized image not only retains the background and anatomical structure information of the original image but also presents the lesion image at the corresponding position, ensuring smooth connection and visual consistency in the edge area.

[0219] Furthermore, after coordinate alignment and boundary fusion, the finally obtained synthesized image can not only accurately display the expected anatomical position of the lesion but also maintain the integrity of the lesion morphology and the background of the original image.

[0220] Exemplarily, the synthesized image data is applicable to the following scenarios:

[0221] Disease monitoring model training: Using the generated simulation cases to expand the training set and improve the recognition ability for multi-stage or rare lesion samples;

[0222] Clinical assistance: Physicians can observe the interaction between the lesion and the surrounding tissues in the synthesized image or compare it with real cases to assist in diagnosis or teaching;

[0223] Algorithm verification: In a research environment, different fusion algorithms (such as gradient correction and seamless cloning) are used to compare the quality of the synthesized image and the improvement effect of detection accuracy.

[0224] In this way, through the coordination of the three steps of lesion pixel coordinate conversion, boundary fusion processing, and synthesized image output in this embodiment, the finally synthesized image can conform to clinical logic in terms of morphology and position, and maintain sufficient pixel continuity, avoiding obvious splicing marks or unnatural transitions.

[0225] As an optional embodiment, after combining the lesion image data that meets the requirements of the multi-level prior information with the original medical image and inputting it into a preset disease monitoring model, generating potential lesion detection results includes:

[0226] Load a preset disease monitoring model and perform image preprocessing operations on the synthesized image data;

[0227] In the model inference stage, perform feature extraction and segmentation calculation on the input synthesized image data to generate the position information, size, and shape of the potential lesion area;

[0228] Based on the potential lesion information obtained from model inference, generate potential lesion detection results; the potential lesion detection results include lesion spatial coordinates, morphological parameters, and staging information.

[0229] In a specific implementation, the disease monitoring model can be a deep learning structure (such as UNet, Mask R-CNN, YOLO series, 3D ResNet, etc.) or a hybrid model combining traditional image processing and machine learning algorithms.

[0230] It can be understood that usually, this model has been previously trained or fine-tuned on a real case set or a data set containing real and synthetic images and stored in a file format (such as.pth,.h5, etc.) in a local or server environment. The system loads this model first in the inference link for subsequent identification or detection of the synthesized image data.

[0231] Before inputting the synthesized image into the model, several basic preprocessing operations need to be performed to ensure the correctness and consistency of the model input format. Exemplarily, the preprocessing methods can include: resolution or size adjustment, normalization, channel number or dimension transformation, and cropping or central alignment, etc.

[0232] Furthermore, in the inference link of the disease monitoring model, it will perform layer-by-layer convolution (in the CNN scenario) on the input image or use structures such as vision transformers to extract feature maps.

[0233] Exemplarily, if the model is of the object detection type (such as YOLO, Faster R-CNN), then the bounding box and confidence of the potential lesion area are output;

[0234] If the model is of the image segmentation type (such as UNet, Mask R-CNN), then a pixel-level segmentation mask (Mask) of the lesion is generated, and the lesion position, size, and shape features can be further calculated.

[0235] Post-process the detection or segmentation output to extract key information such as the center coordinates, major axis, minor axis, and contour shape of the lesion in the image coordinate system. If a 3D model is used, volume, three-dimensional shape features, etc. can also be obtained.

[0236] Through the above inference process, the position information (such as coordinates (x, y) or (x, y, z)), size (such as diameter, volume), and shape features (such as the number of lobes, presence of spicules) of potential lesions in the synthetic image can be identified. This part of the information can not only help verify the consistency of the synthetic lesions with the prior requirements, but also be compared with the results of real clinical cases to further evaluate the performance of the model.

[0237] After completing the inference and post-processing, organize and output each index to form a readable potential lesion detection result. Exemplarily, the result may include: lesion spatial coordinates: indicating the position of the lesion center point or the segmentation mask; morphological parameters: such as major axis, minor axis, contour boundary details, concavity and convexity, etc.; staging information: if the model includes staging discrimination logic or is associated with a medical knowledge graph, the possible clinical stage (such as stage I, stage II) of this lesion can be output; confidence or score: the probability or consistency score of the model for the detection result, assisting the user to evaluate its accuracy.

[0238] In this way, after the synthetic image data is successfully input into the preset disease monitoring model, the system can successfully complete three stages: model loading and preprocessing, model inference, and result output. Finally, a potential lesion detection result is generated, including key information such as the spatial coordinates, morphological parameters, and staging information of the lesion, helping the user or clinician quickly locate and evaluate the morphological features and possible clinical stages of the lesion.

[0239] The lesion synthesis and monitoring process of the present invention can not only address the pain point of the lack of real cases, but also play an important role in AI model evaluation or teaching;

[0240] If the detection result does not match the expected stage, it can also be fed back to the medical knowledge graph or generator to further strengthen the adversarial training or graph update mechanism, forming a closed-loop "data - model - graph" mechanism.

[0241] As an alternative implementation, in response to the staging information, lesion size, and morphological features of the potential lesion detection result not matching the multi-level prior information, record the non-matching information as updated data;

[0242] Send the updated data to the update interface of the medical knowledge graph to adjust the information of the target disease in the medical knowledge graph.

[0243] In the foregoing implementation, it has been described how to input synthetic image data into a preset disease monitoring model and obtain a potential lesion detection result. To achieve continuous update and dynamic improvement of the medical knowledge graph, the present invention further introduces a feedback mechanism of "recording differential information - updating the medical knowledge graph - adjusting the parameters of the generator / monitoring model", so that corrections or expansions can be automatically or semi-automatically performed when the detection result is inconsistent with the prior of the graph.

[0244] In a specific implementation, after the model inference is completed, the system has obtained potential lesion detection results such as the lesion spatial coordinates, morphological parameters, and staging information (see the foregoing implementation for details). Compare this detection result with the corresponding prior information stored in the medical knowledge graph (such as lesion size, morphological coding, staging threshold, etc.). If the detection result has significant differences from the medical knowledge graph in terms of size, morphology, and staging, it is marked as a "non-consistent" or "new knowledge point" situation.

[0245] If the system detects this non-consistency or discovers potential new morphological features, immediately trigger the entry of the medical knowledge graph update process, and package all relevant data for use in the next step of recording and analysis;

[0246] This detection and comparison can be carried out in an automatic mode or reviewed by technical personnel or clinicians in a semi-automatic mode to avoid writing noise or obvious anomalies back to the medical knowledge graph.

[0247] Further, when comparing the potential lesion detection result with the prior information, if it is found that "the staging exceeds the specified range" or "the morphological features are not within the originally defined interval", record the differential information in a structured form, such as "deviation amount", "new morphological description", "inferred staging", etc.;

[0248] For example, for the prior of stage III lung cancer, it is specified that the diameter ≥ 25 mm, while the detected diameter is only 22 mm. The differential item can be recorded as "the diameter is 3 mm short", and the potential staging is "inferred to be stage III but the size does not meet the standard".

[0249] Save the differential information to the system database or cache, and associate it with the unique detection result ID and the node ID (or relationship ID) of the prior information. This can ensure data traceability and facilitate subsequent review or expert review.

[0250] If it is confirmed that the detection results show "lesion boundary morphology not recorded in the prior knowledge" or "larger / smaller staging intervals", additions or corrections can be made to the attributes of the corresponding nodes (such as "Morphological Features of Stage III Lung Cancer") or relationships (such as "Morphology-Staging Association") in the atlas;

[0251] For example, if multiple detections indicate that the diameter of stage III tumors can be as low as 22 mm, but the atlas originally specified it as ≥25 mm, the lower limit of the staging size can be modified to 22 mm, or a note "The lower limit can be extended to 22 mm" can be newly recorded in the attribute.

[0252] If a morphology that has never been recorded before appears, such as a rare honeycomb-like edge in a lung cancer at an advanced stage, the system can add a new node "Honeycomb-like Morphology" in the medical knowledge atlas and establish an association relationship with the node "Stage IV Lung Cancer", and record its possibility and conditions;

[0253] In this way, the medical knowledge atlas can continuously absorb new clinical features or abnormal staging manifestations, enabling more abundant prior support for subsequent lesion synthesis and detection.

[0254] In specific implementation, when performing updates, the original version of the prior information can be retained, and the new version can be marked as "extended" or "corrected", enabling clinicians or researchers to trace the change process and its reasons.

[0255] When the medical knowledge atlas is updated, during the next round of training or fine-tuning, the generator will load new prior constraint rules or temporal attribute intervals, thereby correcting the morphology assignment of lesion image synthesis;

[0256] If new staging thresholds are added to the medical knowledge atlas, in the training loss function, strengthen the penalty or relax the constraint for lesion synthesis cases that exceed or are lower than the threshold.

[0257] If the newly discovered staging or morphological features appear with a certain frequency in the real environment, the disease monitoring model (such as a deep learning detection / segmentation network) can also be allowed to perform a small amount of incremental training (Fine-tuning) on new synthetic images or real data;

[0258] This can enhance the ability to recognize rare staging or brand-new morphologies, enabling the model to continuously evolve and adapt to the latest clinical knowledge.

[0259] After multiple rounds of detection, it is found that a certain staging threshold is too small, resulting in a large amount of "differential information", and the medical knowledge atlas is corrected after review; the next round of training of the generator will use the updated threshold to synthesize more lesion images that conform to the clinical reality;

[0260] After being retrained using these synthetic images, the disease monitoring model showed a significant improvement in the recognition rate of this stage. The entire process forms a closed-loop iteration of "medical knowledge graph - generator - monitoring model".

[0261] In this way, when the present disclosure faces multi-stage, rare lesions or emerging new clinical findings, it can quickly adapt and gradually improve. Its ultimate goal is to build an intelligent ecosystem that can interactively learn with the medical knowledge graph in both the synthesis and detection processes.

[0262] In summary, this application deeply integrates the medical knowledge graph with the generative adversarial network (GAN). By comprehensively utilizing multi-level prior information such as lesion size, shape, temporal progression law, and staging limitations, it significantly enhances the simulation and detection capabilities for multi-stage and multi-shaped lesions. Through the determination of lesion staging limitation rules and morphological dependence relationships, the generator can more accurately generate lesion images that conform to clinical priors during the adversarial training process, reducing the appearance of invalid or unreasonable lesion morphologies, and at the same time improving the acceptability of synthetic images in clinical scenarios. The generator is iteratively optimized by fusing the differential evaluation loss and the compliance determination loss in a preset ratio. Compared with traditional single GAN training methods, it can more ensure the synchronous approximation of lesions in spatial morphology and staging features, providing higher-quality training and inference data for subsequent disease monitoring models. For situations where the monitoring results do not conform to the priors or new morphologies are found, the system can feedback to the medical knowledge graph for update or fine-tuning when necessary, so as to iteratively improve the adaptability to rare lesions or complex staging features, and greatly enhance the self-learning and expansion capabilities of the system. Through multi-algorithm fusion and pixel-level alignment methods, the lesion images that meet the requirements of multi-level prior information are smoothly fused with the original medical images and then input into the disease monitoring model, so as to also achieve good detection and segmentation effects in scenarios where rare lesions or scarce staging data are insufficient, meeting the diverse clinical needs.

[0263] Therefore, through the organic combination of lesion synthesis, staging priors, and monitoring models, this application takes into account morphological constraints and clinical interpretability in the integrated generation-detection process, improves the existing technology in terms of accuracy, applicability, and scalability, and overall enhances the efficiency and reliability of medical image recognition and auxiliary diagnosis.

[0264] Based on the same inventive concept, embodiments of the present disclosure also provide a disease monitoring system for medical image data recognition corresponding to the disease monitoring method for medical image data recognition. Since the principle of solving problems by the system in the embodiments of the present disclosure is similar to that of the above-mentioned disease monitoring method for medical image data recognition in the embodiments of the present disclosure, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be elaborated.

[0265] Refer to Figure 3As shown in the figure, it is a schematic diagram of a disease monitoring system based on medical image data recognition provided by an embodiment of the present disclosure. The system includes: an acquisition module 10, a first processing module 20, a second processing module 30, and a generation module 40;

[0266] The acquisition module 10 is configured to obtain multi-level prior information related to a target disease from a medical knowledge graph. The multi-level prior information includes:

[0267] Lesion attributes, where the lesion attributes include lesion size, shape, location, and anatomical structure dependency;

[0268] At least one temporal attribute describing the progression law of the target disease; and lesion boundary and staging restriction rules for restricting the lesion shape and staging of the target disease;

[0269] The first processing module 20 is configured to use a generative adversarial network as the lesion generation model. The generative adversarial network includes a generator and a discriminator;

[0270] Encode the lesion attributes and temporal attributes in the multi-level prior information into a knowledge prior vector and input it into the generator, so that the generator outputs lesion image data based on the knowledge prior vector to match the anatomical structure and progression law of the target disease;

[0271] While the discriminator evaluates the difference between the lesion image data output by the generator and the knowledge prior vector, the discriminator also calls the lesion boundary and staging restriction rules in the medical knowledge graph to determine the compliance of the shape and staging status of the lesion image data;

[0272] The second processing module 30 is configured to update the network parameters of the generator through multiple rounds of adversarial training based on the results of the difference evaluation and compliance determination output by the discriminator until lesion image data that meets the requirements of the multi-level prior information is obtained;

[0273] The generation module 40 is configured to combine the lesion image data that meets the requirements of the multi-level prior information with the original medical image and input it into a preset disease monitoring model to generate potential lesion detection results.

[0274] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order and does not constitute any limitation to the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic. It should be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0275] In the description of this specification, the descriptions referring to the terms "exemplary", "for example", "specifically", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

Claims

1. A disease monitoring method based on the recognition of medical image data, characterized in that, The method includes: Obtaining multi-level prior information related to a target disease from a medical knowledge graph, where the multi-level prior information includes: Lesion attributes, where the lesion attributes include lesion size, shape, location, and anatomical structure dependency; At least one temporal attribute describing the progression law of the target disease; and lesion boundary and staging restriction rules for restricting the lesion shape and staging of the target disease; Using a generative adversarial network as the lesion generation model, where the generative adversarial network includes a generator and a discriminator; Encoding the lesion attributes and temporal attributes in the multi-level prior information into a knowledge prior vector and inputting it into the generator, so that the generator outputs lesion image data based on the knowledge prior vector to match the anatomical structure and progression law of the target disease; While the discriminator evaluates the difference between the lesion image data output by the generator and the knowledge prior vector, the discriminator also calls the lesion boundary and staging restriction rules in the medical knowledge graph to determine the compliance of the shape and staging status of the lesion image data; Based on the results of the difference evaluation and compliance determination output by the discriminator, update the network parameters of the generator through multiple rounds of adversarial training until lesion image data that meets the requirements of the multi-level prior information is obtained; Combining the lesion image data that meets the requirements of the multi-level prior information with the original medical image and inputting it into a preset disease monitoring model to generate potential lesion detection results; The step of enabling the generator to output lesion image data based on the knowledge prior vector includes: Performing numerical processing on the size, shape, location, and anatomical structure dependency in the lesion attributes to generate a lesion attribute vector, where the lesion attribute vector is used to represent the spatial characteristics and anatomical prior of the target disease lesion; Performing time series feature extraction on the temporal attribute describing the progression law of the target disease, and fusing the obtained temporal features with the lesion attribute vector to form a temporal fusion vector, where the temporal fusion vector is used to represent the dynamic changes of the lesion at different stages and progression phases; In the temporal fusion vector, based on the critical threshold and stage interval defined by the lesion boundary and staging restriction rules obtained from the medical knowledge graph, perform prior restriction processing to label and restrict the temporal fusion vector in the dimensions of lesion shape and staging; Input the temporally fused vector after prior restriction processing into the generator, and use the lesion spatial features and progression stage information contained in the temporally fused vector to generate lesion image data during training and inference phases.

2. The disease monitoring method based on medical image data recognition according to claim 1, wherein The evaluation of the difference between the lesion image data output by the generator and the knowledge prior vector includes: Extracting lesion image features from the lesion image data output by the generator through a feature extraction algorithm to form an image feature vector; Comparing the image feature vector with the knowledge prior vector one by one in the spatial attribute dimension, temporal progression dimension, and staging restriction dimension, and calculating the difference metric respectively; Determine the difference evaluation result based on the difference measurement results of each dimension, and use the difference evaluation result to indicate the deviation degree of the lesion image data from the knowledge prior vector.

3. The disease monitoring method based on medical image data recognition according to claim 2, wherein The compliance determination of the morphology and staging status of the lesion image data includes: Based on the boundary limit rules obtained from the medical knowledge graph, match the edge contour in the lesion image data with a preset lesion morphology threshold, and output a morphology non-compliance indication when the lesion morphology exceeds the lesion morphology threshold; Based on the staging interval in the temporal attributes, compare the staging status of the lesion image data with the staging limit in the medical knowledge graph, and output a staging non-compliance indication when the staging status exceeds the staging interval; Integrate the results of morphology compliance and staging compliance determination to form a compliance determination result, and use the compliance determination result to guide the continuation or correction of the adversarial training process.

4. The disease monitoring method based on medical image data recognition according to claim 3, wherein Updating the network parameters of the generator through multiple rounds of adversarial training based on the results of the difference evaluation and compliance determination output by the discriminator includes: Obtain a difference loss based on the difference evaluation result, obtain a compliance loss based on the compliance determination result, fuse the difference loss and the compliance loss with a preset ratio to generate a total loss; During the training process, perform gradient backpropagation optimization on the generator; When the total loss is lower than a preset threshold or the number of training rounds reaches the upper limit, stop updating the network parameters of the generator to obtain lesion image data that meets the requirements of the multi-level prior information.

5. The disease monitoring method based on medical image data recognition according to claim 4, characterized in that, Combining the lesion image data that meets the requirements of the multi-level prior information with the original medical image includes: Based on the position attribute in the lesion image data, convert and align the lesion pixel coordinates to the corresponding coordinate positions of the original medical image; After coordinate alignment, perform a preset pixel fusion process on the boundary region where the lesion area overlaps with the original medical image. The pixel fusion process includes gradient correction or seamless cloning operations at the fusion boundary; After completing the pixel fusion process of the boundary region, output the merged image data as synthetic image data.

6. The disease monitoring method based on medical image data recognition according to claim 5, wherein After combining the lesion image data that meets the requirements of the multi-level prior information with the original medical image, input it into a preset disease monitoring model to generate potential lesion detection results, including: Load a preset disease monitoring model and perform image preprocessing operations on the synthetic image data; In the model inference stage, perform feature extraction and segmentation calculation on the input synthetic image data to generate the position information, size, and shape of the potential lesion area; Generate potential lesion detection results based on the potential lesion information obtained from model inference; the potential lesion detection results include lesion spatial coordinates, morphological parameters, and staging information.

7. The disease monitoring method based on medical image data recognition according to claim 6, characterized in that It also includes: In response to the staging information, lesion size, and morphological features of the potential lesion detection result not matching the multi-level prior information, record the non-matching information as update data; Send the update data to the update interface of the medical knowledge graph to adjust the information of the target disease in the medical knowledge graph.

8. A disease monitoring system based on medical image data recognition, which is used to implement the disease monitoring method based on medical image data recognition according to any one of claims 1-7, characterized in that, It includes: An acquisition module, a first processing module, a second processing module, and a generation module; The acquisition module is used to obtain multi-level prior information related to the target disease from the medical knowledge graph. The multi-level prior information includes: Lesion attributes, which include lesion size, morphology, location, and anatomical structure dependency; At least one temporal attribute describing the progression law of the target disease; and lesion boundary and staging restriction rules for restricting the lesion morphology and staging of the target disease; The first processing module is used to use a generative adversarial network as the lesion generation model. The generative adversarial network includes a generator and a discriminator; Encode the lesion attributes and temporal attributes in the multi-level prior information into a knowledge prior vector and input it into the generator, so that the generator outputs lesion image data based on the knowledge prior vector to match the anatomical structure and progression law of the target disease; While the discriminator evaluates the difference between the lesion image data output by the generator and the knowledge prior vector, the discriminator also calls the lesion boundary and staging restriction rules in the medical knowledge graph to determine the compliance of the morphology and staging status of the lesion image data; The second processing module is used to update the network parameters of the generator through multiple rounds of adversarial training based on the results of the difference evaluation and compliance determination output by the discriminator until lesion image data that meets the requirements of the multi-level prior information is obtained; The generation module is used to combine the lesion image data that meets the requirements of the multi-level prior information with the original medical image and input it into a preset disease monitoring model to generate potential lesion detection results.

Images