A lesion segmentation and labeling method based on a medical knowledge graph model

Abstract

The application discloses a kind of based on medical knowledge graph model's lesion segmentation annotation method.The method includes: obtaining medical image to be processed and carries out standardization preprocessing;Initial lesion segmentation mask is generated and candidate lesion feature parameters are extracted by inputting standardization medical image into lesion segmentation model;According to image mode, anatomical location and candidate lesion feature parameters, determine lesion feature constraint rule from medical image knowledge graph;According to the matching result of candidate lesion feature parameters and lesion feature constraint rule, calculate knowledge graph constraint score, and based on knowledge graph constraint score and anatomical region constraint, boundary correction is carried out to initial lesion segmentation mask, and the lesion segmentation result after correction is obtained;Lesion annotation information is generated based on the lesion segmentation result after correction and quality evaluation is carried out;When quality evaluation result is unqualified, weight is assigned to error sample and lesion segmentation model is incrementally updated, while adjusting lesion feature threshold boundary in medical image knowledge graph.The method can reduce missed segmentation, over-segmentation and cross-anatomical region mis-segmentation, improve the accuracy and continuous optimization capability of medical image lesion segmentation annotation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and artificial intelligence-assisted medical image analysis technology, and in particular to a lesion segmentation and annotation method based on a medical knowledge graph model. Background Technology

[0002] With the development of medical imaging technologies such as CT, MRI, ultrasound, and DR, lesion region segmentation and annotation have become crucial foundations for lesion quantitative analysis, clinical review, and the construction of medical imaging datasets. Accurately obtaining lesion boundaries, lesion types, anatomical locations, and quantitative parameters helps improve the standardization and utilization efficiency of medical imaging data processing.

[0003] Existing lesion segmentation and annotation methods mainly include manual annotation, traditional image processing methods, and deep learning-based automatic segmentation methods. Manual annotation is inefficient and subject to significant subjective differences; traditional image processing methods are poorly adapted to small lesions, lesions with blurred boundaries, and low-contrast images; although deep learning methods can improve automation, they are still prone to problems such as missed segmentation, over-segmentation, boundary offset, or mis-segmentation across anatomical regions in scenarios with artifact interference, similar features of normal tissue, and complex anatomical structures.

[0004] Furthermore, even when existing solutions incorporate medical knowledge graphs or anatomical prior information, they are mostly used for lesion type reasoning or annotation assistance, and rarely directly for boundary correction of the initial segmentation mask. At the same time, existing solutions are insufficient in quality assessment of segmentation annotation results, error attribution, and continuous model optimization. They are difficult to conduct differentiated training for different error types, and they are also difficult to update the knowledge graph constraint rules in reverse based on the correction results, which makes it easy for the same type of error to recur. Summary of the Invention

[0005] This invention provides a lesion segmentation and annotation method based on a medical knowledge graph model, which addresses the problems existing in the automatic segmentation and annotation of lesion regions in medical images, such as inaccurate initial segmentation boundaries, artifacts or missegmentation of normal tissue, missegmentation across anatomical regions, lack of effective quality assessment of annotation results, difficulty in targeted optimization of erroneous samples, and lack of closed-loop updates between the model and medical knowledge rules.

[0006] This invention is achieved through the following technical solution, providing a lesion segmentation and annotation method based on a medical knowledge graph model, comprising the following steps: S1, acquire the medical image to be processed, and perform standardized preprocessing on the medical image to be processed to obtain a standardized medical image; S2, input the standardized medical image into the lesion segmentation model, generate an initial lesion segmentation mask, determine candidate lesion regions based on the initial lesion segmentation mask, and extract candidate lesion feature parameters of the candidate lesion regions; S3, based on the image modality of the medical image to be processed, the anatomical location of the candidate lesion region, and the characteristic parameters of the candidate lesion, determine the lesion feature constraint rules from the medical image knowledge graph; wherein, the lesion feature constraint rules include lesion feature thresholds and anatomical region constraints; S4, calculate the knowledge graph constraint score of the candidate lesion region based on the matching result between the candidate lesion feature parameters and the lesion feature constraint rules; S5, based on the knowledge graph constraint score and the anatomical region constraint, the initial lesion segmentation mask is modified to obtain the modified lesion segmentation result; wherein, the boundary modification includes at least one of pseudo-boundary removal, missing boundary completion, redundant boundary removal and cross-anatomical region clipping. S6, Based on the corrected lesion segmentation results, generate lesion annotation information, and perform quality assessment on the lesion annotation information to obtain quality assessment results; S7. When the quality assessment result does not meet the preset qualification conditions, the corresponding medical image, the corrected lesion segmentation result and the lesion annotation information are used as samples to be reviewed; after obtaining the standard annotation result or manual review result corresponding to the sample to be reviewed, the error type is determined according to the difference between the corrected lesion segmentation result and the standard annotation result or manual review result. S8. Based on the error type and lesion risk warning level, training weights are assigned to the corresponding error samples, and the lesion segmentation model is incrementally updated based on the weighted error samples. S9. Based on the error sample correction results after incremental update, adjust the feature threshold boundary of the corresponding lesion type in the medical image knowledge graph to update the lesion feature constraint rules.

[0007] Further, the medical image to be processed is one or more of CT images, MRI images, ultrasound images, or DR images; the standardized preprocessing includes one or more of noise suppression, artifact removal, contrast enhancement, pixel normalization, size unification, spatial registration, and background region filtering. For CT images, the standardized preprocessing includes HU value truncation and normalization; for MRI images, the standardized preprocessing includes signal intensity normalization based on the mean and standard deviation of the effective tissue region.

[0008] Furthermore, the lesion segmentation model includes a CNN feature extraction backbone layer, a Transformer global encoding layer, a multi-scale feature fusion layer, and a segmentation output layer; the CNN feature extraction backbone layer is used to extract multi-scale local lesion features, the Transformer global encoding layer is used to extract global context features, the multi-scale feature fusion layer is used to fuse the multi-scale local lesion features and the global context features, and the segmentation output layer is used to generate the initial lesion segmentation mask based on the fused lesion features.

[0009] Further, the multi-scale feature fusion layer fuses the multi-scale local lesion features and the global context features, including: upsampling each layer's feature map to the same spatial resolution and calculating the lesion response value of each layer's feature map at the same pixel location; calculating the adaptive fusion weights corresponding to each layer's feature map based on the lesion response values, and performing pixel-wise weighted fusion of each layer's feature map according to the adaptive fusion weights. The adaptive fusion weights of the i-th layer's feature map at pixel location (x, y) satisfy: Pixel feature response value for the i-th layer feature map Its adaptive weights The calculation formula is:

[0010] Where: n is the number of feature layers participating in the fusion (n=5 in this invention). denoted as the pixel response value of the i-th layer feature at coordinates (x, y). A higher response value indicates a stronger salience of the lesion feature at that location.

[0011] Final fusion features The calculation formula is:

[0012] This weighting mechanism enables dynamic weight adjustment: for lesion areas, it automatically increases the weight of high-recognition features; for normal tissue and artifact areas, it automatically reduces the weight of ineffective features, thus suppressing missegmentation and missed segmentation at the feature level.

[0013] Furthermore, the candidate lesion feature parameters include one or more of the following: grayscale parameters, size parameters, morphological parameters, density uniformity parameters, boundary gradient parameters, and anatomical location parameters.

[0014] Furthermore, the medical imaging knowledge graph includes image modality nodes, anatomical location nodes, lesion type nodes, lesion feature nodes, and risk warning level nodes, and includes modality-lesion relationships, anatomy-lesion relationships, lesion-feature relationships, and lesion-risk relationships. Specifically, the modality-lesion relationship is used to define the identifiable lesion types under different image modalities; the anatomy-lesion relationship is used to define the anatomical regions corresponding to different lesion types; the lesion-feature relationship is used to bind lesion feature thresholds corresponding to different lesion types; and the lesion-risk relationship is used to determine the corresponding lesion risk warning level based on lesion feature parameters.

[0015] Furthermore, the lesion feature threshold includes one or more of the following: grayscale threshold, size threshold, morphology threshold, density uniformity threshold, and boundary gradient threshold; the anatomical region constraint is used to limit the anatomical location range to which the candidate lesion region belongs.

[0016] Furthermore, based on the knowledge graph constraint score and the anatomical region constraint, the initial lesion segmentation mask is modified by boundary correction, including: removing pseudo-boundary regions with knowledge graph constraint scores lower than a preset validity threshold; adding missing regions located in the neighborhood of the initial lesion segmentation mask that are not segmented and whose knowledge graph constraint scores meet a preset completion threshold into the initial lesion segmentation mask; removing extended regions or artifact regions with knowledge graph constraint scores lower than a preset redundancy removal threshold; and cropping regions that exceed the anatomical region constraint.

[0017] Furthermore, the lesion annotation information includes one or more of the following: lesion type annotation, lesion location annotation, lesion quantitative parameter annotation, and lesion risk warning level annotation. The quality assessment includes generating a quality assessment result based on one or more of the following: segmentation accuracy, annotation accuracy, annotation completeness, and annotation consistency. The error types include one or more of the following: missed segmentation, oversegmentation, incorrect annotation classification, and parameter calculation deviation.

[0018] Furthermore, the segmentation accuracy is determined based on the overlap between the corrected lesion segmentation result and the standard segmentation mask; the annotation accuracy is determined based on the accuracy of lesion type annotation and the accuracy of lesion risk warning level annotation; the annotation completeness is determined based on the correspondence between the number of automatically detected lesions and the number of standard lesions; and the annotation consistency is determined based on the dispersion of lesion quantification parameters under the same image modality and the same lesion type.

[0019] Further, steps S8 and S9 include: associating image samples whose quality assessment results do not meet the preset qualification conditions with the corresponding erroneous prediction results, standard segmentation masks, standard annotation parameters, and correct classification labels to form erroneous sample pairs; classifying the erroneous sample pairs according to error type, image modality, and lesion type, and storing them in the incremental training sample library; determining the training weights of the erroneous sample pairs according to the error type and lesion risk warning level; freezing the bottom convolutional parameters of the CNN feature extraction backbone layer in the lesion segmentation model, and opening the multi-scale feature fusion parameters, Transformer attention weight parameters, segmentation output layer parameters, and classification annotation layer parameters in the lesion segmentation model for incremental training; based on the incrementally updated erroneous sample correction results, statistically analyzing the grayscale distribution, size distribution, morphological distribution, density uniformity distribution, and boundary gradient distribution of the corresponding lesion type under the corresponding image modality and anatomical location, and adjusting the feature threshold boundary of the corresponding lesion type in the medical image knowledge graph based on the statistical results.

[0020] Furthermore, severely missed segmentation errors and high-risk warning level lesion classification errors correspond to the first training weight; oversegmentation errors and multiple lesion missed detection errors correspond to the second training weight; severely biased parameter calculation errors and risk warning level labeling errors correspond to the third training weight; and slightly biased parameter deviation errors and small boundary error errors correspond to the fourth training weight. The first training weight is greater than the second training weight, the second training weight is greater than the third training weight, and the third training weight is greater than or equal to the fourth training weight.

[0021] Compared with the prior art, the present invention has at least the following beneficial effects: First, this invention generates an initial lesion segmentation mask through a lesion segmentation model, and uses lesion feature thresholds and anatomical region constraints in the medical imaging knowledge graph to correct the boundaries of the initial lesion segmentation mask. This can reduce artifact areas, normal tissue areas, and missegmentation across anatomical regions, thereby improving the accuracy of lesion segmentation boundaries.

[0022] Second, this invention calculates a knowledge graph constraint score by matching the candidate lesion feature parameters with the lesion feature constraint rules, and performs pseudo-boundary removal, missing boundary completion, redundant boundary removal and cross-anatomical region pruning based on the score, so that the medical image knowledge graph directly participates in the boundary constraints of the lesion segmentation results, thereby enhancing the consistency between the segmentation results and medical prior rules.

[0023] Third, this invention performs quality assessment on the corrected lesion segmentation results and lesion annotation information, and assigns error causes when the quality assessment is unqualified. It can distinguish different error types such as missed segmentation, over-segmentation, incorrect annotation classification, and parameter calculation deviation, providing a clear basis for subsequent model optimization.

[0024] Fourth, this invention assigns differentiated weights to erroneous samples based on error type and lesion risk warning level, and incrementally updates the lesion segmentation model based on the weighted erroneous samples. This can improve the model's learning priority for serious missed segmentation and high-risk annotation errors, and reduce the recurrence of similar errors.

[0025] Fifth, this invention adjusts the threshold boundaries of lesion features in the medical image knowledge graph based on the error sample correction results after incremental updates, enabling the model optimization results to back-update the knowledge graph constraint rules, thereby forming a closed-loop optimization mechanism for lesion segmentation, boundary correction, quality assessment, error attribution, model update, and knowledge graph update. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the intelligent segmentation and annotation method for medical imaging lesion regions provided in an embodiment of the present invention. Figure 2 A flowchart for medical knowledge graph boundary correction provided in an embodiment of the present invention; Figure 3 The closed-loop graph for error attribution, weighted incremental training, and knowledge graph update provided in the embodiments of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature; in the description of this application, unless otherwise stated, "multiple" means two or more.

[0029] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the following embodiments are only used to illustrate the present invention and are not intended to limit the scope of protection of the present invention. Substitutions or adjustments made by those skilled in the art to specific image modalities, lesion types, model structures, threshold parameters, and output formats without departing from the concept of the present invention should all fall within the scope of protection of the present invention.

[0030] This invention provides a lesion segmentation and annotation method based on a medical knowledge graph model, which can be applied to lesion region segmentation, boundary correction, structured annotation, and quality optimization of medical images such as CT, MRI, ultrasound, and DR. The lesion risk warning level is auxiliary prompt information generated based on image features and preset rules, and is not directly used as a disease diagnosis conclusion.

[0031] Example 1: Intelligent Segmentation and Annotation Method for Lesion Regions in Medical Imaging like Figure 1 As shown, this embodiment provides a lesion segmentation and annotation method based on a medical knowledge graph model, including the following steps.

[0032] Step S110: Acquire the medical image to be processed and perform standardized preprocessing to obtain a standardized medical image.

[0033] Specifically, the medical image to be processed can be a CT image, MRI image, ultrasound image, or DR image. After acquiring the medical image to be processed, the system performs at least one of the following processing methods: noise suppression, artifact removal, contrast enhancement, pixel normalization, size unification, spatial registration, and background region filtering, to reduce data differences caused by different acquisition devices, different scanning parameters, and different image modalities. For CT images, standardized preprocessing may include HU value truncation and normalization; for MRI images, standardized preprocessing may include signal intensity normalization based on the mean and standard deviation of the effective tissue region.

[0034] Step S120: Input the standardized medical image into the lesion segmentation model to generate an initial lesion segmentation mask and extract the feature parameters of candidate lesions.

[0035] Specifically, the lesion segmentation model employs an end-to-end hybrid network structure that combines CNN backbone layer feature extraction, Transformer global context encoding, and multi-scale feature weighted fusion. This hybrid network structure includes an input layer, a CNN feature extraction backbone layer, a Transformer global encoding layer, and a multi-scale fusion and output layer. The input layer receives standardized medical image single-channel or multi-channel data with a uniform size of 512×512 or 1024×1024, normalizes the pixels to the [0,1] interval, and outputs a normalized image matrix.

[0036] The CNN feature extraction backbone layer can use an improved lightweight ResNet50 as the backbone network, retaining four levels of downsampling residual modules to extract shallow texture features at 1 / 4 resolution, mid-level edge semantic features at 1 / 8 resolution, deep local lesion morphological features at 1 / 16 resolution, and ultra-deep lesion core features at 1 / 32 resolution. Each level of residual module can add a lesion feature activation layer to enhance the feature response of small lesions and low-contrast lesions, and suppress invalid features such as normal tissue and device artifacts.

[0037] The Transformer global coding layer can be configured with 6 stacked Transformer coding units, while retaining the global attention mechanism. Each coding unit includes a multi-head self-attention module and a feedforward neural network module, which are used to capture the global location association of lesions in the whole image, the spatial distribution relationship of multiple lesions, and the contextual dependencies between lesions and surrounding normal anatomical tissues.

[0038] The multi-scale fusion and output layer fuses the CNN's four-level local features with the Transformer's global context features pixel by pixel. After activation function and convolution dimensionality reduction, it outputs a lesion feature heatmap with the same size as the original image. Then, it completes pixel-level binary classification through the Sigmoid activation function to generate the initial lesion segmentation mask.

[0039] In the multi-scale feature fusion process, the four-level multi-scale local feature maps output by the CNN are first upsampled to be uniform to the original image size. Then, the multi-layer local feature maps with uniform resolution are concatenated with the global context feature maps output by the Transformer to form a fused feature matrix. Then, the multi-dimensional features at each pixel location are effectively filtered by an adaptive feature filtering module, retaining high-response features of lesions and filtering low-response interference features. Finally, a weighted superposition method is used to complete the feature fusion, so that the shallow fine edge features, the middle morphological features, the deep semantic features and the global position features complement each other.

[0040] Pixel feature response value for the i-th layer feature map Its adaptive weights The calculation formula is: (Formula 1), Where: n is the number of feature layers participating in the fusion (n=5 in this invention). denoted as the pixel response value of the i-th layer feature at coordinates (x, y). A higher response value indicates a stronger salience of the lesion feature at that location.

[0041] Final fusion features The calculation formula is: (Formula 2), This weighting mechanism enables dynamic weight adjustment: for lesion areas, it automatically increases the weight of high-recognition features; for normal tissue and artifact areas, it automatically reduces the weight of ineffective features, thus suppressing missegmentation and missed segmentation at the feature level.

[0042] The system determines candidate lesion regions based on the initial lesion segmentation mask, and extracts at least one of the following parameters from the candidate lesion regions: grayscale parameter, size parameter, morphological parameter, density uniformity parameter, boundary gradient parameter, and anatomical location parameter, as the basis for subsequent knowledge graph boundary correction calculations.

[0043] Step S130: Determine lesion feature constraint rules from the medical image knowledge graph based on image modality, anatomical location, and candidate lesion feature parameters.

[0044] Specifically, this invention constructs a medical image knowledge graph adapted to multimodal image lesion segmentation and annotation.

[0045] The medical imaging knowledge graph includes anatomical location nodes, lesion type nodes, imaging modality nodes, lesion feature nodes, and risk warning level nodes. Anatomical location nodes can include sub-nodes for anatomical regions such as the lungs, brain, liver, kidneys, bones, and soft tissues; lesion type nodes can include sub-nodes for lesions such as nodules, tumors, inflammation, effusion, calcification, hyperplasia, infarction, and hemorrhage; imaging modality nodes can include sub-nodes for images such as lung CT, brain MRI, abdominal ultrasound, and DR; lesion feature nodes can include feature sub-nodes for grayscale range, boundary morphology, size range, density uniformity, and growth pattern; and risk warning level nodes can include sub-nodes for low risk, medium risk, and high risk.

[0046] Standardized relationships are established between nodes, including modality-lesion relationships, anatomy-lesion relationships, lesion-feature relationships, and lesion-risk relationships. Modality-lesion relationships are used to define the identifiable lesion types under different imaging modalities; anatomy-lesion relationships are used to define the anatomical regions of various lesions; lesion-feature relationships are used to bind feature thresholds such as grayscale, morphology, size, density uniformity, and boundary gradient for each type of lesion; and lesion-risk relationships are used to map the corresponding lesion risk level based on lesion feature parameters.

[0047] In one implementation, the knowledge graph incorporates image feature threshold ranges for various lesions. For example, for ground-glass nodules on lung CT: grayscale threshold [-600HU, -900HU], diameter threshold [3mm, 30mm], and boundary ambiguity threshold ≤0.3; for solid nodules on lung CT: grayscale threshold [100HU, 300HU], diameter threshold [5mm, 50mm], and density uniformity threshold ≥0.7; for brain tumor lesions on MRI: T1 sequence grayscale threshold [0.2, 0.5], T2 sequence grayscale threshold [0.6, 0.9], and lesion infiltration boundary gradient threshold ≥0.4; for conventional calcified lesions: grayscale threshold ≥200HU, and aspect ratio threshold ≤1.2; for inflammatory effusion lesions: grayscale uniformity threshold ≥0.8, with no obvious hard boundary gradient. These thresholds can be set or adjusted based on the medical knowledge base, standard labeled samples, and subsequent incremental update results.

[0048] Step S140: Calculate the knowledge graph constraint score of the candidate lesion region based on the matching results of the candidate lesion feature parameters and the lesion feature constraint rules.

[0049] Specifically, the system calculates the degree of matching between the grayscale parameters, size parameters, morphological parameters, density uniformity parameters, boundary gradient parameters, and anatomical location parameters of the candidate lesion region and the corresponding lesion feature constraint rules, and performs weighted processing on the degree of matching of each dimension to obtain the knowledge graph constraint score.

[0050] In one implementation, let the candidate lesion region be Ω, and the feature parameter of the j-th type of candidate lesion corresponding to the candidate lesion region Ω be Xj(Ω). The threshold interval of the j-th type of lesion feature corresponding to the lesion type in the medical image knowledge graph is [Lj, Uj]. Then, the matching degree Mj(Ω) between the j-th type of candidate lesion feature parameter and the lesion feature threshold interval satisfies the following calculation: (Formula 3), (Formula 4), (Formula 5), Wherein, δj represents the allowable deviation range corresponding to the feature parameters of the j-th type of candidate lesion, used to control the degree of score decay after the feature parameters of the candidate lesion exceed the threshold range. The feature parameters of the j-th type of candidate lesion include any one of the following: grayscale parameters, size parameters, morphological parameters, density uniformity parameters, and boundary gradient parameters.

[0051] For anatomical location parameters, the system calculates the anatomical location matching degree Ma(Ω) based on the overlap ratio between the candidate lesion region Ω and the corresponding anatomical region A in the medical imaging knowledge graph. The calculation formula is as follows: (Formula 6), Where |Ω∩A| represents the number of pixels in the candidate lesion region Ω that fall within the corresponding anatomical region A, and |Ω| represents the total number of pixels in the candidate lesion region Ω.

[0052] The system calculates the knowledge graph constraint score ScoreKG (Ω) based on the matching degree of various candidate lesion feature parameters and anatomical location matching degree. The calculation formula is as follows: (Formula 7), Where αj represents the weight corresponding to the feature parameter of the j-th candidate lesion, αa represents the weight corresponding to the anatomical location matching degree, and m represents the number of candidate lesion feature parameters participating in the scoring, and The weights for different imaging modalities and lesion types are pre-set based on the importance of lesion features, or updated based on the statistical results of historical labeled samples.

[0053] When there are no corresponding candidate lesion feature parameters in a certain image modality, or when the candidate lesion feature parameters cannot be effectively extracted, the system resets the weight corresponding to the candidate lesion feature parameters to 0 and normalizes the weights corresponding to the other candidate lesion feature parameters.

[0054] The knowledge graph constraint score, ScoreKG(Ω), ranges from [0, 100] and is used to characterize the degree to which a candidate lesion region Ω conforms to the prior rules of the lesion in the medical imaging knowledge graph under the current image modality, anatomical location, and lesion type. The higher the ScoreKG(Ω), the more the candidate lesion region conforms to the image features and anatomical location constraints of the corresponding lesion type; the lower the ScoreKG(Ω), the more likely the candidate lesion region belongs to normal tissue, equipment artifacts, noisy areas, or missegmented areas across anatomical regions.

[0055] Step S150: Based on the knowledge graph constraint score and anatomical region constraint, the initial lesion segmentation mask is boundary-corrected to obtain the corrected lesion segmentation result.

[0056] Specifically, based on the knowledge graph constraint score and anatomical region constraints, the system performs at least one of the following processing steps on the initial lesion segmentation mask: pseudo-boundary removal, missing boundary completion, redundant boundary removal, and cross-anatomical region clipping. These boundary correction actions are performed based on preset validity thresholds, completion thresholds, redundancy removal thresholds, and anatomical region constraints, respectively. For specific boundary correction methods, please refer to Example 2. Step S160: Generate lesion annotation information based on the corrected lesion segmentation results, and evaluate the quality of the lesion annotation information.

[0057] Specifically, the lesion labeling information includes at least one of the following: lesion type labeling, lesion location labeling, lesion quantitative parameter labeling, and lesion risk warning level labeling. The lesion quantitative parameters may include at least one of the following: maximum diameter of the lesion, area, volume, aspect ratio, boundary clarity, and density uniformity.

[0058] This invention constructs a four-dimensional quality assessment system, including segmentation accuracy, annotation accuracy, annotation completeness, and annotation consistency. Segmentation accuracy can be quantified and scored using the Dice similarity coefficient, calculated as follows: (Formula 8), Where Pred is the set of pixels for automatically segmented lesions, and Truth is the set of pixels for standard-labeled lesions.

[0059] The accuracy of lesion classification and risk level labeling is measured by a weighted average of type accuracy and risk level accuracy, calculated using the following formula: (Formula 9), in: Accuracy rate of lesion type labeling (number of correctly labeled samples / total number of samples). The accuracy of risk level labeling is assigned based on clinical importance, with higher weighting for lesion type.

[0060] The core statistic for measuring whether all lesions in a single image are detected without omission is the lesion detection coverage rate, calculated using the following formula: (Formula 10), in: To determine the number of lesions detected by the model, To indicate the true number of lesions using the gold standard To measure the stability of annotation results for lesions of the same type, modality, and characteristics using the same equipment, multi-sample parameter variance quantification is employed, calculated using the following formula: (Formula 11), Where: σ is the standard deviation of quantitative parameters (diameter, area) of the same type of lesion, μ is the mean of parameters of the same type of lesion, and the smaller the coefficient of variation, the higher the consistency score.

[0061] In one implementation, the scores range from [0, 100]. When any index is less than the corresponding preset score threshold, the quality assessment result is deemed not to meet the preset qualification conditions. The preset score threshold can be 80 points, or it can be adjusted according to the image modality, lesion type, or application scenario.

[0062] Step S170: When the quality assessment result does not meet the preset qualification conditions, the error type is determined according to the standard annotation result or the manual review result, and training weights are assigned to the corresponding error samples according to the error type and the lesion risk warning level.

[0063] Specifically, when the quality assessment result does not meet the preset qualification conditions, the system uses the corresponding medical image, the corrected lesion segmentation result, the lesion annotation information, and the quality assessment result as a sample to be reviewed, and obtains the standard annotation result or manual review result corresponding to the sample to be reviewed. The system compares the corrected lesion segmentation result and lesion annotation information with the standard annotation result or manual review result to determine the error type corresponding to the sample to be reviewed.

[0064] The error types include at least one of the following: missed segmentation, over-segmentation, incorrect labeling and classification, and parameter calculation deviation.

[0065] Among them, missed segmentation is used to characterize the situation where the lesion area exists in the standard annotation result but the corresponding lesion area is not detected in the corrected lesion segmentation result; oversegmentation is used to characterize the situation where the corrected lesion segmentation result contains normal tissue area, artifact area, noise area, or area exceeding the anatomical area constraint; incorrect annotation classification is used to characterize the situation where the lesion type annotation, lesion location annotation, or lesion risk warning level annotation is inconsistent with the standard annotation result; parameter calculation deviation is used to characterize the situation where the deviation between the lesion quantification parameter and the standard annotation parameter exceeds the preset deviation threshold.

[0066] Step S180: Incrementally update the lesion segmentation model based on the weighted erroneous samples, and adjust the lesion feature threshold boundary in the medical image knowledge graph according to the erroneous sample correction results after incremental update.

[0067] Specifically, the system associates image samples that fail quality assessment with incorrect prediction results, standard segmentation masks, standard annotation parameters, and correct classification labels to form erroneous sample pairs, and stores them in the incremental training sample library according to error type, image modality, and lesion type. The system determines training weights based on error type and lesion risk warning level, and incrementally updates the lesion segmentation model based on the training weights. After the incremental update is completed, the system statistically analyzes the feature distribution of the corresponding lesion type based on the erroneous sample correction results, and adjusts the lesion feature threshold boundaries in the medical image knowledge graph. For the specific erroneous sample weighting, weighted loss calculation, parameter update, and knowledge graph update process, please refer to Example 3. Example 2: Lesion segmentation boundary correction under medical knowledge graph constraints like Figure 2As shown in the example, this embodiment, based on Example 1, further illustrates the application process of medical image knowledge graph in lesion segmentation boundary correction. In this embodiment, the medical image knowledge graph is used to provide association rules between image modalities, anatomical locations, lesion types, lesion feature thresholds, and anatomical region constraints. The generated lesion risk warning level is only used as auxiliary information and is not directly used as a disease diagnosis conclusion.

[0068] In this embodiment, the data for constructing the medical image knowledge graph can come from one or more of the following: preset medical image annotation rules, historical standard annotation samples after desensitization, publicly available medical image samples, manually reviewed lesion annotation results, and subsequent error sample correction results. After de-identification, format standardization, and field normalization, the above data is used to form the node, relationship, and lesion feature constraint rules in the medical image knowledge graph.

[0069] The medical imaging knowledge graph includes image modality nodes, anatomical location nodes, lesion type nodes, lesion feature nodes, and risk warning level nodes, and establishes modality-lesion relationships, anatomy-lesion relationships, lesion-feature relationships, and lesion-risk relationships. Through these nodes and relationships, the system can determine the identifiable lesion types under different image modalities, the allowed anatomical regions for different lesion types, the image feature thresholds corresponding to different lesion types, and the risk warning levels corresponding to candidate lesion regions. In one implementation, each lesion feature constraint rule can include a rule identifier, image modality, anatomical location, candidate lesion type, lesion feature threshold, allowed deviation range, anatomical region constraint, and rule version number, for subsequent rule tracing and threshold updates.

[0070] When correcting lesion segmentation boundaries, the system obtains the initial lesion segmentation mask generated in Example 1 and determines candidate lesion regions based on the initial lesion segmentation mask. The system extracts at least one of the following parameters from the standardized medical images: grayscale parameters, size parameters, morphological parameters, density uniformity parameters, boundary gradient parameters, and anatomical location parameters of the candidate lesion regions. It then combines the image modality of the medical image to be processed with the anatomical location of the candidate lesion regions and matches the corresponding lesion feature constraint rules from the medical image knowledge graph.

[0071] Specifically, the system first filters candidate lesion types that can be identified under the current image modality, then filters lesion types that are allowed to appear within the anatomical region based on the anatomical location of the candidate lesion region, and further obtains the corresponding grayscale threshold, size threshold, morphology threshold, density uniformity threshold, boundary gradient threshold, and anatomical region constraints for the lesion type. When a candidate lesion region may correspond to multiple lesion types, the system can calculate the knowledge graph constraint score corresponding to each candidate lesion type separately, and select the lesion feature constraint rule with the higher score as the basis for boundary correction.

[0072] After obtaining the lesion feature constraint rules, the system calculates knowledge graph constraint scores for the overall candidate lesion region, the set of boundary pixels, the candidate completion region, the abnormal extension region, and the suspected artifact region. The knowledge graph constraint scores are used to characterize the degree of matching between the above regions and the lesion feature constraint rules under the current image modality, anatomical location, and candidate lesion type.

[0073] For boundary pixel sets or local boundary regions in the initial lesion segmentation mask, if their knowledge graph constraint score is lower than a preset validity threshold, the system classifies them as false boundary regions and removes them from the initial lesion segmentation mask. For candidate completion regions located in the neighborhood outside the initial lesion segmentation mask, if they are located within the corresponding anatomical region and their knowledge graph constraint score reaches a preset completion threshold, the system adds them to the initial lesion segmentation mask to reduce under-segmentation of small lesions, weak contrast lesions, or lesions with blurred boundaries.

[0074] For abnormally extended regions or suspected artifact regions in the initial lesion segmentation mask, if their knowledge graph constraint score is lower than the preset redundancy removal threshold, the system will remove them from the initial lesion segmentation mask to reduce oversegmentation caused by normal tissue, vascular sections, equipment artifacts, or noisy regions. The system also trims portions of the initial lesion segmentation mask that exceed the allowed anatomical regions based on anatomical region constraints in the medical imaging knowledge graph; for candidate completion regions that do not belong to the allowed anatomical regions, the system will not perform completion.

[0075] After completing the above corrections, the system outputs the corrected lesion segmentation results. The corrected lesion segmentation results include pixel distribution information, lesion contour information, and corresponding anatomical location information of the lesion region. Simultaneously, the system records the atlas rule identifier, rule version number, candidate lesion type, knowledge graph constraint score, correction object, correction type, and mask changes before and after correction used in this boundary correction. This recorded information can be used for subsequent quality assessment, error attribution, erroneous sample inclusion in the database, and updating the threshold boundaries of the medical imaging knowledge graph.

[0076] For example, in a lung CT nodule scenario, the system can match the corresponding grayscale threshold, size threshold, boundary morphology threshold, and lung anatomical region constraints from a medical image knowledge graph based on the lung CT image modality, lung anatomical region, and candidate nodule features. When the initial lesion segmentation mask mistakenly includes vascular sections, tracheal walls, or bone edge regions, the system can remove them based on the knowledge graph constraint score. When a local area of ​​the nodule boundary is not covered by the initial mask due to weak contrast, but its image features conform to the nodule prior rules and are located within the lung anatomical region, the system can add this local area to the lesion segmentation mask.

[0077] For example, in the context of brain MRI tumors, the system can match the corresponding grayscale threshold, boundary gradient threshold, infiltration morphology constraint, and brain anatomical region constraint from the medical image knowledge graph based on the brain MRI image modality, brain anatomical region, and candidate tumor features. When the initial lesion segmentation mask mistakenly covers the ventricles, skull, or extracranial background region, the system can crop the erroneous region based on the anatomical region constraint and knowledge graph constraint score. When a local area of ​​the tumor infiltration boundary conforms to the graph constraint rules but is not covered by the initial mask, the system can add it to the lesion segmentation mask based on the completion threshold.

[0078] Example 3: Error Attribution, Weighted Incremental Training, and Knowledge Graph Update Closed Loop like Figure 3 As shown in the example, this embodiment further illustrates the process of error attribution, error sample weighting, incremental update of lesion segmentation model, and threshold update of medical image knowledge graph after the quality assessment fails, based on the first embodiment.

[0079] In this embodiment, the system performs a quality assessment on the corrected lesion segmentation results and lesion annotation information. When at least one of the segmentation accuracy, annotation accuracy, annotation completeness, and annotation consistency is lower than the corresponding preset scoring threshold, the system uses the corresponding medical image, the corrected lesion segmentation results, lesion annotation information, and quality assessment results as samples to be reviewed, and obtains the standard annotation results or manual review results corresponding to the samples to be reviewed.

[0080] The system compares the corrected lesion segmentation results and lesion annotation information with the standard annotation results or manual review results to determine the error type. The error types include at least one of the following: missed segmentation, oversegmentation, incorrect annotation classification, and parameter calculation deviation. Missed segmentation indicates that a lesion region exists in the standard annotation results but the system fails to detect the corresponding lesion region; oversegmentation indicates that the system identifies normal tissue, artifact regions, noise regions, or regions exceeding anatomical constraints as lesions; incorrect annotation classification indicates that the lesion type, anatomical location, or risk warning level is incorrectly labeled; parameter calculation deviation indicates that the deviation between quantitative parameters such as the maximum diameter, area, volume, aspect ratio, or boundary clarity of the lesion and the standard annotation parameters exceeds a preset deviation threshold. If multiple error types exist for the same sample to be reviewed, the system can determine the primary error type based on the severity of the error and retain other error types as auxiliary error labels.

[0081] After determining the error type, the system associates the sample to be reviewed with its corresponding erroneous prediction result, standard segmentation mask, standard annotation parameters, and correct classification label to form erroneous sample pairs. These pairs are then classified according to error type, image modality, and lesion type before being stored in the incremental training sample library. To avoid duplicate samples interfering with model training, the system can deduplicate erroneous sample pairs based on medical image identifiers, lesion region locations, and standard annotation results.

[0082] The system determines the training weights of erroneous sample pairs based on the error type and the lesion risk level. Specifically, severely missed segmentation erroneous samples and lesion classification erroneous samples with high risk levels are assigned higher training weights; oversegmentation erroneous samples and missed detection of multiple lesions are assigned medium training weights; severely biased parameter calculation erroneous samples and risk level labeling erroneous samples are assigned lower training weights; and slightly biased parameter calculation erroneous samples and small boundary error erroneous samples are assigned the lowest training weights. Through this differentiated weighting method, the system can give high-risk, high-harm erroneous samples higher learning priority in subsequent model updates.

[0083] In one implementation, the training weights Ws of the erroneous sample pairs are jointly determined based on the error type base weights We and the lesion risk warning level correction coefficient Wr, satisfying the calculation formula: (Formula 12), Where We represents the base weight corresponding to the error type, and Wr represents the correction coefficient corresponding to the lesion risk warning level; the Wr corresponding to the high risk warning level is greater than the Wr corresponding to the medium risk warning level, and the Wr corresponding to the medium risk warning level is greater than the Wr corresponding to the low risk warning level.

[0084] During incremental training, the system locally updates the lesion segmentation model based on weighted error samples. Specifically, the system can weight the training loss according to the training weights of error sample pairs, allowing high-weight error samples to have a greater impact during model parameter updates. When updating model parameters, the system can freeze the low-level convolutional parameters of the CNN feature extraction backbone layer to retain general image feature extraction capabilities; simultaneously, it can open the parameters of the multi-scale feature fusion layer, Transformer attention weights, segmentation output layer, and classification annotation layer to specifically correct the model's deficiencies in lesion boundary recognition, lesion detection, and annotation classification. The system can also employ low-learning-rate fine-tuning and learning-rate decay to ensure smooth model updates and reduce model forgetting caused by incremental training.

[0085] After the incremental update is completed, the system statistically analyzes at least one of the following distributions based on the correction results of the erroneous samples: grayscale distribution, size distribution, morphological distribution, density uniformity distribution, and boundary gradient distribution, for the same image modality, anatomical location, and lesion type. When multiple erroneous samples, confirmed by standard annotation results or manual review, show a stable deviation between the feature distribution of a certain lesion type and the feature threshold boundary in the original medical imaging knowledge graph, the system adjusts the feature threshold boundary corresponding to that lesion type based on the statistical results of the corrected samples. For example, when multiple lung nodule samples, after review, show a deviation between their grayscale distribution range and the original knowledge graph threshold, the system can adjust the grayscale threshold boundary corresponding to the lung nodules; when multiple brain MRI tumor samples show changes in the boundary gradient feature range, the system can adjust the boundary gradient threshold boundary of the corresponding lesion type.

[0086] After updating the lesion segmentation model and medical image knowledge graph, the system can verify the update results. Verification methods can include at least one of the following: retesting with an error sample set, verification with a general dataset, and verification through sampling of newly added medical images. When the verification results meet preset requirements, the system uses the updated lesion segmentation model and medical image knowledge graph for subsequent lesion segmentation and annotation of medical images; when the verification results do not meet preset requirements, the system can re-screen error samples, adjust training weights, or adjust incremental training parameters before updating again.

[0087] Through the above method, this embodiment can transform samples that fail the quality assessment into training data that can be used for model optimization and knowledge graph updates, so that the error attribution results, lesion risk warning levels, incremental model updates and medical image knowledge graph threshold adjustments form a closed loop, thereby reducing the recurrence of similar segmentation and annotation errors and improving the stability and accuracy of subsequent medical image lesion segmentation and annotation results.

[0088] Based on the above embodiments, the present invention can realize the entire process from medical image input, initial segmentation, knowledge graph constraint boundary correction, lesion annotation, quality assessment, error attribution, weighted incremental training to knowledge graph threshold update, thereby improving the accuracy, standardization and continuous optimization capability of lesion segmentation and annotation results.

[0089] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. This application is not limited to the exact structures described above and illustrated in the accompanying drawings, and it should not be considered that the specific implementation of this application is limited to these descriptions. For those skilled in the art, various changes and modifications made without departing from the concept of this application should be considered to fall within the protection scope of this application.

Claims

1. A lesion segmentation and annotation method based on a medical knowledge graph model, characterized in that, Includes the following steps: S1, acquire the medical image to be processed, and perform standardized preprocessing on the medical image to be processed to obtain a standardized medical image; S2, input the standardized medical image into the lesion segmentation model, generate an initial lesion segmentation mask, determine candidate lesion regions based on the initial lesion segmentation mask, and extract candidate lesion feature parameters of the candidate lesion regions; S3, based on the image modality of the medical image to be processed, the anatomical location of the candidate lesion region, and the characteristic parameters of the candidate lesion, determine the lesion feature constraint rules from the medical image knowledge graph; wherein, the lesion feature constraint rules include lesion feature thresholds and anatomical region constraints; S4, calculate the knowledge graph constraint score of the candidate lesion region based on the matching result between the candidate lesion feature parameters and the lesion feature constraint rules; S5, based on the knowledge graph constraint score and the anatomical region constraint, the initial lesion segmentation mask is modified to obtain the modified lesion segmentation result; wherein, the boundary modification includes at least one of pseudo-boundary removal, missing boundary completion, redundant boundary removal and cross-anatomical region clipping. S6, Generate lesion annotation information based on the corrected lesion segmentation results, and perform quality assessment on the lesion annotation information to obtain quality assessment results; S7. When the quality assessment result does not meet the preset qualification conditions, the corresponding medical image, the corrected lesion segmentation result and the lesion annotation information are used as samples to be reviewed; after obtaining the standard annotation result or manual review result corresponding to the sample to be reviewed, the error type is determined according to the difference between the corrected lesion segmentation result and the standard annotation result or manual review result. S8. Based on the error type and lesion risk warning level, training weights are assigned to the corresponding error samples, and the lesion segmentation model is incrementally updated based on the weighted error samples. S9. Based on the error sample correction results after incremental update, adjust the feature threshold boundary of the corresponding lesion type in the medical image knowledge graph to update the lesion feature constraint rules.

2. The lesion segmentation and annotation method based on a medical knowledge graph model according to claim 1, characterized in that, The medical image to be processed is one or more of CT images, MRI images, ultrasound images, or DR images; The standardized preprocessing includes one or more of the following: noise suppression, artifact removal, contrast enhancement, pixel normalization, size unification, spatial registration, and background region filtering. For CT images, the standardized preprocessing includes HU value truncation and normalization; for MRI images, the standardized preprocessing includes signal intensity standardization based on the mean and standard deviation of the effective tissue region.

3. The lesion segmentation and annotation method based on a medical knowledge graph model according to claim 1, characterized in that, The lesion segmentation model includes a CNN feature extraction backbone layer, a Transformer global encoding layer, a multi-scale feature fusion layer, and a segmentation output layer; The CNN feature extraction backbone layer is used to extract multi-scale local lesion features, the Transformer global encoding layer is used to extract global context features, the multi-scale feature fusion layer is used to fuse the multi-scale local lesion features and the global context features, and the segmentation output layer is used to generate the initial lesion segmentation mask based on the fused lesion features.

4. The lesion segmentation and annotation method based on a medical knowledge graph model according to claim 3, characterized in that, The multi-scale feature fusion layer fuses the multi-scale local lesion features and the global context features, including: Upsample the feature maps of each layer to the same spatial resolution, and calculate the lesion response value at the same pixel position in the feature maps of each layer; The adaptive fusion weights corresponding to each layer of feature maps are calculated based on the lesion response values, and the feature maps of each layer are weighted and fused pixel by pixel according to the adaptive fusion weights. Pixel feature response value for the i-th layer feature map Its adaptive weights Calculation formula: Where: n is the number of feature layers participating in the fusion (n=5 in this invention). denoted as the pixel response value of the i-th layer feature at coordinates (x, y). A higher response value indicates a stronger salience of the lesion feature at that location. Final fusion features Calculation formula: This weighting mechanism enables dynamic weight adjustment: for lesion areas, it automatically increases the weight of high-recognition features; for normal tissue and artifact areas, it automatically reduces the weight of ineffective features, thus suppressing missegmentation and missed segmentation at the feature level.

5. The lesion segmentation and annotation method based on a medical knowledge graph model according to claim 1, characterized in that, The candidate lesion feature parameters include one or more of the following: grayscale parameters, size parameters, morphological parameters, density uniformity parameters, boundary gradient parameters, and anatomical location parameters.

6. The lesion segmentation and annotation method based on a medical knowledge graph model according to claim 1, characterized in that, The medical imaging knowledge graph includes image modality nodes, anatomical location nodes, lesion type nodes, lesion feature nodes, and risk warning level nodes, and includes modality-lesion relationships, anatomy-lesion relationships, lesion-feature relationships, and lesion-risk relationships; The modality-lesion relationship is used to define the identifiable lesion types under different imaging modalities, the anatomy-lesion relationship is used to define the anatomical occurrence areas corresponding to different lesion types, the lesion-feature relationship is used to bind lesion feature thresholds corresponding to different lesion types, and the lesion-risk relationship is used to determine the corresponding lesion risk warning level based on lesion feature parameters.

7. The lesion segmentation and annotation method based on a medical knowledge graph model according to claim 1, characterized in that, The lesion feature thresholds include one or more of the following: grayscale threshold, size threshold, morphology threshold, density uniformity threshold, and boundary gradient threshold. The anatomical region constraint is used to limit the anatomical location range to which the candidate lesion region belongs.

8. The lesion segmentation and annotation method based on a medical knowledge graph model according to claim 1, characterized in that, In step S5, based on the knowledge graph constraint score and the anatomical region constraint, the initial lesion segmentation mask is boundary-corrected, including: Remove pseudo-boundary regions whose knowledge graph constraint scores are below a preset legality threshold; The missing regions located in the neighborhood of the initial lesion segmentation mask that are not segmented and whose knowledge graph constraint scores meet the preset completion threshold are filled into the initial lesion segmentation mask; Remove extended regions or artifact regions whose knowledge graph constraint scores are lower than the preset redundancy removal threshold; And to trim areas that exceed the constraints of the anatomical region.

9. The lesion segmentation and annotation method based on a medical knowledge graph model according to claim 1, characterized in that, The lesion labeling information includes one or more of the following: lesion type labeling, lesion location labeling, lesion quantitative parameter labeling, and lesion risk warning level labeling; The quality assessment includes generating quality assessment results based on one or more of segmentation accuracy, annotation accuracy, annotation completeness, and annotation consistency. The error types include one or more of the following: missed segmentation, over-segmentation, labeling and classification errors, and parameter calculation bias. Specifically, the segmentation accuracy is determined based on the overlap between the corrected lesion segmentation result and the standard segmentation mask; the annotation accuracy is determined based on the accuracy of lesion type annotation and the accuracy of lesion risk warning level annotation; the annotation completeness is determined based on the correspondence between the number of automatically detected lesions and the number of standard lesions; and the annotation consistency is determined based on the dispersion of lesion quantification parameters under the same image modality and the same lesion type.

10. The lesion segmentation and annotation method based on a medical knowledge graph model according to claim 1, characterized in that, Steps S8 and S9 include: Image samples whose quality assessment results do not meet the preset qualification conditions are associated with the corresponding incorrect prediction results, standard segmentation mask, standard annotation parameters and correct classification labels to form incorrect sample pairs; The erroneous sample pairs are classified according to error type, image modality, and lesion type, and stored in the incremental training sample library; The training weights of the error sample pairs are determined based on the error type and the lesion risk warning level. Specifically, the first training weight corresponds to the severely missed segmentation error sample and the high-risk warning level lesion classification error sample; the second training weight corresponds to the oversegmentation error sample and the multiple lesion missed detection error sample; the third training weight corresponds to the parameter calculation severely biased error sample and the risk warning level labeling error sample; and the fourth training weight corresponds to the slight parameter deviation error sample and the boundary small error error sample. The first training weight is greater than the second training weight, the second training weight is greater than the third training weight, and the third training weight is greater than or equal to the fourth training weight. Freeze the bottom convolutional parameters of the CNN feature extraction backbone layer in the lesion segmentation model, and open the multi-scale feature fusion parameters, Transformer attention weight parameters, segmentation output layer parameters and classification labeling layer parameters in the lesion segmentation model for incremental training; Based on the error sample correction results after incremental updates, the grayscale distribution, size distribution, morphological distribution, density uniformity distribution, and boundary gradient distribution of the corresponding lesion type under the corresponding image modality and anatomical location are statistically analyzed, and the feature threshold boundary of the corresponding lesion type in the medical image knowledge graph is adjusted based on the statistical results.

Images