A medical image auxiliary annotation method and system based on image processing

Abstract

The present invention relates to the field of image annotation technology, and specifically to a medical image auxiliary annotation method and system based on image processing, comprising the following steps: obtaining medical image data, performing edge enhancement processing on the image data; performing preliminary annotation on the central necrotic layer, the marginal active layer and the peripheral normal tissue layer, and generating preliminary annotation areas of different lesion levels; extracting texture features, morphological features, and density features of each level, and converting them into feature vectors for storage; forming a multi-level lesion association model; adjusting the level boundary position, and ensuring that the relative position relationship between the levels is reasonable; dynamically updating the level annotation according to the change characteristics; and outputting an annotation layer containing the multi-level lesion features of the central necrotic area and the marginal active area. Compared with the existing single-level or coarse-grained annotation method, the present invention improves the hierarchical expression effect of the lesion area, so that the detail levels of the complex morphological lesions are clearly presented.

Description

Technical Field

[0001] The present invention relates to the technical field of image annotation, and in particular to a medical image auxiliary annotation method and system based on image processing. Background Art

[0002] In medical image analysis, accurate labeling of lesion areas is of great significance for clinical diagnosis, treatment planning and disease progression monitoring. Existing medical image labeling technologies are usually completed through single-level or two-dimensional plane labeling, which is sufficient for the identification and positioning of simple lesions. However, in the face of lesion areas with complex morphology, especially lesions with different lesion levels (such as necrotic areas, active areas and normal tissue boundaries), single-level labeling often fails to provide complete structural information, resulting in blurred lesion boundaries or mislabeling. This deficiency makes it difficult for doctors to accurately assess the internal state and development trend of lesions, affecting the accuracy of diagnosis and subsequent treatment effects.

[0003] In order to improve the accuracy and hierarchical expression ability of annotation, some technologies use deep learning models to extract features from image data, but these technologies mostly focus on single lesion segmentation and fail to effectively distinguish the structural features of different levels within the lesion. In addition, the lesion annotation of follow-up images lacks a dynamic update mechanism and cannot effectively reflect the changing trend of lesions over time, resulting in a lack of temporal consistency in the annotation results. At the same time, the existing annotation file formats often lack structured hierarchical information, making it difficult to present the internal features of the lesions in layers in the image management system, limiting the clinical application value of multi-level annotation results. Summary of the invention

[0004] The present invention provides a medical image auxiliary annotation method and system based on image processing.

[0005] A medical image auxiliary annotation method based on image processing comprises the following steps:

[0006] S1, image preprocessing and multi-scale enhancement: obtain medical image data and perform edge enhancement on the image data;

[0007] S2, preliminary segmentation of multi-level lesion areas: Convolutional neural network (CNN) and density clustering algorithm are used to perform multi-level segmentation on the image data after edge enhancement. The lesion area is subdivided into a central necrotic layer, a peripheral active layer, and a peripheral normal tissue layer according to the grayscale gradient characteristics and density distribution characteristics. The central necrotic layer, the peripheral active layer, and the peripheral normal tissue layer are preliminarily labeled to generate preliminary labeled areas of different lesion levels;

[0008] S3, hierarchical feature vectorization processing: feature extraction and vectorization are performed on each segmented preliminary annotated area. The texture features, morphological features, and density features of each layer are extracted through principal component analysis and local binary pattern analysis, and converted into feature vectors for storage;

[0009] S4, modeling of the correlation relationship between layers: Based on the extracted feature vectors of each layer, combined with the graph structure analysis method, a correlation model between each lesion layer is constructed, and the spatial and structural relationship between the central necrotic layer, the marginal active layer and the surrounding normal tissue layer is established to form a multi-level lesion correlation model;

[0010] S5, hierarchical features and boundary optimization: Under the guidance of the multi-level lesion association model, the preliminary annotations of each level are optimized, and the boundary positions of the preliminary annotations of different levels are checked to ensure that the relative positions of the annotation levels are consistent with the actual pathology. The level boundary positions are adjusted to improve the accuracy of the annotations, and the relative position relationship between the levels is ensured to be reasonable.

[0011] S6, Dynamic tracking and updating of lesion level changes: In the process of labeling multiple scanned images or follow-up images, the dynamic changes of lesion levels are identified through a multi-level lesion association model, including the expansion of the central necrotic area or the shrinking trend of the marginal active area. The level annotation is dynamically updated according to the change characteristics to ensure the temporal consistency and structural coherence of the annotation;

[0012] S7, output and storage of multi-level annotation results: the multi-level annotation results finally generated are stored as a standardized label file, and the annotation layer containing the multi-level lesion features of the central necrotic area and the marginal active area is output.

[0013] Optionally, the preliminary segmentation of the multi-level lesion area in S2 specifically includes:

[0014] S21, for the image data after edge enhancement, using the pre-trained convolutional neural network to extract grayscale gradient features and density distribution features to obtain structural feature maps of different levels in the lesion area, and for each layer of structural feature map F of the input image data, performing a convolution operation through a convolution layer;

[0015] The convolution operation identifies different grayscale gradient features and density distribution features in the image. After being processed by several convolutional layers, pooling layers, and activation layers in the convolutional neural network, the output structure feature map set {F (1) , F (2) , ..., F (n)}, each structural feature map represents the features of different levels of the image, including grayscale gradient features and density distribution features;

[0016] S22, the feature map set {F (1) , F(2) , ..., F (n)} Input density clustering algorithm, according to the density and grayscale gradient of pixel points in the lesion area, the lesion area is subdivided into central necrotic layer, edge active layer and surrounding normal tissue layer. The density clustering algorithm depends on the neighborhood radius ∈ and the minimum number of neighborhood points MinPts. For each pixel point p in the image, its neighborhood range N in the feature space is calculated. ∈ (p), the density condition is: if the neighborhood N of p ∈ If (p) contains at least MinPts points, then p is regarded as a core point and belongs to a high-density area, and is then clustered into one category. According to the grayscale gradient and density distribution characteristics of the lesion area, the areas obtained by density clustering are divided into:

[0017] Central necrosis layer: contains the densest pixels (high-density area), which is the necrotic area in the center of the lesion with low gray value;

[0018] Marginal active layer: located outside the necrotic layer, the area with relatively high gray value and density, reflecting the active area of ​​the lesion;

[0019] Peripheral normal tissue layer: The peripheral area with lower density forms a natural boundary with the lesion;

[0020] S23, multi-level preliminary annotation generation: According to the segmentation results of density clustering, the central necrotic layer, the marginal active layer and the surrounding normal tissue layer are respectively preliminarily annotated to generate preliminary annotation areas of different lesion levels.

[0021] Define annotation categories: According to the different levels of regions obtained by segmentation, define three annotation categories:

[0022] Optionally, the hierarchical feature vectorization processing in S3 specifically includes:

[0023] S31, texture feature extraction: Apply the local binary pattern LBP to each preliminary marked area to generate a local binary pattern histogram to represent the texture features in the area. The local binary pattern LBP is expressed as: Where (x, y) represents the pixel position in the labeled area, I k is the gray value of the kth neighboring pixel around the pixel (x, y), I c is the gray value of the center pixel, s(x) is the sign function, defined as: s(x) = 1 when x ≥ 0, otherwise s(x) = 0;

[0024] S32, morphological feature extraction: perform principal component analysis (PCA) on the contour of each marked area to extract the morphological features of the area, including the main morphological direction and area size, and convert the principal component feature vector of the area into PCA shape It is expressed as:

[0025] PCA shape =(λ1, λ2, θ), where λ1 and λ2 represent the first and second principal component eigenvalues ​​of the region, representing the main direction of the shape; θ is the direction angle of the principal component, used to describe the directional characteristics of the region;

[0026] S33, density feature calculation: in each marked area, the average gray value and standard deviation in the area are calculated as density features to form a density feature vector Density: Density = (μ, σ), where μ represents the average gray value of the area, σ represents the standard deviation of the gray value, and reflects the gray distribution of the area;

[0027] S34, the extracted texture features, morphological features and density features are combined to form a feature vector V, which is expressed as: V = (LBP, PCA shape , Density), and stores the feature vector in the database for subsequent modeling of inter-level association relationships and annotation optimization.

[0028] Optionally, the S4 specifically includes:

[0029] S41, feature node definition: Based on the preliminary labeled regions of the central necrotic layer, the marginal active layer, and the surrounding normal tissue layer, the feature vector V of each labeled region is calculated as (LBP, PCA shape , Density) as the node feature, defining each hierarchical area as a graph node N i , where N1, N2, and N3 represent the central necrotic layer, marginal active layer, and surrounding normal tissue layer, respectively;

[0030] Spatial relationship edge construction: calculate the spatial distance of each level area and determine the edge connection relationship between nodes;

[0031] Structural relationship weight calculation: Based on the similarity of the feature vectors of nodes at each level, the structural relationship weight w is calculated i，j , defined as: Among them, V i and V j The nodes N i and N j The characteristic vector of ||V i -V j || represents the Euclidean distance between feature vectors, σ is a parameter that controls the attenuation of similarity, and the structural relationship weight is used to quantify the feature similarity between each level;

[0032] Construction of a multi-level lesion association model: The constructed nodes, edges, and weights form a graph structure. The graph structure establishes the spatial and structural relationship between the central necrotic layer, the marginal active layer, and the surrounding normal tissue layer, forming a multi-level lesion association model to provide a reference for subsequent annotation optimization and dynamic tracking.

[0033] Optionally, the edge connection relationship between the nodes is expressed as:

[0034] Among them, d i，j Represents node N i and N j The spatial distance between i , r i and c j , r j The nodes N are i and N j The centroid coordinates of i，j If the value is less than the preset threshold, an edge E is established between the nodes. i，j , used to represent the spatial adjacency relationship between levels;

[0035] The graph structure is expressed as: G=(N, E, W), where N is a node set, E is an edge set, and W is an edge weight set.

[0036] Optionally, the hierarchical feature and boundary optimization in S5 specifically includes:

[0037] S51, boundary overlap check: Based on the node and edge relationship in the multi-level lesion association model, the boundary overlap of the central necrotic layer, the marginal active layer and the surrounding normal tissue layer is calculated. i and N j , calculate the boundary overlap, that is, the area ratio of the boundary overlap area R i，j :

[0038] Among them, B i and B j Represents node N respectively i and N j The boundary area of ​​the corresponding level, |B i ∩B j | is the overlapping area of ​​the two boundary regions, |B i | and |B j | is the area of ​​each boundary region; if R i，j If the preset overlap threshold is exceeded, it is identified as a boundary overlap anomaly;

[0039] S52, boundary distance check: According to the node centroid coordinates and spatial relationship edges of the multi-level lesion association model, the relative boundary distances between the central necrotic layer, the marginal active layer and the surrounding normal tissue layer are calculated. i and N j , calculate the minimum distance d between boundaries min (B i , Bj );

[0040] S53, Boundary position adjustment: For boundary areas with excessive overlap or too close distance, the boundary position is fine-tuned using the boundary offset method based on the results of boundary overlap and distance inspection, so that the relative positions of the boundaries between the central necrotic layer, the marginal active layer and the surrounding normal tissue layer are reasonable. The boundary fine-tuning is expressed as: in, represents the adjusted boundary position, δ is the adjustment offset, and the boundary overlap R i，j and the boundary distance d min (B i , B j )’s feedback dynamic setting ensures that the annotation hierarchy is reasonable and the boundaries are clear.

[0041] Optionally, the minimum distance d between the boundaries min (B i , B j ) is calculated as:

[0042] If d min (B i , B j ) is less than the preset distance threshold, the boundary position is adjusted to meet the separation requirements of the pathological structure. dist(p, q) represents the spatial distance between two pixel points p and q, which is consistent with the above centroid dist(N i , N j ) is calculated in the same way and is defined using the Euclidean distance.

[0043] Optionally, the S6 specifically includes:

[0044] S61, hierarchical region change detection: based on the node N of each hierarchical region in the multi-level lesion association model i The center of mass position (c i , r i ) and area A i , compare the same level area at different times and calculate the centroid shift Δd i and area change ratio ΔA i :

[0045] S62, dynamic change trend identification: according to the centroid offset Δd i and area change ratio ΔA i Determine the changing trend of lesion levels;

[0046] If the central necrosis layer node N 坏死 The area change ratio ΔA 坏死 If it increases significantly, it is identified as a trend of expansion of the necrotic area;

[0047] If the edge active layer node N 活性 The area change ratio ΔA 活性 If it decreases significantly, it is identified as a trend of shrinking active area;

[0048] If the center of mass shifts Δd i If the set threshold is exceeded, it means that the position of the hierarchical area has changed and needs to be adjusted.

[0049] Optionally, the center of mass offset Δd i and area change ratio ΔA i The calculation is expressed as:

[0050]

[0051]

[0052] Where, Δd i Indicates area N i The offset distance between the center of mass of the current time point and the previous time point, and Respectively represent the centroid position at the current time t and the previous time t-1, and Respectively represent the area at the current time t and the previous time t-1, d i =(c i , r i ).

[0053] A medical image assisted annotation system based on image processing, used to implement the above-mentioned medical image assisted annotation method based on image processing, includes the following modules:

[0054] An image preprocessing module is used to enhance the acquired medical image data to highlight the details of the lesion edge and reduce noise interference;

[0055] The multi-level segmentation module uses convolutional neural networks and density clustering methods to perform multi-level segmentation on the enhanced image data and generate preliminary annotated regions including the central necrotic layer, the marginal active layer, and the surrounding normal tissue layer;

[0056] The feature vectorization module is used to perform feature vectorization on the segmented labeled regions at each level to generate feature vectors containing texture, morphology and density information;

[0057] The association relationship modeling module builds the association relationship model between each lesion level based on the feature vector and graph structure analysis method to ensure that the spatial and structural relationships between the annotation levels are reasonable;

[0058] The dynamic annotation update module is used to identify the dynamic changes of the lesion layer according to the association relationship model during the annotation process of multiple scan images or follow-up images, and adjust the boundary position of the annotation area to ensure time consistency;

[0059] The annotation result storage and output module stores the final generated multi-level annotation results as standardized label files for compatible display in the medical image management system.

[0060] Beneficial effects of the present invention:

[0061] The present invention combines a convolutional neural network (CNN) with a density clustering method to achieve multi-level segmentation of the lesion area, subdivide the lesion area into a central necrotic layer, an edge active layer and a surrounding normal tissue layer, and perform feature extraction and annotation on each level area respectively. Compared with the existing single-level or coarse-grained annotation method, the layered expression effect of the lesion area is improved, and the detailed levels of complex morphological lesions are clearly presented, especially the accurate depiction of the boundaries of necrotic and active areas. This multi-level annotation not only provides accurate lesion distribution information for clinical diagnosis, but also facilitates doctors to fully understand the internal structure of the lesion, providing strong support for formulating more targeted treatment plans.

[0062] The present invention introduces a multi-level lesion association model, dynamically detects the centroid position and boundary overlap of different levels of lesions, can track the trend of lesions changing over time in real time, and identify the expansion of the necrotic area in the center of the lesion or the contraction of the active area at the edge. In the follow-up images, the hierarchical boundary position is dynamically adjusted based on the comprehensive feedback of the centroid offset and the area change ratio to ensure the dynamic consistency of the annotation, so that the continuity and temporal coherence of the follow-up image annotation can be maintained. The dynamic update mechanism improves the clinical practicality of the annotation, can effectively reflect the progress or changes of the lesions at different time points, and provides a reliable basis for the dynamic monitoring of the disease and the evaluation of the efficacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 A schematic diagram of a marking method flow chart of an embodiment of the present invention;

[0065] Figure 2 Schematic diagram of a module of a labeling system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0067] It should be noted that the references to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0068] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0069] like Figure 1 As shown, a medical image auxiliary annotation method based on image processing includes the following steps:

[0070] S1, Image preprocessing and multi-scale enhancement: Obtain medical image data, perform edge enhancement on the image data, improve the clarity of the edge of the lesion, reduce artifacts and noise interference, provide a clear image basis for subsequent segmentation and annotation, implement a local contrast enhancement algorithm in the lesion edge area, enhance the edge details based on the grayscale difference of image pixels, and make the boundary between the lesion area and the surrounding normal tissue clearer;

[0071] The local contrast enhancement algorithm magnifies the grayscale difference at the edge of the lesion area to make the edge clearer and enhance the grayscale contrast of the local area.

[0072] Grayscale difference calculation: a local window W (3 × 3 pixels) is set at the edge of the lesion area. For each pixel in the window, the difference between the pixel and the average grayscale value in the window is calculated to highlight the contrast of the pixel relative to its neighborhood.

[0073] Define I(x, y) to represent the grayscale value of the pixel at the coordinate (x, y). Represents the average grayscale value of all pixels in window W, and the grayscale difference formula is: The grayscale difference ΔI(x, y) between the pixel I(x, y) and its neighborhood is calculated. When the grayscale difference is large, it means that the pixel is in the edge area. In order to enhance the contrast of the edge of the lesion, the grayscale difference of each pixel is amplified. The enhancement coefficient is set to α (1<α≤3), and the following enhancement is applied to each pixel: Among them, I enhanced (x, y) is the enhanced pixel grayscale value, and α·ΔI(x, y) represents the amplified grayscale difference to enhance the contrast of the pixel relative to its neighborhood. The grayscale difference between the edge of the lesion area and its surrounding normal tissue is amplified, making the boundary easier to identify in subsequent segmentation and annotation.

[0074] S2, preliminary segmentation of multi-level lesion areas: Convolutional neural network (CNN) and density clustering algorithm are used to perform multi-level segmentation on the image data after edge enhancement. The lesion area is subdivided into a central necrotic layer, a peripheral active layer, and a peripheral normal tissue layer according to the grayscale gradient characteristics and density distribution characteristics. The central necrotic layer, the peripheral active layer, and the peripheral normal tissue layer are preliminarily labeled to generate preliminary labeled areas of different lesion levels;

[0075] S3, hierarchical feature vectorization processing: feature extraction and vectorization are performed on each segmented preliminary annotated area. The texture features, morphological features, and density features of each layer are extracted through principal component analysis and local binary pattern analysis, and converted into feature vectors for storage;

[0076] S4, modeling of the correlation relationship between layers: Based on the extracted feature vectors of each layer, combined with the graph structure analysis method, a correlation model between each lesion layer is constructed, and the spatial and structural relationship between the central necrotic layer, the marginal active layer and the surrounding normal tissue layer is established to form a multi-level lesion correlation model;

[0077] S5, hierarchical features and boundary optimization: Under the guidance of the multi-level lesion association model, the preliminary annotations of each level are optimized, and the boundary positions of the preliminary annotations of different levels are checked to ensure that the relative positions of the annotation levels are consistent with the actual pathology. The level boundary positions are adjusted to improve the accuracy of the annotations, and the relative position relationship between the levels is ensured to be reasonable.

[0078] S6, Dynamic tracking and updating of lesion level changes: In the process of labeling multiple scanned images or follow-up images, the dynamic changes of lesion levels are identified through a multi-level lesion association model, including the expansion of the central necrotic area or the shrinking trend of the marginal active area. The level annotation is dynamically updated according to the change characteristics to ensure the temporal consistency and structural coherence of the annotation;

[0079] S7, output and storage of multi-level annotation results: The final generated multi-level annotation results are stored as standardized label files, and the annotation layers containing the multi-level lesion features of the central necrotic area and the marginal active area are output for reference in clinical diagnosis and treatment plan formulation. Metadata or annotations are added to the generated label file to indicate the lesion level corresponding to each label value for subsequent reading and analysis. For example, label 1 represents the "central necrotic area", label 2 represents the "marginal active area", and label 3 represents the "peripheral normal tissue area".

[0080] The preliminary segmentation of the multi-level lesion area in S2 specifically includes:

[0081] S21, for the image data after edge enhancement, the grayscale gradient features and density distribution features are extracted using the pre-trained convolutional neural network to obtain the structural feature maps of different levels in the lesion area. For each layer of the structural feature map F of the input image data, a convolution operation is performed through the convolution layer. The convolution operation is expressed as:

[0082] Among them, F i，j is the value of the feature map output by the convolution at position (i, j), I i+m，j+n is the pixel grayscale value of the input image data at position (i+m, j+n), K m，n is the weight of the convolution kernel, the size of the convolution kernel is (2k+1)×(2k+1), and b is the bias of the convolution layer;

[0083] The convolution operation identifies different grayscale gradient features and density distribution features in the image. After being processed by several convolutional layers, pooling layers, and activation layers in the convolutional neural network, the output structure feature map set {F (1) , F (2) , ..., F (n)}, each structural feature map represents the features of different levels of the image, including grayscale gradient features and density distribution features;

[0084] S22, the feature map set {F (1) , F (2) , ..., F (n)} Input density clustering algorithm, according to the density and grayscale gradient of pixel points in the lesion area, the lesion area is subdivided into central necrotic layer, edge active layer and surrounding normal tissue layer. The density clustering algorithm depends on the neighborhood radius ∈ and the minimum number of neighborhood points MinPts. For each pixel point p in the image, its neighborhood range N in the feature space is calculated. ∈ (p): N ∈ (p) = {q∈D|dist(p,q)≤∈}, where D is the set of all pixels and dist(p,q) is the Euclidean distance between pixels p and q: Among them, F (i) (p) and F (i) (q) are the pixels p and q in the structural feature map F (i) The density condition is: if the neighborhood N of p ∈ If (p) contains at least MinPts points, then p is regarded as a core point and belongs to a high-density area, and is then clustered into one category. According to the grayscale gradient and density distribution characteristics of the lesion area, the areas obtained by density clustering are divided into:

[0085] Central necrosis layer: contains the densest pixels (high-density area), which is the necrotic area in the center of the lesion with low gray value;

[0086] Marginal active layer: located outside the necrotic layer, the area with relatively high gray value and density, reflecting the active area of ​​the lesion;

[0087] Peripheral normal tissue layer: The peripheral area with lower density forms a natural boundary with the lesion;

[0088] S23, multi-level preliminary annotation generation: According to the segmentation results of density clustering, the central necrotic layer, the marginal active layer and the surrounding normal tissue layer are respectively preliminarily annotated to generate preliminary annotation areas of different lesion levels.

[0089] Define annotation categories: According to the different levels of regions obtained by segmentation, define three annotation categories:

[0090] Category 1 (central necrosis layer): represents the core area of ​​the lesion, usually appearing as a low-density, low-gray value area.

[0091] Category 2 (edge ​​active layer): represents the active area outside the lesion, with high density and obvious changes in grayscale and texture.

[0092] Category 3 (surrounding normal tissue layer): represents the normal tissue area surrounding the lesion, which has a lower density and is connected to the boundary of the lesion.

[0093] Pixel classification and labeling: According to the results of density clustering, each pixel in the image is classified into the corresponding labeling level. Assuming that each pixel p belongs to a certain category C, then:

[0094] If pixel p has the highest density and meets the low gray value criteria, it is marked as category 1 (central necrosis layer).

[0095] If the density and grayscale of pixel p are in the middle range and located outside the necrotic layer, it is marked as category 2 (edge ​​active layer).

[0096] If the density of pixel p is low and it is adjacent to the outer boundary of the active layer, it is marked as category 3 (surrounding normal tissue layer).

[0097] The hierarchical feature vectorization processing in S3 specifically includes:

[0098] S31, texture feature extraction: Apply the local binary pattern LBP to each preliminary marked area to generate a local binary pattern histogram to represent the texture features in the area. The local binary pattern LBP is expressed as: Where (x, y) represents the pixel position in the labeled area, I k is the gray value of the kth neighboring pixel around the pixel (x, y), I c is the gray value of the center pixel, s(x) is the sign function, defined as: s(x) = 1 when x ≥ 0, otherwise s(x) = 0;

[0099] S32, morphological feature extraction: perform principal component analysis (PCA) on the contour of each marked area to extract the morphological features of the area, including the main morphological direction and area size, and convert the principal component feature vector of the area into PCA shape It is expressed as:

[0100] PCA shape =(λ1, λ2, θ), where λ1 and λ2 represent the first and second principal component eigenvalues ​​of the region, representing the main direction of the shape; θ is the direction angle of the principal component, used to describe the directional characteristics of the region;

[0101] S33, density feature calculation: in each marked area, the average gray value and standard deviation in the area are calculated as density features to form a density feature vector Density: Density = (μ, σ), where μ represents the average gray value of the area, σ represents the standard deviation of the gray value, and reflects the gray distribution of the area;

[0102] S34, the extracted texture features, morphological features and density features are combined to form a feature vector V, which is expressed as: V = (LBP, PCA shape , Density), and stores the feature vector in the database for subsequent modeling of inter-level association relationships and annotation optimization.

[0103] The aforementioned convolutional neural network and density clustering extract the grayscale gradient and density distribution of image data, which is used for global feature extraction before segmentation. The purpose is to obtain a preliminary feature map within the lesion area, which is used for the segmentation of multi-level regions and is the basis for identifying different levels of regions during the segmentation process. However, these features are high-dimensional.

[0104] The feature extraction table and feature vector quantization processing here are to make more detailed feature expressions for each specific segmented labeled area after the segmentation is completed. The features extracted here are low-dimensional features, which are used to describe the independent characteristics of each labeled area; the feature vector quantization processing is to extract the local structural features of the specific labeled areas obtained by segmentation (central necrotic layer, edge active layer and surrounding normal tissue layer). These features are quantitative descriptions of the specific labeled areas, which are convenient for storage and subsequent analysis.

[0105] S4 specifically includes:

[0106] S41, feature node definition: Based on the preliminary labeled regions of the central necrotic layer, the marginal active layer, and the surrounding normal tissue layer, the feature vector V of each labeled region is calculated as (LBP, PCA shape , Density) as the node feature, defining each hierarchical area as a graph node N i , where N1, N2, and N3 represent the central necrotic layer, marginal active layer, and surrounding normal tissue layer, respectively;

[0107] Spatial relationship edge construction: calculate the spatial distance of each level area and determine the edge connection relationship between nodes;

[0108] Structural relationship weight calculation: Based on the similarity of the feature vectors of nodes at each level, the structural relationship weight w is calculated i，j , defined as: Among them, V i and V j The nodes N i and N j The characteristic vector of ||V i -V j || represents the Euclidean distance between feature vectors, σ is a parameter that controls the attenuation of similarity, and the structural relationship weight is used to quantify the feature similarity between each level;

[0109] Construction of a multi-level lesion association model: The constructed nodes, edges, and weights form a graph structure. The graph structure establishes the spatial and structural relationship between the central necrotic layer, the marginal active layer, and the surrounding normal tissue layer, forming a multi-level lesion association model to provide a reference for subsequent annotation optimization and dynamic tracking.

[0110] The edge connection relationship between nodes is expressed as:

[0111] Among them, d i，j Represents node N i and N j The spatial distance between i , r i and c j , r j The nodes N arei and N j The centroid coordinates of i，j If the value is less than the preset threshold, an edge E is established between the nodes. i，j , used to represent the spatial adjacency relationship between levels;

[0112] The graph structure is expressed as: G = (N, E, W), where N is the node set, E is the edge set, and W is the edge weight set.

[0113] The hierarchical features and boundary optimization in S5 specifically include:

[0114] S51, boundary overlap check: Based on the node and edge relationship in the multi-level lesion association model, the boundary overlap of the central necrotic layer, the marginal active layer and the surrounding normal tissue layer is calculated. i and N j , calculate the boundary overlap, that is, the area ratio of the boundary overlap area R i，j :

[0115] Among them, B i and B j Represents node N respectively i and N j The boundary area of ​​the corresponding level, |B i ∩B j | is the overlapping area of ​​the two boundary regions, |B i | and |B j | is the area of ​​each boundary region; if R i，j If the preset overlap threshold is exceeded, it is identified as a boundary overlap anomaly;

[0116] S52, boundary distance check: According to the node centroid coordinates and spatial relationship edges of the multi-level lesion association model, the relative boundary distances between the central necrotic layer, the marginal active layer and the surrounding normal tissue layer are calculated. i and N j , calculate the minimum distance d between boundaries min (B i , B j );

[0117] S53, Boundary position adjustment: For boundary areas with excessive overlap or too close distance, the boundary position is fine-tuned using the boundary offset method based on the results of boundary overlap and distance inspection, so that the relative positions of the boundaries between the central necrotic layer, the marginal active layer and the surrounding normal tissue layer are reasonable. The boundary fine-tuning is expressed as: in, represents the adjusted boundary position, δ is the adjustment offset, and the boundary overlap R i，jand the boundary distance d min (B i , B j )’s feedback dynamic setting ensures that the annotation hierarchy is reasonable and the boundaries are clear.

[0118] The minimum distance d between boundaries min (B i , B j ) is calculated as:

[0119] If d min (B i , B j ) is less than the preset distance threshold, the boundary position is adjusted to meet the separation requirements of the pathological structure. dist(p, q) represents the spatial distance between two pixel points p and q, which is consistent with the above centroid dist(N i , N j ) is calculated in the same way, using the Euclidean distance definition:

[0120] Among them, c p , r p are the column and row coordinates of pixel point p, c q , r q are the column and row coordinates of pixel q. Different from the centroid calculation, dist(N i , N j ) represents the distance between the centroids of the hierarchical regions, which is used to represent the overall spatial relationship of different levels of the lesion (such as the central necrotic layer and the marginal active layer); dist(p, q) represents the distance between two pixel points p and q, which is used to calculate the minimum distance d of the boundary area. min (B i , B j ), which is mainly used to check whether the boundaries of different levels are too close or overlap.

[0121] According to the boundary overlap R i，j and the boundary distance d min (B i , B j )’s feedback dynamic setting adjustment offset δ is as follows:

[0122] 1. If the overlap R of adjacent boundaries is detected i，j If the threshold is exceeded, it means that the boundary areas of adjacent layers overlap excessively. In this case, δ needs to be set to reduce the overlap. The setting rules are as follows: δ i，j =β·(R i，j -R threshold ), where δ i，j From area N i To area Nj The adjustment offset, β is the adjustment scale factor (between 0 and 1), which is used to control the size of the offset. i，j Beyond R threshold When i，j It will increase by the amount that the threshold is exceeded, thereby reducing the overlap of the boundaries.

[0123] 2. If the minimum distance d between adjacent boundaries is detected min (B i , B j ) is less than the preset minimum distance threshold d min-threshold This means that the boundaries between the two hierarchical areas are too close and need to be pulled apart to ensure a clear structure. The setting rules are as follows:

[0124] δ i，j =α·(d min-threshold -d min (B i , B j )), where d min-threshol d is the minimum distance threshold allowed by the boundary, d min (B i , B j ) is the minimum distance currently calculated, and α is the distance adjustment coefficient, which controls the magnitude of the offset. min (B i , B j ) is less than d min-threshol d, δ i，j The deviation will increase to widen the boundary distance.

[0125] S6 specifically includes:

[0126] S61, hierarchical region change detection: based on the node N of each hierarchical region in the multi-level lesion association model i The center of mass position (c i , r i ) and area A i , compare the same level area at different times and calculate the centroid shift Δd i and area change ratio ΔA i :

[0127] S62, dynamic change trend identification: according to the centroid offset Δd i and area change ratio ΔA i Determine the changing trend of lesion levels;

[0128] If the central necrosis layer node N 坏死 The area change ratio ΔA 坏死 If it increases significantly, it is identified as a trend of expansion of the necrotic area;

[0129] If the edge active layer node N 活性 The area change ratio ΔA 活性 If it decreases significantly, it is identified as a trend of shrinking active area;

[0130] If the center of mass shifts Δd i If the threshold is exceeded, it means that the position of the hierarchical area has changed and needs to be adjusted;

[0131] According to the detected expansion or contraction trend, the hierarchical annotation boundary position is dynamically adjusted to reflect the actual changes in the lesion hierarchy. The specific update formula is as follows: in, is the final updated boundary position, is the current boundary position (the updated result in S5).

[0132] In the process of feature vectorization, the area A i It is calculated as one of the morphological features of each labeled area and stored in the feature vector. Specifically, the area A i The calculation method is as follows:

[0133] Pixel counting method: For each marked area (central necrotic layer, marginal active layer, and peripheral normal tissue layer), the area is calculated by counting the number of all pixels in the area: 1, where A i Indicates the level area N i The area of ​​​​p∈N i Indicates that it belongs to area N i For each pixel point, the number of pixels in the region is actually accumulated to obtain the area of ​​the region. In the feature vectorization step, A i It is stored in the feature vector V as a morphological feature of the region.

[0134] Center of mass deviation Δd i and area change ratio ΔA i The calculation is expressed as:

[0135]

[0136]

[0137] Where, Δd i Indicates area N i The offset distance between the center of mass of the current time point and the previous time point, and Respectively represent the centroid position at the current time t and the previous time t-1, and Respectively represent the area at the current time t and the previous time t-1, d i =(ci , r i ).

[0138] like Figure 2 As shown, a medical image assisted annotation system based on image processing is used to implement the above-mentioned medical image assisted annotation method based on image processing, and includes the following modules:

[0139] An image preprocessing module is used to enhance the acquired medical image data to highlight the details of the lesion edge and reduce noise interference;

[0140] The multi-level segmentation module uses convolutional neural networks and density clustering methods to perform multi-level segmentation on the enhanced image data and generate preliminary annotated regions including the central necrotic layer, the marginal active layer, and the surrounding normal tissue layer;

[0141] The feature vectorization module is used to perform feature vectorization on the segmented labeled regions at each level to generate feature vectors containing texture, morphology and density information;

[0142] The association relationship modeling module builds the association relationship model between each lesion level based on the feature vector and graph structure analysis method to ensure that the spatial and structural relationships between the annotation levels are reasonable;

[0143] The dynamic annotation update module is used to identify the dynamic changes of the lesion layer according to the association relationship model during the annotation process of multiple scan images or follow-up images, and adjust the boundary position of the annotation area to ensure time consistency;

[0144] The annotation result storage and output module stores the final generated multi-level annotation results as standardized label files for compatible display in the medical image management system.

[0145] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0146] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A medical image auxiliary annotation method based on image processing, characterized in that: The following steps are involved: S1, image preprocessing and multi-scale enhancement: obtain medical image data and perform edge enhancement on the image data; S2, preliminary segmentation of multi-level lesion areas: Use convolutional neural network and density clustering algorithm to perform multi-level segmentation on the image data after edge enhancement processing, subdivide the lesion area into a central necrotic layer, a marginal active layer and a peripheral normal tissue layer according to the grayscale gradient characteristics and density distribution characteristics, and preliminarily mark the central necrotic layer, the marginal active layer and the peripheral normal tissue layer to generate preliminary marked areas of different lesion levels; S3, hierarchical feature vectorization processing: feature extraction and vectorization are performed on each segmented preliminary annotated area. The texture features, morphological features, and density features of each layer are extracted through principal component analysis and local binary pattern analysis, and converted into feature vectors for storage; S4, modeling of the correlation relationship between layers: Based on the extracted feature vectors of each layer, combined with the graph structure analysis method, a correlation model between each lesion layer is constructed, and the spatial and structural relationship between the central necrotic layer, the marginal active layer and the surrounding normal tissue layer is established to form a multi-level lesion correlation model; S5, hierarchical features and boundary optimization: Under the guidance of the multi-level lesion association model, the preliminary annotations of each level are optimized, and the boundary positions of the preliminary annotations of different levels are checked to ensure that the relative positions of the annotation levels are consistent with the actual pathology, adjust the level boundary positions, and ensure that the relative position relationship between the levels is reasonable; S6, Dynamic tracking and updating of lesion level changes: In the process of labeling multiple scanned images or follow-up images, the dynamic changes of lesion levels are identified through a multi-level lesion association model, including the expansion of the central necrotic area or the shrinking trend of the marginal active area, and the level annotation is dynamically updated according to the change characteristics; S7, output and storage of multi-level annotation results: the multi-level annotation results finally generated are stored as a standardized label file, and the annotation layer containing the multi-level lesion features of the central necrotic area and the marginal active area is output.

2. The medical image auxiliary annotation method based on image processing according to claim 1, characterized in that: The preliminary segmentation of the multi-level lesion area in S2 specifically includes: S21, for the image data after edge enhancement, using the pre-trained convolutional neural network to extract grayscale gradient features and density distribution features to obtain structural feature maps of different levels in the lesion area, and for each layer of structural feature map F of the input image data, performing a convolution operation through a convolution layer; The convolution operation identifies different grayscale gradient features and density distribution features in the image. After being processed by several convolutional layers, pooling layers, and activation layers in the convolutional neural network, the output structure feature map set {F (1) ,F (2) ,...,F (n) }, each structural feature map represents the features of different levels of the image, including grayscale gradient features and density distribution features; S22, the feature map set {F (1) ,F (2) ,...,F (n) } Input density clustering algorithm, according to the density and grayscale gradient of pixel points in the lesion area, the lesion area is subdivided into central necrotic layer, edge active layer and surrounding normal tissue layer. The density clustering algorithm depends on the neighborhood radius ∈ and the minimum number of neighborhood points MinPts. For each pixel point p in the image, its neighborhood range N in the feature space is calculated. ∈ (p), the density condition is: if p's neighborhood N ∈ If (p) contains at least MinPts points, then p is regarded as a core point and belongs to a high-density area, and then clustered into one category. According to the grayscale gradient and density distribution characteristics of the lesion area, the area obtained by density clustering is divided into: central necrotic layer, marginal active layer and peripheral normal tissue layer; S23, multi-level preliminary annotation generation: According to the segmentation results of density clustering, the central necrotic layer, the marginal active layer and the surrounding normal tissue layer are respectively preliminarily annotated to generate preliminary annotation areas of different lesion levels.

3. The medical image auxiliary annotation method based on image processing according to claim 1, characterized in that: The hierarchical feature vectorization processing in S3 specifically includes: S31, texture feature extraction: Apply the local binary pattern LBP to each preliminary marked area to generate a local binary pattern histogram to represent the texture features in the area. The local binary pattern LBP is expressed as: Where (x, y) represents the pixel position in the labeled area, I k is the gray value of the kth neighboring pixel around the pixel (x, y), Ic is the gray value of the central pixel, s(x) is the sign function, defined as: s(x) = 1 when x ≥ 0, otherwise s(x) = 0; S32, morphological feature extraction: perform principal component analysis (PCA) on the contour of each marked area to extract the morphological features of the area, including the main morphological direction and area size, and convert the principal component feature vector of the area into PCA shape It is expressed as: PCA shape =(λ1,λ2,θ), where λ1 and λ2 represent the first and second principal component eigenvalues ​​of the region, representing the main directions of the shape; θ is the direction angle of the principal component, used to describe the directional characteristics of the region; S33, density feature calculation: in each marked area, the average gray value and standard deviation in the area are calculated as density features to form a density feature vector Density: Density = (μ, σ), where μ represents the average gray value of the area, σ represents the standard deviation of the gray value, and reflects the gray distribution of the area; S34, the extracted texture features, morphological features and density features are combined to form a feature vector V, which is expressed as: V = (LBP, PCA shape ,Density), and store the feature vector in the database.

4. The medical image auxiliary annotation method based on image processing according to claim 3 is characterized in that: The S4 specifically includes: S41, feature node definition: Based on the preliminary labeled regions of the central necrotic layer, the marginal active layer, and the surrounding normal tissue layer, the feature vector V of each labeled region is calculated as (LBP, PCA shape ,Density) as the node feature, defining each hierarchical area as a graph node N i , where N1, N2 and N3 represent the central necrotic layer, marginal active layer and surrounding normal tissue layer, respectively; Spatial relationship edge construction: calculate the spatial distance of each level area and determine the edge connection relationship between nodes; Structural relationship weight calculation: Based on the similarity of the feature vectors of nodes at each level, the structural relationship weight w is calculated i,j , defined as: Among them, V i and V j The nodes N i and N j The characteristic vector of ||V i -V j || represents the Euclidean distance between feature vectors, σ is a parameter that controls the attenuation of similarity, and the structural relationship weight is used to quantify the feature similarity between each level; Construction of a multi-level lesion association model: The constructed nodes, edges, and weights form a graph structure, which establishes the spatial and structural relationship between the central necrotic layer, the marginal active layer, and the surrounding normal tissue layer to form a multi-level lesion association model.

5. The medical image auxiliary annotation method based on image processing according to claim 4, characterized in that: The edge connection relationship between the nodes is expressed as: Among them, d i,j Represents node N i and N j The spatial distance between i ,r i and c j ,r j They are node N i and N j The centroid coordinates of i,j If the value is less than the preset threshold, an edge E is established between the nodes. i,j , used to represent the spatial adjacency relationship between levels; The graph structure is represented as: G=(N, E, W), where N is a node set, E is an edge set, and W is an edge weight set.

6. The medical image auxiliary annotation method based on image processing according to claim 1, characterized in that: The hierarchical features and boundary optimization in S5 specifically include: S51, boundary overlap check: Based on the node and edge relationship in the multi-level lesion association model, the boundary overlap of the central necrotic layer, the marginal active layer and the surrounding normal tissue layer is calculated. i and N j , calculate the boundary overlap, that is, the area ratio of the boundary overlap area R i,j : Among them, B i and B j Represents node N respectively i and N j The boundary area of ​​the corresponding level, |B i ∩B j | is the overlapping area of ​​the two boundary regions, |B i | and |B j | is the area of ​​each boundary region; if R i,j If the preset overlap threshold is exceeded, it is identified as a boundary overlap anomaly; S52, boundary distance check: According to the node centroid coordinates and spatial relationship edges of the multi-level lesion association model, the relative boundary distances between the central necrotic layer, the marginal active layer and the surrounding normal tissue layer are calculated. i and N j , calculate the minimum distance d between boundaries min (B i ,B j ); S53, Boundary position adjustment: For boundary areas with excessive overlap or too close distance, the boundary position is fine-tuned using the boundary offset method based on the results of boundary overlap and distance inspection, so that the relative positions of the boundaries between the central necrotic layer, the marginal active layer and the surrounding normal tissue layer are reasonable. The boundary fine-tuning is expressed as: in, represents the adjusted boundary position, δ is the adjustment offset, and the boundary overlap R i,j and the boundary distance d min (B i ,B j )’s feedback dynamic setting.

7. The medical image auxiliary annotation method based on image processing according to claim 6, characterized in that: The minimum distance d between the boundaries min (B i ,B j ) is calculated as: If d min (B i ,B j ) is less than the preset distance threshold, the boundary position is adjusted to meet the separation requirements of the pathological structure, and dist(p,q) represents the spatial distance between two pixel points p and q.

8. The medical image auxiliary annotation method based on image processing according to claim 7, characterized in that: The S6 specifically includes: S61, hierarchical region change detection: based on the node N of each hierarchical region in the multi-level lesion association model i The center of mass position (c i ,r i ) and area A i , compare the same level area at different times and calculate the centroid shift Δd i and area change ratio ΔA i : S62, dynamic change trend identification: according to the centroid offset Δd i and area change ratio ΔA i Determine the changing trend of lesion levels; If the central necrosis layer node N 坏死 The area change ratio ΔA 坏死 If it increases, it is identified as a trend of expansion of the necrotic area; If the edge active layer node N 活性 The area change ratio ΔA 活性 If it decreases, it is identified as a trend of shrinking active area; If the center of mass shifts Δd i Exceeding the set threshold indicates that the hierarchical area position has changed and needs to be adjusted.

9. The medical image auxiliary annotation method based on image processing according to claim 8, characterized in that: The mass center offset Δd i and area change ratio ΔA i The calculation is expressed as: Where, Δd i Indicates area N i The offset distance between the center of mass of the current time point and the previous time point, and Respectively represent the centroid position at the current time t and the previous time t-1, and Respectively represent the area at the current time t and the previous time t-1, d i =(c i ,r i ).

10. A medical image assisted annotation system based on image processing, used to implement a medical image assisted annotation method based on image processing as claimed in any one of claims 1 to 9, characterized in that: Includes the following modules: An image preprocessing module is used to enhance the acquired medical image data to highlight the details of the lesion edge and reduce noise interference; The multi-level segmentation module uses convolutional neural networks and density clustering methods to perform multi-level segmentation on the enhanced image data and generate preliminary annotated regions including the central necrotic layer, the marginal active layer, and the surrounding normal tissue layer; The feature vectorization module is used to perform feature vectorization on the segmented labeled regions at each level to generate feature vectors containing texture, morphology and density information; The correlation modeling module builds the correlation model between each lesion level based on the feature vector and graph structure analysis method; The dynamic annotation update module is used to identify the dynamic changes of the lesion layer according to the association relationship model during the annotation process of multiple scan images or follow-up images, and adjust the boundary position of the annotation area to ensure time consistency; The annotation result storage and output module stores the final generated multi-level annotation results as a standardized label file.

Images