Medical image organ segmentation method based on semantic context relation and geometric perception prior

Abstract

The invention relates to the field of medical image processing, and provides a medical image organ segmentation method based on a semantic context relationship and geometric perception prior, which comprises the following steps of: dividing geometric information from multi-organ medical image data with a segmentation truth value as geometric prior constraints; the method comprises the following steps: preprocessing to-be-segmented multi-organ medical image data, and inputting a coding and decoding network based on three-dimensional convolution to extract multi-level features; calculating a spatial context semantic relationship for the multi-organ medical image data, and injecting the spatial context semantic relationship into multi-level features; and carrying out feature fusion on the geometric priori constraint and the multi-level features through a coding and decoding network, and carrying out organ segmentation on to-be-segmented multi-organ medical image data based on an activation function to obtain a segmentation result. According to the invention, the problem of inaccurate three-dimensional organ image division in the prior art can be effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a medical image organ segmentation method based on semantic contextual relationship and geometric perception prior. Background Art

[0002] Automatic multi-organ segmentation is crucial to promote computer-aided medical image analysis and aid disease diagnosis. Its goal is to accurately classify voxels in medical 3D volume data into multiple organ regions and correctly restore the geometric shapes of organs. In practice, automatic multi-organ segmentation can not only reduce the workload of manual annotation for doctors and radiologists, but also provide possibilities for clinical applications such as surgery, radiotherapy, and morphological assessment. Thanks to the powerful generalization ability of deep neural networks, encoder-decoder models based on convolutional neural networks and visual transformers (ViT) have been widely used in medical image segmentation tasks and achieved remarkable performance.

[0003] However, the lack of modeling of 3D spatial context information limits the model's ability to capture complex 3D structures; and existing models tend to use local perception and global dependencies to implicitly learn the underlying contextual features, but these models cannot correctly restore the shape and structure of organs, and result in 3D volume segmented organs with disconnected regions and incorrect topological structures. In practical applications, pathological diagnosis usually requires consideration of the semantic structure of organs and their surrounding tissues. On the other hand, anatomical prior knowledge of abdominal organs is also crucial for radiologists to annotate voxels with correct labels. Summary of the invention

[0004] In order to overcome the defect of the above-mentioned prior art that organs cannot be correctly divided, the present invention provides a medical image organ segmentation method based on semantic context relationship and geometric perception prior.

[0005] In order to achieve the above technical effects, the technical solution of the present invention is as follows:

[0006] A medical image organ segmentation method based on semantic contextual relationship and geometric perception prior includes the following steps:

[0007] Divide the multi-organ medical imaging data with segmentation truth values ​​into geometric information as geometric prior constraints;

[0008] Preprocess the multi-organ medical image data to be segmented and input them into a 3D convolution-based encoding and decoding network to extract multi-level features;

[0009] Calculating spatial context semantic relationships for multi-organ medical image data, and injecting the spatial context semantic relationships into the multi-level features;

[0010] The geometric prior constraints are fused with the multi-level features through the encoding and decoding network, and organ segmentation is performed on the multi-organ medical image data to be segmented based on the activation function to obtain a segmentation result.

[0011] The present invention also proposes a medical image organ segmentation system based on semantic context relationship and geometric perception prior, and applies a medical image organ segmentation method based on semantic context relationship and geometric perception prior proposed by the present invention. The system comprises:

[0012] Geometric information constraint module: used to divide the geometric information of multi-organ medical imaging data with segmentation truth values ​​as geometric prior constraints;

[0013] Multi-level feature extraction module: used to extract multi-level features from input medical organ imaging data through a three-dimensional convolutional encoding and decoding network;

[0014] A semantic relationship prior propagation module: used for calculating the spatial context semantic relationship of the input medical organ image data, and injecting the spatial context semantic relationship into the multi-level features step by step;

[0015] Organ segmentation module: The geometric prior constraints and multi-level features are fused through the encoding and decoding network, and the multi-organ medical imaging data to be segmented is segmented based on the activation function to obtain the segmentation results.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] The present invention extracts geometric information from images with existing segmented data as a geometric prior constraint, thereby ensuring the consistency of the geometric information and improving the model's ability to understand the geometric features of the image, thereby making the final image processing result more accurate; under the framework of the deep three-dimensional convolutional coding and decoding model, the accuracy of the voxel spatial contextual semantic relationship is considered, and the model's understanding of the spatial structure is enhanced by analyzing the spatial relationship between voxels; the spatial contextual semantic relationship is then injected into the multi-level features to enhance the model's ability to express features, thereby providing more accurate semantic segmentation results when performing image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flowchart of the medical image organ segmentation method based on semantic context relationship and geometric perception prior of Example 1.

[0019] Figure 2 This is an architectural diagram of the medical image organ segmentation system based on semantic contextual relationships and geometric perception priors of Example 2. DETAILED DESCRIPTION

[0020] The accompanying drawings are only used for illustrative purposes and are not to be construed as limiting the present invention;

[0021] It is understandable to those skilled in the art that some well-known descriptions may be omitted in the drawings.

[0022] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0023] Example 1

[0024] This embodiment proposes a medical image organ segmentation method based on semantic context relationship and geometric perception prior. Figure 1 , which is a flow chart of the medical image organ segmentation method based on semantic contextual relationship and geometric perception priors of this embodiment.

[0025] The medical image organ segmentation method based on semantic context relationship and geometric perception prior proposed in this embodiment includes the following steps:

[0026] Divide the multi-organ medical imaging data with segmentation truth values ​​into geometric information as geometric prior constraints;

[0027] Preprocess the multi-organ medical image data to be segmented and input them into a 3D convolution-based encoding and decoding network to extract multi-level features;

[0028] Calculating spatial context semantic relationships for multi-organ medical image data, and injecting the spatial context semantic relationships into the multi-level features;

[0029] The geometric prior constraints are fused with the multi-level features through the encoding and decoding network, and organ segmentation is performed on the multi-organ medical image data to be segmented based on the activation function to obtain a segmentation result.

[0030] In this embodiment, geometric information is first extracted from the multi-organ medical imaging data with existing segmented data as a geometric prior constraint for subsequent model prediction, thereby ensuring the consistency of the geometric information and improving the model's ability to understand the geometric features of the image; then, under the framework of the deep three-dimensional convolutional coding and decoding model, by extracting spatial context semantic relationships and analyzing the spatial relationship between voxels, the model's understanding of the spatial structure is enhanced, and the spatial hierarchy and structure implicit in the image can be identified and understood, showing higher accuracy and efficiency; the spatial context semantic relationship is then injected into the multi-level features, and the multi-level feature fusion strategy enables the model to not only capture multi-scale information from local to global, but also provide more refined and accurate segmentation according to semantic information at different levels when performing tasks such as image segmentation, thereby providing more accurate semantic segmentation results when performing image segmentation; finally, the data to be segmented is segmented by fusing the spatial context semantic relationship and the geometric prior constraint to obtain the final result, and the method of comprehensively considering geometric and semantic information enables the model to effectively reduce errors and improve segmentation quality when processing highly complex and variable image data.

[0031] In an optional embodiment, the geometric information includes a geometric contour and a geometric normal.

[0032] Specifically, for a given three-dimensional medical image dataset with voxel-by-voxel annotation and the corresponding voxel-level multi-organ segmentation truth map calibrated by voxel, data augmentation is performed through random voxel cutting (Random Crop), and the voxel data is normalized by truncation values ​​before geometric contour and geometric normal division.

[0033] In this embodiment, multiple organs can be accurately divided through geometric contours; the geometric normal can reflect the directionality and structural information of the organ surface, generate more realistic surface texture and shadow effects in the model, and improve the visualization quality of the image.

[0034] In an optional embodiment, the step of dividing the geometric contour includes: labeling the segmentation truth value of the multi-organ medical imaging data, determining the boundary voxels of each organ one by one, and obtaining the geometric contour.

[0035] Specifically, the boundary voxels of each organ are determined organ by organ through the segmentation truth annotations corresponding to the voxels in the multi-organ medical imaging data. The 3D segmentation annotations are sliced ​​into 2D slices from the axial, sagittal and coronal planes; in each view, the boundaries of the 2D semantic mask are extracted slice by slice, and the corresponding slice index number is recorded; then all the boundary points on the axial, sagittal and coronal slices are collected according to the recorded 3D coordinates, and the 2D slices are fused into 3D volume data to generate the geometric contours of the 3D organs.

[0036] In this implementation, by combining boundary information from different perspectives, a more accurate and comprehensive three-dimensional reconstruction can be achieved. Multi-perspective fusion helps to overcome the spatial information that may be ignored by a single perspective, thereby generating a more accurate three-dimensional model of the organ or lesion.

[0037] In an optional embodiment, the step of dividing the geometric normal includes:

[0038] Based on the geometric contour, for any single organ, a search ball in a specific radius space is selected, and the nearest point of the voxel point on the current organ contour to the center of the search ball is retrieved to obtain the nearest neighbor voxel;

[0039] According to the nearest neighbor voxels, a surface normal vector is estimated using a covariance analysis method to obtain a geometric normal.

[0040] Furthermore, in order to ensure that the normal direction is facing outside the voxel, it is necessary to further analyze the semantic information of the voxel block pointed by the normal to decide whether to reverse the surface normal.

[0041] In this embodiment, by using a search ball to retrieve the nearest neighbor voxel point based on the geometric contour and using the covariance analysis method to estimate the surface normal vector, the normal direction of the organ or lesion surface can be determined more accurately, reducing the error caused by local surface unevenness or noise, making the obtained normal vector more accurate and reliable. In addition, by analyzing the semantic information of the voxel block pointed by the normal vector, it is ensured that the normal direction is correctly facing outside the voxel, solving the problem of inconsistent normal estimation direction in traditional methods.

[0042] In an optional embodiment, the step of calculating spatial context semantic relationships includes: setting a domain range for voxel blocks in multi-organ medical imaging data, recording the semantic labels of neighboring voxels within the neighborhood range of each voxel block in turn, and constructing voxel-level spatial context semantic relationships.

[0043] As an exemplary illustration, voxel points are selected as the center in sequence, and the semantic labels of 26 three-dimensional neighboring voxels of 3×3×3 around the central voxel are recorded in sequence as multi-scale semantic relationship prior truth information.

[0044] In this embodiment, local semantic understanding is enhanced:

[0045] By analyzing the semantic labels of the neighboring voxels of the central voxel, the model can obtain detailed information about the surrounding environment of each voxel, which helps to better understand the semantic and structural characteristics of local areas, helps to understand complex or closely connected multi-organ areas, and improves its ability to discriminate new samples.

[0046] In an optional embodiment, the step of injecting spatial contextual semantic relations into the multi-level features includes: injecting contextual semantic relations into the multi-level features step by step through a multi-level bypass output structure and a feature pyramid within the encoding and decoding network.

[0047] Further optionally, the step of injecting contextual semantic relations into multi-level features step by step includes: using a multi-level bypass output structure to predict semantic relations at different levels, and by introducing recurrent connections and three-dimensional convolutions, propagating the predicted semantic relation information of high-level feature layers along the feature pyramid structure to low-level feature expressions, which is expressed as follows:

[0048]

[0049] in, Represents the prediction result of the i-th level semantic context relationship; Represents the i-th level semantic contextual relationship prediction network; Conv i represents the three-dimensional convolutional layer of the i-th level feature; CAT represents the feature cascade operation; l represents the number of feature layers, Represents the i-1th layer features.

[0050] In this embodiment, by injecting contextual semantic relationships into multi-level features, the hierarchy and detail richness of feature expression are enhanced, so that the model can more accurately capture structural details of different scales and depths when processing complex medical images. The use of recurrent connections and three-dimensional convolution technology can effectively transfer high-level semantic information to low-level feature expressions, which helps to improve the accuracy of the model when performing organ segmentation. In addition, the feature pyramid structure optimizes the flow of information from high-level to low-level, ensuring the effective integration of information between different levels. This structure enables high-level global information to guide low-level local processing and improve the overall network's ability to process complex images.

[0051] In an optional embodiment, the step of feature fusion includes: firstly fusing the geometric prior constraint into the first-level feature in the multi-level feature, and fusing the multi-level feature injected with the spatial context semantic relationship and the first-level feature fused with the geometric prior constraint by using a three-dimensional convolution layer, and the expression is as follows:

[0052]

[0053] in, It represents the features that integrate the spatial contextual semantic relationship and geometric prior constraints. Conv(·) represents the three-dimensional convolutional layer. CAT represents the feature cascade operation. Represents the prediction result of the i-th level semantic context relationship; Represents the first-level features with geometric prior constraints.

[0054] In this embodiment, by combining geometric priors with spatial context information, the model can more comprehensively understand and analyze objects and structures in three-dimensional space. Using three-dimensional convolutional layers to fuse features and geometric information at different levels helps the model maintain high recognition and segmentation accuracy when facing image noise, blur or other quality issues, and enhances the robustness of features, improving the reliability of the model in practical applications.

[0055] In an optional embodiment, the step of performing organ segmentation on the multi-organ medical image data to be segmented based on the activation function comprises:

[0056] Use convolutional layers and activation functions to construct semantic segmentation prediction heads;

[0057] The probability or score of multiple organs at each voxel point is calculated by the semantic segmentation prediction head and the multi-level features, and the organ classification is achieved by setting the threshold; its expression is as follows:

[0058]

[0059] in, The probability or score of multiple organs at each voxel point, P (seg) is the semantic segmentation prediction head.

[0060] In this embodiment, the specially designed semantic segmentation prediction head can effectively calculate the probability or score of each voxel corresponding to multiple organs, making the classification process more accurate and reducing classification errors. In particular, when the organs are morphologically similar or adjacent, the accuracy of segmentation is significantly improved. And by operating directly on the feature map through the convolution layer, less computing resources are required compared to the fully connected layer, which makes the entire segmentation process more efficient and can achieve fast image processing and analysis.

[0061] Example 2

[0062] This embodiment proposes a medical image organ segmentation system based on semantic context relationship and geometric perception prior, and applies the medical image organ segmentation method based on semantic context relationship and geometric perception prior proposed in Embodiment 1. Figure 2 , which is an architecture diagram of the medical image organ segmentation system based on semantic contextual relationship and geometric perception priors of this embodiment.

[0063] The present embodiment proposes a medical image organ segmentation system based on semantic contextual relationship and geometric perception prior, including:

[0064] Geometric information constraint module: used to divide the geometric information of multi-organ medical imaging data with segmentation truth values ​​as geometric prior constraints;

[0065] Multi-level feature extraction module: used to extract multi-level features from input medical organ imaging data through a three-dimensional convolutional encoding and decoding network;

[0066] A semantic relationship prior propagation module: used for calculating the spatial context semantic relationship of the input medical organ image data, and injecting the spatial context semantic relationship into the multi-level features step by step;

[0067] Organ segmentation module: The geometric prior constraints and multi-level features are fused through the encoding and decoding network, and the multi-organ medical imaging data to be segmented is segmented based on the activation function to obtain the segmentation results.

[0068] It can be understood that the system of this embodiment corresponds to the method of the above-mentioned embodiment 1, and the options in the above-mentioned embodiment 1 are also applicable to this embodiment, so they will not be described repeatedly here.

[0069] The terms in the drawings are for illustrative purposes only and should not be construed as limiting this patent;

[0070] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A medical image organ segmentation method based on semantic contextual relationship and geometric perception prior, characterized in that: The following steps are involved: Divide the multi-organ medical imaging data with segmentation truth values ​​into geometric information as geometric prior constraints; Preprocess the multi-organ medical image data to be segmented and input them into a 3D convolution-based encoding and decoding network to extract multi-level features; Calculating spatial context semantic relationships for multi-organ medical image data, and injecting the spatial context semantic relationships into the multi-level features; The geometric prior constraints are fused with the multi-level features through the encoding and decoding network, and organ segmentation is performed on the multi-organ medical image data to be segmented based on the activation function to obtain a segmentation result.

2. The medical image organ segmentation method based on semantic context and geometric perception prior according to claim 1, characterized in that: The geometric information includes a geometric outline and a geometric normal.

3. The medical image organ segmentation method based on semantic context and geometric perception prior according to claim 2, characterized in that: The step of dividing the geometric contour includes: marking the segmentation true value of the multi-organ medical image data, determining the boundary voxels of each organ one by one, and obtaining the geometric contour.

4. The medical image organ segmentation method based on semantic context and geometric perception prior according to claim 2, characterized in that: The step of dividing the geometric normal comprises: Based on the geometric contour, for any single organ, a search ball in a specific radius space is selected, and the nearest point of the voxel point on the current organ contour to the center of the search ball is retrieved to obtain the nearest neighbor voxel; According to the nearest neighbor voxels, a surface normal vector is estimated using a covariance analysis method to obtain a geometric normal.

5. The medical image organ segmentation method based on semantic context and geometric perception prior according to claim 1, characterized in that: The step of calculating the spatial context semantic relationship includes: setting a domain range for the voxel blocks in the multi-organ medical imaging data, sequentially recording the semantic labels of the neighboring voxels within the neighborhood range of each voxel block, and constructing the voxel-level spatial context semantic relationship.

6. The medical image organ segmentation method based on semantic context and geometric perception prior according to claim 1, characterized in that: The step of injecting spatial contextual semantic relations into the multi-level features includes: injecting contextual semantic relations into the multi-level features step by step through the multi-level bypass output structure and feature pyramid in the encoding and decoding network.

7. The medical image organ segmentation method based on semantic context and geometric perception prior according to claim 6, characterized in that: The step of injecting contextual semantic relations into multi-level features step by step includes: using a multi-level bypass output structure to predict semantic relations at different levels, and by introducing recurrent connections and three-dimensional convolutions, propagating the predicted semantic relation information of high-level feature layers along the feature pyramid structure to low-level feature expressions, and the expression is: in, represents the prediction result of the i-th level semantic context relationship; P i (r) (·) represents the i-th level semantic contextual relationship prediction network; Conv i represents the three-dimensional convolutional layer of the i-th level feature; CAT represents the feature cascade operation; l represents the number of feature layers, Represents the i-1th layer features.

8. The medical image organ segmentation method based on semantic context and geometric perception prior according to claim 1, characterized in that: The step of feature fusion includes: firstly fusing the geometric prior constraint into the first-level feature in the multi-level feature, and fusing the multi-level feature injected with the spatial context semantic relationship and the first-level feature fused with the geometric prior constraint by using a three-dimensional convolutional layer, and the expression is as follows: in, It represents the features that integrate the spatial contextual semantic relationship and geometric prior constraints. Conv(·) represents the three-dimensional convolutional layer. CAT represents the feature cascade operation. Represents the prediction result of the i-th level semantic context relationship; Represents the first-level features with geometric prior constraints.

9. The medical image organ segmentation method based on semantic context and geometric perception prior according to claim 8, characterized in that: The step of performing organ segmentation on the multi-organ medical image data to be segmented based on the activation function comprises: Use convolutional layers and activation functions to construct semantic segmentation prediction heads; The probability or score of multiple organs at each voxel point is calculated by the semantic segmentation prediction head and the multi-level features, and the organ classification is achieved by setting the threshold; its expression is as follows: in, The probability or score of multiple organs at each voxel point, P (seg) is the semantic segmentation prediction head.

10. A medical image organ segmentation system based on semantic context and geometric perception prior, applied to the medical image organ segmentation method based on semantic context and geometric perception prior as claimed in any one of claims 1 to 9, characterized in that: The system comprises: Geometric information constraint module: used to divide the geometric information of multi-organ medical imaging data with segmentation truth values ​​as geometric prior constraints; Multi-level feature extraction module: used to extract multi-level features from input medical organ imaging data through a three-dimensional convolutional encoding and decoding network; A semantic relationship prior propagation module: used for calculating the spatial context semantic relationship of the input medical organ image data, and injecting the spatial context semantic relationship into the multi-level features step by step; Organ segmentation module: The geometric prior constraints and multi-level features are fused through the encoding and decoding network, and the multi-organ medical imaging data to be segmented is segmented based on the activation function to obtain the segmentation results.

Images