Image segmentation method, device and computer program product

Abstract

The invention discloses an image segmentation method and device and a computer program product, and relates to the technical field of image processing, and the method comprises the steps: obtaining an original medical image; and inputting the original medical image into a pre-constructed image segmentation model for image segmentation processing to obtain a segmented image, the image segmentation model being obtained by training a semantic segmentation neural network framework based on a marked image data set after data enhancement processing, the semantic segmentation neural network framework is constructed through a residual block, a spatial pyramid pool and an attention mechanism. The semantic segmentation neural network framework significantly improves the segmentation precision, and the marked image data set after data enhancement realizes training of the semantic segmentation neural network framework with a small amount of data. Therefore, the image segmentation model achieves high-precision segmentation of the original medical image with a small amount of marked data.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image segmentation method, device and computer program product. Background Art

[0002] Preoperative planning in total knee replacement requires segmentation of the original CBCT medical images to identify the lesion site or target area. Currently, the segmentation of CBCT medical images mainly relies on traditional image feature recognition, but the segmentation accuracy of medical images using traditional image feature recognition theories such as threshold, edge, region, wavelet analysis and wavelet change is not high. On the basis of artificial intelligence models and image annotation, the segmentation of CBCT medical images requires a large amount of manually annotated data, which results in the high cost of CBCT medical image segmentation and makes it difficult to be widely used.

[0003] Therefore, how to use a small amount of labeled data to achieve high-precision medical image segmentation processing has become an urgent problem to be solved in this application.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide an image segmentation method, device and computer program product, aiming to solve the technical problem of how to achieve high-precision medical image segmentation processing using a small amount of labeled data.

[0006] To achieve the above object, the present application proposes an image segmentation method, which comprises:

[0007] Obtaining original medical images;

[0008] The original medical image is input into a pre-built image segmentation model for image segmentation processing to obtain a segmented image. The image segmentation model is trained by a semantic segmentation neural network framework based on a labeled image data set after data enhancement processing. The semantic segmentation neural network framework is constructed by residual blocks, spatial pyramid pools, and attention mechanisms.

[0009] In one embodiment, the step of inputting the original medical image into a pre-built image segmentation model for image segmentation processing to obtain a segmented image further includes:

[0010] Before the step of inputting the original medical image into a pre-built image segmentation model for image segmentation processing to obtain a segmented image, the step further includes:

[0011] Get the original sample image dataset;

[0012] Preprocessing the original sample image data set to obtain a preprocessed image data set;

[0013] Annotating the contour lines of the original sample image dataset by a preset open source image segmentation tool to obtain a labeled image dataset;

[0014] Establishing a residual block, a spatial pyramid pool, and an attention mechanism according to the preprocessed image dataset;

[0015] Building a semantic segmentation neural network framework based on the residual block, the spatial pyramid pool and the attention mechanism;

[0016] Performing data enhancement processing on the labeled image dataset;

[0017] The labeled image dataset after data enhancement processing is fed to the semantic segmentation neural network architecture for training to obtain an image segmentation model.

[0018] In one embodiment, the step of preprocessing the original sample image dataset to obtain a preprocessed image dataset includes:

[0019] Distinguishing the original sample image data set based on the file sequence of the original sample image data set to obtain a differentiation result;

[0020] executing a region growing algorithm on the original sample image data set in combination with the distinction result to obtain first sample image data;

[0021] Establishing a segmentation search algorithm according to the coordinate axis of the first sample image data, and executing the segmentation search algorithm in the first sample image data to obtain second sample image data;

[0022] Establishing a special spatial binary tree according to the point cloud data of the second sample image data;

[0023] Performing unordered point cloud denoising on the second sample image data by using the spatial binary tree to obtain third sample image data;

[0024] The third sample image data is smoothed based on a preset triangular mesh hole filling algorithm to obtain a preprocessed image data set.

[0025] In one embodiment, the step of performing a region growing algorithm on the original sample image data set in combination with the distinction result to obtain the first sample image data comprises:

[0026] Reading the original sample image data set into a specified array in combination with the distinction result;

[0027] Determine the current unit cell by reading the original sample image data set into a specified array;

[0028] Rendering an isosurface image based on the current unit volume;

[0029] A region growing algorithm with input thresholds and splicing vectors is performed on the original sample image data set through the isosurface image to obtain first sample image data.

[0030] In one embodiment, the step of establishing a segmentation search algorithm according to the coordinate axis of the first sample image data and executing the segmentation search algorithm in the first sample image data to obtain the second sample image data includes:

[0031] determining a space dividing line according to a coordinate axis of the first sample image data;

[0032] Establishing a segmentation search algorithm based on the spatial segmentation line;

[0033] A segmentation search algorithm is executed on the first sample image data to obtain second sample image data.

[0034] In one embodiment, the step of performing unordered point cloud denoising on the second sample image data by using the spatial binary tree to obtain the third sample image data includes:

[0035] Establishing the point cloud three-dimensional coordinates of the second sample image data and the topological relationship of the point cloud three-dimensional coordinates associated with each other in the three-dimensional space through the spatial binary tree;

[0036] The second sample image data is subjected to disordered point cloud denoising according to the point cloud three-dimensional coordinates and the topological relationship to obtain third sample image data.

[0037] In one embodiment, the step of performing smoothing processing on the third sample image data based on a preset triangular mesh hole filling algorithm to obtain a preprocessed image data set includes:

[0038] Get the triangular mesh hole filling algorithm based on Poisson equation;

[0039] Detect hole boundaries of the third sample image data according to the triangular mesh hole filling algorithm and generate an initialization hole filling mesh;

[0040] Adjusting the initial hole-filling grid according to the hole boundary to obtain a final hole-filling grid;

[0041] The third sample image data is smoothed based on the final hole filling grid to obtain a preprocessed image data set.

[0042] In one embodiment, the step of building a semantic segmentation neural network framework based on the preprocessed image data set includes:

[0043] Establishing a residual block according to the preprocessed image data set;

[0044] Establishing spatial pyramid pooling and attention mechanism through the residual block;

[0045] A semantic segmentation neural network framework is built based on the residual block, the spatial pyramid pool, and the attention mechanism.

[0046] In one embodiment, the step of performing data enhancement processing on the labeled image dataset includes:

[0047] Setting a cropping margin based on the labeled image dataset;

[0048] Adjusting pixels of the labeled image dataset;

[0049] Perform data augmentation on the pixel-adjusted labeled image dataset.

[0050] In addition, to achieve the above objectives, the present application also proposes an image segmentation device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the image segmentation method described above.

[0051] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the image segmentation method described above are implemented.

[0052] One or more technical solutions proposed in this application have at least the following technical effects:

[0053] The original medical image is input into the pre-built image segmentation model, and the image segmentation model quickly and accurately segments the original CBCT medical image to obtain a segmented image. Among them, a semantic segmentation neural network framework is built according to the residual block, spatial pyramid pool, and attention mechanism, and a high-quality labeled image dataset processed by data enhancement is input into the semantic segmentation neural network framework, and the labeled image dataset is used to train the semantic segmentation neural network framework, thereby obtaining an image segmentation model. Since the semantic segmentation neural network framework built according to the residual block, spatial pyramid pool, and attention mechanism significantly improves the segmentation effect of the femur and tibia in the CBCT medical image; the high-quality labeled image dataset processed by data enhancement realizes the training of the semantic segmentation neural network framework with a small amount of data. The image segmentation model is trained by the semantic segmentation neural network framework based on the labeled image dataset processed by data enhancement. Therefore, the image segmentation model can realize high-precision medical image segmentation with a small amount of labeled data by processing the original medical image through the image segmentation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0056] Figure 1 A schematic diagram of a flow chart of the first embodiment of the image segmentation method of the present application;

[0057] Figure 2 A schematic diagram of a flow chart provided for the second embodiment of the image segmentation method of the present application;

[0058] Figure 3 Schematic diagram of the verification of the results of ITK-Snap annotation of CBCT contour lines;

[0059] Figure 4 This is a schematic diagram of the ResUNet++ model framework;

[0060] Figure 5 A flowchart for building a semantic segmentation neural network framework;

[0061] Figure 6 This is a schematic diagram of the module structure of the image segmentation device according to an embodiment of the present application;

[0062] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the image segmentation method in the embodiment of the present application.

[0063] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0064] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0065] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0066] The main solution of the embodiment of the present application is: input the original medical image into a pre-built image segmentation model, and the image segmentation model quickly and accurately segments the original CBCT medical image to obtain a segmented image. To build the image segmentation model, it is first necessary to build a semantic segmentation neural network framework, and obtain a high-quality labeled image dataset and a pre-processed image dataset by pre-processing and annotating the sample image dataset. The semantic segmentation neural network framework is built according to the pre-processed image dataset, and the high-quality labeled image dataset after data enhancement processing is input to the semantic segmentation neural network framework, and the semantic segmentation neural network framework is trained using the labeled image dataset to obtain an image segmentation model. Since the semantic segmentation neural network framework significantly improves the segmentation effect of the femur and tibia in CBCT medical images, the high-quality labeled image dataset after data enhancement processing realizes the training of the semantic segmentation neural network framework with a small amount of data. The image segmentation model is a semantic segmentation neural network trained with a high-quality labeled dataset, so the image segmentation model can achieve high-precision segmentation of CBCT medical images with a small amount of labeled data.

[0067] The embodiment of the present application takes into account that: currently, the segmentation of CBCT medical images mainly relies on traditional image feature recognition, but the segmentation accuracy of medical images is not high through traditional image feature recognition theories such as threshold, edge, region, wavelet analysis and wavelet change. On the basis of artificial intelligence models and image annotation, the segmentation of CBCT medical images requires a large amount of manually annotated data, resulting in the high cost of CBCT medical image segmentation processing, which is difficult to be widely used.

[0068] Therefore, the present application provides a solution, after a series of preprocessing such as region growing, segmentation search algorithm, denoising and smoothing are performed on the acquired original sample image data, a preprocessed data set is obtained, and a semantic segmentation neural network model framework including residual blocks, spatial pyramid pools and attention mechanisms is built using the preprocessed data set. The contours of the original sample image data are marked using the preset open source image segmentation tool: ITK-Snap to obtain a marked image data set, and the marked image data set is subjected to data enhancement processing, and the semantic segmentation neural network model framework is trained using the processed marked image data set. When the marked image data set is fed to the semantic segmentation neural network model framework for training, the number of training samples is increased by setting the cropping margin, adjusting the image pixels, and using a series of operations such as data enhancement, so that the trained semantic segmentation neural network model can achieve high-precision image segmentation with less labeled data. The image segmentation model is a trained semantic segmentation neural model, so the image segmentation model can achieve higher-precision CBCT image segmentation with less labeled data.

[0069] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, an image segmentation device, etc. The following takes the image segmentation device as an example to illustrate this embodiment and the following embodiments.

[0070] Based on this, the present application embodiment provides an image segmentation method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the image segmentation method of the present application.

[0071] In this embodiment, the image segmentation method includes steps S10 to S20:

[0072] Step S10, obtaining an original medical image;

[0073] The original medical image is a CBCT medical image. CBCT (Cone Beam Computed Tomography) is a commonly used imaging diagnostic technology in the medical field, providing three-dimensional, high-resolution anatomical structure images. Since the lesion site and target area in the CBCT medical image need to be identified in preoperative planning, the CBCT original medical image needs to be segmented.

[0074] Step S20, inputting the original medical image into a pre-built image segmentation model for image segmentation processing to obtain a segmented image, wherein the image segmentation model is trained by a semantic segmentation neural network framework based on a labeled image data set after data enhancement processing, and the semantic segmentation neural network framework is constructed by residual blocks, spatial pyramid pools, and attention mechanisms.

[0075] The original CBCT medical image is input into a pre-built image segmentation model for image segmentation processing. The image segmentation model automatically identifies and segments the lesion site or target area to obtain a segmented image that meets the requirements.

[0076] It should be noted that the image segmentation model is an artificial intelligence model obtained by training a semantic segmentation neural network framework with a labeled image dataset after data enhancement processing, wherein the semantic segmentation neural network framework is built by preprocessing the image dataset; the labeled image dataset and the preprocessed image dataset are high-quality segmented images obtained by a series of image segmentation processing operations on the original sample image dataset.

[0077] A semantic segmentation neural network framework is built by preprocessing the image data set. The semantic segmentation neural network framework is specifically the ResUNet++ model framework. As a semantic segmentation neural network framework built using residual blocks, squeezing and excitation blocks, spatial pyramid pooling and attention blocks, the ResUNet++ model framework can significantly improve the tibia and femur segmentation effect in CBCT original medical images, and the ResUNet++ model framework can efficiently process images with a small amount of labeled data.

[0078] Therefore, in order to further improve the segmentation accuracy of the image segmentation model for the original medical images, on the basis of ensuring the segmentation accuracy and robustness, the semantic segmentation neural network framework is trained by labeling the image dataset, so that the trained semantic segmentation neural network, that is, the image segmentation model, can achieve high-precision image segmentation with less labeled data.

[0079] This embodiment provides an image segmentation method, wherein the original medical image is input into a pre-built image segmentation model, and the image segmentation model quickly and accurately segments the original CBCT medical image to obtain a segmented image. A semantic segmentation neural network framework is constructed based on residual blocks, spatial pyramid pools, and an attention mechanism, and a high-quality labeled image data set that has been processed with data enhancement is input into the semantic segmentation neural network framework, and the labeled image data set is used to train the semantic segmentation neural network framework, thereby obtaining an image segmentation model. Since the semantic segmentation neural network framework constructed based on the residual block, spatial pyramid pool, and attention mechanism significantly improves the segmentation effect of the femur and tibia in the CBCT medical image; the high-quality labeled image data set that has been processed with data enhancement realizes the training of the semantic segmentation neural network framework with a small amount of data. The image segmentation model is obtained by training the semantic segmentation neural network framework based on the labeled image data set after data enhancement. Therefore, processing the original medical image through the image segmentation model can realize high-precision medical image segmentation with a small amount of labeled data.

[0080] Based on the first embodiment of the present application, the second embodiment of the present application is proposed. In the second embodiment of the present application, the same or similar contents as those of the above-mentioned first embodiment can be referred to the above introduction, and will not be repeated in the following.

[0081] On this basis, please refer to Figure 2 The original medical image is input into a pre-built image segmentation model for image segmentation processing, and before step S20 of obtaining the segmented image, steps S101 to S105 are also included:

[0082] Step S101, obtaining an original sample image data set;

[0083] Since it is necessary to build an image segmentation model that can automatically and efficiently segment CBCT medical images, the original sample image dataset is a sample set of CBCT medical image data.

[0084] Step S102, preprocessing the original sample image data set to obtain a preprocessed image data set;

[0085] The original sample image dataset is subjected to a series of preprocessing operations including denoising and smoothing to obtain a preprocessed image dataset. A semantic segmentation neural network framework is built based on the preprocessed data to make the semantic segmentation neural network framework converge faster and achieve better performance.

[0086] Step S103, annotating the contour lines of the original sample image dataset by a preset open source image segmentation tool to obtain a labeled image dataset;

[0087] The preset open source image segmentation tool is ITK-Snap. As an easy-to-use and powerful 3D image segmentation tool, ITK-Snap can accurately segment and measure medical images. Use ITK-Snap to annotate the contours of medical images in the original sample image dataset. Select Save Segmentation Image in Segmentation in the ITK-Snap toolbar, determine the path (Browse to select the save path) and name (cannot contain Chinese characters) of the annotated data, and save the annotated image data in .nii.gz format.

[0088] like Figure 3 As shown, Figure 3 The figure is a schematic diagram of the verification of the results of ITK-Snap labeling CBCT contour lines. In order to ensure the image quality of the labeled image data set obtained after ITK-Snap processing, the image data processed by ITK-Snap is verified. By opening the saved .nii.gz file, setting the image aspect ratio, the content of the current view is automatically adjusted to the size that best suits the current display area, and checking whether the display edge of the corresponding position of the window is smooth and complete, such as Figure 3 The aspect ratio shown is 742 of 867. The verified image data constitutes a labeled image dataset.

[0089] Step S104, establishing a residual block, a spatial pyramid pool and an attention mechanism according to the preprocessed image data set;

[0090] First, a residual block is established based on the preprocessed image dataset. Since the residual block propagates information on each layer, a deeper neural network can be built to solve the degradation problem of each encoder. Building a deep neural network improves the interdependence of channels while reducing computational costs. The ResUnet++ semantic segmentation neural network framework proposed based on the characteristics of the preprocessed image data contains a residual unit block, followed by an encoder block and a decoder block.

[0091] Since the residual unit is a combination of batch normalization, rectified linear unit (ReLU) activation, and convolutional layers. Therefore, each encoder block consists of two consecutive 3×3 convolutional blocks and a unit map; each convolutional block includes a batch normalization layer, a ReLU activation layer, and a convolutional layer; the identity map connects the input and output of the encoder block. In the first convolutional layer of the encoder block, a strided convolutional layer is used to halve the spatial dimension of the feature map. The output of the encoder block passes through the compression and excitation blocks.

[0092] Further, spatial pyramid pooling and attention mechanisms are established through residual blocks. Spatial pyramid pooling (ASPP) and attention mechanisms are further constructed by building on the features embodied in residual blocks. Among them, ASPP acts as a bridge to expand the field of view of the filter to include a wider range of contexts. Correspondingly, the decoding path also consists of residual units. Before each unit, the attention block increases the effectiveness of the feature map. Next is the nearest neighbor upsampling of the feature map from the lower level and concatenation with the feature map from the corresponding encoding path.

[0093] The output of the decoding block is passed through ASPP, and a 1×1 sigmoid activation convolution is applied to obtain the segmentation map. The extension of ResUNet++ is the squeeze and excitation block, the ASPP block, and the attention block. The squeeze and excitation network improves the representativeness of the network by recalibrating the feature responses and taking advantage of the accurate modeling interdependencies between channels. The purpose of the squeeze and excitation block is to ensure that the network's sensitivity to relevant features is increased and unwanted features are suppressed. This goal will be achieved in two steps: the first step is compression (global information embedding), where each channel is compressed by using global average pooling to generate channel statistics. The second step is excitation (active calibration), which aims to fully capture channel correlations. In the proposed structure, the squeeze and excitation block is stacked with the residual block to improve the effective generalization of different datasets and improve the performance of the architecture.

[0094] Step S105, building a semantic segmentation neural network framework based on the residual block, the spatial pyramid pool and the attention mechanism;

[0095] The basic semantic segmentation neural network framework is built based on the constructed residual block, spatial pyramid pool, and attention mechanism. Figure 4 As shown, Figure 4 The following is a schematic diagram of the ResUNet++ model framework, where the convolution operation (Conv2D) represents the basic operation used in neural networks to extract image features. Batch Normalization (Batch Norm.) indicates that this module is used to accelerate the training process by normalizing the input of each small batch to reduce internal covariate shift. The ReLU activation function is a commonly used nonlinear activation function used to introduce nonlinearity into neural networks. Figure 4 The Squeeze&Excite module is a module used to enhance the network representation capability by compressing (squeeze) the channel dimension of the feature map and then learning the dependencies between channels (excitation) through two fully connected layers. Upsampling is used to increase the size of the feature map and is usually used in the decoder part. Figure 4 In the example, it is represented by a rectangular box with the word "UpSampling". The attention mechanism is used to make the network pay attention to important features. Figure 4 As shown in the figure, the attention mechanism is represented by the word "Attention".

[0096] It should be noted that the idea of ​​ASPP comes from spatial pyramid pooling, which successfully resamples features at multiple scales. In ASPP, contextual information is captured at different scales, and many parallel Atrous convolutions with different rates are fused in the input feature map. Atrous convolution allows the field of view to be controlled to accurately capture multi-scale information. In the architecture of building ResUNet++, ASPP acts as a bridge between the encoder and the decoder, and ASPP shows good results in various segmentation tasks by providing multi-scale information. Therefore, ASPP is used to capture useful multi-scale information in semantic segmentation tasks.

[0097] Attention mechanisms are most popular in Natural Language Processing (NLP). Attention mechanisms focus on a subset of the input and are often used for semantic segmentation tasks such as pixel-by-pixel prediction. The attention mechanism determines which parts of the neural network need more attention. The attention mechanism also reduces the computational cost of encoding the information in each polyp image into a fixed-dimensional vector. The main advantage of the attention mechanism is its simplicity and can be applied to any input size, improving the quality of features and thus improving the results.

[0098] Step S106, performing data enhancement processing on the labeled image data set;

[0099] By setting cropping margins, pixel adjustments, data enhancement and other processing on the labeled image dataset, the amount of labeled sample data is increased during the model training process, avoiding the overfitting problem and achieving the goal of higher-precision image segmentation with less labeled data.

[0100] Step S107, feeding the labeled image data set after data enhancement processing to the semantic segmentation neural network architecture for training to obtain an image segmentation model.

[0101] The semantic segmentation neural network framework is trained using the labeled image dataset after data augmentation to improve the generalization ability and performance of the model, and a segmentation image model is obtained that can achieve high-precision image segmentation with a small amount of labeled data.

[0102] The data augmented labeled image dataset is fed to the semantic segmentation neural network architecture, and the semantic segmentation neural network model is trained using the data augmented labeled image dataset, where 80% of the labeled image dataset is used for training, 10% for validation, and 10% for testing. In order to improve the performance of the final image segmentation model, the training process combines stochastic gradient descent with restarts (SGDR) to improve model performance.

[0103] In this embodiment, after a series of preprocessing such as region growing, segmentation search algorithm, denoising and smoothing are performed on the acquired original sample image data, a preprocessed data set is obtained, and a semantic segmentation neural network model framework including residual blocks, spatial pyramid pools and attention mechanisms is built using the preprocessed data set. The original sample image data contours are marked using the preset open source image segmentation tool: ITK-Snap to obtain a marked image data set, and the semantic segmentation neural network model framework is trained using the marked image data set, and finally an image segmentation model that achieves high-precision image segmentation using a small amount of labeled data is obtained.

[0104] A specific implementation of step S106 in the second embodiment of the present application is described in detail below:

[0105] Specifically, in a feasible implementation of step S106, step S106 of performing data enhancement processing on the labeled image dataset may include steps S161 to S163:

[0106] Step S161, setting a cropping margin based on the marked image data set;

[0107] In the process of training the semantic segmentation neural network according to the labeled image dataset, the labeled image dataset is divided into multiple batches of image data to train the semantic segmentation neural network framework. During the training process, the Adam optimizer is used to optimize the architecture of the semantic segmentation neural network and set a suitable algorithm learning rate. For example, the learning rate of the algorithm is set to 1e-4.

[0108] During training, in order to effectively utilize the GPU and reduce training time, set the crop margin to crop the images in the labeled image dataset to increase the training data and achieve the effect of high-precision segmentation with less data. For example, set a 320×320 crop margin to increase the data volume of the training dataset.

[0109] Step S162, adjusting the pixels of the labeled image data set;

[0110] Since the images in the same dataset have different sizes, and the preprocessed image dataset for building the semantic segmentation neural network model and the labeled image dataset for training the semantic segmentation neural network model consist of images of different resolutions, the pixels of the labeled image dataset are adjusted before feeding the images of the labeled image dataset to the semantic segmentation neural network architecture. For example, the image pixels of the labeled image dataset can be adjusted to 256×256 pixels.

[0111] Step S163, performing data enhancement processing on the pixel-adjusted labeled image data set.

[0112] After adjusting the pixels of the labeled image dataset, data augmentation techniques such as center cropping, random cropping, horizontal flipping, vertical flipping, scale enhancement, random rotation, cut-out, brightness enhancement, etc. are used to increase the number of samples of the labeled image dataset.

[0113] In this embodiment, by setting a series of processing such as cropping margin, pixel adjustment, and data enhancement on the labeled image data set, the amount of labeled sample data is increased during the model training process, the overfitting problem is avoided, and the purpose of achieving higher-precision image segmentation with less labeled data is achieved.

[0114] Based on the first embodiment and / or the second embodiment of the present application, the third embodiment of the present application is proposed. In the third embodiment of the present application, the same or similar contents as those of the above-mentioned first and second embodiments can be referred to the above introduction, and will not be repeated in the following.

[0115] In this embodiment, the step S102 of preprocessing the original sample image data set to obtain the preprocessed image data set may include steps S121 to S126:

[0116] Step S121, distinguishing the original sample image data set based on the file sequence of the original sample image data set to obtain a distinction result;

[0117] The original sample image dataset is a collection of CBCT sample medical image data. The original sample image dataset is distinguished between left and right legs according to the CBCT file sequence to achieve a better segmentation effect.

[0118] The process of distinguishing the original sample image dataset based on the CBCT file sequence includes: locating the key frames of the occlusal and sagittal planes through VTK, executing the restricted region growing algorithm; removing the noise image of the cropped dental model, and smoothing the denoised image.

[0119] Step S122, performing a region growing algorithm on the original sample image data set in combination with the distinction result to obtain first sample image data;

[0120] The region growing algorithm divides the image into different regions based on the similarity of pixels in the original sample image data set or some consistency criteria. The basic idea of ​​the region growing algorithm is to start from one or more seed points and gradually add neighboring pixels with similar properties to the corresponding regions until a certain stopping condition is met.

[0121] After the original sample image data set is distinguished, a region growing algorithm is performed on the original sample image data set according to the distinction result to obtain the first sample image data. When the region growing algorithm is performed, the threshold is controlled in the range of 770-25584 to reconstruct a 3D model to view the main anatomical structure of the CBCT.

[0122] Step S123, establishing a segmentation search algorithm according to the coordinate axis of the first sample image data, and executing the segmentation search algorithm in the first sample image data to obtain second sample image data;

[0123] A segmentation search algorithm is established according to the coordinate axis of the first sample image data. The segmentation process of executing the segmentation search algorithm in the first sample image data corresponds to a binary tree. Each segmentation line in the segmentation search algorithm corresponds to a branch in the binary tree, and each point in the space corresponds to a leaf node. The second sample image data is obtained by executing the segmentation search algorithm in the first sample image data.

[0124] Step S124, establishing a special spatial binary tree according to the point cloud data of the second sample image data;

[0125] A special spatial binary tree, namely a kd tree, is generated based on the point cloud data of the second sample image data. The kd tree is a data structure that organizes points in a k-dimensional Euclidean space, and is a special case of a spatial binary tree (Binary space partitioning). The kd tree is a binary tree in which each node is a k-dimensional point. All non-leaf nodes can be regarded as a hyperplane that divides the space into two half-spaces. The subtree on the left side of the node represents the point on the left side of the hyperplane, and the subtree on the right side of the node represents the point on the right side of the hyperplane. By establishing a kd tree, the second sample image data is searched for noise points, thereby achieving the purpose of denoising.

[0126] Step S125, performing unordered point cloud denoising on the second sample image data by using the spatial binary tree to obtain third sample image data;

[0127] The second sample image data is denoised according to the established spatial binary tree, namely, the kd tree, to obtain the third sample image data.

[0128] Step S126: Smoothing the third sample image data based on a preset triangular mesh hole filling algorithm to obtain a preprocessed image data set.

[0129] The preset triangular mesh hole filling algorithm is a triangular mesh hole filling algorithm based on the Poisson equation. The triangular mesh hole filling algorithm based on the Poisson equation adjusts the geometric shape of the triangular facets in the hole filling mesh by solving the Poisson equation so that it can be smoothly integrated with the surrounding original mesh.

[0130] The main steps of smoothing the third sample image data according to the preset triangular mesh hole-filling algorithm based on the Poisson equation include: detecting the hole boundaries of the third sample image data and initializing the hole-filling mesh; further, generating the final hole-filling mesh by constructing the Laplace equation and the Poisson equation; the final generated hole-filling mesh will fill the holes in the original mesh while maintaining the overall consistency and quality of the mesh, thereby obtaining a preprocessed image data set.

[0131] In this embodiment, a series of preprocessing operations such as region growing algorithm, segmentation search algorithm, establishment of kd tree, triangle mesh hole filling algorithm, etc. are performed on the original sample image data set to obtain a high-quality preprocessed image data set, so that a semantic segmentation neural network framework can be built based on the preprocessed image data set. The high-quality preprocessed image data set enables the semantic segmentation neural network framework to have a powerful image precision segmentation capability.

[0132] Based on the above embodiments of the present application, a fourth embodiment of the present application is proposed. In the fourth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0133] In this embodiment, the step S122 of performing a region growing algorithm on the original sample image data set in combination with the distinction result to obtain the first sample image data may include steps A11 to A14:

[0134] Step A11, reading the original sample image data set into a specified array in combination with the distinction result;

[0135] Since the original sample image dataset is differentiated, the differentiation includes locating the key frames of the occlusal plane and sagittal plane through VTK, executing the restricted region growing algorithm, removing the noise image of the cropped dental model, and smoothing it. Combined with the differentiation results, the original sample image dataset is read into the specified array.

[0136] Step A12, determining the current unit cell by reading the original sample image data set into a specified array;

[0137] By reading the original sample image data set into the specified array, a unit body is extracted from the grid data body to become the current unit body, and all the information of the unit body is obtained at the same time, and the information includes the vertex function value of the boundary and the point cloud coordinate position of the unit body.

[0138] Step A13, drawing an isosurface image based on the current unit volume;

[0139] By comparing the function value of the vertex in the current unit body with the given isosurface value C, the state table of the unit body is obtained; according to the state table index of the current unit body, the unit body edge intersecting with the isosurface is found, and the position coordinates of each intersection point are calculated using the linear interpolation method.

[0140] The central difference method is used to find the normal vector of the vertex in the current unit body, and then the linear interpolation method is used to obtain the normal vector of each vertex of the triangle; the isosurface image is drawn according to the coordinates of each triangle vertex and the vertex normal vector.

[0141] Step A14, performing a region growing algorithm with changing input thresholds and stitching vectors on the original sample image data set through the isosurface image to obtain first sample image data.

[0142] When executing the region growing algorithm, the threshold is controlled in the range of 770-25584, and a 3D model is reconstructed according to the Marching Cube algorithm, that is, the process described in steps A11 to A13 to view the main anatomical structure of the CBCT, and the region growing algorithm with changing the input threshold and the splicing vector is executed on the original sample image data set according to the isosurface image constructed by the MarchingCube algorithm.

[0143] The region growing algorithm includes the change of input threshold and splicing vector. The splicing vector formula is as follows:

[0144] O1={(x,y)F(x,y)≤s3};

[0145] O2={(x,y)s3<F(x,y)≤(s3+σ)∧G(x,y)> t3};

[0146] O=O1∪O2={(x,y)F(x,y)≤s3}∪{(x,y)s3<F(x,y)≤(s3+σ)∧G(x,y)> t3};

[0147] B wf=1,(x,y)∈O,0,(x,y)∈others;

[0148] It should be noted that (s3, t3) is a two-dimensional threshold vector, s3 is the grayscale threshold, t3 is the gradient threshold, and the image points with a threshold value less than s3 are taken as the target image point set; σ is a variable parameter, and its size corresponds to the degree of image blur; Bwf is the grayscale image value in CT; (x, y) represents the coordinates in the original sample image data set.

[0149] In this embodiment, the original CBCT image is reconstructed using the Marching Cube algorithm, and the original sample image data is processed according to the isosurface image obtained after reconstruction and in combination with the region growing algorithm to obtain the first sample image data. The accurate isosurface image is reconstructed by the MC algorithm, and the original sample image data is further processed in combination with the region growing algorithm to further improve the accuracy of three-dimensional reconstruction. The region growing algorithm can segment the region of interest more finely and retain more detail information, which is the basis for obtaining a high-quality preprocessed image data set, so as to build a model framework with high-precision image segmentation capabilities through the preprocessed image data set.

[0150] Based on the above embodiments of the present application, a fifth embodiment of the present application is proposed. In the fifth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0151] In this embodiment, the step S123 of establishing a segmentation search algorithm according to the coordinate axis of the first sample image data and executing the segmentation search algorithm in the first sample image data to obtain the second sample image data may include steps A21 to A23:

[0152] Step A21, determining a space dividing line according to the coordinate axis of the first sample image data;

[0153] The space is divided according to the dividing line of the XX axis of the first sample image data, the average value of the XX coordinates of all points is calculated, and the point closest to the average value among all points is selected as the space dividing line to divide the points in the space.

[0154] Step A22, establishing a segmentation search algorithm based on the space segmentation line;

[0155] The steps of establishing a segmentation search algorithm according to the space dividing line are as follows: segment the space according to the segmentation line of the XX axis of the first sample image data, calculate the average value of the XX coordinates of all points, select the point closest to the average value among all points as the space dividing line, and after segmenting the points in the space, further segment the segmented space according to the segmentation line of the YY axis; calculate the average value of the XX coordinates of all points, select the point closest to the average value among all points as the segmentation line, and segment the points in the space; continue to segment the segmented space according to the XX axis, and so on, cyclically segment with the XX axis and the YY axis as the target objects, and end the segmentation when each space is segmented to a point.

[0156] Step A23, executing a segmentation search algorithm on the first sample image data to obtain second sample image data.

[0157] The segmentation process of executing the segmentation search algorithm on the first sample image data corresponds to a binary tree, each segmentation line corresponds to a branch in the binary tree, and each point in the space corresponds to a leaf node.

[0158] The binary tree includes a spatial point (X, Y), and the neighboring points of the spatial point (X, Y) are searched. The search method steps are as follows: traverse the binary tree to reach the leaf node (X', Y'); calculate the distance between (X, Y) and (X', Y'); backtrack and calculate the distance between the upper layer node (x", y") and (X, Y); compare the distances between (X', Y') and (x", y") and (X, Y), respectively, and draw a circle with (X, Y) as the center and the closest distance between the two as the radius; if the circle in the above step has an intersection with the dividing line of the cube in the three-dimensional space and the same length as the circle diameter, then traverse the points with the closest distance compared in the above step, compare the distance between the traversed point and it, continue to backtrack, and draw a circle with (X, Y) as the center and the distance between the traversed point and it as the radius. When the drawn circle has no intersection with the dividing line, the traversed point is the nearest neighbor point and the search is completed. After the traversal search is completed, the second sample image data is obtained.

[0159] In this embodiment, although the region growing algorithm can segment the image into different regions based on the similarity of pixels, its segmentation results may be affected by factors such as the selection of the initial seed point, the similarity metric, and the growth stop condition. Further execution of the segmentation search algorithm can refine and optimize the segmentation results on the basis of region growing to improve the precision and accuracy of the segmentation. Therefore, the segmentation search algorithm is further executed on the sample image data after executing the region growing algorithm to obtain the second sample image data. Compared with the first sample image data, the second sample image data has a more optimized segmentation effect. Combining the segmentation search algorithm can obtain a higher quality pre-processed image data set, so as to build a model framework with high-precision image segmentation capabilities through the pre-processed image data set.

[0160] Based on the above embodiments of the present application, a sixth embodiment of the present application is proposed. In the sixth embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0161] In this embodiment, the step S125 of performing unordered point cloud denoising on the second sample image data by using the spatial binary tree to obtain the third sample image data may include steps A31-A32:

[0162] Step A31, establishing the point cloud three-dimensional coordinates of the second sample image data and the topological relationship of the point cloud three-dimensional coordinates associated with each other in the three-dimensional space through the spatial binary tree;

[0163] A kd tree is generated based on the point cloud data. The kd tree is a special spatial binary tree. The three-dimensional coordinates of the point cloud of the entire 3D graphics and their interrelated topological relationships in three-dimensional space are established through the kd tree.

[0164] Step A32, performing disordered point cloud denoising on the second sample image data according to the point cloud three-dimensional coordinates and the topological relationship to obtain third sample image data.

[0165] Find the neighborhood of any point through topological relationship; calculate the distance between the point and each point in the neighborhood and take the average value; determine whether the average value exceeds the threshold, if so, determine that the point is a noise point and remove it. The image after the noise point is removed is the third sample image data.

[0166] In this embodiment, the noise or outliers introduced in the process of executing the region growth algorithm and the segmentation search algorithm are processed using a kd space binary tree. The kd tree denoising algorithm can effectively identify and remove noise points in the image by constructing a kd tree structure and utilizing its spatial partitioning characteristics, thereby improving the overall quality of the image. In medical image analysis, noise may mask the lesion area or lead to misdiagnosis. Through kd tree denoising, the segmentation results can be further optimized, the impact of noise on subsequent analysis can be reduced, and the accuracy and reliability of the analysis can be improved, so as to build a model framework with high-precision image segmentation capabilities by preprocessing image data sets.

[0167] Based on the above embodiments of the present application, a seventh embodiment of the present application is proposed. In the seventh embodiment of the present application, the same or similar contents as those of the above embodiments can be referred to the above introduction, and will not be described in detail later.

[0168] In this embodiment, the step S126 of performing smoothing processing on the third sample image data based on a preset triangular mesh hole filling algorithm to obtain a preprocessed image data set may include steps A41 to A44:

[0169] Step A41, obtaining a triangular mesh hole filling algorithm based on the Poisson equation;

[0170] Since there are many types of triangular mesh hole filling algorithms, such as hole filling algorithms based on projected feature planes and methods based on voxels (Volumetric Representation), the triangular mesh hole filling algorithm based on the Poisson equation is selected for smoothing based on the hole shape, size, boundary complexity and computing resources of the CBCT sample image data.

[0171] The Poisson equation-based triangular mesh hole filling algorithm is a technique commonly used in 3D geometry processing, especially when processing triangular meshes with missing parts. The Poisson equation hole filling algorithm estimates the vertex positions on the missing area by solving a Poisson equation, thereby smoothly filling the holes in the mesh. This method can maintain the topological structure and geometric continuity of the mesh.

[0172] Step A42, detecting the hole boundaries of the third sample image data according to the triangular mesh hole filling algorithm and generating an initialization hole filling mesh;

[0173] According to the triangular mesh hole filling algorithm, unclosed boundary rings in the mesh are found, hole boundaries are detected and an initial hole filling mesh is generated.

[0174] Step A43, adjusting the initialized hole-filling mesh according to the hole boundary to obtain a final hole-filling mesh;

[0175] Based on the normal information of the hole boundary, the Laplace equation is constructed to solve the normal distribution of the vertices inside the hole-filling mesh. According to the expected normal of the vertex, the triangles in the hole-filling mesh are rotated to make them consistent with the normal of the surrounding original mesh. In order to repair the mesh tearing that may be caused by the rotation operation, the Poisson equation is used to reconstruct the mesh surface, that is, the final hole-filling mesh is obtained, so that a smooth transition is achieved between the final hole-filling mesh and the original mesh.

[0176] Calculate the expected normal of the vertices in the hole-filling mesh, and construct the Laplace equation Δf=0 to solve the normal distribution of the vertices inside the hole-filling mesh. The Laplace operator is:

[0177]

[0178] Among them, N1(xi) represents the 1-ring neighboring point of vertex xi, α ij and β ij For edge e ij The corresponding two diagonals.

[0179] Furthermore, the triangular facets in the hole-filling mesh are rotated based on the desired normal direction; the vertex positions of the hole-filling mesh are adjusted based on the Poisson equation, and the gradient field of the three-dimensional mesh is calculated. The formula is:

[0180]

[0181] Among them, f is the vertex position of the adjusted mesh to be calculated, and w is the gradient field of the torn mesh.

[0182] Furthermore, the calculation formula of the gradient operator is:

[0183] Among them, the basis function gradient The expression is:

[0184] It means rotating the vector 90 degrees counterclockwise, and AT represents the area of ​​triangle T;

[0185] The calculation formula of the divergence operator is:

[0186]

[0187] Among them, T1(x i ) represents the vertex x i The 1-ring neighborhood triangle of A T represents the area of ​​triangle T.

[0188] The geometric shape of the triangular facets in the hole-filling mesh is adjusted according to the desired normal, which includes rotating the triangular facets to match the desired normal and adjusting the vertex positions through the Poisson equation to obtain the final hole-filling mesh, so that the hole-filling mesh and the original mesh can have a smooth transition.

[0189] Step A44: Smoothing the third sample image data based on the final hole-filling grid to obtain a preprocessed image data set.

[0190] The third sample image data is smoothed by the final hole-filling grid, so that the hole-filling grid and the original grid can smoothly transition to obtain a preprocessed image data set.

[0191] In this embodiment, the sample image data is smoothed using a triangular mesh hole filling algorithm based on the Poisson equation to improve the image quality and segmentation effect of the preprocessed image data set, and finally the preprocessed image data set is obtained. By smoothing, noise, non-uniformity and other interference factors in the image can be reduced. These improved images are visually clearer, which helps the semantic segmentation neural network framework to more accurately capture the features and information in the image. Using the smoothed image data set to train the model can reduce the errors caused by factors such as image noise during the training process of the model. This helps the model to better learn the features in the image and improve the generalization ability and accuracy of the model.

[0192] The following combination Figure 5 , Figure 5 A flowchart for building a semantic segmentation neural network framework. Figure 5 , the process of building a semantic segmentation neural network framework based on the preprocessed image dataset is explained in detail.

[0193] First, a region growing algorithm is performed on the original sample image data set. When performing the region growing algorithm, the threshold is controlled in the range of 770-25584, and a 3D model is reconstructed to view the main anatomical structure of CBCT.

[0194] Further, the segmentation search algorithm is executed. The segmentation search algorithm is continued to be executed on the image after the region growing algorithm is executed. Further execution of the segmentation search algorithm can refine and optimize the segmentation result based on the region growing to improve the precision and accuracy of the segmentation.

[0195] Furthermore, a kd tree is established. A kd tree is generated based on the point cloud data to establish the three-dimensional coordinates of the point cloud of the entire 3D graphics and their topological relationships in three-dimensional space; find the neighborhood of any point; calculate the distance between the point and each point in the neighborhood, and take the average value; determine whether the average value exceeds the threshold, if so, the point is determined to be a noise point and removed.

[0196] Furthermore, a correction and hole-filling process is performed to obtain a preprocessed image data set. An initial hole-filling mesh is generated according to the hole boundary in the triangular mesh, and then the geometric shape of the triangular facets in the hole-filling mesh is corrected by normal estimation and Poisson equation, so that it can adapt to and merge with the surrounding original mesh, and the final preprocessed image data set is obtained.

[0197] Finally, residual blocks, spatial pyramid pooling, and attention mechanisms are established by preprocessing the image dataset. The ResUNet++ model is built by establishing residual blocks, spatial pyramid pooling, and attention mechanisms. The ResUNet++ model is a semantic segmentation neural network that uses residual blocks, squeeze and excitation blocks, spatial pyramid pooling (ASPP), and attention blocks. Compared with other advanced methods, ResUNet++ significantly improves the segmentation effect of tibia and femur and can handle a small number of images well.

[0198] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the image segmentation method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0199] This application also provides an image segmentation device, please refer to Figure 6 , the image segmentation device comprises:

[0200] The original medical image acquisition module 10 is used to acquire the original medical image;

[0201] The image segmentation module 20 is used to input the original medical image into a pre-built image segmentation model for image segmentation processing to obtain a segmented image. The image segmentation model is trained by a semantic segmentation neural network framework based on a labeled image data set after data enhancement processing. The semantic segmentation neural network framework is built through residual blocks, spatial pyramid pools, and attention mechanisms.

[0202] The image segmentation device provided by the present application adopts the image segmentation method in the above embodiment to solve the technical problem of image segmentation. Compared with the prior art, the beneficial effects of the image segmentation device provided by the present application are the same as the beneficial effects of the image segmentation method provided by the above embodiment, and other technical features in the image segmentation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0203] The present application provides an image segmentation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image segmentation method in the above-mentioned embodiment 1.

[0204] Reference below Figure 7 , which shows a schematic diagram of the structure of an image segmentation device suitable for implementing an embodiment of the present application. The image segmentation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The image segmentation device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0205] like Figure 7 As shown, the image segmentation device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the image segmentation device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the image segmentation device to communicate with other devices wirelessly or by wire to exchange data. Although the image segmentation device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0206] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0207] The image segmentation device provided by the present application adopts the image segmentation method in the above embodiment to solve the technical problem of image segmentation. Compared with the prior art, the beneficial effects of the image segmentation device provided by the present application are the same as the beneficial effects of the image segmentation method provided by the above embodiment, and other technical features in the image segmentation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0208] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0209] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0210] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the image segmentation method in the above-mentioned embodiment.

[0211] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0212] The computer-readable storage medium may be included in the image segmentation device; or may exist independently without being assembled into the image segmentation device.

[0213] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the image segmentation device, the image segmentation device: obtains an original medical image; inputs the original medical image into a pre-built image segmentation model for image segmentation processing to obtain a segmented image, wherein the image segmentation model is trained by a semantic segmentation neural network framework based on a labeled image data set after data enhancement processing, and the semantic segmentation neural network framework is constructed by residual blocks, spatial pyramid pools, and attention mechanisms.

[0214] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0215] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0216] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0217] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned image segmentation method, and can solve the technical problem of image segmentation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the image segmentation method provided in the above-mentioned embodiment, and will not be described in detail here.

[0218] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned image segmentation method when executed by a processor.

[0219] The computer program product provided by the present application can solve the technical problem of image segmentation. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the image segmentation method provided by the above embodiment, which will not be repeated here.

[0220] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An image segmentation method, characterized in that: The method comprises: Obtaining original medical images; The original medical image is input into a pre-built image segmentation model for image segmentation processing to obtain a segmented image. The image segmentation model is trained by a semantic segmentation neural network framework based on a labeled image data set after data enhancement processing. The semantic segmentation neural network framework is constructed by residual blocks, spatial pyramid pools, and attention mechanisms.

2. The method according to claim 1, characterized in that Before the step of inputting the original medical image into a pre-built image segmentation model for image segmentation processing to obtain a segmented image, the step further includes: Get the original sample image dataset; Preprocessing the original sample image data set to obtain a preprocessed image data set; Annotating the contour lines of the original sample image dataset by a preset open source image segmentation tool to obtain a labeled image dataset; Establishing a residual block, a spatial pyramid pool, and an attention mechanism according to the preprocessed image dataset; Building a semantic segmentation neural network framework based on the residual block, the spatial pyramid pool and the attention mechanism; Performing data enhancement processing on the labeled image dataset; The labeled image dataset after data enhancement processing is fed to the semantic segmentation neural network architecture for training to obtain an image segmentation model.

3. The method according to claim 2, characterized in that The step of preprocessing the original sample image data set to obtain a preprocessed image data set comprises: Distinguishing the original sample image data set based on the file sequence of the original sample image data set to obtain a differentiation result; executing a region growing algorithm on the original sample image data set in combination with the distinction result to obtain first sample image data; Establishing a segmentation search algorithm according to the coordinate axis of the first sample image data, and executing the segmentation search algorithm in the first sample image data to obtain second sample image data; Establishing a special spatial binary tree according to the point cloud data of the second sample image data; Performing unordered point cloud denoising on the second sample image data by using the spatial binary tree to obtain third sample image data; The third sample image data is smoothed based on a preset triangular mesh hole filling algorithm to obtain a preprocessed image data set.

4. The method according to claim 3, characterized in that The step of performing a region growing algorithm on the original sample image data set in combination with the distinction result to obtain first sample image data comprises: Reading the original sample image data set into a specified array in combination with the distinction result; Determine the current unit cell by reading the original sample image data set into a specified array; Rendering an isosurface image based on the current unit volume; A region growing algorithm with input thresholds and splicing vectors is performed on the original sample image data set through the isosurface image to obtain first sample image data.

5. The method according to claim 3, characterized in that The step of establishing a segmentation search algorithm according to the coordinate axis of the first sample image data and executing the segmentation search algorithm in the first sample image data to obtain the second sample image data comprises: determining a space dividing line according to a coordinate axis of the first sample image data; Establishing a segmentation search algorithm based on the spatial segmentation line; A segmentation search algorithm is executed on the first sample image data to obtain second sample image data.

6. The method according to claim 3, characterized in that The step of performing disordered point cloud denoising on the second sample image data by using the spatial binary tree to obtain third sample image data comprises: Establishing the point cloud three-dimensional coordinates of the second sample image data and the topological relationship of the point cloud three-dimensional coordinates associated with each other in the three-dimensional space through the spatial binary tree; The second sample image data is subjected to disordered point cloud denoising according to the point cloud three-dimensional coordinates and the topological relationship to obtain third sample image data.

7. The method according to claim 3, characterized in that The step of performing smoothing processing on the third sample image data based on a preset triangular mesh hole filling algorithm to obtain a preprocessed image data set comprises: Get the triangular mesh hole filling algorithm based on Poisson equation; Detect hole boundaries of the third sample image data according to the triangular mesh hole filling algorithm and generate an initialization hole filling mesh; Adjusting the initial hole-filling grid according to the hole boundary to obtain a final hole-filling grid; The third sample image data is smoothed based on the final hole filling grid to obtain a preprocessed image data set.

8. The method according to claim 2, characterized in that The step of performing data enhancement processing on the labeled image data set comprises: Setting a cropping margin based on the labeled image dataset; Adjusting pixels of the labeled image dataset; Perform data augmentation on the pixel-adjusted labeled image dataset.

9. An image segmentation device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the image segmentation method according to any one of claims 1 to 8.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the image segmentation method according to any one of claims 1 to 8 are implemented.

Images