Spine segmentation method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides a spine segmentation method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring a spine region image; for the two-dimensional slice image of the spine region image, performing feature extraction by using a pre-trained coding model to obtain a target slice embedding representation; inputting the target slice embedding representation into a pre-trained first decoding model for centrum key point detection to obtain point embedding representation of centrum key points contained in the spine region image; and inputting the target slice embedding representation and the point embedding representation into a pre-trained second decoding model for spine segmentation to obtain a spine mask. The method is used for achieving the effects of reducing the repeated image coding process, reducing the calculation overhead and improving the spine segmentation efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a spine segmentation method, apparatus, device, storage medium and program product. Background Art

[0002] With the continuous development of imaging, the application of CT (Computed Tomography) technology in clinical practice has become more and more common. Among them, accurately locating and segmenting the mask of each spine from CT medical images plays an important role in clinical scenarios.

[0003] In the existing technology, the key points or areas of each vertebra are determined by positioning the model, and then the corresponding areas are cropped and input into the segmentation model for processing to obtain the mask of each vertebra. The vertebral segmentation process involves multiple models, such as detection model, positioning model, vertebral naming model, segmentation model, etc. The current vertebral segmentation has the problems of repeated calculation and low efficiency. Summary of the invention

[0004] The embodiments of the present application provide a spine segmentation method, apparatus, device, storage medium and program product to reduce repeated image encoding processes, reduce computing overhead and improve spine segmentation efficiency.

[0005] In a first aspect, an embodiment of the present application provides a spine segmentation method, comprising:

[0006] Obtaining images of the spinal region;

[0007] For the two-dimensional slice image of the spinal region image, a pre-trained encoding model is used to perform feature extraction to obtain an embedded representation of the target slice;

[0008] Inputting the target slice embedding representation into a pre-trained first decoding model to perform vertebral key point detection, and obtaining a point embedding representation of the vertebral key points contained in the spinal region image;

[0009] The target slice embedding representation and the point embedding representation are input into a pre-trained second decoding model to perform spine segmentation and obtain a spine mask.

[0010] In a possible implementation, the two-dimensional slice image of the spinal region image is subjected to feature extraction using a pre-trained encoding model to obtain a target slice embedding representation, including:

[0011] Get the current available computing resource information;

[0012] Based on the computing resource available information, selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal region image;

[0013] The target two-dimensional slice image is input into the pre-trained encoding model for feature extraction to obtain a target slice embedding representation.

[0014] In a possible implementation, selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal region image based on the computing resource available information includes:

[0015] If the computing resource availability information meets the preset computing resource condition, two-dimensional slice images of different perspectives are selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice images of different perspectives are used as target two-dimensional slice images.

[0016] In a possible implementation, inputting the target two-dimensional slice image into the pre-trained encoding model for feature extraction to obtain a target slice embedding representation includes:

[0017] The two-dimensional slice images of different perspectives are respectively input into the pre-trained encoding model for feature extraction to obtain slice embedding representations corresponding to different perspectives, and the extracted slice embedding representations of different perspectives are spliced ​​to obtain the target slice embedding representation.

[0018] In a possible implementation, selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal region image based on the computing resource available information includes:

[0019] If the computing resource availability information does not meet the preset computing resource condition, a two-dimensional slice image of a target perspective is selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice image of the target perspective is used as a target two-dimensional slice image.

[0020] In a possible implementation, inputting the target two-dimensional slice image into the pre-trained encoding model for feature extraction to obtain a target slice embedding representation includes:

[0021] The two-dimensional slice image of the target perspective is input into the pre-trained encoding model for feature extraction to obtain a slice embedding representation of the target perspective, and the extracted slice embedding representation of the target perspective is used as the target slice embedding representation.

[0022] In a possible implementation, inputting the target slice embedding representation into a pre-trained first decoding model to perform vertebral key point detection to obtain a point embedding representation of the vertebral key points contained in the spinal region image includes:

[0023] The target slice is embedded into the first decoding model that is pre-trained to detect vertebral key points, so as to obtain a vertebral key point heat map;

[0024] Feature extraction is performed on the vertebral key point heat map to obtain a point embedding representation of the vertebral key points contained in the spinal column region image.

[0025] In a second aspect, an embodiment of the present application provides a spinal segmentation device, comprising:

[0026] An acquisition module, used for acquiring an image of a spinal region;

[0027] An extraction module, for extracting features of the two-dimensional slice image of the spinal region image using a pre-trained encoding model to obtain an embedded representation of the target slice;

[0028] A detection module, configured to input the target slice embedding representation into a pre-trained first decoding model to perform vertebral key point detection, and obtain a point embedding representation of the vertebral key points contained in the spinal region image;

[0029] The segmentation module is used to input the target slice embedding representation and the point embedding representation into a pre-trained second decoding model to perform spine segmentation and obtain a spine mask.

[0030] In a possible implementation, the extraction module is specifically used to:

[0031] Get the current available computing resource information;

[0032] Based on the computing resource available information, selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal region image;

[0033] The target two-dimensional slice image is input into the pre-trained encoding model for feature extraction to obtain a target slice embedding representation.

[0034] In a possible implementation, the extraction module is specifically used to:

[0035] If the computing resource availability information meets the preset computing resource condition, two-dimensional slice images of different perspectives are selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice images of different perspectives are used as target two-dimensional slice images.

[0036] In a possible implementation, the extraction module is specifically used to:

[0037] The two-dimensional slice images of different perspectives are respectively input into the pre-trained encoding model for feature extraction to obtain slice embedding representations corresponding to different perspectives, and the extracted slice embedding representations of different perspectives are spliced ​​to obtain the target slice embedding representation.

[0038] In a possible implementation, the extraction module is specifically used to:

[0039] If the computing resource availability information does not meet the preset computing resource condition, a two-dimensional slice image of a target perspective is selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice image of the target perspective is used as a target two-dimensional slice image.

[0040] In a possible implementation, the extraction module is specifically used to:

[0041] The two-dimensional slice image of the target perspective is input into the pre-trained encoding model for feature extraction to obtain a slice embedding representation of the target perspective, and the extracted slice embedding representation of the target perspective is used as the target slice embedding representation.

[0042] In a possible implementation manner, the detection module is specifically used to:

[0043] The target slice is embedded into the first decoding model that is pre-trained to detect vertebral key points, so as to obtain a vertebral key point heat map;

[0044] Feature extraction is performed on the vertebral key point heat map to obtain a point embedding representation of the vertebral key points contained in the spinal column region image.

[0045] In a third aspect, an embodiment of the present application provides a spine segmentation device, including: a memory, a processor;

[0046] The memory stores computer-executable instructions;

[0047] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0049] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0050] The spine segmentation method, device, equipment, storage medium and program product provided in the embodiments of the present application can obtain a spinal region image. The spinal region image can be cropped from the original medical image. The spinal region image is composed of a series of two-dimensional slice images. For the two-dimensional slice image, a pre-trained encoding model can be used to extract features to obtain a target slice embedding representation. Then, the target slice embedding representation can be input into a pre-trained first decoding model to perform vertebral key point detection, and obtain a point embedding representation of the vertebral key points contained in the spinal region image. The target slice embedding representation and the point embedding representation are input into a pre-trained second decoding model for spinal segmentation, and the spinal mask corresponding to each vertebra can be obtained. In the embodiments of the present application, vertebral key point detection and spinal segmentation are realized by using the target slice embedding representation. By sharing the embedding representation, a faster and resource-saving image processing process can be achieved without affecting the accuracy, thereby reducing repeated image encoding processes, reducing computational overhead, and improving the efficiency of spinal segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0052] Figure 1 A schematic diagram of a scenario for the spine segmentation method provided in this application;

[0053] Figure 2 Schematic diagram of the spine segmentation method provided in this application Figure 1 ;

[0054] Figure 3 Schematic diagram of the spine segmentation method provided in this application Figure 2 ;

[0055] Figure 4 A schematic diagram of cutting out a spinal region image from an original medical image provided for this application;

[0056] Figure 5 Schematic diagram of two-dimensional slice images at different viewing angles provided for this application;

[0057] Figure 6 A schematic diagram of the center point of the vertebral body provided for this application;

[0058] Figure 7 A schematic diagram of the structure of the first decoding model provided by this application;

[0059] Figure 8 A schematic diagram of the structure of the second decoding model provided by this application;

[0060] Fig. 9 A schematic diagram of the segmentation process of the spine segmentation method provided in this application;

[0061] Fig.10 A schematic diagram of the structure of the spine segmentation device provided in this application;

[0062] Fig.11 This is a schematic diagram of the structure of the spine segmentation device provided in this application.

[0063] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0064] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0065] The application scenario of the present application is specifically the scenario of performing spinal segmentation. In the process of spinal segmentation, the detection model, positioning model, vertebra naming model, segmentation model, etc. each play a different role. The detection model is usually used to preliminarily identify the spinal region or related structures. Working principle: By inputting a medical image, the detection model can identify the area containing the spine and may provide the approximate position and range of the spine. The positioning model is used to accurately determine the key points or areas of each spine. In the spinal region, the positioning model can identify and mark the key points of each spine (such as cervical vertebrae, thoracic vertebrae, lumbar vertebrae, etc.), such as the center, edge or specific anatomical landmark of the vertebral body. These key points are usually used in subsequent cropping and segmentation steps. The vertebra naming model is used to identify and name each vertebra. Based on the key points determined by the positioning model, the vertebra naming model can further identify the specific name of each vertebra. The segmentation model is used to segment each vertebra from the surrounding tissue and generate a corresponding mask. The segmentation model can identify and segment the precise boundaries of each vertebra. This usually involves pixel-level classification of medical images to distinguish spinal pixels from pixels of surrounding tissues. The currently used detection models, positioning models, vertebra naming models, and segmentation models all require preprocessing of medical images and training the models from scratch. Each model contains an encoder and a decoder, which leads to repeated calculations, low computational efficiency, a large total number of model parameters, and long training time.

[0066] The spine segmentation method provided in the present application can obtain a spinal region image. The spinal region image can be cropped from the original medical image. The spinal region image is composed of a series of two-dimensional slice images. For the two-dimensional slice image, a pre-trained encoding model can be used to extract features to obtain a target slice embedding representation. Then, the target slice embedding representation can be input into a pre-trained first decoding model for vertebral key point detection to obtain a point embedding representation of the vertebral key points contained in the spinal region image. The target slice embedding representation and the point embedding representation are input into a pre-trained second decoding model for spinal segmentation, and the spinal mask corresponding to each vertebra can be obtained. In an embodiment of the present application, vertebral key point detection and spinal segmentation are achieved by using the target slice embedding representation. By sharing the embedding representation, a faster and more resource-saving image processing process can be achieved without affecting the accuracy, thereby solving the problem of low spinal segmentation efficiency.

[0067] Figure 1 A schematic diagram of a spine segmentation method provided in this application, such as Figure 1 As shown, the medical image is processed in the terminal device to distinguish the spine pixels from the pixels of the surrounding tissues in the medical image. The terminal device can be a desktop computer, a laptop computer, a tablet computer, a smart phone, a robot, etc.

[0068] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0069] Figure 2 Schematic diagram of the spine segmentation method provided in this application Figure 1 ,like Figure 2 As shown, the method includes:

[0070] S201, obtaining a spinal region image.

[0071] In the embodiment of the present application, the original medical image obtained by photographing the spine can be first obtained, wherein the original medical image can be a three-dimensional CT image, which refers to a series of two-dimensional CT images obtained by CT scanning technology, and these two-dimensional CT images are combined into a three-dimensional image in a computer through three-dimensional reconstruction technology. The spine position can be identified for the original medical image, and the spine area image can be cropped therefrom.

[0072] S202: For the two-dimensional slice image of the spinal region image, a pre-trained encoding model is used to perform feature extraction to obtain an embedded representation of the target slice.

[0073] The cropped spinal region image is composed of a series of two-dimensional slice images. For the two-dimensional slice images, a pre-trained encoding model can be used for feature extraction. After training, the encoding model can effectively identify and extract important features in the two-dimensional slice images. Through this feature extraction process, the slice embedding representation of the two-dimensional slice image can be obtained, and then the target slice embedding representation for subsequent vertebral key point detection and spine segmentation can be determined.

[0074] The encoding model can be the Segment Anything Model (SAM), which is a natural image segmentation model that can segment images based on text instructions and other methods, and has strong zero-sample generalization capabilities. The SAM model has been trained with more than 1 billion masks on more than 11 million images, which enables it to understand and segment a variety of image contents including unknown objects and scenes.

[0075] S203: Input the target slice embedding representation into a pre-trained first decoding model to perform vertebral key point detection, and obtain a point embedding representation of the vertebral key points contained in the spinal column region image.

[0076] In an embodiment of the present application, by inputting the target slice embedding representation into the first decoding model, the first decoding model can identify and detect the key points of the vertebrae / vertebrae in the spinal region image. These key points may be the edges, centers or other anatomically significant landmarks of the vertebrae.

[0077] S204, input the target slice embedding representation and the point embedding representation into a pre-trained second decoding model to perform spine segmentation and obtain a spine mask.

[0078] The second decoding model uses the target slice embedding representation and the point embedding representation of the vertebral key points to identify and segment the structure of the spine. By combining these two types of information, the second decoding model can better understand the morphology and position of the spine, thereby generating a more accurate segmentation result, namely the spine mask corresponding to the spine, which identifies the area in the image that belongs to the spine.

[0079] The spine segmentation method provided in the embodiment of the present application can obtain a spine region image. The spine region image can be cropped from the original medical image. The spine region image is composed of a series of two-dimensional slice images. For the two-dimensional slice image, a pre-trained encoding model can be used to extract features to obtain a target slice embedding representation. Then, the target slice embedding representation can be input into a pre-trained first decoding model to perform vertebral key point detection, and obtain a point embedding representation of the vertebral key points contained in the spine region image. The target slice embedding representation and the point embedding representation are input into a pre-trained second decoding model for spine segmentation, and the spine mask corresponding to each vertebra can be obtained. In the embodiment of the present application, vertebral key point detection and spine segmentation are realized by using the target slice embedding representation. By sharing the embedding representation, a faster and resource-saving image processing process can be achieved without affecting the accuracy, thereby reducing repeated image encoding processes, reducing computational overhead, and improving the efficiency of spine segmentation.

[0080] Figure 3 Schematic diagram of the spine segmentation method provided in this application Figure 2 ,like Figure 3 As shown, in this embodiment Figure 2 Based on the embodiment, a spine segmentation method is described in detail, and the method includes:

[0081] S301, obtaining a spinal region image.

[0082] Specifically, the spine bounding box can be calculated based on the spine mask obtained by rough segmentation, and the spine region image can be cropped out from the original medical image based on the bounding box coordinates. By cropping out the spine region image, the amount of data to be processed can be reduced, while focusing on the spine region of interest, improving the efficiency and accuracy of subsequent processing.

[0083] Reference Figure 4 As shown, it is a schematic diagram of cropping the spinal region image from the original medical image provided by the present application, wherein the 3D U-Net model can be used to perform rough spinal segmentation to obtain a spinal mask. 3D U-Net is a deep learning architecture for 3D medical image segmentation tasks, which is developed based on the classic 2D U-Net architecture. Compared with 2D U-Net, 3D U-Net can better utilize the information in three-dimensional data, so that it is more robust and accurate in segmentation tasks.

[0084] The training process of the 3D U-Net model includes:

[0085] First, a segmentation mask of each vertebra can be annotated on the medical images used for training. This process requires accurately drawing the boundaries of the vertebra to indicate its presence or absence. The annotation covers all regions of the spine, including:

[0086] Cervical vertebrae: The seven cervical vertebrae (C1 to C7) located in the neck area.

[0087] Thoracic vertebrae: The twelve thoracic vertebrae (T1 to T12) located in the chest area.

[0088] Lumbar vertebrae: The five lumbar vertebrae (L1 to L5) located in the lower back area.

[0089] Sacrum: The first vertebra (S1) at the base of the spine.

[0090] Secondly, after the annotation is completed, these segmentation masks need to be binarized to generate the gold standard for binary segmentation. Binarization refers to converting the pixel values ​​in the segmentation mask into only two states (0 or 1), where 0 represents the background and 1 represents the spine.

[0091] Finally, once the binary segmentation gold standard is obtained, the 3D U-Net model can be trained. The specific process may include:

[0092] 1. Data preparation: Normalize the medical images used for training, normalize their pixel values ​​to the [0,1] interval, and divide the medical image dataset into training set, validation set, and test set. The training set is used to train the model parameters, the validation set is used to monitor the model performance and adjust the parameters during the training process, and the test set is used to finally evaluate the generalization ability of the model. The common division ratio is 80% for training, 10% for validation, and 10% for testing. The division process is random to ensure data diversity.

[0093] 2. Model construction: The 3D U-Net model consists of an encoder and a decoder. The encoder extracts image features through a series of 3D convolutional layers and pooling layers. The decoder restores the image features to the same size as the input image through a series of upsampling layers (such as transposed convolution) and jump connections, and maps the output to the [0,1] interval through convolution kernels and activation functions (such as softmax), indicating the probability that each pixel belongs to the target category.

[0094] 3. Loss function and optimizer: You can use Dice Loss as the loss function, Adam as the optimizer, and set parameters such as the learning rate. The Dice coefficient is used to measure the similarity between the predicted segmentation and the true segmentation. Its calculation formula is Where X is the predicted target area pixel set, Y is the real target area pixel set, and |·| represents the number of elements in the set. Dice Loss is a loss function based on the Dice coefficient, defined as DiceLoss = 1-Dice, so that during the training process, the model will be optimized in the direction of making the predicted segmentation more similar to the real segmentation.

[0095] 4. Training process: For each training epoch, traverse the batch data in the training set. Input each batch of medical image data and its corresponding binary segmentation gold standard data into the 3D U-Net model to be trained. Calculate the output prediction segmentation probability map of the model through forward propagation, and then use the backpropagation algorithm to calculate the gradient of Dice Loss to the model parameters. Update the parameters of the model according to the gradient and optimizer update rules. Repeat the above steps until all training epochs are completed or the performance of the model on the validation set no longer improves.

[0096] 5. Validation and testing: Tune the model performance on the validation set and evaluate the generalization ability of the model on an independent test set to ensure that it can accurately perform spine segmentation on unseen data.

[0097] S302: Obtain current available computing resource information.

[0098] In an embodiment of the present application, available information of computing resources of the current device may be obtained, such as CPU information, memory information, and GPU information.

[0099] S303 : Selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal column region image based on the available computing resource information.

[0100] In an embodiment of the present application, the availability of computing resources of the current device is evaluated, and a strategy for selecting two-dimensional slice images is determined based on the availability of computing resources, thereby ensuring that an appropriate number of slice images can be flexibly selected and processed under different computing resource conditions, thereby optimizing performance and resource utilization.

[0101] S304: Input the target two-dimensional slice image into a pre-trained encoding model for feature extraction to obtain an embedded representation of the target slice.

[0102] The pre-trained encoding model can be a segmentation-everything model, which can extract the slice embedding representation of the target two-dimensional slice image through the segmentation-everything model. It can not only effectively capture the local features within the slice, but also use its powerful multimodal capabilities and additional prompt information (such as point prompts, box prompts, and mask prompts) to reduce dependence on a large amount of labeled data and improve the generalization ability of abnormal vertebrae.

[0103] In a possible implementation, based on the available computing resource information, selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal region image may specifically include:

[0104] If the available computing resource information meets the preset computing resource condition, two-dimensional slice images of different perspectives are selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice images of different perspectives are used as target two-dimensional slice images.

[0105] In a possible implementation, the target two-dimensional slice image is input into a pre-trained encoding model for feature extraction to obtain an embedded representation of the target slice, which may specifically include:

[0106] The two-dimensional slice images of different perspectives are respectively input into the pre-trained encoding model for feature extraction to obtain the slice embedding representations corresponding to different perspectives, and the extracted slice embedding representations of different perspectives are spliced ​​to obtain the target slice embedding representation.

[0107] In a possible implementation, based on the available computing resource information, selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal region image may specifically include:

[0108] If the available computing resource information does not meet the preset computing resource condition, a two-dimensional slice image of a target perspective is selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice image of the target perspective is used as the target two-dimensional slice image.

[0109] In a possible implementation, the target two-dimensional slice image is input into a pre-trained encoding model for feature extraction to obtain an embedded representation of the target slice, which may specifically include:

[0110] The two-dimensional slice image of the target perspective is input into a pre-trained encoding model for feature extraction to obtain a slice embedding representation of the target perspective, and the extracted slice embedding representation of the target perspective is used as the target slice embedding representation.

[0111] The present application provides two methods for generating target slice embedding representations. One is that the two-dimensional slice images of three perspectives (cross section, sagittal plane and coronal plane) are all input into the encoding model for feature extraction, and then the slice embedding representations extracted from these three perspectives are spliced ​​to obtain the final target slice embedding representation. In this way, the model can integrate spatial information from different perspectives and capture more comprehensive features. The second is to input only the two-dimensional slice image of one of the perspectives (any one of the cross section, sagittal plane or coronal plane) into the encoding model for feature extraction, and then use the slice embedding representation extracted from this perspective as the final target slice embedding representation. By focusing on one perspective, the model can deeply explore the local features and detail information under that perspective. The single-perspective method may have an advantage in computational efficiency, while the multi-perspective method performs better in accuracy and information completeness.

[0112] When computing resources are limited, a single-view method can be used. This method can effectively extract image features at a low computing cost and is suitable for resource-constrained environments or application scenarios with high real-time requirements. When computing resources are sufficient, a multi-view method can be used.

[0113] Reference Figure 5 The figure shows a schematic diagram of two-dimensional slice images of different perspectives provided by the present application, including a two-dimensional slice image of a sagittal perspective, a two-dimensional slice image of a transverse perspective, and a two-dimensional slice image of a coronal perspective. Wherein, D-Depth, represents depth; W-Width, represents width; H-Height, represents height; I-Image, represents image.

[0114] S305: Input the target slice embedding representation into a pre-trained first decoding model to perform vertebral key point detection, and obtain a point embedding representation of the vertebral key points contained in the spinal column region image.

[0115] As a pre-task for fine spine segmentation, spine key point detection is of great significance for the subsequent spine naming and segmentation. By accurately locating the key points of each spine, key point prompt information can be provided for the subsequent segmentation task, thereby improving the accuracy and robustness of the segmentation.

[0116] In a possible implementation, the target slice embedding representation is input into a pre-trained first decoding model to perform vertebral key point detection, and the point embedding representation of the vertebral key points contained in the spinal region image is obtained, which may specifically include:

[0117] The target slice is embedded into the representation input into the pre-trained first decoding model to detect the vertebral key points and obtain a vertebral key point heat map;

[0118] Feature extraction is performed on the vertebral key point heat map to obtain a point embedding representation of the vertebral key points contained in the spinal region image.

[0119] This application uses the above-obtained target slice embedding representation as the input of vertebral key point detection, and uses a lightweight CNN (Convolutional Neural Network) decoder (i.e., the first decoding model) to convert the input into a heat map of vertebral key points, thereby obtaining the key point coordinates. By combining the powerful feature extraction capability of the segmentation model with the lightweight CNN decoder, it can not only reduce the consumption of computing resources, but also improve the accuracy and robustness of key point detection.

[0120] The training process of the first decoding model includes:

[0121] First, the key points of each vertebra can be annotated on the medical images used for training, including the cervical, thoracic, lumbar and sacral vertebrae. Figure 6 As shown, it is a schematic diagram of the vertebral center point provided in this application. The vertebral key points marked here are the vertebral center points, and each vertebra contains a center point.

[0122] Secondly, after the labeling is completed, a heat map is generated for each key point based on the labeled vertebral key point locations. A heat map is a probability distribution map representing the locations of vertebral key points, usually forming a Gaussian distribution near the key point locations. The heat maps of all key points are combined to form a complete gold standard heat map. This gold standard heat map can be used to train and evaluate the model to help the model learn how to accurately locate and identify vertebral key points.

[0123] Furthermore, the structure of the first decoding model is defined, referring to Figure 7 As shown, it is a structural diagram of the first decoding model provided by the present application. The design of the first decoding model includes the following key components: three-dimensional transposed convolution, instance normalization, leaky rectified linear unit and three-dimensional convolution. Among them, the three-dimensional transposed convolution (also called deconvolution or upsampling convolution) is used to convert the low-resolution embedded representation into a high-resolution output. Instance normalization is a normalization technique that is generally used to improve the training stability and convergence speed of the model. It eliminates the statistical differences between samples by normalizing each sample individually. Leaky ReLU (Leaky Rectified Linear Unit) can allow small gradients to pass when the input is negative, thereby alleviating the "dead ReLU" problem in the ReLU activation function.

[0124] Finally, once the heatmap dataset is obtained, the first decoding model can be trained. The first decoding model can be trained using the generated gold standard heatmap. This process involves setting hyperparameters such as loss function, optimizer, and learning rate, and may require multiple training cycles to minimize the training error.

[0125] S306: Input the target slice embedding representation and the point embedding representation into a pre-trained second decoding model to perform spine segmentation and obtain a spine mask.

[0126] Fine segmentation of the spine is the last key task in the spine segmentation task, which aims to segment and obtain the mask of each spine. This application uses the target slice embedding representation and the point embedding representation of the vertebral key points as the input of fine segmentation of the spine, and uses a lightweight CNN decoder (here refers to the second decoding model) to convert the input into a spine mask for each spine. By using shared slice embedding representations and lightweight CNN decoders, the consumption of computing resources is greatly reduced, and the accuracy and robustness of fine segmentation are further improved through the powerful feature extraction capabilities of the segmentation model.

[0127] The training process of the second decoding model includes:

[0128] First, the segmentation mask of each spine can be annotated on the medical image used for training, and the vertebral key points of each spine can be annotated on the medical image used for training. This process has been completed during the training of the 3D U-Net model and the first decoding model.

[0129] Next, define the structure of the second decoding model, referring to Figure 8 As shown, it is a structural diagram of the second decoding model provided in the present application. The design of the second decoding model includes the following key components: three-dimensional transposed convolution, instance normalization, leaky rectified linear unit and three-dimensional convolution.

[0130] Finally, the second decoding model can be trained. This process includes setting hyperparameters such as loss function, optimizer, and learning rate, and may require multiple training cycles to minimize the training error.

[0131] Reference Fig. 9 As shown, it is a schematic diagram of the segmentation process of the spine segmentation method provided by the present application. For details, please refer to the above description. In order to avoid repetition, it will not be repeated here. Through the present application, the technical problem that not only the parameters of model training are large, but also the consumption of computer video memory is increased in the existing spine segmentation method is solved, and the technical effect of reducing the parameters of model training and reducing the consumption of computer video memory by the model is achieved.

[0132] The spine segmentation method provided in the embodiment of the present application can obtain a spine region image. The spine region image can be cropped from the original medical image. The spine region image is composed of a series of two-dimensional slice images. For the two-dimensional slice image, a pre-trained encoding model can be used to extract features to obtain a target slice embedding representation. Then, the target slice embedding representation can be input into a pre-trained first decoding model to perform vertebral key point detection, and obtain a point embedding representation of the vertebral key points contained in the spine region image. The target slice embedding representation and the point embedding representation are input into a pre-trained second decoding model for spine segmentation, and the spine mask corresponding to each vertebra can be obtained. In the embodiment of the present application, vertebral key point detection and spine segmentation are realized by using the target slice embedding representation. By sharing the embedding representation, a faster and resource-saving image processing process can be achieved without affecting the accuracy, thereby reducing repeated image encoding processes, reducing computational overhead, and improving the efficiency of spine segmentation.

[0133] Fig.10 The schematic diagram of the structure of the spine segmentation device provided in this application is as follows: Fig.10 As shown, the spine segmentation device 100 provided in this embodiment includes:

[0134] An acquisition module 1001 is used to acquire an image of the spinal region;

[0135] An extraction module 1002 is used to extract features of a two-dimensional slice image of a spinal region image using a pre-trained encoding model to obtain an embedded representation of a target slice;

[0136] A detection module 1003 is used to input the target slice embedding representation into a pre-trained first decoding model to perform vertebral key point detection, and obtain a point embedding representation of the vertebral key points contained in the spinal region image;

[0137] The segmentation module 1004 is used to input the target slice embedding representation and the point embedding representation into a pre-trained second decoding model to perform spine segmentation and obtain a spine mask.

[0138] In a possible implementation, the extraction module is specifically used for:

[0139] Get the current available computing resource information;

[0140] Selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal region image based on available computing resource information;

[0141] The target 2D slice image is input into the pre-trained encoding model for feature extraction to obtain the target slice embedding representation.

[0142] In a possible implementation, the extraction module is specifically used for:

[0143] If the available computing resource information meets the preset computing resource condition, two-dimensional slice images of different perspectives are selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice images of different perspectives are used as target two-dimensional slice images.

[0144] In a possible implementation, the extraction module is specifically used for:

[0145] The two-dimensional slice images of different perspectives are respectively input into the pre-trained encoding model for feature extraction to obtain the slice embedding representations corresponding to different perspectives, and the extracted slice embedding representations of different perspectives are spliced ​​to obtain the target slice embedding representation.

[0146] In a possible implementation, the extraction module is specifically used for:

[0147] If the available computing resource information does not meet the preset computing resource condition, a two-dimensional slice image of a target perspective is selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice image of the target perspective is used as the target two-dimensional slice image.

[0148] In a possible implementation, the extraction module is specifically used for:

[0149] The two-dimensional slice image of the target perspective is input into a pre-trained encoding model for feature extraction to obtain a slice embedding representation of the target perspective, and the extracted slice embedding representation of the target perspective is used as the target slice embedding representation.

[0150] In a possible implementation, the detection module is specifically used to:

[0151] The target slice is embedded into the representation input into the pre-trained first decoding model to detect the vertebral key points and obtain a vertebral key point heat map;

[0152] Feature extraction is performed on the vertebral key point heat map to obtain a point embedding representation of the vertebral key points contained in the spinal region image.

[0153] The spine segmentation device provided in this embodiment can execute the method provided in the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0154] Fig.11 This is a schematic diagram of the structure of the spine segmentation device provided in this application. Fig.11 As shown, the spine segmentation device 110 provided in this embodiment includes: at least one processor 1101 and a memory 1102. Optionally, the device 110 further includes a communication component 1103. The processor 1101, the memory 1102 and the communication component 1103 are connected via a bus.

[0155] In a specific implementation process, at least one processor 1101 executes the computer-executable instructions stored in the memory 1102, so that at least one processor 1101 executes the above method.

[0156] The specific implementation process of the processor 1101 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0157] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0158] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0159] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0160] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0161] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0162] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0163] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0164] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0165] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0166] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0167] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0168] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0169] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A spine segmentation method, characterized in that: include: Obtaining images of the spinal region; For the two-dimensional slice image of the spinal region image, a pre-trained encoding model is used to perform feature extraction to obtain an embedded representation of the target slice; Inputting the target slice embedding representation into a pre-trained first decoding model to perform vertebral key point detection, and obtaining a point embedding representation of the vertebral key points contained in the spinal region image; The target slice embedding representation and the point embedding representation are input into a pre-trained second decoding model to perform spine segmentation and obtain a spine mask.

2. The method according to claim 1, characterized in that The method of extracting features of the two-dimensional slice image of the spinal region image using a pre-trained encoding model to obtain a target slice embedding representation includes: Get the current available computing resource information; Based on the computing resource available information, selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal region image; The target two-dimensional slice image is input into the pre-trained encoding model for feature extraction to obtain a target slice embedding representation.

3. The method according to claim 2, characterized in that The step of selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal column region image based on the available computing resource information includes: If the computing resource availability information meets the preset computing resource condition, two-dimensional slice images of different perspectives are selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice images of different perspectives are used as target two-dimensional slice images.

4. The method according to claim 3, characterized in that The step of inputting the target two-dimensional slice image into the pre-trained encoding model for feature extraction to obtain a target slice embedding representation includes: The two-dimensional slice images of different perspectives are respectively input into the pre-trained encoding model for feature extraction to obtain slice embedding representations corresponding to different perspectives, and the extracted slice embedding representations of different perspectives are spliced ​​to obtain the target slice embedding representation.

5. The method according to claim 2, characterized in that: The step of selecting a target two-dimensional slice image from the two-dimensional slice images of the spinal column region image based on the available computing resource information includes: If the computing resource availability information does not meet the preset computing resource condition, a two-dimensional slice image of a target perspective is selected from the two-dimensional slice images of the spinal region image, and the two-dimensional slice image of the target perspective is used as a target two-dimensional slice image.

6. The method according to claim 5, characterized in that The step of inputting the target two-dimensional slice image into the pre-trained encoding model for feature extraction to obtain a target slice embedding representation includes: The two-dimensional slice image of the target perspective is input into the pre-trained encoding model for feature extraction to obtain a slice embedding representation of the target perspective, and the extracted slice embedding representation of the target perspective is used as the target slice embedding representation.

7. The method according to any one of claims 1 to 6, characterized in that: The step of inputting the target slice embedding representation into a pre-trained first decoding model to detect vertebral key points and obtaining a point embedding representation of the vertebral key points contained in the spinal region image comprises: The target slice is embedded into the first decoding model that is pre-trained to detect vertebral key points, so as to obtain a vertebral key point heat map; Feature extraction is performed on the vertebral key point heat map to obtain a point embedding representation of the vertebral key points contained in the spinal column region image.

8. A spinal segmentation device, characterized in that: include: An acquisition module, used for acquiring an image of a spinal region; An extraction module, for extracting features of the two-dimensional slice image of the spinal region image using a pre-trained encoding model to obtain an embedded representation of the target slice; A detection module, configured to input the target slice embedding representation into a pre-trained first decoding model to perform vertebral key point detection, and obtain a point embedding representation of the vertebral key points contained in the spinal region image; The segmentation module is used to input the target slice embedding representation and the point embedding representation into a pre-trained second decoding model to perform spine segmentation and obtain a spine mask.

9. A spine segmentation device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

10. A computer-readable storage medium / computer program product, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor; and / or, The computer program product comprises a computer program, which implements the method according to any one of claims 1 to 7 when executed by a processor.

Images