Dual-channel complementary spine image segmentation method for vertebral body and intervertebral disk segmentation

Abstract

To provide a dual-channel complementary spine image segmentation method for vertebral body and intervertebral disc segmentation.SOLUTION: A dual-channel complementary spine image segmentation method includes the steps of: acquiring some magnetic resonance images of spine and corresponding label images, extracting a to-be-segmented slice and a segmented slice, and preprocessing them; constructing a residual neural network to extract detail features, constructing a spatial feature extractor to extract relation features, constructing a feature fusion module to fuse the detail features and the relation features, and obtaining a segmentation feature map; outputting a multi-class segmentation result via a network output layer, and constructing a dual-channel complementary spine image segmentation module; and training and obtaining an optimal dual-channel complementary spine image segmentation module, and obtaining a multi-class segmentation result for the magnetic resonance image slice to be segmented.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the technical field of spine image segmentation, and in particular to a dual channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs. [Background technology]

[0002] With the aging of society and changes in work and lifestyle, the incidence of spinal diseases is increasing. Computed tomography (CT) and magnetic resonance imaging (MRI) play an important role in the diagnosis and treatment of spinal diseases. Automatic segmentation of vertebrae and intervertebral discs is an important step in analyzing spinal images and constructing modeling, such as identifying abnormalities in vertebrae, image-based biomechanical module analysis, detecting fractures in vertebrae, herniated discs, and image-guided spinal intervention. For example, radiofrequency ablation (RFA) is one of the minimally invasive intervention methods commonly used to treat back pain and dysfunction caused by lumbar disc herniation. Because CT has high sensitivity to vertebrae, it is common to use CT to guide the needle tip of RFA to a precise position. In addition, MRI helps visualize the intervertebral disc in the image, so it is used to accurately locate the herniated site during RFA. And fine segmentation of spine images is the key to ensure smooth RFA.

[0003] Automatic segmentation of vertebrae and intervertebral discs in magnetic resonance images can save doctors' time and provide a multiplexable solution for spine diagnosis and treatment. Therefore, this field has attracted keen attention from researchers. In segmenting spine images, the tasks involved are mainly divided into segmenting vertebrae and intervertebral discs in magnetic resonance images separately (i.e., segmenting the images of vertebrae or intervertebral discs in magnetic resonance images separately) and segmenting vertebrae and intervertebral discs simultaneously in magnetic resonance images (i.e., segmenting the images of vertebrae and intervertebral discs simultaneously in magnetic resonance images). The method of segmenting vertebrae or intervertebral discs independently in magnetic resonance images cannot provide doctors with sufficient pathological features. In addition, the method of segmenting vertebrae and intervertebral discs simultaneously in magnetic resonance images can not only avoid overlapping and misjudgment, but also extract more accurate feature fusion by their connections.

[0004] However, in the conventional method, all vertebrae or all intervertebral discs are classified into the same category, but there are few studies on classifying different vertebrae into different categories, and there is little detailed differentiation of intervertebral discs. This is still lacking in the positioning of vertebrae and intervertebral discs and the formulation of surgical procedures. The detailed segmentation of the spine in magnetic resonance images faces two challenges: intra-category similarity and intra-category variability. Intra-category similarity is manifested by the similarity of shapes and patterns between the disc portion and the vertebral portion in magnetic resonance images, and intra-category variability is manifested by the obvious differences between the intervertebral discs or vertebrae that belong to the same category in the magnetic resonance images of different patients.

[0005] Therefore, in order to solve the problems of intra-class similarity and intra-class variability, there is an ever-imminent need for a dual-channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs, which can assist doctors and physicians in quickly finding the regions of vertebral bodies and intervertebral discs, and simultaneously and effectively segment different classes of vertebral bodies and intervertebral discs in magnetic resonance images. Summary of the Invention [Problem to be solved by the invention]

[0006] SUMMARY OF THE PRESENT DISCLOSURE The present invention provides a dual-channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs to overcome the above-mentioned deficiencies existing in the prior art. [Means for solving the problem]

[0007] The method includes: obtaining a few magnetic resonance images and corresponding label images of the spine, extracting and initially processing the segments to be segmented and the segmented segments, creating a residual neural network to extract detailed features, creating a spatial feature extractor to extract related features, creating a feature fusion module to fuse the detailed features and related features to obtain a segmentation feature map, outputting the segmentation results of only a plurality of categories through the network output layer, creating a dual-channel complementary spine image segmentation module, and training and obtaining an optimal dual-channel complementary spine image segmentation module to obtain the segmentation results of only a plurality of categories for the magnetic resonance image slices to be segmented. The present invention uses strip convolution to extract vertical and horizontal relationships at each point to account for the similarity of categories and intra-category variability, and uses attention mechanism to gradually guide the two paths to learn specific features at each stage to achieve better feature accumulation and improve the segmentation accuracy.

[0008] The present invention provides a dual-channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs, which includes the following steps.

[0009] Step S1 is to initially process the magnetic resonance images and corresponding label images of the spine, obtain a few magnetic resonance images and corresponding label images of the spine, extract all the segmentation waiting sections that contain the spine parts in all the magnetic resonance images, extract all the segmentation waiting sections and corresponding segmentation sections in all the label images, normalize and process all the segmentation waiting sections, arrange and crop the size of the segmentation waiting sections and the segmentation section size, and divide into a training data set and a test data set.

[0010] Step S2 creates a dual-channel complementary spine image segmentation module for segmenting the vertebral body and the intervertebral disc into multiple categories, the spine image segmentation module including a residual neural network, a spatial feature extraction device, a feature fusion module and a network output layer. When the dual-channel complementary spine image segmentation module receives image data of a segmentation-awaiting slice, the dual-channel complementary spine image segmentation module inputs the image data of the segmentation-awaiting slice into a residual neural network and a spatial feature extraction device, respectively, and then fuses the outputs of the residual neural network and the spatial feature extraction device and inputs them into a feature fusion module, and the output of the feature fusion module is used as the input of the network output layer. Finally, the network output layer outputs the segmentation results of the segmentation-awaiting slice into which the vertebral body and the intervertebral disc are segmented into multiple categories. Among them, the residual neural network is for extracting detailed features of the vertebral body and the intervertebral disc in the segmentation-awaiting slice. The spatial feature extraction device is for extracting the relationship features between the vertebral body and the intervertebral disc in the segmentation-awaiting slice. The feature fusion module is for fusing the detailed features and the relationship features to obtain a segmentation feature map. The network output layer is for outputting the division results of only a plurality of categories for the segments waiting to be divided.

[0011] Step S21: Create a residual neural network as a context path, and extract detailed features of vertebral body and intervertebral disc in the segment to be segmented. The residual neural network includes an encoder and a decoder, the encoder and the decoder are respectively residual-based encoder and decoder, the encoder includes a first stage encoder, a second stage encoder, a third stage encoder, a fourth stage encoder and a fifth stage encoder in order, the decoder includes a first stage decoder, a second stage decoder, a third stage decoder and a fourth stage decoder in order, and the input of the first stage encoder is the first input of the dual-channel complemented spine image segmentation module.

[0012] Step S22: create a spatial feature extraction device as a spatial path, and extract the relationship features between the vertebrae and the intervertebral disc in the slice to be segmented, the spatial feature extraction device includes a first stage strip extraction device, a second stage strip extraction device, a third stage strip extraction device, a fourth stage strip extraction device and a fifth stage strip extraction device, in order, the input of the first stage strip extraction device is the second input of the dual channel complemented spine image segmentation module.

[0013] Step S23 extracts a feature fusion module to fuse the detailed features in the context path and the relationship features with the spatial path to obtain segmentation features, the feature fusion module including, in order, a first attention refining module, a second attention refining module, a third attention refining module, a fourth attention refining module, a fifth attention refining module, a first segmentation module, a second segmentation module, a third segmentation module, a fourth segmentation module and a fifth segmentation module.

[0014] Step S3: training a dual-channel complementary spine image segmentation module to obtain an optimal dual-channel complementary spine image segmentation module, setting a loss function, setting module parameters and evaluation indexes, training the dual-channel complementary spine image segmentation module according to a training data set and a test data set, obtaining weight parameters corresponding to the dual-channel complementary spine image segmentation module when the evaluation index of the test data set is maximum, and obtaining an optimal dual-channel complementary spine image segmentation module.

[0015] Step S4: According to the optimal dual-channel complementary spine image segmentation module, obtain a segmentation result of the magnetic resonance image slice of the spine to be segmented. According to the optimal dual-channel complementary spine image segmentation module, the magnetic resonance image slice of the spine to be segmented is normalized, arranged and cropped in step S1, and then obtained as a first input and a second input of the dual-channel complementary spine image segmentation module, which is the output of the dual-channel complementary spine image segmentation module.

[0016] Furthermore, the step S1 specifically includes the following steps:

[0017] Step S11: obtain a magnetic resonance image and a corresponding label image of the spine, and construct a spine data set. Obtain a few magnetic resonance images and corresponding label images of the spine, extract all segmentation waiting sections that include a spine part in every magnetic resonance image, extract all segmentation sections that correspond to the segmentation waiting sections in every label image, and convert all the segmentation waiting sections into grayscale and process them to construct a spine data set together with every segmentation section.

[0018] JPEG2025022815000002.jpg19170JPEG2025022815000003.jpg12170 JPEG2025022815000004.jpg11170

[0019] In step S13, the size of the segment waiting to be divided and the size of the divided segments are adjusted and cropped to obtain a second spine data set. The size of the segment waiting to be divided and the corresponding divided segments in the first spine data set are adjusted and the background on both the left and right sides is cropped to obtain a second spine data set. The size of the segment waiting to be divided and the corresponding divided segments after the size adjustment is both 256×256, and the size of the segment waiting to be divided and the corresponding divided segments after the cropping is both 128×256.

[0020] Step S14 divides the second spine data set into a training data set and a test data set according to the second spine data set, and divides every segmentation waiting section and corresponding segmentation section in the second spine data set according to the correspondence between the segmentation waiting section and the segmentation section in a ratio of 9:1, respectively, to obtain a training data set and a test data set for segmenting the spine image.

[0021] Preferably, in step S21, the output of the fifth stage encoder is pixel-overlapped with the output after average pooling to become a first network output of the residual neural network, the outputs of the first stage decoder, the second stage decoder, the third stage decoder and the fourth stage decoder respectively become a second network output, a third network output, a fourth network output and a fifth network output in the residual neural network, the outputs of the first stage encoder, the second stage encoder, the third stage encoder and the fourth stage encoder respectively become inputs of the second stage encoder, the third stage encoder, the fourth stage encoder and the fifth stage encoder, The output of the encoder is sampled through bilinear interpolation and then blended with the output of the fourth stage encoder into a channel to be input to the first stage decoder; the output of the first stage decoder is sampled through bilinear interpolation and then blended with the output of the third stage encoder into a channel to be input to the second stage decoder; the output of the second stage decoder is sampled through bilinear interpolation and then blended with the output of the second stage encoder into a channel to be input to the third stage decoder; the output of the third stage decoder is sampled through bilinear interpolation and then blended with the output of the first stage encoder into a channel to be input to the fourth stage decoder.

[0022] In step S22, the outputs of the first stage strip extraction device, the second stage strip extraction device, the third stage strip extraction device, the fourth stage strip extraction device and the fifth stage strip extraction device respectively become the first spatial output, the second spatial output, the third spatial output, the fourth spatial output and the fifth spatial output of the spatial feature extraction device, and the outputs of the first stage strip extraction device, the second stage strip extraction device, the third stage strip extraction device and the fourth stage strip extraction device respectively become the inputs of the second stage strip extraction device, the third stage strip extraction device, the fourth stage strip extraction device and the fifth stage strip extraction device, and each of the first stage strip extraction device, the second stage strip extraction device, the third stage strip extraction device, the fourth stage strip extraction device and the fifth stage strip extraction device includes strip convolution.

[0023] In step S23, the input of the feature fusion module includes the first network output, the second network output, the third network output, the fourth network output, the fifth network output, the first space output, the second space output, the third space output, the fourth space output and the fifth space output, and the output of the fifth segmentation module is the output of the feature fusion module as a segmentation feature map, the first network output and the fifth space output are fused into a channel and then input to the first attention refinement module, the output of the first attention refinement module is sampled through bilinear interpolation and then input to the first segmentation module, the second network output and the fourth space output are fused into a channel and then input to the second attention refinement module, the output of the second attention refinement module and the output of the first segmentation module are fused into a channel and then sampled through bilinear interpolation and then input to the second segmentation module, the third network output and the third space output are fused into a channel and then input to the third attention refinement module, and the output of the third attention refinement module is sampled through bilinear interpolation and then input to the second segmentation module. the input of the fourth network and the output of the second segmentation module are fused into a channel and then input to the fourth attention refinement module; the output of the fourth attention refinement module and the output of the third segmentation module are fused into a channel and then input to the fourth attention refinement module; the output of the fourth attention refinement module and the output of the third segmentation module are fused into a channel and then input to the fourth attention refinement module; the fifth network output and the first space output are fused into a channel and then input to the fifth attention refinement module; the output of the fifth attention refinement module and the output of the fourth segmentation module are fused into a channel and then input to the fifth attention refinement module; the first attention refinement module, the second attention refinement module, the third attention refinement module, the fourth attention refinement module and the fifth attention refinement module all include an attention module, and the first segmentation module, the second segmentation module, the third segmentation module, the fourth segmentation module and the fifth segmentation module all include a convolution means.

[0024] The step S2 further includes outputting the segmentation results of only a plurality of categories for the segmentation-awaiting sections by a network output layer, the input of the network output layer being the segmentation feature map, the output of the network output layer being the segmentation results of only a plurality of categories for the segmentation-awaiting sections, which is the output of the dual-channel complemented spine image segmentation module, and the number of channels of the output of the network output layer being the number of segmentation categories.

[0025] Preferably, step S3 specifically includes the following steps: A step S31 sets a loss function DFKL. Step S32 sets the module parameters and the evaluation index Dice. Step S33 is to input the segments to be segmented in the training data set into the dual-channel complementary spine image segmentation module to perform training based on the training data set and the test data set. Step S331: the segmentation-ready slice in the training data set is used as the first input of the dual-channel complementary spine image segmentation module, and the residual neural network is used to extract detailed features of the vertebral body and the intervertebral disc in the segmentation-ready slice. In step S332, the segmentation-ready slices in the training data set are used as the second input of the dual-channel complementary spine image segmentation module, and the spatial feature extraction device extracts the relationship features between the vertebrae and the intervertebral discs in the segmentation-ready slices. In step S333, the detailed features and the related features are input to the feature fusion module, and the fifth segmentation module outputs a segmentation feature map. Step S334 outputs the segmentation feature map as the output of the dual-channel complementary spine image segmentation module, which is the segmentation result of only multiple categories for the segments to be segmented by the network output layer, and obtains weight parameters of the dual-channel complementary spine image segmentation module. In step S334, the segmentation segments are one-hot encoded based on the segmentation results of only multiple categories for the segment waiting to be segmented and the corresponding segmentation segments in the training data set, a segmentation segment whose number of output channels is the number of segmentation categories is obtained, and an evaluation index Dice of the training data set is calculated. In step S335, the segments waiting to be split in the test dataset are used as the first and second inputs of the dual-channel complementary spine image splitting module, and based on the weight parameters of the dual-channel complementary spine image splitting module, split results for only a plurality of categories are obtained for the segments waiting to be split in the test dataset, and the corresponding split segments in the test dataset are one-hot encoded, and an evaluation index Dice for the test dataset is calculated. Step S336 repeats steps S331 to S335 until all training generations are completed, and when the evaluation index Dice of the test data set in all training generations is maximum, the dual-channel complementary spine image segmentation module and corresponding weight parameters are obtained, and an optimal dual-channel complementary spine image segmentation module is obtained.

[0026] Preferably, in step S22, the strip convolution includes a first convolution operation, a horizontal strip pooling operation, a vertical strip pooling operation, a horizontal strip convolution operation, a vertical strip convolution operation, a batch normalization operation, a second convolution operation, a first activation function and a second activation function, an input of the first convolution operation is an input of the strip convolution, an output of the first convolution operation passes through the horizontal strip pooling operation, the horizontal strip convolution operation and the batch processing operation in order to obtain a horizontal strip output, an output of the first convolution operation passes through the vertical strip pooling operation, the vertical strip convolution operation and the batch processing operation in order to obtain a vertical strip output, the horizontal strip output and the vertical strip output are superimposed to obtain a strip output by the first activation function, and the strip output passes through the second convolution operation plus a second activation function in order to multiply each term with the strip output one by one to obtain an output of the strip convolution.

[0027] In step S23, the attention module includes a third convolution operation, a global pooling operation, a batch normalization operation, a fourth convolution operation, and a third activation function, the input of the third convolution operation is the input of the attention module, and the output of the third convolution operation passes through the global pooling operation, the fourth convolution operation, the batch normalization operation, and the third activation function in order, and is multiplied by each item one by one together with the output of the third convolution operation to obtain the output of the attention module.

[0028] Preferably, in step S11, the number of channels to be grayscale-processed in the waiting-for-division segments is 1 and 3, and the number of channels in the divided segments is 1; in step S21, the number of output channels of the first-stage encoder, the second-stage encoder, the third-stage encoder, the fourth-stage encoder, the fifth-stage encoder, the first-stage decoder, the second-stage decoder, the third-stage decoder and the fourth-stage decoder are 32, 64, 128, 256, 512, 256, 128, 64 and 32, respectively; and in step S22, the number of output channels of the first-stage strip is 1 and 3, respectively. The numbers of output channels of the first attention refining module, the second attention refining module, the third attention refining module, the fourth attention refining module and the fifth attention refining module are 32, 64, 128, 256 and 512, respectively, and in step S23, the numbers of output channels of the first attention refining module, the second attention refining module, the third attention refining module, the fourth attention refining module, the fifth attention refining module, the first segmentation module, the second segmentation module, the third segmentation module, the fourth segmentation module and the fifth segmentation module are all 64.

[0029] Preferably, in step S22, the convolution kernel of the first convolution operation is 3×3, the convolution kernel of the second convolution operation is 1×1, the convolution kernel of the horizontal strip convolution operation is 3×1, the convolution kernel of the vertical strip convolution operation is 1×3, the first activation function is selected as a rectified linear function ReLU, and the second activation function is selected as an S-type function Sigmoid; in step S23, the convolution kernel of the third convolution operation is 3×3, the convolution kernel of the fourth convolution operation is 1×1, and the third activation function is selected as an S-type function Sigmoid; and in step S24, the network output layer is a convolution operation with a convolution kernel of 1×1.

[0030] Preferably, in step S21, the numbers of residual blocks in the first stage encoder, second stage encoder, third stage encoder, fourth stage encoder, fifth stage encoder, first stage decoder, second stage decoder, third stage decoder and fourth stage decoder are set as 3, 4, 6, 3, 3, 2, 2, 2, 2, respectively.

[0031] Preferably, in the step S31, the loss function DFKL is calculated based on a similarity loss DL, a dynamic scaling cross entropy loss FL and a relative entropy loss KL as follows: It is set as JPEG2025022815000005.jpg16170, In step S32, the evaluation index Dice is It is shown as JPEG2025022815000006.jpg11170, Of these, TP refers to true positive, FP refers to false positive, and FN refers to false negative. Effect of the Invention

[0032] Compared with the prior art, the technical advantages of the present invention are as follows:

[0033] 1. The dual-channel complementary spine image segmentation method for vertebral body and intervertebral disc segmentation provided by the present invention solves the problem of segmenting images of vertebral body and intervertebral disc into multiple categories, and based on the BiSeNet framework, introduces a residual neural network as a context path to achieve the advantage of synergizing the encoder and decoder, and combines a spatial feature extractor as a spatial path to provide a feature fusion module to replace the original feature fusion module of the context path and the spatial path. With the advantages of these means complementing each other, the method can improve the performance of spine image segmentation.

[0034] 2. The dual-channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs provided by the present invention addresses the challenges of class similarity and intra-class variability when segmenting spine images at fine grain. In this method, different sized strip pooling blocks are introduced into the spatial feature extraction device to extract vertical and horizontal relationships for each point through different directional strip convolution, thereby capturing the long-range relationship between vertebral bodies and intervertebral discs and the inter-relationship within the same class. This effectively complements the feature extraction ability of residual neural network, and can better capture the complex details and structure of the spine in magnetic resonance images, and improve the accuracy of image segmentation.

[0035] 3. The dual-channel complementary spine image segmentation method for segmenting vertebral body and intervertebral disc provided by the present invention introduces the attention refinement module in the BiSeNet framework into the feature fusion module to utilize the complementary features of context path and spatial path. Through the attention mechanism, the residual neural network and spatial feature extraction device are gradually guided to learn specific features at each stage, instead of simultaneously fusing the context path and spatial path features using the feature fusion module in the original BiSeNet framework, so as to better integrate features and improve the accuracy of image segmentation. [Brief description of the drawings]

[0036] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of the non-limiting embodiments thereof and upon reference to the following drawings. [Figure 1] 1 is a flow chart illustrating a dual channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs according to the present invention. [Diagram 2] FIG. 2 illustrates the configuration of a dual-channel complementary spine image segmentation module for segmenting vertebral bodies and intervertebral discs according to the present invention. [Diagram 3]FIG. 2 is a schematic diagram showing the configuration of a strip convolution created in the present invention. [Figure 4] FIG. 2 is a schematic diagram showing the configuration of an attention module created in the present invention. [Diagram 5] FIG. 2 is a comparison diagram showing the effect of segmenting an image using a segmentation module according to a specific embodiment of the present invention and another typical module; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] The present application will be described in more detail below with reference to the drawings and examples. It should be understood that the specific examples described herein are merely for the purpose of interpreting the relevant invention, and are not intended to limit the invention. In addition, it should be noted that, for the sake of convenience, only parts related to the invention of the present application are shown in the drawings.

[0038] It should be noted that, unless there is a contradiction, the embodiments and features in the embodiments in the present application can be combined with each other. Hereinafter, the present application will be described in detail with reference to the drawings together with the embodiments.

[0039] FIG. 1 illustrates a dual-channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs according to the present invention, which includes the following steps.

[0040] In the present invention, segmentation means the identification of spinal portions and the division of different vertebrae and intervertebral disc regions in a magnetic resonance image, and as shown in the right-side segmented section in FIG. 2, L1 indicates the region of the first lumbar vertebra, L2 indicates the region of the second lumbar vertebra, L3 indicates the region of the third lumbar vertebra, L4 indicates the region of the fourth lumbar vertebra, L5 indicates the region of the fifth lumbar vertebra, T9 indicates the region of the ninth thoracic vertebra, T10 indicates the region of the tenth thoracic vertebra, T11 indicates the region of the eleventh thoracic vertebra, T12 indicates the region of the twelfth thoracic vertebra, T9 / T10 indicates the region of the intervertebral disc between the ninth and tenth thoracic vertebra, T10 / T11 indicates the region of the intervertebral disc between the tenth thoracic vertebra and the eleventh thoracic vertebra. T11 / T12 indicates the intervertebral disc area between the eleventh and twelfth thoracic vertebrae, T12 / L1 indicates the intervertebral disc area between the twelfth thoracic vertebra and the first lumbar vertebra, L1 / L2 indicates the intervertebral disc area between the first and second lumbar vertebra, L2 / L3 indicates the intervertebral disc area between the second and third lumbar vertebra, L3 / L4 indicates the intervertebral disc area between the third and fourth lumbar vertebra, L4 / L5 indicates the intervertebral disc area between the fourth and fifth lumbar vertebra, L5 / S indicates the intervertebral disc area between the fifth lumbar vertebra and the coccygeal vertebra, and S indicates the coccygeal vertebra.

[0041] Step S1 is to initially process magnetic resonance images and corresponding label images of the spine, obtain a few magnetic resonance images and corresponding label images of the spine, extract all segmentation waiting sections containing the spine part in every magnetic resonance image, extract all segmentation waiting sections corresponding to the segmentation waiting sections in every label image, normalize and process all segmentation waiting sections, adjust and crop the size of the segmentation waiting sections and the size of the segmentation waiting sections, and divide them into a training data set and a test data set.

[0042] Step S11 acquires magnetic resonance images and corresponding label images for the spine, and constructs a spine data set. A few magnetic resonance images and corresponding label images for the spine are acquired, and all the division waiting sections that include the spine part in all the magnetic resonance images are extracted, and all the division sections that correspond to the division waiting sections in all the label images are extracted, and when all the division waiting sections are grayscale processed, a spine data set is constructed together with all the division sections. The number of channels of the division waiting sections before and after the grayscale processing is 1 and 3, respectively, and the number of channels of the division sections is 1.

[0043] JPEG2025022815000007.jpg19170JPEG2025022815000008.jpg12170 JPEG2025022815000009.jpg11170

[0044] In step S13, the sizes of the segments to be divided and the segments are arranged and cropped to obtain a second spine data set. The sizes of the segments to be divided and the corresponding segments in the first spine data set are arranged, and the backgrounds on both the left and right sides are cropped to obtain a second spine data set, and the sizes of the segments to be divided and the corresponding segments after the arrangement are both 256×256, and the sizes of the cropped segments to be divided and the corresponding segments are both 128×256.

[0045] Step S14 divides the second spine data set into a training data set and a test data set, and divides every segmentation waiting section and corresponding segmentation section in the second spine data set according to the correspondence between the segmentation waiting section and the segmentation section in a ratio of 9:1, respectively, to obtain a training data set and a test data set for segmenting the spine image.

[0046] Step S2, as shown in FIG. 2, creates a dual-channel complementary spine image segmentation module for segmenting vertebral bodies and intervertebral discs into multiple categories. The spine image segmentation module includes a residual neural network, a spatial feature extraction device, a feature fusion module, and a network output layer. When the dual-channel complementary spine image segmentation module receives image data of a slice to be segmented, it inputs the image data of the slice to be segmented into the residual neural network and the spatial feature extraction device, respectively, fuses the outputs of the residual neural network and the spatial feature extraction device, and inputs them into the feature fusion module, and uses the output of the feature fusion module as the input of the network output layer. Finally, the network output layer outputs the segmentation results of only multiple categories for the slice to be segmented into which the vertebral bodies and intervertebral discs are segmented. Among them, the residual neural network is for extracting detailed features of the vertebral bodies and intervertebral discs in the slice to be segmented. The spatial feature extraction device is for extracting the relationship features between the vertebral bodies and intervertebral discs in the slice to be segmented. The feature fusion module is for fusing the detailed features and the relationship features to obtain a segmentation feature map. The network output layer is for outputting the division results obtained by dividing the segments waiting to be divided into a plurality of categories.

[0047] Step S21 outputs the residual neural network as a context path to extract detailed features of the vertebral body and the intervertebral disc in the segment to be divided. The residual neural network includes an encoder and a decoder, and the encoder and the decoder are encoders and decoders based on the residual, respectively, and the encoder includes a first stage encoder, a second stage encoder, a third stage encoder, a fourth stage encoder and a fifth stage encoder, respectively, and the decoder includes a first stage decoder, a second stage decoder, a third stage decoder and a fourth stage decoder, respectively. The first stage encoder, the second stage encoder, the third stage encoder, the fourth stage encoder, the fifth stage encoder, the first stage decoder, the second stage decoder, the third stage decoder and the fourth stage decoder have output channel numbers of 32, 64, 128, 256, 512, 256, 128, 64 and 32, respectively.

[0048] The input of the first stage encoder is the first input of the dual channel complemented spine image segmentation module, the output of the fifth stage encoder and the average pooled output are pixel-overlapped to be the first network output of the residual neural network, and the outputs of the first stage decoder, the second stage decoder, the third stage decoder and the fourth stage decoder are the second network output, the third network output, the fourth network output and the fifth network output of the residual neural network, respectively.

[0049] The outputs of the first stage encoder, the second stage encoder, the third stage encoder and the fourth stage encoder are respectively inputs to the second stage encoder, the third stage encoder, the fourth stage encoder and the fifth stage encoder. The output of the fifth stage encoder is sampled through bilinear interpolation and merged with the output of the fourth stage encoder into a channel to be input to the first stage decoder. The output of the first stage decoder is sampled through bilinear interpolation and merged with the output of the third stage encoder into a channel to be input to the second stage decoder. The output of the second stage decoder is sampled through bilinear interpolation and merged with the output of the second stage encoder into a channel to be input to the third stage decoder. The output of the third stage decoder is sampled through bilinear interpolation and merged with the output of the first stage encoder into a channel to be input to the fourth stage decoder.

[0050] The first-stage encoder, the second-stage encoder, the third-stage encoder, the fourth-stage encoder, the fifth-stage encoder, the first-stage decoder, the second-stage decoder, the third-stage decoder and the fourth-stage decoder each include only a few residual blocks, and the number of residual blocks in the first-stage encoder, the second-stage encoder, the third-stage encoder, the fifth-stage encoder, the first-stage decoder, the second-stage decoder, the third-stage decoder and the fourth-stage decoder is set as 3, 4, 6, 3, 3, 2, 2, 2, 2, respectively. The residual block includes two convolution means, and the convolution means includes one convolution operation, one batch normalization operation and one activation function. The convolution kernel of the convolution operation is 3×3. The activation function is selected as a rectified linear function ReLU. In the first stage encoder, the second stage encoder, the third stage encoder, the fourth stage encoder, and the fifth stage encoder, the step length of the convolution operation of the first convolution means in the first residual block is all set as 2.

[0051] Step S22 creates a spatial feature extraction device as a spatial path, and extracts the relationship features between the vertebrae and the intervertebral disc in the slice to be divided. The spatial feature extraction device includes a first stage strip extraction device, a second stage strip extraction device, a third stage strip extraction device, a fourth stage strip extraction device, and a fifth stage strip extraction device, respectively. The first stage strip extraction device, the second stage strip extraction device, the third stage strip extraction device, the fourth stage strip extraction device, and the fifth stage strip extraction device have output channel numbers set as 32, 64, 128, 256, and 512, respectively.

[0052] The input of the first stage strip extractor is the second input of the dual channel complementary spine image segmentation module. The outputs of the first stage strip extractor, the second stage strip extractor, the third stage strip extractor, the fourth stage strip extractor and the fifth stage strip extractor are the first spatial output, the second spatial output, the third spatial output, the fourth spatial output and the fifth spatial output of the spatial feature strip extractor, respectively.

[0053] The outputs of the first stage strip extractor, the second stage strip extractor, the third stage strip extractor and the fourth stage strip extractor are inputs to the second stage strip extractor, the third stage strip extractor, the fourth stage strip extractor and the fifth stage strip extractor, respectively, each of which includes one strip convolution.

[0054] As shown in FIG. 3, the strip convolution includes a first convolution operation, a horizontal strip pooling operation, a vertical strip pooling operation, a horizontal strip convolution operation, a vertical strip convolution operation, a batch normalization operation, a second convolution operation, a first activation function, and a second activation function. The input of the first convolution operation is the input of the strip convolution. The output of the first convolution operation is passed through a horizontal strip pooling operation, a horizontal strip convolution operation, and a batch processing operation in order to become a horizontal strip output. The output of the first convolution operation is passed through a vertical strip pooling operation, a vertical strip convolution operation, and a batch processing operation in order to become a vertical strip output. When the horizontal strip output and the vertical strip output are superimposed, the strip output is passed through the first activation function in order. The strip output is passed through a second convolution operation and a second activation function in order to multiply each item together with the strip output, and becomes the output of the strip convolution.

[0055] The convolution kernel of the first convolution operation is 3×3, the convolution kernel of the second convolution operation is 1×1, the convolution kernel of the horizontal strip convolution operation is 3×1, the convolution kernel of the vertical strip convolution operation is 1×3, the first activation function is selected as a rectified linear function ReLU, and the second activation function is selected as an S-type function Sigmoid.

[0056] Step S23 creates a feature fusion module, fuses the detailed features in the context path and the relationship features with the spatial route, and obtains segmented features. The feature fusion module includes a first attention refining module, a second attention refining module, a third attention refining module, a fourth attention refining module, a fifth attention refining module, a first division module, a second division module, a third division module, a fourth division module, and a fifth division module, in order. The first attention refining module, the second attention refining module, the third attention refining module, the fourth attention refining module, the fifth attention refining module, the first division module, the second division module, the third division module, the fourth division module, and the fifth division module each have 64 output channels.

[0057] The inputs of the feature fusion module include the first network output, the second network output, the third network output, the fourth network output, the fifth network output, the first spatial output, the second spatial output, the third spatial output, the fourth spatial output and the fifth spatial output, and the output of the fifth segmentation module becomes the output of the feature fusion module as a segmented feature map.

[0058] The first network output and the fifth space output are fused into a channel and then input to the first attention refinement module, the output of the first attention refinement module is sampled through bilinear interpolation and then input to the first division module, the second network output and the fourth space output are fused into a channel and then input to the second attention refinement module, the output of the second attention refinement module and the output of the first division module are fused into a channel and then sampled through bilinear interpolation and then input to the second division module, the third network output and the third space output are fused into a channel and then input to the third attention refinement module, the output of the third attention refinement module and the second The output of the segmentation module is merged into a channel, and then undergoes sampling operation through bilinear interpolation to become the input of the third segmentation module; the fourth network output and the second space output are merged into a channel, and then undergoes sampling operation through bilinear interpolation to become the input of the fourth segmentation module; the output of the fourth attention refinement module and the output of the third segmentation module are merged into a channel, and then undergoes sampling operation through bilinear interpolation to become the input of the fourth segmentation module; the fifth network output and the first space output are merged into a channel, and then undergoes sampling operation through bilinear interpolation to become the input of the fifth attention refinement module; the output of the fifth attention refinement module and the output of the fourth segmentation module are merged into a channel, and then undergoes sampling operation through bilinear interpolation to become the input of the fifth segmentation module. The first attention refinement module, the second attention refinement module, the third attention refinement module, the fourth attention refinement module and the fifth attention refinement module each include one attention module.

[0059] As shown in FIG. 4, the attention module includes a third convolution operation, a global pooling operation, a batch normalization operation, a fourth convolution operation, and a third activation function. The input of the third convolution operation becomes the input of the attention module. The output of the third convolution operation passes through a global pooling operation, a fourth convolution operation, a batch normalization operation, and a third activation function in order, and is multiplied by each item one by one together with the output of the third convolution operation to obtain the output of the attention module.

[0060] The convolution kernel of the third convolution operation is 3×3, the convolution kernel of the fourth convolution operation is 1×1, the third activation function is selected as an S-type function Sigmoid, and the first division module, the second division module, the third division module, the fourth division module and the fifth division module each include one convolution means.

[0061] Step S24: output the segmentation results of the segments to be segmented into multiple categories by the network output layer. The input of the network output layer is the segmentation feature map, and the output of the network output layer is the segmentation results of the segments to be segmented into multiple categories, which is the output of the spine image segmentation module with dual channel complementation, and the network output layer is a convolution operation with a convolution kernel of 1×1, and the number of output channels is the number of segmentation categories.

[0062] Step S3: training a dual-channel complementary spine image segmentation module to obtain an optimal dual-channel complementary spine image segmentation module, setting a loss function, setting module parameters and evaluation indexes, setting a training dataset and a test dataset in the second spine dataset, training a dual-channel complementary spine image segmentation module, obtaining the dual-channel complementary spine image segmentation module and corresponding weight parameters when the evaluation index in the test dataset is maximum, and obtaining an optimal dual-channel complementary spine image segmentation module.

[0063] In step S31, a loss function is set as DFKL based on the similarity loss DL, the dynamic scaling cross entropy loss FL and the relative entropy loss KL. JPEG2025022815000010.jpg15170

[0064] Step S32 sets the module parameters and evaluation indexes. The optimizer is set as the adaptive moment estimation Adam, the number of batch processes is set to 20, the weight decay is set to 0.002, and the training generation is set to 500. The learning rate of the training i-th generation is set to 0.001. If the evaluation indexes of the test data set for the i+1st generation to the i+10th generation do not increase more than the evaluation index of the test data set for the i-th generation, the learning rate of the i+11th generation is updated to one-tenth of the learning rate of the i-th generation.

[0065] The evaluation index Dice is It is shown as JPEG2025022815000011.jpg11170, Of these, TP is indicated as true positive, FP as false positive, and FN as false negative.

[0066] Step S33 inputs the segments to be segmented in the training data set into a dual-channel complementary spine image segmentation module for training based on the training data set and the test data set in the second spine data set.

[0067] In step S331, the segmentation-ready slices in the training data set are used as the first input of a dual-channel complementary spine image segmentation module, which extracts detailed features of the vertebral bodies and intervertebral discs in the segmentation-ready slices through a residual neural network.

[0068] In step S332, the segmentation-ready slices in the training data set are used as a second input of the dual-channel complementary spine image segmentation module, and the spatial feature extraction device extracts the relationship features between the vertebrae and the intervertebral discs in the segmentation-ready slices.

[0069] In step S333, the detailed feature and the relational feature are input to the feature fusion module, and a fifth segmentation module outputs a segmented feature map.

[0070] Step S334 outputs the segmentation feature map as the output of the dual-channel complementary spine image segmentation module through the network output layer, which is the segmentation result of only multiple categories for the segment to be segmented, and obtains the weight parameters of the dual-channel complementary spine image segmentation module.

[0071] In step S334, the partitioned segments are one-hot encoded based on the partitioning results of only multiple categories for the segment waiting to be partitioned and the corresponding partitioned segments in the training data set, a partitioned segment whose number of output channels is the number of partitioned categories is obtained, and the evaluation index Dice of the training data set is calculated.

[0072] In step S335, the segments waiting to be split in the test data set are used as the first and second inputs of the dual-channel complementary spine image splitting module, and based on the weight parameters of the dual-channel complementary spine image splitting module, multiple classification segmentation results are obtained for the segments waiting to be split in the test data set, and the corresponding split segments in the test data set are one-hot encoded to calculate the evaluation index Dice of the test data set.

[0073] Step S336 repeats steps S331 to S335 until all training generations are completed, and when the evaluation index Dice of the test data set in all training generations is maximum, a dual-channel complementary spine image segmentation module and corresponding weight parameters are obtained, and an optimal dual-channel complementary spine image segmentation module is obtained.

[0074] Step S4: According to the optimal dual channel complementary spine image segmentation module, obtain a segmentation result of the magnetic resonance image slice for the spine to be segmented. According to the optimal dual channel complementary spine image segmentation module, the magnetic resonance image slice for the spine to be segmented is normalized, arranged and cropped in step S1 as the first input and the second input of the dual channel complementary spine image segmentation module, and obtain a segmentation result of the magnetic resonance image slice for the spine to be segmented, which is the output of the dual channel complementary spine image segmentation module.

[0075] In a specific embodiment, the acquired spine magnetic resonance images and corresponding label images are from the MRSpineSeg challenge dataset (https: / / www.spine segmentation-Challenge.com), which includes only 172 T2-weighted magnetic resonance images and corresponding label images. The resolution of the magnetic resonance images is 512×512 to 1024×1024, and the number of slices is 12 to 18. If the dataset includes 20 categories, the number of division categories, which is the number of output channels of the network output layer, will be 20, of which there are only 10 vertebral regions and only 9 intervertebral disc regions and background, but not all magnetic resonance images have 20 categories.

[0076] Extract all the segmentation waiting sections containing the spine in all the magnetic resonance images, extract all the segmentation sections corresponding to the segmentation waiting sections in all the label images, normalize and process all the segmentation waiting sections, organize and crop the sizes of the segmentation waiting sections and the segmentation sections to 128 × 256, and divide the data set into a training dataset (1953 pairs of segmentation waiting sections and segmentation sections) and a testing dataset (216 pairs of segmentation waiting sections and segmentation sections).

[0077] The network parameters are set as follows: the optimizer is set to adaptive moment estimation Adam, the number of batches is set to 20, the weight decay is set to 0.002, and the training generations are set to 500. The learning rate of the training i-th generation is set to 0.001. If the evaluation indexes of the test dataset for the i+1st generation to the i+10th generation do not increase relative to the evaluation indexes of the test dataset for the i-th generation, the learning rate of the i+11th generation is updated to one tenth of the learning rate of the i-th generation. The video card is selected as NVIDIA GeForce RTX 3090 with 24G video memory.

[0078] Based on the training dataset and the test dataset, the commonly used cross entropy loss CE and the associated loss function DFKL are selected to train each of the created dual-channel complementary spine image segmentation modules, and obtain the optimal Dice value in the test dataset, respectively.

[0079] The optimal test dataset Dice value using a typical U-shaped network (U-Net) trained with cross entropy loss CE in the 16-layer network architecture VGG16 (module 1) and the 34-layer residual network architecture ResNet34 (module 2) provided by the Oxford University VGG Group (Visual Geometry Group) and the optimal test dataset Dice value using a typical dual-channel network (BiSeNet) trained with cross entropy loss CE under the 34-layer residual network architecture ResNet34 (module 3) are compared in Table 1. The dual-channel complementary spine image segmentation module trained with cross entropy loss CE is described as module 4, and the dual-channel complementary spine image segmentation module trained with the loss function DFKL is described as module 5. JPEG2025022815000012.jpg59170

[0080] As shown in Table 1, the dual-channel complementary spine image segmentation module (Module 5) provided by the loss function DFKL can achieve the highest Dice value (82.802%), and the cross-entropy loss CE (Module 4) can achieve the second highest Dice value (81.691%). This proves that the provided dual-channel complementary spine image segmentation module has superiority. In addition, when observing the statistical data variance of the results, Module 5 obtains the lowest variance (0.0148). This shows that the segmentation performance of the created dual-channel complementary spine image segmentation module on the test data set has relatively strong robustness. The smaller the variance, the more consistent and stable the segmentation performance of the provided module is among different samples. This consistency shows that the module can accurately and reliably segment the vertebrae and disc regions in multiple classifications. It plays an important role in helping doctors identify and apply vertebrae and discs in practice. FIG. 5 shows the concrete result of visual segmentation performed by another module.

[0081] The dual channel complementary spine image segmentation method for vertebral body and intervertebral disc segmentation of the present invention solves the problem of segmenting images of vertebral body and intervertebral disc into multiple categories, and based on the BiSeNet framework, introduces residual neural network as a context path to achieve the advantage of synergizing encoder and decoder, and combines spatial feature extraction device as a spatial path to provide a feature fusion module to replace the original feature fusion module of context path and spatial path. With the advantages of these means complementing each other, the method can improve the performance of spine image segmentation. In order to meet the challenges of class similarity and intra-class variability in segmenting spine images at fine level, different sizes of strip pooling blocks are introduced into the spatial feature extraction device, and different directions of strip convolution are used to extract vertical and horizontal relationships for each point, so as to capture the long-distance relationship between vertebrae and intervertebral discs and the interrelationships within the same class, which effectively complements the feature extraction ability of residual neural network, and can better capture the complex details and configurations of spine in magnetic resonance images, and improve the accuracy of image segmentation. In order to utilize the complementary features of context path and spatial path, the attention refinement module in the BiSeNet framework is introduced into the feature fusion module, and the attention mechanism gradually guides the residual neural network and spatial feature extraction device to learn specific features at each stage, instead of fusing the context path and spatial path features simultaneously in the original BiSeNet framework, which can realize better feature integration and improve the accuracy of image segmentation.

[0082] Finally, it should be noted that the above embodiments are merely for illustrating the technical means of the present invention, and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art can still make modifications or equivalent substitutions to the present invention, and any modifications or local substitutions that do not deviate from the spirit and scope of the present invention are all included in the scope of the claims of the present invention.

Claims

1. Step S1 of initial processing magnetic resonance images and corresponding label images of the spine, including acquiring a few magnetic resonance images and corresponding label images of the spine, extracting all segmentation waiting sections including the spine part in every magnetic resonance image, extracting all segmentation waiting sections corresponding to the segmentation waiting sections in every label image, normalizing and processing all the segmentation waiting sections, adjusting and cropping the size of the segmentation waiting sections and the size of the segmentation waiting sections, and dividing into a training data set and a test data set; Step S2 of creating a dual-channel complementary spine image segmentation module for segmenting vertebral bodies and intervertebral discs into multiple categories, the spine image segmentation module including a residual neural network, a spatial feature extraction device, a feature fusion module and a network output layer, in which the dual-channel complementary spine image segmentation module receives image data of a slice to be segmented, inputs the image data of the slice to be segmented into the residual neural network and the spatial feature extraction device respectively, then fuses the outputs from the residual neural network and the spatial feature extraction device and inputs them into the feature fusion module, and outputs the result from the feature fusion module. Step S2, in which the outputs from the residual neural network are input to a network output layer, and finally, the network output layer outputs only a plurality of classification segmentation results for the segments into which the vertebral bodies and the intervertebral discs are divided, in which the residual neural network is for extracting detailed features of the vertebral bodies and the intervertebral discs in the segments to be divided, the spatial feature extraction device is for extracting relationship features between the vertebral bodies and the intervertebral discs in the segments to be divided, the feature fusion module is for fusing the detailed features and the relationship features to obtain a segmentation feature map, and the network output layer is for outputting only a plurality of classification segmentation results for the segments to be divided; Step S21: creating a residual neural network as a context path to extract detailed features of the vertebral body and the intervertebral disc in the segment to be segmented, the residual neural network includes an encoder and a decoder, the encoder and the decoder are a residual-based encoder and a residual-based decoder, respectively, the encoder includes a first-stage encoder, a second-stage encoder, a third-stage encoder, a fourth-stage encoder and a fifth-stage encoder in sequence, the decoder includes a first-stage decoder, a second-stage decoder, a third-stage decoder and a fourth-stage decoder in sequence, and the input of the first-stage encoder is a first input of the dual-channel complemented spine image segmentation module; Step S22: creating a spatial feature extraction device as a spatial path, and extracting the relationship features between the vertebral body and the intervertebral disc in the slice to be divided, the spatial feature extraction device sequentially includes a first stage strip extraction device, a second stage strip extraction device, a third stage strip extraction device, a fourth stage strip extraction device and a fifth stage strip extraction device, and the input of the first stage strip extraction device is the second input of the dual channel complemented spine image division module; Step S23 of creating a feature fusion module to fuse the detailed features of the context path and the relationship features of the spatial path to obtain segmentation features, the feature fusion module including a first attention refining module, a second attention refining module, a third attention refining module, a fourth attention refining module, a fifth attention refining module, a first segmentation module, a second segmentation module, a third segmentation module, a fourth segmentation module and a fifth segmentation module in this order; training a dual channel complementary spine image segmentation module to obtain an optimal dual channel complementary spine image segmentation module, in step S3, setting a loss function, setting module parameters and evaluation indexes, training the dual channel complementary spine image segmentation module according to a training data set and a test data set, obtaining a weight parameter corresponding to the dual channel complementary spine image segmentation module when the evaluation index of the test data set is the maximum, and obtaining an optimal dual channel complementary spine image segmentation module; and step S4 of obtaining segmentation results of magnetic resonance image slices of the spine according to the optimal dual channel complementary spine image segmentation module, in which the magnetic resonance image slices of the spine are normalized, arranged and cropped according to the optimal dual channel complementary spine image segmentation module in step S1, and then the segmentation results of magnetic resonance image slices of the spine according to the optimal dual channel complementary spine image segmentation module are obtained as a first input and a second input of the dual channel complementary spine image segmentation module, which are the output from the dual channel complementary spine image segmentation module. A dual channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs, comprising:

2. Specifically, step S1 includes the following steps: Step S11 of acquiring magnetic resonance images and corresponding label images of the spine to construct a spine data set, in which a few magnetic resonance images and corresponding label images of the spine are acquired, every segmentation waiting section including a spine portion in every magnetic resonance image is extracted, every segmentation section corresponding to the segmentation waiting section in every label image is extracted, and every segmentation waiting section is grayscaled and processed to construct a spine data set together with every segmentation section; Step S13 of arranging and cropping the segment size and the divided segment size to obtain a second spine data set, in which the size of the segment to be divided and the size of the corresponding divided segment in the first spine data set are arranged and the background on both the left and right sides are cropped to obtain a second spine data set, the arranged size of the segment to be divided and the size of the corresponding divided segment are both 256×256, and the cropped size of the segment to be divided and the size of the corresponding divided segment are both 128×256; The dual-channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs according to claim 1, further comprising: a step S14 of dividing into a training data set and a test data set based on a second spine data set, in which every segmentation-awaiting slice and corresponding segmentation slice in the second spine data set are divided according to the correspondence between the segmentation-awaiting slice and the segmentation slice in a ratio of 9:1, respectively, to obtain a training data set and a test data set for segmenting spine images.

3. In step S21, the output from the fifth stage encoder and its average pooled output are pixel-overlapped to become a first network output of the residual neural network, the outputs of the first stage decoder, the second stage decoder, the third stage decoder and the fourth stage decoder respectively become a second network output, a third network output, a fourth network output and a fifth network output in the residual neural network, the outputs of the first stage encoder, the second stage encoder, the third stage encoder and the fourth stage encoder respectively become inputs of the second stage encoder, the third stage encoder, the fourth stage encoder and the fifth stage encoder, the output from the first stage decoder is sampled via bilinear interpolation and fused to a channel as an input to the first stage decoder together with the output from the fourth stage encoder, the output from the first stage decoder is sampled via bilinear interpolation and fused to a channel as an input to the second stage decoder together with the output from the third stage encoder, the output from the second stage decoder is sampled via bilinear interpolation and fused to a channel as an input to the third stage decoder together with the output from the second stage encoder, and the output from the third stage decoder is sampled via bilinear interpolation and fused to a channel as an input to the fourth stage decoder together with the output from the first stage encoder, In step S22, outputs from the first stage strip extraction device, the second stage strip extraction device, the third stage strip extraction device, the fourth stage strip extraction device and the fifth stage strip extraction device respectively become a first spatial output, a second spatial output, a third spatial output, a fourth spatial output and a fifth spatial output of the spatial feature extraction device, respectively, outputs from the first stage strip extraction device, the second stage strip extraction device, the third stage strip extraction device and the fourth stage strip extraction device respectively become inputs of the second stage strip extraction device, the third stage strip extraction device, the fourth stage strip extraction device and the fifth stage strip extraction device, each of which includes strip convolution; In step S23, the input of the feature fusion module includes the first network output, the second network output, the third network output, the fourth network output, the fifth network output, the first space output, the second space output, the third space output, the fourth space output and the fifth space output, and the output from the fifth segmentation module is the output from the feature fusion module as a segmentation feature map, the first network output and the fifth space output are fused into a channel and then input to the first attention refinement module, the output from the first attention refinement module is sampled through bilinear interpolation and then input to the first segmentation module, the second network output and the fourth space output are fused into a channel and then input to the second attention refinement module, the output from the second attention refinement module and the output from the first segmentation module are fused into a channel and then sampled through bilinear interpolation and then input to the second segmentation module, the third network output and the third space output are fused into a channel and then input to the third attention refinement module, and the third attention refinement module is sampled through bilinear interpolation and then input to the second segmentation module. The output from the second segmentation module and the output from the second segmentation module are fused into a channel, and then undergo a sampling operation through bilinear interpolation to become the input of the third segmentation module; the fourth network output and the second space output are fused into a channel and then input to the fourth attention refining module; the output from the fourth attention refining module and the output from the third segmentation module are fused into a channel, and then undergo a sampling operation through bilinear interpolation to become the input of the fourth segmentation module; the fifth network output and the first space output are fused into a channel and then input to the fifth attention refining module; the output from the fifth attention refining module and the output from the fourth segmentation module are fused into a channel and then undergo a sampling operation through bilinear interpolation to become the input of the fifth segmentation module; the first attention refining module, the second attention refining module, the third attention refining module, the fourth attention refining module and the fifth attention refining module all include an attention module; the first segmentation module, the second attention refining module, the third segmentation module, the fourth segmentation module and the fifth segmentation module areEach of them includes a convolution means, Step S2 further includes outputting a segmentation result of only a plurality of categories for the segmentation-awaiting section by a network output layer, an input of the network output layer is the segmentation feature map, an output from the network output layer is a segmentation result of only a plurality of categories for the segmentation-awaiting section, which is an output from the spine image segmentation module with complementation by the dual channel, and the number of output channels from the network output layer is the number of segmentation categories. The dual channel complementary spine image segmentation method for segmenting vertebral body and intervertebral disc as claimed in claim 1.

4. Specifically, step S3 is Step S31 of setting a loss function DFKL; Step S32: setting module parameters and evaluation index Dice; S33: inputting the segments to be segmented in the training data set into the dual-channel complementary spine image segmentation module to perform training based on the training data set and the test data set; In step S331, the segmentation-ready slices in the training data set are used as a first input of the dual-channel complementary spine image segmentation module, and the residual neural network is used to extract detailed features of the vertebral bodies and intervertebral discs in the segmentation-ready slices; S332: the training data set is used as a second input of the dual-channel complementary spine image segmentation module, and the spatial feature extraction device extracts the relationship between the vertebrae and the intervertebral discs in the training data set; In step S333, the detailed features and the related features are input to the feature fusion module, and a segmentation feature map is output by the fifth segmentation module; Step S334: outputting the segmentation feature map as an output of the dual-channel complementary spine image segmentation module, which is a segmentation result of only a plurality of categories of the segments to be segmented by the network output layer, and obtaining weight parameters of the dual-channel complementary spine image segmentation module; Step S334: one-hot encoding the segmentation segments based on the segmentation results of only the multiple categories for the segmentation waiting to be segmented and the corresponding segmentation segments in the training data set, to obtain a segmentation segment whose number of output channels is the number of the segmentation categories, and to calculate an evaluation index Dice of the training data set; Step S335: the test data set awaiting division slices are used as the first and second inputs of the dual-channel complementary spine image division module, and the test data set awaiting division slices are divided into a plurality of categories based on the weight parameters of the dual-channel complementary spine image division module, and the corresponding divided slices in the test data set are one-hot encoded to calculate the evaluation index Dice of the test data set; Repeating steps S331 to S335 until all training generations are completed, and when the evaluation index Dice of the test data set in all training generations is the largest, obtaining the dual-channel complemented spine image segmentation module and corresponding weight parameters, and obtaining an optimal dual-channel complemented spine image segmentation module. The dual channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs according to claim 1 or 3.

5. In step S22, the strip convolution includes a first convolution operation, a horizontal strip pooling operation, a vertical strip pooling operation, a horizontal strip convolution operation, a vertical strip convolution operation, a batch normalization operation, a second convolution operation, a first activation function and a second activation function, the input of the first convolution operation is the input of the strip convolution, the output of the first convolution operation passes through the horizontal strip pooling operation, the horizontal strip convolution operation and the batch processing operation in order to obtain a horizontal strip output, the output of the first convolution operation passes through the vertical strip pooling operation, the vertical strip convolution operation and the batch processing operation in order to obtain a vertical strip output, the horizontal strip output and the vertical strip output are superimposed to obtain a strip output by the first activation function, the strip output passes through the second convolution operation and the second activation function in order to multiply each term with the strip output one by one to obtain the output of the strip convolution; In step S23, the attention module includes a third convolution operation, a global pooling operation, a batch normalization operation, a fourth convolution operation, and a third activation function, the input of the third convolution operation is the input of the attention module, and the output of the third convolution operation passes through the global pooling operation, the fourth convolution operation, the batch normalization operation, and the third activation function in order, and is multiplied by each item one by one together with the output of the third convolution operation to obtain the output of the attention module; The dual channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs according to claim 3.

6. In step S11, the number of channels of the waiting-to-be-divided segment before and after the grayscale processing is 1 and 3, respectively, and the number of channels of the divided segment is 1; In step S21, the numbers of output channels of the first stage encoder, the second stage encoder, the third stage encoder, the fourth stage encoder, the fifth stage encoder, the first stage decoder, the second stage decoder, the third stage decoder and the fourth stage decoder are 32, 64, 128, 256, 512, 256, 128, 64 and 32, respectively; In step S22, the numbers of output channels of the first stage strip extractor, the second stage strip extractor, the third stage strip extractor, the fourth stage strip extractor, and the fifth stage strip extractor are 32, 64, 128, 256, and 512, respectively; In step S23, the number of output channels of the first attention refining module, the second attention refining module, the third attention refining module, the fourth attention refining module, the fifth attention refining module, the first division module, the second division module, the third division module, the fourth division module and the fifth division module is all 64.

3. The dual channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs according to claim 1 or 2.

7. In step S22, the convolution kernel of the first convolution operation is 3×3, the convolution kernel of the second convolution operation is 1×1, the convolution kernel of the horizontal strip convolution operation is 3×1, the convolution kernel of the vertical strip convolution operation is 1×3, the first activation function is selected as a rectified linear function ReLU, and the second activation function is selected as an S-type function Sigmaid; In the step S23, the convolution kernel of the third convolution operation is 3×3, the convolution kernel of the fourth convolution operation is 1×1, and the third activation function is selected as an S-type function Sigmoid; In step S24, the network output layer is a convolution operation with a convolution kernel of 1×1. The dual channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs according to claim 3.

8. In step S21, the numbers of residual blocks in the first-stage encoder, the second-stage encoder, the third-stage encoder, the fourth-stage encoder, the fifth-stage encoder, the first-stage decoder, the second-stage decoder, the third-stage decoder and the fourth-stage decoder are set as 3, 4, 6, 3, 3, 2, 2, 2, 2, respectively; The dual channel complementary spine image segmentation method for segmenting vertebral bodies and intervertebral discs according to claim 1 or 3.

9. In the step S31, the loss function DFKL is calculated based on the similarity loss DL, the dynamic scaling cross entropy loss FL, and the relative entropy loss KL as follows: is set as In step S32, the evaluation index Dice is It is shown as Among them, TP refers to true positive, FP refers to false positive, and FN refers to false negative. The dual channel complementary spine image segmentation method for segmenting vertebral body and intervertebral disc as claimed in claim 1.

Images