Vertebral body identifying and positioning method, device and equipment and storage medium

Abstract

The invention discloses a centrum recognition and positioning method, device and equipment and a storage medium, and the method comprises the steps: inputting a to-be-processed spine image into an image encoder for processing, extracting an image feature representing centrum information from the spine image, fusing the image feature, a centrum embedding feature and a centrum position embedding feature based on a cross attention mechanism, and carrying out the recognition and positioning of the centrum. The method comprises the following steps: acquiring a centrum identification feature, fusing an image feature and a centrum position embedding feature based on a cross attention mechanism to acquire a centrum positioning feature, and predicting the existence probability value of each cone and the position information of each cone based on the centrum identification feature and the centrum positioning feature. According to the method, centrum identification and positioning are completed at the same time based on a cross attention mechanism, all possible centrum are regarded as cone embedding features, probability prediction is carried out in an attention decoding learning mode, the accuracy of centrum identification is improved, position prediction is carried out after image features are combined with the cone embedding features, and the accuracy of centrum identification is improved. And the accuracy of centrum position prediction is greatly improved.

Description

Technical Field

[0001] The present invention relates to machine learning technology, and in particular to a vertebral body identification and positioning method, device, equipment and storage medium. Background Art

[0002] The localization, identification and segmentation of spinal vertebrae based on CT (Computed Tomography) images play an important role in clinical applications such as spinal deformity assessment, bone density measurement and related surgical planning.

[0003] In recent years, the development of deep learning has brought significant progress in the automatic positioning, identification and segmentation of spinal vertebrae, but it still faces huge challenges in real-world applications. The recognition and positioning accuracy of traditional detection and classification methods is not high, and anatomical consistency cannot be guaranteed. Duplicate identification numbers, missed identification numbers, and positioning errors may occur. Summary of the invention

[0004] The present invention provides a vertebral identification and positioning method, device, equipment and storage medium to improve the accuracy of vertebral identification and the accuracy of vertebral position prediction.

[0005] In a first aspect, the present invention provides a vertebral body identification and positioning method, comprising:

[0006] Inputting the spinal column image to be processed into an image encoder for processing, and extracting image features representing vertebral information from the spinal column image;

[0007] The image features, the cone embedding features representing the vertebral identification information, and the vertebral position embedding features representing the cone position information are input into a decoder, the decoder fuses the image features, the cone embedding features, and the vertebral position embedding features based on a cross-attention mechanism to obtain a vertebral identification feature, and the decoder fuses the image features and the vertebral position embedding features based on a cross-attention mechanism to obtain a vertebral positioning feature;

[0008] The probability value of the existence of each of the cones and the position information of each of the cones are predicted based on the vertebral identification features and the vertebral positioning features.

[0009] In a second aspect, the present invention further provides a vertebral body identification and positioning device, comprising:

[0010] An image feature extraction module is used to input the spinal image to be processed into the image encoder for processing, and extract image features representing vertebral information from the spinal image;

[0011] a feature processing module, configured to input the image feature, the cone embedding feature representing the vertebral identification information, and the vertebral position embedding feature representing the cone position information into a decoder, wherein the decoder fuses the image feature, the cone embedding feature, and the vertebral position embedding feature based on a cross-attention mechanism to obtain a vertebral identification feature, and the decoder fuses the image feature and the vertebral position embedding feature based on a cross-attention mechanism to obtain a vertebral positioning feature;

[0012] The cone prediction module is used to predict the probability value of the existence of each cone and the position information of each cone based on the vertebral identification feature and the vertebral positioning feature.

[0013] In a third aspect, the present invention further provides an electronic device, comprising:

[0014] one or more processors;

[0015] A memory for storing one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the vertebral identification and positioning method provided in the first aspect of the present invention.

[0017] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the vertebral identification and positioning method provided in the first aspect of the present invention.

[0018] The vertebral identification and positioning method provided by the present invention inputs the spinal image to be processed into an image encoder for processing, extracts image features representing vertebral information from the spinal image, inputs image features, cone embedding features representing vertebral identification information, and vertebral position embedding features representing cone position information into a decoder, the decoder fuses image features, cone embedding features, and vertebral position embedding features based on a cross-attention mechanism to obtain vertebral identification features, and the decoder fuses image features and vertebral position embedding features based on a cross-attention mechanism to obtain vertebral positioning features, and predicts the probability value of each cone's existence and the position information of each cone based on the vertebral identification features and vertebral positioning features. The present invention simultaneously completes the identification and positioning of vertebrae based on the cross-attention mechanism, innovatively regards all possible vertebrae as cone embedding features, extracts the information of the vertebral existence through attention decoding learning, and performs probability prediction, thereby improving the accuracy of vertebral identification, and performs position prediction after combining image features with cone embedding features, which greatly improves the accuracy of vertebral position prediction.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a vertebral identification and positioning method provided by an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of vertebral segmentation provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a model structure for vertebral identification and positioning provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of the structure of a decoding module provided by an embodiment of the present invention;

[0025] Figure 5 A prediction result correction flow chart provided by an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of a single vertebra segmentation provided by an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the structure of a vertebral body identification and positioning device provided by an embodiment of the present invention;

[0028] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0029] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Figure 1 This is a flow chart of a vertebral identification and positioning method provided in an embodiment of the present invention. This embodiment can be applied to identifying and positioning each vertebra in a spinal CT image. The method can be performed by a vertebral identification and positioning device provided in an embodiment of the present invention. The device can be implemented in software and / or hardware and is usually configured in an electronic device, such as Figure 1 As shown, the vertebral body identification and positioning method specifically includes the following steps:

[0033] S101, inputting a spinal column image to be processed into an image encoder for processing, and extracting image features representing vertebral information from the spinal column image.

[0034] The spinal image is a computer tomography image (CT image), which uses a precisely collimated X-ray beam, gamma ray, ultrasound, etc., together with a highly sensitive detector to perform cross-sectional scans around a certain part of the human body. The detector receives the X-rays that pass through the layer, converts them into visible light, and then converts them into electrical signals through photoelectric conversion, which are then converted into digital signals through an analog / digital converter and input into a computer for processing to obtain a CT image.

[0035] In an embodiment of the present invention, the spinal image may be a two-dimensional CT image or a three-dimensional CT image, which is not limited in the embodiment of the present invention. A two-dimensional CT image, that is, an ordinary CT image, is a CT plain scan image, which only requires one scan of the organ. A three-dimensional CT image requires multiple continuous scans for different areas to obtain multiple two-dimensional CT images, and then the multiple two-dimensional CT images are reconstructed to obtain a three-dimensional three-dimensional CT image. The three-dimensional CT image can make the original planar image three-dimensional, increase the density difference between the diseased tissue and the adjacent normal tissue, and more clearly display the condition of the diseased tissue. Therefore, in a specific embodiment of the present invention, a three-dimensional CT image is selected as the spinal image.

[0036] In an embodiment of the present invention, the spinal image to be processed is input into an image encoder for processing, and image features representing vertebral information are extracted from the spinal image. Exemplarily, the encoder may perform feature extraction on the spinal image after preprocessing, cropping, etc., for example, convolution, pooling, normalization, etc., to extract image features representing vertebral information (including but not limited to CT values ​​of each sampling point in the vertebral body, position information, etc.) from the spinal image.

[0037] Exemplarily, in one embodiment of the present invention, the above step S101 includes the following sub-steps:

[0038] S1011. Extract image information of the area where the vertebral body is located from the spinal column image to be processed to obtain a target image.

[0039] Exemplarily, based on segmentation methods such as visual image processing technology or neural network segmentation models, the region where the vertebrae are located can be segmented and extracted from the spinal column image to be processed, and image information of the region where the vertebrae are located can be extracted to obtain a target image.

[0040] Exemplarily, in a specific embodiment of the present invention, the above step S1011 includes the following sub-steps:

[0041] 1. Preprocess the spinal image to be processed to obtain a preprocessed image including the cone.

[0042] Exemplarily, the preprocessing process includes resampling and normalization. Specifically, the spinal image to be processed is resampled to obtain a resampled image with a preset resolution. Exemplarily, the resampled image is a 3D image with a resolution of [2, 2, 2]. The CT value of the resampled image is normalized to obtain a preprocessed image including a cone. Exemplarily, the resampled image with a CT value in the range of [-500, 1300] is normalized to [0, 1] to obtain a processed preprocessed image image_ct.

[0043] 2. Divide the preprocessed image into multiple sub-images of preset sizes.

[0044] In an embodiment of the present invention, since the size of the entire pre-processed image is very large, the pre-processed image cannot be directly processed in the subsequent vertebral segmentation process, and therefore, the pre-processed image needs to be cut into a plurality of sub-images of preset sizes. Exemplarily, in a specific embodiment of the present invention, the pre-processed image is cut into sub-images of [128×128×128], i.e., [256mm×256mm×256mm].

[0045] 3. Segment the area where the cone is located from the sub-image to obtain a sub-segmented image.

[0046] Exemplarily, based on a segmentation method such as a visual image processing technology or a neural network segmentation model, the region where the vertebral body is located can be segmented from the sub-image to obtain a sub-segmented image.

[0047] Figure 2 A schematic diagram of vertebral segmentation provided by an embodiment of the present invention is shown in FIG. Figure 2 As shown, in a specific embodiment of the present invention, the sub-image ( Figure 2 The area selected by the dashed line in the middle) is input into the image segmentation network for binary segmentation, and the area where the cone is located is segmented from the background of the sub-image to obtain a sub-segmented image, that is, the specific vertebrae are not distinguished, and a binary segmentation map is output. Figure 2 As shown in FIG. 1 , the area where the vertebral body is located in the image after vertebral body segmentation is indicated by white, and the rest is filled with black. For example, in a specific embodiment of the present invention, the image segmentation network can be 3D Unet ( Figure 2 Spine-Unet in .

[0048] 4. Concatenate the sub-segmented images of the multiple sub-images into a segmented image.

[0049] After the sub-segmented images of all sub-images are obtained, the sub-segmented images of multiple sub-images are spliced ​​into a segmented image.

[0050] For example, random cutting is performed during training, and Dice_loss and CE_loss are used as objective functions for optimization training. During inference, the preprocessed image is sequentially divided into multiple sub-images, and the sub-images overlap by 1 / 4. The prediction value of the overlapping part is the average value of the sub-segmented images. All sub-segmented images are combined to obtain the binary segmentation image mask_spine of the spinal vertebra.

[0051] 5. Concatenate the preprocessed image and the segmented image in the channel dimension to obtain the target image.

[0052] Figure 3A schematic diagram of a model structure for vertebral identification and positioning provided by an embodiment of the present invention. After obtaining the binary segmentation image mask_spine of the spinal vertebra, the region of interest ( Figure 3 The image information of the vertebral area in the preprocessed image image_ct and the segmented image mask_spine is extracted, and then the target image image_mask_spine is obtained by scaling them respectively to obtain a uniform size of D×H×W. Then, the preprocessed image image_ct and the segmented image mask_spine are concatenated in the channel dimension to extract the image information of the vertebral area in the preprocessed image image_ct.

[0053] S1012: Input the target image into an image encoder to extract features, thereby obtaining image features representing vertebral information.

[0054] like Figure 3 As shown, the target image image_mask_spine is input into the image encoder Encoder, and the image encoder Encoder extracts features of the target image image_mask_spine to obtain image features image embedding that extracts and represents vertebral information.

[0055] Exemplarily, in a specific embodiment of the present invention, a residual network model (ResNet) is used to extract features of a target image. Specifically, the image encoder includes a three-stage residual module resnet block, each resnet block performs 2 times downsampling, and finally obtains an 8 times downsampled image feature image embedding.

[0056] Specifically, the target image is input into the residual module of the first stage for processing to obtain a first feature map with a resolution half of the target image, the first feature map is input into the residual module of the second stage for processing to obtain a second feature map with a resolution half of the first feature map, the second feature map is input into the residual module of the third stage for processing to obtain image features with a resolution half of the second feature map.

[0057] S102. Input the image features, the cone embedding features representing the vertebral identification information, and the vertebral position embedding features representing the cone position information into the decoder. The decoder fuses the image features, the cone embedding features, and the vertebral position embedding features based on a cross-attention mechanism to obtain the vertebral identification features. The decoder also fuses the image features and the vertebral position embedding features based on a cross-attention mechanism to obtain the vertebral positioning features.

[0058] In an embodiment of the present invention, image features, cone embedding features representing vertebral identification information, and vertebral position embedding features representing cone position information are input into a decoder, and the decoder fuses image features, cone embedding features, and vertebral position embedding features based on a cross-attention mechanism, and the cone embedding features learn image information from image features, and the cone embedding features learn position information from vertebral position embedding features, thereby obtaining vertebral identification features with vertebral identification. The decoder fuses image features and vertebral position embedding features based on a cross-attention mechanism, and the image features obtain vertebral positioning information from the vertebral position embedding features, thereby obtaining vertebral positioning features with vertebral positioning.

[0059] Among them, the cone embedding feature includes 26 vectors, corresponding to the 26 vertebrae of the adult spine, including 7 cervical vertebrae (C1-C7), 12 thoracic vertebrae (T1-T12), 5 lumbar vertebrae (L1-L5), 1 sacrum and 1 coccyx. During the training process, all possible vertebrae (cervical vertebrae, thoracic vertebrae, lumbar vertebrae, including transitional vertebrae T13 and L6)) are regarded as learnable cone embedding features, and the cone embedding features are initialized. Through continuous training and learning, the cone embedding features that characterize the vertebral identification information are obtained. The vertebral position embedding feature is obtained by a similar method, which will not be repeated in the present invention.

[0060] For example, Figure 3 As shown, the decoder Decoder includes multiple layers ( Figure 3 6 layers are used as an example) and the decoding modules are stacked in sequence. Each layer of the decoding modules includes a first decoding unit ( Figure 3 Token attention in the middle and the second decoding unit ( Figure 3 The process of extracting vertebral recognition features and vertebral positioning features is as follows:

[0061] like Figure 3 As shown in FIG. 1 , the image feature image embedding is concatenated with the image position embedding feature image position embedding representing the position information of each element in the image feature to obtain a first concatenated feature. The cone embedding feature token embedding is concatenated with the cone position embedding feature token position embedding to obtain a second concatenated feature.

[0062] like Figure 3As shown, in the first decoding unit (token attention) of the first layer decoding module, the first concatenated feature is used as the key vector K and the value vector V in the cross attention mechanism, and the second concatenated feature is used as the query vector Q in the cross attention mechanism to calculate the output feature of the first decoding unit (token attention) in the first layer decoding module.

[0063] like Figure 3 As shown, in the first decoding unit (tokenattention) of the target decoding module (subsequent decoding module), the output features of the second decoding unit (image attention) in the previous layer decoding module are used as the key vector K and the value vector V in the cross attention mechanism, the output features of the first decoding unit (tokenattention) in the previous layer decoding module are used as the query vector Q in the cross attention mechanism, the output features of the first decoding unit (tokenattention) in the target decoding module are calculated, and the output features of the first decoding unit (tokenattention) in the last layer decoding module are used as the vertebral recognition features, wherein the target decoding module is any one of the multiple decoding modules except the first layer decoding module.

[0064] like Figure 3 As shown, the output feature of the first decoding unit (token attention) of the first layer decoding module is concatenated with the vertebral position embedding feature token position embedding to obtain a third concatenated feature.

[0065] In the second decoding unit (image attention) of the first-layer decoding module, the third concatenated feature is used as the key vector K and value vector V in the cross-attention mechanism, and the image feature image embedding is used as the query vector Q in the cross-attention mechanism to calculate the output features of the second decoding unit (image attention) in the first-layer decoding module.

[0066] like Figure 3As shown, in the second decoding unit (imageattention) of the object decoding module (subsequent decoding module), the output features of the first decoding unit (token attention) in the object decoding module are used as the key vector K and the value vector V in the cross attention mechanism, the output features of the second decoding unit (imageattention) in the previous layer of decoding module are used as the query vector Q in the cross attention mechanism, the output features of the second decoding unit (imageattention) in the object decoding module are calculated, and the output features of the second decoding unit (imageattention) in the last layer of decoding module are used as the vertebral positioning features, wherein the object decoding module is any one of the multiple decoding modules except the first layer decoding module.

[0067] In this way, the multi-layer first decoding units (token attention) continuously update and learn the cone embedding features, and the multi-layer second decoding units (image attention) continuously update and learn the image features. At the same time, the first decoding unit (token attention) and the second decoding unit (image attention) of the same layer cross-learn, that is, the cone embedding feature token embedding representing the vertebral recognition information is integrated into the image feature imageembedding, and the image feature is integrated into the cone embedding feature token embedding, thereby improving the accuracy of vertebral recognition and positioning.

[0068] Figure 4 A schematic diagram of the structure of a decoding module provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the first decoding unit (token attention) includes a cross-attention layer (cross-attention), a first normalization layer (add&norm1), a self-attention layer (self-attention), a second normalization layer (add&norm 2), a feedforward layer (FFN) and a third normalization layer (add&norm 3) stacked in sequence. Correspondingly, the second decoding unit includes a cross-attention layer (cross-attention), a first normalization layer (add&norm 1), a self-attention layer (self-attention), a second normalization layer (add&norm 2), a feedforward layer (FFN) and a third normalization layer (add&norm 3) stacked in sequence.

[0069] The processing of the decoding module after the first layer decoding module is as follows:

[0070] In the cross-attention layer of the first decoding unit (token attention), the output features of the second decoding unit (image attention) in the previous decoding module are used as the key vector K and value vector V in the cross-attention mechanism, and the output features of the first decoding unit (token attention) in the previous decoding module are used as the query vector Q in the cross-attention mechanism to calculate the output features of the cross-attention layer.

[0071] In the first normalization layer (add&norm1), the output features of the cross-attention layer (cross-attention) are concatenated with the output features of the first decoding unit (token attention) in the previous layer of decoding module, and the concatenated features are normalized to obtain the first feature.

[0072] In the self-attention layer, the first feature is processed based on the self-attention mechanism to obtain the second feature. The self-attention layer learns the information exchange between the pyramid embedding features and token embeddings based on the self-attention mechanism.

[0073] In the second normalization layer (add&norm2), the first feature is concatenated with the second feature, and the concatenated features are normalized to obtain the third feature.

[0074] In the feed-forward layer (FFN), the third feature is fully connected and mapped to obtain the fourth feature.

[0075] In the third normalization layer (add&norm3), the third feature is concatenated with the fourth feature, and the concatenated features are normalized to obtain the output features of the first decoding unit (token attention) in the target decoding module.

[0076] Similarly, in the cross-attention layer of the second decoding unit (image attention), the output features of the first decoding unit (token attention) of the same layer are used as the key vector K and value vector V in the cross-attention mechanism, and the output features of the second decoding unit (image attention) in the previous decoding module are used as the query vector Q in the cross-attention mechanism to calculate the output features of the cross-attention layer.

[0077] In the first normalization layer (add&norm1), the output features of the cross-attention layer are concatenated with the output features of the second decoding unit (image attention) in the previous layer of decoding module, and the concatenated features are normalized to obtain the fifth feature.

[0078] In the self-attention layer, the normalized features are processed based on the self-attention mechanism to obtain the sixth feature. The self-attention layer learns the information exchange between the elements in the image features based on the self-attention mechanism.

[0079] In the second normalization layer (add&norm2), the fifth feature and the sixth feature are concatenated, and the concatenated features are normalized to obtain the seventh feature.

[0080] In the feed-forward layer (FFN), the seventh feature is fully connected and mapped to obtain the eighth feature.

[0081] In the third normalization layer (add&norm3), the seventh feature is concatenated with the eighth feature, and the concatenated features are normalized to obtain the output features of the second decoding unit (image attention).

[0082] The processing process of the first-layer decoding module is similar to the processing process of the subsequent decoding modules. The difference is that the cross-attention layer of the first decoding unit (token attention) in the first-layer decoding module uses the first splicing feature as the key vector K and the value vector V in the cross-attention mechanism, and uses the second splicing feature as the query vector Q in the cross-attention mechanism; the cross-attention layer of the second decoding unit (image attention) in the first-layer decoding module uses the third splicing feature as the key vector K and the value vector V in the cross-attention mechanism, and uses the image feature image embedding as the query vector Q in the cross-attention mechanism. The embodiments of the present invention are not repeated here.

[0083] S103: predicting the probability value of each cone existing and the position information of each cone based on the vertebral body recognition features and the vertebral body positioning features.

[0084] In the embodiment of the present invention, the vertebral body identification features and vertebral body positioning features are input into the detection head for prediction, and the probability value of the existence of each cone and the position information of each cone are predicted.

[0085] In some embodiments of the present invention, the detection head fuses vertebral identification features and vertebral positioning features based on a cross-attention mechanism to obtain fused features, and predicts the probability value of the existence of each cone based on the fused features, and predicts the position information of each cone based on the fused features and vertebral positioning features.

[0086] For example, Figure 3 As shown in the figure, the detection head (head) includes an attention module (attention), a first multi-layer perceptron (MLP1), a second multi-layer perceptron (MLP2), a deconvolution layer (trans-conv) and a convolution layer (conv). The processing process of the detection head (head) is as follows:

[0087] In the attention module, the vertebral recognition features and vertebral positioning features are fused based on the attention mechanism to obtain the fused features.

[0088] The fused features are processed in the first multi-layer perceptron (MLP1) to predict the probability value S of the existence of each cone.

[0089] The fused features are processed in the second multi-layer perceptron (MLP2) to obtain perceptual features.

[0090] The cone localization features are upsampled in the deconvolution layer (trans-conv) to obtain upsampled features with the same dimension as the perceptual features.

[0091] Perform a dot multiplication operation on the upsampled feature and the perceptual feature to obtain the product feature.

[0092] The product features are convolved in the convolution layer (conv) to obtain a position heatmap representing the position information of each cone.

[0093] Exemplarily, during training, the cross entropy loss function is used to train the recognition branch, and the L2 loss function is used to train the positioning branch to achieve joint training. The label of the positioning branch uses a Gaussian function to generate a heat map to supervise the predicted position heatmap. During reasoning, the predicted position heatmap is converted into position coordinates and restored to the image coordinate system of the original spinal image to obtain the position coordinates P. Combined with the results of the recognition branch, the positioning recognition information (P, S) of each vertebra can be obtained.

[0094] In the embodiment of the present invention, the attention module in the detection head has the same structure as the first decoding unit (token attention) in the decoder. The attention module includes a cross attention layer, a first normalization layer, a self-attention layer, a second normalization layer, a feedforward layer, and a third normalization layer stacked in sequence. The processing of the attention module is as follows:

[0095] In the cross-attention layer, the vertebral positioning features are used as the key vector K and value vector V in the cross-attention mechanism, and the vertebral identification features are used as the query vector Q in the cross-attention mechanism to calculate the output features of the cross-attention layer.

[0096] In the first normalization layer, the output features of the cross attention layer are concatenated with the vertebral body recognition features, and the concatenated features are normalized to obtain the ninth feature.

[0097] In the self-attention layer, the ninth feature is processed based on the self-attention mechanism to obtain the tenth feature.

[0098] In the second normalization layer, the ninth feature and the tenth feature are concatenated, and the concatenated features are normalized to obtain the eleventh feature.

[0099] In the feed-forward layer, the eleventh feature is fully connected and mapped to obtain the twelfth feature.

[0100] In the third normalization layer, the eleventh feature and the twelfth feature are concatenated, and the concatenated features are normalized to obtain a fused feature.

[0101] The vertebral identification and positioning method provided by the embodiment of the present invention inputs the spinal image to be processed into the image encoder for processing, extracts the image features representing the vertebral information from the spinal image, inputs the image features, the cone embedding features representing the vertebral identification information and the vertebral position embedding features representing the cone position information into the decoder, the decoder fuses the image features, the cone embedding features and the vertebral position embedding features based on the cross-attention mechanism to obtain the vertebral identification features, and the decoder fuses the image features and the vertebral position embedding features based on the cross-attention mechanism to obtain the vertebral positioning features, and predicts the probability value of each cone's existence and the position information of each cone based on the vertebral identification features and the vertebral positioning features. The present invention simultaneously completes the identification and positioning of the vertebrae based on the cross-attention mechanism, innovatively regards all possible vertebrae as cone embedding features, extracts the information of the vertebral existence through the attention decoding learning method, and performs probability prediction, thereby improving the accuracy of vertebral identification, and performs position prediction after combining the image features with the cone embedding features, which greatly improves the accuracy of vertebral position prediction.

[0102] In some embodiments of the present invention, after predicting the probability value of each cone and the position information of each cone, the method further includes: correcting the predicted result to further improve the accuracy of the predicted result. Exemplarily, the correction process of the predicted result is as follows:

[0103] S104: Filter out target vertebrae whose probability values ​​are greater than a preset value from all vertebrae.

[0104] In an embodiment of the present invention, a target vertebra with a probability value greater than a preset value is selected from all predicted vertebrae, and a predicted probability value greater than the preset value indicates that the vertebra exists, and a predicted probability value greater than or equal to the preset value indicates that the vertebra does not exist. Exemplarily, in a specific embodiment of the present invention, the preset value can be set to 70%.

[0105] S105. Based on prior knowledge, correct the position information of the target vertebra so that the target vertebra conforms to the prior arrangement order and the prior vertebral spacing.

[0106] After the target vertebra actually exists, based on the prior knowledge, the position information of the target vertebra is corrected so that the target vertebra conforms to the prior arrangement order and the prior vertebral spacing. For example, the normal thoracic vertebrae are arranged in the order of T1-T12, and the spacing between the vertebrae is in the range of [dmin-dmax].

[0107] Figure 5 A prediction result correction flow chart provided by an embodiment of the present invention. Exemplarily, in some embodiments of the present invention, the above step S105 includes the following sub-steps:

[0108] 1. Sort the target vertebrae according to the prior vertebrae arrangement order to obtain the vertebrae sequence.

[0109] like Figure 5 As shown, the target vertebrae are sorted according to the prior vertebrae arrangement order to obtain the vertebrae sequence seq.

[0110] 2. Referring to the corrected position information of the target vertebra, correct the position information of the target vertebra to be corrected in descending order according to the probability value, so that the target vertebra meets the prior vertebral spacing.

[0111] Exemplarily, the target vertebra is activated in order from high to low probability values. When activated, the position information of the target vertebra is corrected so that the distance between it and the previously activated target vertebra (i.e., the target vertebra that has completed position correction) is within the distance range [dmin-dmax], and the displacement of the target vertebra is recorded to complete the activation of the target vertebra. This cycle is repeated until the position correction of all target vertebrae is completed.

[0112] 3. Determine whether there is a transitional vertebra based on the target vertebra.

[0113] Transitional vertebrae refer to the phenomenon of mutual transition at the junction of each segment of the spine, with some or all of the morphological structures having adjacent vertebrae, the total number of the entire spine remains unchanged, and the number of vertebrae in each segment increases or decreases. For example, the absence of T12 or the presence of T13 / L6 will lead to errors in the vertebral number based on sequential inference. In the embodiment of the present invention, the cone embedding feature token embedding already includes the possible occurrence of transitional vertebrae T13 and L6, so the probability value of the transitional vertebra will also be predicted in the prediction result.

[0114] 4. If it exists, add or reduce the corresponding vertebra at the corresponding position of the transitional vertebra.

[0115] If a transitional vertebra appears in the prediction result, the corresponding vertebra is added or subtracted at the position corresponding to the transitional vertebra. Figure 5 As shown, if transitional vertebra T13 appears, T13 will be added to the vertebral sequence (highlighted).

[0116] 5. Correct the positions of all target vertebrae uniformly to correct the overall movement dislocation that occurs during the process of correcting the positions of the target vertebrae one by one.

[0117] After correcting the target vertebrae one by one, the positions of all the target vertebrae are further corrected uniformly to correct the overall movement dislocation that occurs during the process of correcting the positions of the target vertebrae one by one.

[0118] The embodiment of the present invention corrects the prediction results based on prior knowledge. The model prediction is based on data-driven inference. When a certain paradigm does not exist or exists very rarely in the training data, the reasoning will be wrong. The prediction results are further optimized and corrected for rationality to meet the requirements of medical anatomical consistency, thereby enhancing the practicality and accuracy of the prediction results.

[0119] In some embodiments of the present invention, after the prediction result is corrected, the vertebral body can be individually segmented based on the prediction result and annotated into the original spinal image so that the user (doctor or researcher) can view and study the individual vertebral body. Exemplarily, after the prediction result is corrected, the following steps are further included:

[0120] S106. Based on the position information of each cone, segment the area where a single cone is located from the spinal column image.

[0121] After the prediction result is corrected, the region where a single target cone is located is segmented from the spinal column image based on the corrected position information of the target vertebra, that is, the region where each target cone is located is segmented from the spinal column image.

[0122] Figure 6 A schematic diagram of a single vertebra segmentation provided by an embodiment of the present invention. Exemplarily, the above step S106 includes the following sub-steps:

[0123] 1. Extract the region of interest where a single cone is located from the preprocessed image of the spine image.

[0124] like Figure 6 As shown, in this embodiment of the present invention, for each target vertebra, a region of interest including the target vertebra is cropped from the pre-processed image image_ct in the aforementioned embodiment.

[0125] 2. Resample the region of interest to obtain a resampled image block.

[0126] Exemplarily, the captured region of interest is resampled to obtain a 3D image block with a resolution of [1, 1, 1].

[0127] 2. Input the position information of the image block and the cone into the image segmentation network for binary segmentation, and segment the area where the cone is located from the background of the image block.

[0128] like Figure 6 As shown, the image block and the position heatmap representing the position information of the target vertebra are spliced ​​in the channel dimension, and the position heatmap indicates the vertebra that needs to be segmented. Then, the spliced ​​features are input into the image segmentation network for binary segmentation, and the area where the target cone is located is segmented from the background of the image block. Exemplarily, in a specific embodiment of the present invention, the image segmentation network can be 3D Unet ( Figure 6 Vertebrae-Unet in ).

[0129] S107, scaling the area where each single cone is located back to the spinal column image, and assigning the cone a corresponding vertebral body identification.

[0130] Exemplarily, the area where each individual target cone is located is zoomed back into the spinal image, and the target cone is assigned a corresponding vertebral body identifier (e.g., the number of the vertebral body). In this way, the user (doctor or researcher) can view and study the individual vertebral bodies. For example, when the user clicks on a vertebral body, the vertebral body is highlighted and the number of the vertebral body is displayed.

[0131] The embodiment of the present invention further provides a vertebral body identification and positioning device, Figure 7 A schematic diagram of the structure of a vertebral body identification and positioning device provided by an embodiment of the present invention is shown in FIG. Figure 7 As shown, the vertebral body identification and positioning device includes:

[0132] An image feature extraction module 201 is used to input the spinal image to be processed into an image encoder for processing, and to extract image features representing vertebral information from the spinal image;

[0133] The feature processing module 202 is used to input the image feature, the cone embedding feature representing the vertebral identification information, and the vertebral position embedding feature representing the cone position information into a decoder, wherein the decoder fuses the image feature, the cone embedding feature, and the vertebral position embedding feature based on a cross-attention mechanism to obtain a vertebral identification feature, and the decoder fuses the image feature and the vertebral position embedding feature based on a cross-attention mechanism to obtain a vertebral positioning feature;

[0134] The cone prediction module 203 is used to predict the probability value of the existence of each cone and the position information of each cone based on the vertebral identification feature and the vertebral positioning feature.

[0135] In some embodiments of the present invention, the image feature extraction module 201 includes:

[0136] A target image extraction submodule is used to extract image information of the area where the vertebral body is located from the spinal column image to be processed to obtain a target image;

[0137] The image feature extraction submodule is used to input the target image into an image encoder for feature extraction, so as to obtain image features representing vertebral information.

[0138] In some embodiments of the present invention, the target image extraction submodule includes:

[0139] A preprocessing unit, used for preprocessing the spinal column image to be processed to obtain a preprocessed image including a cone;

[0140] An image segmentation unit, used for segmenting the preprocessed image into a plurality of sub-images of preset sizes;

[0141] An image segmentation unit, used for segmenting the area where the cone is located from the sub-image to obtain a sub-segmented image;

[0142] An image stitching unit, used for stitching the sub-segmented images of the plurality of sub-images into a segmented image;

[0143] A dimension splicing unit is used to splice the preprocessed image and the segmented image in the channel dimension to obtain a target image.

[0144] In some embodiments of the present invention, the pre-processing unit comprises:

[0145] A resampling subunit, used for resampling the spinal column image to be processed to obtain a resampled image with a preset resolution;

[0146] The normalization subunit is used to normalize the CT value of the resampled image to obtain a preprocessed image including a cone.

[0147] In some embodiments of the present invention, the image segmentation unit includes:

[0148] The image segmentation subunit is used to input the sub-image into the image segmentation network for binary segmentation, and segment the area where the cone is located from the background of the sub-image to obtain a sub-segmented image.

[0149] In some embodiments of the present invention, the image encoder includes a three-stage residual module, and the image feature extraction submodule includes:

[0150] A first residual processing unit, used for inputting the target image into the residual module of the first stage for processing, to obtain a first feature map with a resolution half of the target image;

[0151] A second residual processing unit, used for inputting the first feature map into a residual module of a second stage for processing, so as to obtain a second feature map having a resolution half of the first feature map;

[0152] The third residual processing unit is used to input the second feature map into the residual module of the third stage for processing to obtain image features with a resolution half of the second feature map.

[0153] In some embodiments of the present invention, the decoder includes multiple layers of decoding modules stacked in sequence, the decoding module includes a first decoding unit and a second decoding unit, and the feature processing module 202 includes:

[0154] A first splicing submodule, configured to splice the image feature with an image position embedding feature representing position information of each element in the image feature to obtain a first splicing feature;

[0155] A second splicing submodule is used to splice the cone embedding feature and the vertebral position embedding feature to obtain a second splicing feature;

[0156] A first cross attention processing submodule, configured to calculate, in a first decoding unit of a first layer decoding module, an output feature of a first decoding unit in the first layer decoding module by using the first concatenated feature as a key vector and a value vector in a cross attention mechanism and using the second concatenated feature as a query vector in the cross attention mechanism;

[0157] The second cross-attention processing submodule is used to use the output features of the second decoding unit in the previous layer of decoding modules as the key vector and value vector in the cross-attention mechanism in the first decoding unit of the target decoding module, the output features of the first decoding unit in the previous layer of decoding modules as the query vector in the cross-attention mechanism, calculate the output features of the first decoding unit in the target decoding module, and use the output features of the first decoding unit in the last layer of decoding modules as the vertebral recognition features, wherein the target decoding module is any one of the multiple decoding modules except the first layer of decoding modules.

[0158] In some embodiments of the present invention, the first decoding unit includes a cross attention layer, a first normalization layer, a self-attention layer, a second normalization layer, a feedforward layer and a third normalization layer stacked in sequence, and the second cross attention processing submodule includes:

[0159] A first cross-attention processing unit is used to calculate, in a cross-attention layer of the first decoding unit, an output feature of the second decoding unit in a previous decoding module as a key vector and a value vector in a cross-attention mechanism, and an output feature of the first decoding unit in a previous decoding module as a query vector in the cross-attention mechanism, to calculate an output feature of the cross-attention layer;

[0160] A first normalization processing unit, used for, in the first normalization layer, concatenating the output features of the cross attention layer with the output features of the first decoding unit in the previous layer decoding module, and performing normalization processing on the concatenated features to obtain a first feature;

[0161] A first self-attention processing unit, configured to process the first feature in a self-attention layer based on a self-attention mechanism to obtain a second feature;

[0162] a second normalization processing unit, configured to concatenate the first feature with the second feature in a second normalization layer, and perform normalization processing on the concatenated features to obtain a third feature;

[0163] A first fully connected processing unit, configured to perform fully connected mapping on the third feature in a feed-forward layer to obtain a fourth feature;

[0164] The third normalization processing unit is used to concatenate the third feature with the fourth feature in a third normalization layer, and perform normalization processing on the concatenated features to obtain output features of the first decoding unit in the target decoding module.

[0165] In some embodiments of the present invention, the feature processing module 202 further includes:

[0166] A third splicing submodule, used for splicing the output feature of the first decoding unit of the first layer decoding module with the vertebral position embedding feature to obtain a third splicing feature;

[0167] A third cross attention processing submodule is used to calculate the output feature of the second decoding unit in the first layer decoding module in the second decoding unit of the first layer decoding module by using the third splicing feature as a key vector and a value vector in the cross attention mechanism and the image feature as a query vector in the cross attention mechanism;

[0168] The fourth cross-attention processing submodule is used to use the output features of the first decoding unit in the object decoding module as the key vector and value vector in the cross-attention mechanism in the second decoding unit of the object decoding module, use the output features of the second decoding unit in the previous layer decoding module as the query vector in the cross-attention mechanism, calculate the output features of the second decoding unit in the object decoding module, and use the output features of the second decoding unit in the last layer decoding module as the vertebral positioning feature, wherein the object decoding module is any one of the multiple decoding modules except the first layer decoding module.

[0169] In some embodiments of the present invention, the second decoding unit includes a cross attention layer, a first normalization layer, a self-attention layer, a second normalization layer, a feedforward layer and a third normalization layer stacked in sequence, and the fourth cross attention processing submodule includes:

[0170] A second cross-attention processing unit is used to calculate the output features of the cross-attention layer of the second decoding unit by using the output features of the first decoding unit in the object decoding module as the key vector and the value vector in the cross-attention mechanism and the output features of the second decoding unit in the previous layer of decoding modules as the query vector in the cross-attention mechanism;

[0171] A fourth normalization processing unit is used to concatenate the output features of the cross attention layer with the output features of the second decoding unit in the previous layer decoding module in the first normalization layer, and perform normalization processing on the concatenated features to obtain a fifth feature;

[0172] A second self-attention processing unit, configured to process the normalized feature based on a self-attention mechanism in a self-attention layer to obtain a sixth feature;

[0173] a fifth normalization processing unit, configured to concatenate the fifth feature with the sixth feature in a second normalization layer, and perform normalization processing on the concatenated features to obtain a seventh feature;

[0174] A second fully connected processing unit is used to perform fully connected mapping on the seventh feature in a feed-forward layer to obtain an eighth feature;

[0175] The sixth normalization processing unit is used to concatenate the seventh feature with the eighth feature in the third normalization layer, and perform normalization processing on the concatenated features to obtain the output features of the second decoding unit in the object decoding module.

[0176] In some embodiments of the present invention, the cone prediction module 203 includes:

[0177] The cone prediction submodule is used to input the vertebral identification features and the vertebral positioning features into the detection head. The detection head fuses the vertebral identification features and the vertebral positioning features based on the attention mechanism to obtain fused features, and predicts the probability value of the existence of each cone based on the fused features, and predicts the position information of each cone based on the fused features and the vertebral positioning features.

[0178] In some embodiments of the present invention, the detection head includes an attention module, a first multi-layer perceptron, a second multi-layer perceptron, a deconvolution layer and a convolution layer, and the cone prediction submodule includes:

[0179] An attention processing unit, configured to fuse the vertebral identification feature and the vertebral positioning feature based on an attention mechanism in an attention module to obtain a fused feature;

[0180] A first multi-layer perception unit is used to process the fusion features in a first multi-layer perception machine to predict the probability value of the existence of each of the cones;

[0181] A second multi-layer perception unit is used to process the fused features in a second multi-layer perception machine to obtain perception features;

[0182] An upsampling unit, used for upsampling the vertebral positioning feature in the deconvolution layer to obtain an upsampled feature;

[0183] A dot multiplication operation unit, used for performing a dot multiplication operation on the up-sampled feature and the perceptual feature to obtain a product feature;

[0184] A convolution processing unit is used to perform convolution processing on the product features in the convolution layer to obtain a position heat map representing the position information of each of the cones.

[0185] In some embodiments of the present invention, the attention module includes a cross attention layer, a first normalization layer, a self-attention layer, a second normalization layer, a feedforward layer and a third normalization layer stacked in sequence, and the attention processing unit includes:

[0186] A cross-attention processing subunit, used to calculate the output features of the cross-attention layer in the cross-attention layer by using the vertebral positioning features as the key vector and the value vector in the cross-attention mechanism and using the vertebral identification features as the query vector in the cross-attention mechanism;

[0187] a first normalization subunit, configured to concatenate the output features of the cross attention layer with the vertebral body recognition features in the first normalization layer, and perform normalization processing on the concatenated features to obtain a ninth feature;

[0188] A self-attention processing subunit, configured to process the ninth feature based on a self-attention mechanism in a self-attention layer to obtain a tenth feature;

[0189] a second normalization subunit, configured to concatenate the ninth feature with the tenth feature in a second normalization layer, and perform normalization processing on the concatenated features to obtain an eleventh feature;

[0190] A fully connected processing subunit, used for performing fully connected mapping on the eleventh feature in a feed-forward layer to obtain a twelfth feature;

[0191] The third normalization subunit is used to splice the eleventh feature with the twelfth feature in a third normalization layer, and normalize the spliced ​​features to obtain a fused feature.

[0192] In some embodiments of the present invention, the vertebral body identification and positioning device further comprises:

[0193] A target vertebra screening module, configured to screen out target vertebrae with a probability value greater than a preset value from all vertebrae after predicting the probability value of the existence of each of the cones and the position information of each of the cones based on the vertebrae identification features and the vertebrae positioning features;

[0194] The position correction module is used to correct the position information of the target vertebra based on prior knowledge, so that the target vertebra conforms to the prior arrangement order and the prior vertebral spacing.

[0195] In some embodiments of the present invention, the position correction module includes:

[0196] The vertebral body sorting submodule is used to sort the target vertebral bodies according to the prior vertebral body arrangement order to obtain a vertebral body sequence;

[0197] The position correction submodule is used to refer to the position information of the target vertebra that has been corrected, and correct the position information of the target vertebra to be corrected in descending order according to the probability value, so that the target vertebra meets the prior vertebral spacing;

[0198] A transitional vertebra judgment submodule, used for judging whether there is a transitional vertebra based on the target vertebra;

[0199] The transitional vertebra correction submodule is used to increase or decrease the corresponding vertebral body at the position corresponding to the transitional vertebra when there is a transitional vertebra;

[0200] The unified correction submodule is used to uniformly correct the positions of all target vertebrae to correct the overall movement dislocation that occurs during the process of correcting the positions of the target vertebrae one by one.

[0201] In some embodiments of the present invention, the vertebral body identification and positioning device further comprises:

[0202] A single vertebra segmentation module is used to segment the area where a single cone is located from the spinal column image based on the position information of each cone after predicting the probability value of the existence of each cone and the position information of each cone based on the vertebra identification feature and the vertebra positioning feature;

[0203] The vertebral identification module is used to scale the area where each single cone is located back to the spinal column image and assign the corresponding vertebral identification to the cone.

[0204] In some embodiments of the present invention, the single vertebra segmentation module includes:

[0205] A region of interest interception submodule, used for intercepting a region of interest where a single cone is located from a preprocessed image of the spinal column image;

[0206] A resampling submodule, used for resampling the region of interest to obtain a resampled image block;

[0207] The single cone segmentation submodule is used to input the position information of the image block and the cone into the image segmentation network for binary segmentation, and segment the area where the cone is located from the background of the image block.

[0208] The above-mentioned vertebral body identification and positioning device can execute the vertebral body identification and positioning method provided by the aforementioned embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the vertebral body identification and positioning method.

[0209] Figure 8A schematic diagram of the structure of an electronic device provided for an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0210] like Figure 8 As shown, the electronic device includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0211] A number of components in the electronic device are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0212] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a vertebral identification and positioning method.

[0213] In some embodiments, the vertebral identification and positioning method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the vertebral identification and positioning method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the vertebral identification and positioning method in any other appropriate manner (for example, by means of firmware).

[0214] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0215] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0216] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0217] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0218] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0219] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0220] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the vertebral identification and positioning method provided in any embodiment of the present application.

[0221] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0222] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0223] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A vertebral body identification and positioning method, characterized in that: include: Inputting the spinal column image to be processed into an image encoder for processing, and extracting image features representing vertebral information from the spinal column image; The image features, the cone embedding features representing the vertebral identification information, and the vertebral position embedding features representing the cone position information are input into a decoder, the decoder fuses the image features, the cone embedding features, and the vertebral position embedding features based on a cross-attention mechanism to obtain a vertebral identification feature, and the decoder fuses the image features and the vertebral position embedding features based on a cross-attention mechanism to obtain a vertebral positioning feature; The probability value of the existence of each of the cones and the position information of each of the cones are predicted based on the vertebral identification features and the vertebral positioning features.

2. The vertebral body identification and positioning method according to claim 1, characterized in that: Image features representing vertebral information are extracted from the spinal image to be processed, including: Extracting image information of the area where the vertebral body is located from the spinal column image to be processed to obtain a target image; The target image is input into an image encoder for feature extraction to obtain image features representing vertebral information.

3. The vertebral body identification and positioning method according to claim 2, characterized in that: Extracting the image information of the vertebral area from the spinal image to be processed to obtain the target image includes: Preprocessing the spinal column image to be processed to obtain a preprocessed image including a cone; Dividing the preprocessed image into a plurality of sub-images of preset sizes; Segmenting the area where the cone is located from the sub-image to obtain a sub-segmented image; splicing the sub-segmented images of the plurality of sub-images into a segmented image; The preprocessed image and the segmented image are concatenated in the channel dimension to obtain a target image.

4. The vertebral body identification and positioning method according to claim 3, characterized in that: The spinal column image to be processed is preprocessed to obtain a preprocessed image including a cone, including: Resampling the spinal column image to be processed to obtain a resampled image with a preset resolution; The CT value of the resampled image is normalized to obtain a preprocessed image including a cone.

5. The vertebral body identification and positioning method according to claim 3, characterized in that: Segmenting the area where the cone is located from the sub-image to obtain a sub-segmented image, including: The sub-image is input into an image segmentation network for binary segmentation, and the area where the cone is located is segmented from the background of the sub-image to obtain a sub-segmented image.

6. The vertebral body identification and positioning method according to claim 2, characterized in that: The image encoder includes a three-stage residual module, and the target image is input into the image encoder for feature extraction to obtain image features representing vertebral information, including: Inputting the target image into the residual module of the first stage for processing to obtain a first feature map with a resolution half of the target image; Inputting the first feature map into the residual module of the second stage for processing to obtain a second feature map having a resolution half of the first feature map; The second feature map is input into the residual module of the third stage for processing to obtain image features with a resolution half of the second feature map.

7. The vertebral body identification and positioning method according to any one of claims 1 to 6, characterized in that: The decoder includes multiple layers of sequentially stacked decoding modules, the decoding module includes a first decoding unit and a second decoding unit, and the decoder fuses the image features, the cone embedding features and the vertebral position embedding features based on a cross attention mechanism to obtain vertebral recognition features, including: splicing the image feature with an image position embedding feature representing position information of each element in the image feature to obtain a first splicing feature; Splicing the cone embedding feature and the vertebral position embedding feature to obtain a second splicing feature; In a first decoding unit of a first layer decoding module, the first concatenated feature is used as a key vector and a value vector in a cross attention mechanism, and the second concatenated feature is used as a query vector in the cross attention mechanism to calculate an output feature of the first decoding unit in the first layer decoding module; In the first decoding unit of the target decoding module, the output features of the second decoding unit in the previous layer decoding module are used as the key vector and the value vector in the cross attention mechanism, the output features of the first decoding unit in the previous layer decoding module are used as the query vector in the cross attention mechanism, the output features of the first decoding unit in the target decoding module are calculated, and the output features of the first decoding unit in the last layer decoding module are used as the vertebral recognition features, wherein the target decoding module is any one of the multiple decoding modules except the first layer decoding module.

8. The vertebral body identification and positioning method according to claim 7, characterized in that: The first decoding unit includes a cross attention layer, a first normalization layer, a self-attention layer, a second normalization layer, a feedforward layer, and a third normalization layer stacked in sequence. In the first decoding unit of the target decoding module, the output features of the second decoding unit in the previous decoding module are used as the key vector and the value vector in the cross attention mechanism, and the output features of the first decoding unit in the previous decoding module are used as the query vector in the cross attention mechanism. The output features of the first decoding unit in the target decoding module are calculated, including: In the cross-attention layer of the first decoding unit, the output features of the second decoding unit in the previous decoding module are used as the key vector and value vector in the cross-attention mechanism, and the output features of the first decoding unit in the previous decoding module are used as the query vector in the cross-attention mechanism to calculate the output features of the cross-attention layer; In the first normalization layer, the output features of the cross attention layer are concatenated with the output features of the first decoding unit in the previous layer of decoding module, and the concatenated features are normalized to obtain the first features; In the self-attention layer, the first feature is processed based on the self-attention mechanism to obtain the second feature; In a second normalization layer, the first feature and the second feature are concatenated, and the concatenated features are normalized to obtain a third feature; In the feed-forward layer, the third feature is fully connected and mapped to obtain a fourth feature; In the third normalization layer, the third feature is concatenated with the fourth feature, and the concatenated features are normalized to obtain the output features of the first decoding unit in the target decoding module.

9. The vertebral body identification and positioning method according to claim 7, characterized in that: The decoder fuses the image features and the vertebral position embedding features based on the cross attention mechanism to obtain vertebral positioning features, including: splicing the output feature of the first decoding unit of the first layer decoding module with the vertebral position embedding feature to obtain a third splicing feature; In a second decoding unit of the first layer decoding module, the third concatenated feature is used as a key vector and a value vector in a cross attention mechanism, and the image feature is used as a query vector in the cross attention mechanism to calculate an output feature of the second decoding unit in the first layer decoding module; In the second decoding unit of the object decoding module, the output features of the first decoding unit in the object decoding module are used as the key vector and the value vector in the cross-attention mechanism, the output features of the second decoding unit in the previous layer of decoding module are used as the query vector in the cross-attention mechanism, the output features of the second decoding unit in the object decoding module are calculated, and the output features of the second decoding unit in the last layer of decoding module are used as the vertebral positioning features, wherein the object decoding module is any one of the multiple decoding modules except the first layer decoding module.

10. The vertebral body identification and positioning method according to claim 9, characterized in that: The second decoding unit includes a cross attention layer, a first normalization layer, a self-attention layer, a second normalization layer, a feedforward layer, and a third normalization layer stacked in sequence. In the second decoding unit of the object decoding module, the output features of the first decoding unit in the object decoding module are used as the key vector and the value vector in the cross attention mechanism, and the output features of the second decoding unit in the previous decoding module are used as the query vector in the cross attention mechanism. The output features of the second decoding unit in the object decoding module are calculated, including: In the cross-attention layer of the second decoding unit, the output features of the first decoding unit in the object decoding module are used as the key vector and value vector in the cross-attention mechanism, and the output features of the second decoding unit in the previous layer of decoding modules are used as the query vector in the cross-attention mechanism to calculate the output features of the cross-attention layer; In the first normalization layer, the output features of the cross attention layer are concatenated with the output features of the second decoding unit in the previous layer decoding module, and the concatenated features are normalized to obtain the fifth features; In the self-attention layer, the normalized feature is processed based on the self-attention mechanism to obtain a sixth feature; In the second normalization layer, the fifth feature and the sixth feature are concatenated, and the concatenated features are normalized to obtain a seventh feature; In the feed-forward layer, the seventh feature is fully connected and mapped to obtain an eighth feature; In the third normalization layer, the seventh feature and the eighth feature are concatenated, and the concatenated features are normalized to obtain the output features of the second decoding unit in the object decoding module.

11. The vertebral body identification and positioning method according to any one of claims 1-6, 8-10, characterized in that: Predicting the probability value of the existence of each of the cones and the position information of each of the cones based on the vertebral identification features and the vertebral positioning features includes: The vertebral identification features and the vertebral positioning features are input into the detection head, and the detection head fuses the vertebral identification features and the vertebral positioning features based on the attention mechanism to obtain fused features, and predicts the probability value of the existence of each cone based on the fused features, and predicts the position information of each cone based on the fused features and the vertebral positioning features.

12. The vertebral body identification and positioning method according to claim 11, characterized in that: The detection head includes an attention module, a first multi-layer perceptron, a second multi-layer perceptron, a deconvolution layer and a convolution layer. The detection head fuses the vertebral identification features and the vertebral positioning features based on the attention mechanism to obtain fused features, and predicts the probability value of the existence of each of the cones based on the fused features, and predicts the position information of each of the cones based on the fused features and the vertebral positioning features, including: In the attention module, the vertebral body recognition feature and the vertebral body positioning feature are fused based on the attention mechanism to obtain a fused feature; Processing the fused features in a first multi-layer perceptron to predict the probability value of the existence of each of the cones; Processing the fused features in a second multi-layer perceptron to obtain perceptual features; Upsampling the vertebral positioning features in a deconvolution layer to obtain upsampled features; Performing a dot multiplication operation on the up-sampled feature and the perception feature to obtain a product feature; The product features are convolved in the convolution layer to obtain a position heat map representing the position information of each of the cones.

13. The vertebral body identification and positioning method according to claim 12, characterized in that: The attention module includes a cross attention layer, a first normalization layer, a self-attention layer, a second normalization layer, a feedforward layer and a third normalization layer stacked in sequence. In the attention module, the vertebral recognition feature and the vertebral positioning feature are fused based on the attention mechanism to obtain a fused feature, including: In the cross-attention layer, the vertebral positioning feature is used as a key vector and a value vector in the cross-attention mechanism, and the vertebral identification feature is used as a query vector in the cross-attention mechanism to calculate the output feature of the cross-attention layer; In the first normalization layer, the output feature of the cross attention layer is concatenated with the vertebral body recognition feature, and the concatenated feature is normalized to obtain the ninth feature; In the self-attention layer, the ninth feature is processed based on the self-attention mechanism to obtain a tenth feature; In a second normalization layer, the ninth feature and the tenth feature are concatenated, and the concatenated features are normalized to obtain an eleventh feature; In the feed-forward layer, the eleventh feature is fully connected and mapped to obtain the twelfth feature; In the third normalization layer, the eleventh feature and the twelfth feature are concatenated, and the concatenated features are normalized to obtain a fused feature.

14. The vertebral body identification and positioning method according to any one of claims 1-6, 8-10, 12, and 13, characterized in that: After predicting the probability value of the existence of each cone and the position information of each cone based on the vertebral body identification feature and the vertebral body positioning feature, the method further includes: Filter out target vertebrae whose probability values ​​are greater than a preset value from all vertebrae; Based on prior knowledge, the position information of the target vertebra is corrected so that the target vertebra conforms to the prior arrangement order and the prior vertebral spacing.

15. The vertebral body identification and positioning method according to claim 14, characterized in that: Based on prior knowledge, the position information of the target vertebra is corrected so that the target vertebra conforms to the prior arrangement order and the prior vertebral spacing, including: Sort the target vertebrae according to the prior vertebrae arrangement order to obtain a vertebrae sequence; Referring to the corrected position information of the target vertebra, the position information of the target vertebra to be corrected is corrected in descending order according to the probability values, so that the target vertebra meets the prior vertebral spacing; Determining whether there is a transitional vertebra based on the target vertebra; If it exists, the corresponding vertebra is added or reduced at the position corresponding to the transitional vertebra; The positions of all target vertebrae are uniformly corrected to correct the overall movement dislocation that occurs during the process of correcting the positions of the target vertebrae one by one.

16. The vertebral body identification and positioning method according to any one of claims 1-6, 8-10, 12, 13, and 15, characterized in that: After predicting the probability value of the existence of each cone and the position information of each cone based on the vertebral body identification feature and the vertebral body positioning feature, the method further includes: Based on the position information of each of the cones, segmenting the area where a single cone is located from the spinal column image; The region where each individual cone is located is scaled back into the spinal column image, and the cone is assigned a corresponding vertebral body label.

17. The vertebral body identification and positioning method according to any one of claim 16, characterized in that: Based on the position information of each of the cones, segmenting the area where a single cone is located from the spinal column image includes: Cutting out a region of interest where a single cone is located from a preprocessed image of the spinal column image; Resampling the region of interest to obtain a resampled image block; The position information of the image block and the cone is input into an image segmentation network for binary segmentation, and the area where the cone is located is segmented from the background of the image block.

18. A vertebral body identification and positioning device, characterized in that: include: An image feature extraction module is used to input the spinal image to be processed into the image encoder for processing, and extract image features representing vertebral information from the spinal image; a feature processing module, configured to input the image features, the cone embedding features representing the vertebral identification information, and the vertebral position embedding features representing the cone position information into a decoder, wherein the decoder fuses the image features, the cone embedding features, and the vertebral position embedding features based on a cross-attention mechanism to obtain a vertebral identification feature, and the decoder fuses the image features and the vertebral position embedding features based on a cross-attention mechanism to obtain a vertebral positioning feature; The cone prediction module is used to predict the probability value of the existence of each cone and the position information of each cone based on the vertebral identification feature and the vertebral positioning feature.

19. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the vertebral identification and positioning method as described in any one of claims 1-17.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the vertebral body identification and positioning method as described in any one of claims 1 to 17 is implemented.

Images