Spine image recognition method and related equipment
By directly outputting the positioning frame and vertebral information through an improved key point recognition model and combining it with the CBAM module to enhance feature capture, the problems of poor robustness and low efficiency of intervertebral disc positioning and vertebral identification in the existing technology are solved, and efficient and accurate intervertebral disc positioning and vertebral identification are achieved.
Patent Information
- Application Number
- CN202510721537.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-30
- Publication Date
- 2025-10-14
AI Technical Summary
Existing intervertebral disc localization and vertebrae identification methods rely on segmentation models and complex post-processing algorithms. They have poor robustness, difficulty in handling mild lesions, and are inefficient, unable to meet the needs of large-scale data processing.
Using an improved key point recognition model (such as YOLOv8-Pose with CBAM), without relying on a segmentation model, it directly outputs the positioning box, vertebral name, and corner coordinates. Combined with the CBAM module, the model captures vertebral regional features, simplifies the process, and improves detection speed and robustness.
The robustness and detection speed of intervertebral disc positioning and vertebral identification are improved, especially for lesion images, which solves the problem of poor robustness of traditional automatic methods.
Smart Images

Figure CN120782706A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer and communication technology, and in particular to a spinal image recognition method and related equipment. Background Art
[0002] Spinal diseases (such as herniated discs, degenerative lesions, and spinal stenosis) are common clinical conditions with high morbidity, severely impacting patients' quality of life. Disc localization and vertebral identification are two crucial steps in the examination and diagnosis process, directly impacting the accuracy of disease assessment, the rationality of surgical planning, and the effectiveness of postoperative follow-up.
[0003] Currently, methods for intervertebral disc localization and vertebral identification are divided into two categories: manual and automatic. Traditional manual methods require experienced technicians and have problems such as low efficiency and significant influence from subjective factors, making them difficult to meet the needs of large-scale data processing and analysis. Automatic methods are auxiliary positioning methods based on artificial intelligence. Such methods can significantly reduce time consumption, but they often rely on segmentation models and complex post-processing processes. Such methods have poor robustness and may not be able to process data from some minor lesions (such as compression fractures). Therefore, a highly robust and generalizable method that does not rely on excessive post-processing algorithms is needed. Summary of the Invention
[0004] The embodiments of the present application provide a spine image recognition method and related equipment, which do not rely on segmentation models and complex post-processing algorithms, have low labeling costs and high robust performance.
[0005] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.
[0006] According to one aspect of an embodiment of the present application, a spinal image recognition method is provided, comprising: acquiring a spinal image to be tested, preprocessing the spinal image to be tested, and obtaining a target spinal image; inputting the target spinal image into a key point recognition model, and obtaining position information output by the key point recognition model for each vertebra in the target spinal image, wherein the position information is marked on the target spinal image and includes at least positioning frame information, vertebral name, and corner point coordinates; correcting and adjusting the position information to obtain corrected position information; and performing intervertebral disc positioning and vertebra identification according to the corrected position information.
[0007] According to an aspect of some embodiments of the present application, a spinal image recognition device is provided, which comprises: a spine image acquisition module, configured to acquire a to-be-tested spine image, and to pre-process the to-be-tested spine image to obtain a target spine image; a spine image input module, configured to input the target spine image into a key point recognition model to obtain position information output by the key point recognition model for each vertebral body in the target spine image, the position information being marked on the target spine image and comprising at least positioning box information, a vertebral body name and corner point coordinates; a spine image adjustment module, configured to correct and adjust the position information to obtain corrected position information; and a spine image identification module, configured to perform intervertebral disc positioning and vertebral identification according to the corrected position information.
[0008] According to an aspect of some embodiments of the present application, a computer readable medium having a computer program stored thereon is provided, the computer program being executed by a processor to implement the spinal image recognition method as described in the above embodiments.
[0009] According to an aspect of some embodiments of the present application, an electronic device is provided, which comprises: one or more processors; and a storage device configured to store one or more programs, the one or more programs being executed by the one or more processors to cause the one or more processors to implement the spinal image recognition method as described in the above embodiments.
[0010] According to an aspect of some embodiments of the present application, a computer program product is provided, which comprises one or more computer programs, the one or more computer programs being executed by one or more processors to implement the steps of the spinal image recognition method as described in the above embodiments.
[0011] In the technical solutions provided in some embodiments of the present application, an improved key point recognition model (such as YOLOv8-Pose containing CBAM) is adopted, without relying on a segmentation model, to directly output a positioning box, a vertebral body name and corner point coordinates, thereby simplifying the process and improving the detection speed and robustness. The CBAM module enhances the capture of vertebral region features by the model and suppresses background interference, especially for diseased images, thereby achieving better results and solving the problem of poor robustness of traditional automatic methods.
[0012] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present application. BRIEF DESCRIPTION OF DRAWINGS
[0013] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments that conform to the present application and, together with the description, further serve to explain the principles of the present application. It is to be noted, as
[0014] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied is shown.
[0015] Figure 2 A flowchart of a spine image recognition method provided by an embodiment of the present application is shown.
[0016] Figure 3 A flowchart of a specific implementation of step S200 in the spine image recognition method shown in the corresponding embodiment is shown. Figure 2 A flowchart of a specific implementation of step S200 in the spine image recognition method shown in the corresponding embodiment is shown.
[0017] Figure 4 A structure diagram of a key point recognition model provided by an embodiment of the present application is shown.
[0018] Figure 5 A structure diagram of a CBAM module provided by an embodiment of the present application is shown.
[0019] Figure 6 A flowchart of a specific implementation of step S300 in the spine image recognition method shown in the corresponding embodiment is shown. Figure 2 A flowchart of a specific implementation of step S300 in the spine image recognition method shown in the corresponding embodiment is shown.
[0020] Figure 7 A structure diagram of a spine image recognition device provided by an embodiment of the present application is shown.
[0021] Figure 8 A structure diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION
[0022] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art.
[0023] Moreover, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures, devices, implementations, and operations have not been shown or described in detail to avoid obscuring aspects of the application.
[0024] The block diagrams in the drawings show only the functionality of the features and can not imply that the functionality must be implemented in the order shown. In some embodiments, functional elements can be implemented in hardware, software, or a combination of both hardware and software. Further, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures, devices, implementations, and operations have not been shown or described in detail to avoid obscuring aspects of the application.
[0025] The flow diagrams in the drawings show example processes. The processes can not necessarily be performed in the order shown, and can include additional or fewer steps than those shown. For example, some steps can be performed in parallel or in a different order. In some embodiments, the processes can be performed by hardware, software, or a combination of both hardware and software.
[0026] Figure 1 A schematic diagram of an example system architecture to which the techniques of embodiments of the application can be applied is shown.
[0027] As shown in Figure 1 , the system architecture can include one or more of a terminal device (such as a smartphone 101, a tablet computer 102, and a portable computer 103 as shown in Figure 1 , of course a desktop computer, and so on), a network 104, and a server 105. The network 104 is a medium to provide a communication link between the terminal device and the server 105. The network 104 can include various connection types, such as a wired communication link, a wireless communication link, and so on.
[0028] It should be understood that the number of terminal devices, networks, and servers in Figure 1 is merely illustrative. Any number of terminal devices, networks, and servers can be present according to implementation needs. For example, the server 105 can be a server cluster composed of multiple servers, and so on.
[0029] The user can use the terminal device to interact with the server 105 through the network 104 to receive or send messages, etc. The server 105 can be a server that provides various services. For example, the user uploads a to-be-tested spine image to the server 105 by using the terminal device 103 (which can also be the terminal device 101 or 102), the server 105 can pre-process the to-be-tested spine image to obtain a target spine image; input the target spine image into a key point recognition model to obtain position information output by the key point recognition model for each vertebral body in the target spine image, the position information is marked on the target spine image, and at least includes positioning box information, vertebral body name and corner point coordinates, the key point recognition model includes a CBAM module; correct and adjust the position information to obtain corrected position information; and perform intervertebral disc positioning and vertebral identification according to the corrected position information.
[0030] It should be noted that the spine image recognition method provided in the embodiments of the present application is generally executed by the server 105, and correspondingly, the spine image recognition device is generally arranged in the server 105. However, in other embodiments of the present application, the terminal device can also have similar functions as the server, so as to execute the scheme of the spine image recognition provided in the embodiments of the present application.
[0031] The implementation details of the technical scheme of the embodiments of the present application are described in detail as follows:
[0032] Figure 2 A flowchart of a spine image recognition method according to an embodiment of the present application is shown, which can be executed by a server, which can be the server shown in Figure 1 . Referring to Figure 2 , the spine image recognition method at least includes:
[0033] S100, a to-be-tested spine image is obtained, and the to-be-tested spine image is pre-processed to obtain a target spine image.
[0034] S200, the target spine image is input into a key point recognition model to obtain position information output by the key point recognition model for each vertebral body in the target spine image, the position information is marked on the target spine image, and at least includes positioning box information, vertebral body name and corner point coordinates, the key point recognition model includes a CBAM module.
[0035] S300, the position information is corrected and adjusted to obtain corrected position information.
[0036] S400, intervertebral disc positioning and vertebral identification are performed according to the corrected position information.
[0037] In this embodiment, an improved keypoint recognition model (such as YOLOv8-Pose with CBAM) is used, eliminating the need for a segmentation model and directly outputting positioning boxes, vertebral names, and corner coordinates. This simplifies the process and improves detection speed and robustness. The CBAM module enhances the model's capture of vertebral regional features and suppresses background interference, achieving particularly effective results for images of lesions, addressing the poor robustness of traditional automatic methods.
[0038] In S100, the spinal image to be tested is an MR spinal positioning image obtained by scanning the sagittal plane of the cervical, thoracic, or lumbar spine in Dicom format. The above-mentioned preprocessing includes data mapping processing, image size adjustment, etc., so as to transform the spinal image to be tested into a target spinal image that meets the input requirements of the adaptation model.
[0039] Specifically, in some embodiments, the specific implementation of step S100 can refer to the following embodiments. Figure 2 The detailed description of step S100 in the spine image recognition method shown in the corresponding embodiment, in the spine image recognition method, step S100 may include the following steps:
[0040] Acquire a spinal column image to be measured, and determine a pixel value to be measured, wherein the pixel value to be measured is a pixel value of each pixel point of the spinal column image to be measured;
[0041] According to the pixel values to be measured, data mapping processing is performed on the spinal column image to be measured to obtain an adjusted spinal column image.
[0042] The adjusted spinal column image is resized to obtain a target spinal column image.
[0043] In this embodiment, a specific data mapping formula is used to map the pixel values to the corresponding intervals, eliminating differences in image data across different devices or acquisition conditions. This unifies the image's grayscale range, improves the model's adaptability to images from different sources, resolves the image data normalization issue, and enhances the model's generalization capabilities. The adjusted spinal image is resized to the target size, ensuring that the image meets the input requirements of the keypoint recognition model. This resolves the issue of inconsistent image sizes and improves the model's efficiency and accuracy in processing images.
[0044] Specifically, a medical image reading tool is used to read the MR spinal positioning image (Dicom format) and convert it into a computer-processable image data structure. Each pixel in the image is traversed to obtain its corresponding pixel value to form a set of pixel values to be tested.
[0045] Then the above to be measured spine image is processed by data mapping. The data mapping processing is to convert the data to the interval of 0-255. The maximum pixel value and the minimum pixel value of the to-be-measured spine image can be determined first. Then, for the pixel value of each pixel point of the to-be-measured spine image, the data mapping calculation is performed according to the maximum pixel value and the minimum pixel value, to obtain the mapped pixel value. All the mapped pixel values are recombined into an image to obtain an adjusted spine image.
[0046] It should be noted that the calculation formula of the above data mapping calculation is as follows:
[0047]
[0048] Wherein, p max represents the maximum pixel value of the original image, p min represents the minimum pixel value of the original image, and p' represents the current original pixel value. P' represents the value after pixel mapping processing.
[0049] Finally, the adjusted spine image is resized, and the aspect ratio of the adjusted spine image is calculated. According to the target size (such as 640x640), it is determined whether to scale the long side or to fill the short side. If the long side is greater than the target size, the long side is scaled down by the same proportion, and the short side is scaled by the same proportion; if the short side is less than the target size, the pixel value 114 is uniformly filled on both sides of the short side until the image size reaches the target size, to obtain a target spine image. The size is 640x640, and the pixel value range is between 0-255, which is suitable as the input of the key point recognition model.
[0050] In S200, the preprocessed target spine image is input into the trained key point recognition model, and the key point recognition model can output the positioning box information of each vertebral segment, the vertebral name and the corner point coordinates. Among them, the vertebral positioning box is the circumscribed positioning box information and the corresponding confidence score output of each vertebral segment. There are 24 positioning boxes in total, including 6 cervical vertebrae (C2-C7), 12 thoracic vertebrae (T1-T12), 5 lumbar vertebrae (L1-L5), and 1 sacral vertebra (S1). The corner point coordinates are the 4 corner point coordinate information outputs of each vertebral segment.
[0051] Specifically, in some embodiments, the specific implementation of step S200 can refer to Figure 3 . Figure 3 is according to Figure 2 the details of step S200 in the spine image recognition method according to the corresponding embodiment, wherein the key point recognition model comprises a backbone network, a feature fusion network and a detection head. Step S200 can comprise the following steps:
[0052] S210, performing feature extraction and cross-stage feature fusion on the target spine image through the backbone network to obtain a multi-scale feature map.
[0053] S220, performing fusion feature processing on the multi-scale feature map through the feature fusion network to obtain a fusion feature image, wherein the fusion feature image contains semantic information and spatial details.
[0054] S230, performing prediction processing and coordinate regression on the fusion feature image according to the semantic information and the spatial details through the detection head to obtain position information corresponding to each vertebral body in the target spine image.
[0055] In this embodiment, in combination with the description of Figures 4-5 , first, the target spine image is subjected to feature extraction through the backbone network, and a multi-scale feature map is extracted, and cross-stage feature fusion is performed, thereby retaining information of different scales, especially considering different features of different vertebral bodies, so that the details of the image are effectively extracted. Then, the multi-scale feature map is subjected to fusion processing through the feature fusion network (Neck Network) to obtain a fusion feature image. This image not only contains spatial detail information (such as the edges and spatial layout of the vertebral body), but also contains semantic information (such as the type and position of the vertebral body). Finally, through the detection head (Head), the fusion feature image is subjected to prediction processing and coordinate regression, and finally the position information of each vertebral body in the target spine image is output. The detection head accurately regresses the position of each vertebral body by comprehensively considering the semantic information and the spatial details, thereby achieving accurate key point positioning.
[0056] In S210, the backbone network extracts features of different levels from the image, enabling the model to effectively learn the details of the image at multiple scales, and through cross-stage feature fusion, these detail information can be supplemented and enhanced. In this way, the model can better capture the features of each vertebral body when processing complex spine images, thereby improving the accuracy of subsequent recognition.
[0057] In this step, the backbone network (usually a convolutional neural network, CNN) inputs the target spine image and processes it layer by layer to extract low-level to high-level feature information. For example, low-level features may involve edges, textures, and other simple information, while high-level features involve more complex structures. The backbone network not only extracts these features, but also performs cross-stage feature fusion to ensure that the feature information of each stage can be fully combined. Finally, a multi-scale feature map is obtained, which represents various details and macro structures in the image.
[0058] Specifically, in some embodiments, the specific implementation of step S210 can refer to the following embodiments. Figure 3 The detailed description of step S210 in the spine image recognition method shown in the corresponding embodiment, in the spine image recognition method, step S210 may include the following steps:
[0059] An initial convolution is performed on the target spine image to obtain a preliminary convolution image.
[0060] The preliminary convolution image is subjected to cross-stage feature enhancement through CSPDarknet to obtain a feature enhanced image.
[0061] The feature-enhanced image is subjected to channel attention and spatial attention enhancement through the CBAM module to obtain a multi-scale feature map.
[0062] In this example, the CSPDarknet architecture is used to perform cross-stage feature enhancement on the preliminary convolutional image, reducing computational complexity while enhancing feature expression capabilities. This addresses the computational redundancy and insufficient feature extraction issues of traditional models, improving both model efficiency and accuracy. Furthermore, the CBAM module is used to enhance channel and spatial attention on the feature-enhanced image, enabling the model to focus more on features in the vertebral region, suppressing background noise and, in particular, increasing sensitivity to lesions. This addresses the traditional model's insufficient ability to detect lesions and improves both the model's robustness and accuracy.
[0063] Specifically, a convolutional layer (e.g., a 7×7 kernel with a stride of 2) is used to convolve the target spine image. This convolution operation is typically followed by batch normalization (BN) and an activation function (e.g., ReLU) to accelerate model convergence and enhance nonlinear expressiveness. This produces a preliminary convolutional image, which is typically halved in size and has an increased number of channels.
[0064] The preliminary convolutional image is then subjected to cross-stage feature enhancement using the CSPDarknet architecture. The CSPDarknet architecture comprises multiple Convolutional-to-Fractional (C2f) modules and a Specialized Perceptual Function (SPPF) layer. First, the preliminary convolutional image is fed into the first C2f module. This module splits the input image across stages, passing one portion directly while extracting features from the remaining portion through convolution and concatenating it with the original. This reduces computational complexity while enhancing information flow. After processing by multiple C2f modules, the image's features are further enhanced. Finally, the image processed by the C2f modules is fed into the Specialized Perceptual Function (SPPF) layer, which performs spatial pyramid pooling on the image to enhance multi-scale perception and produce a feature-enhanced image.
[0065] Finally, attention enhancement is performed, including channel attention enhancement and spatial attention enhancement.
[0066] For channel attention enhancement, first, global average pooling and global max pooling are performed on the feature enhanced image to obtain two different feature descriptors. The two feature descriptors are processed through a shared multi-layer perceptron (MLP) respectively to obtain two channel attention weight vectors. The two channel attention weight vectors are added and activated to obtain the final channel attention weight matrix Mc. The feature enhanced image is element-wise multiplied with the channel attention weight matrix Mc to obtain the channel attention enhanced image I'.
[0067] For spatial attention enhancement, first, average pooling and max pooling in the channel dimension are performed on the channel attention enhanced image I' to obtain two different feature descriptors. The two feature descriptors are spliced and processed through a convolution layer to obtain the spatial attention weight matrix Ms.
[0068] The channel attention enhanced image I' is element-wise multiplied with the spatial attention weight matrix Ms to obtain the final multi-scale feature map I''.
[0069] In S220, the introduction of the feature fusion network enables the network to maintain spatial details and semantic information when fusing feature images. In this way, the fused image can more comprehensively represent the complexity of the spine structure, providing a more accurate information basis for subsequent key point prediction, thereby improving the image recognition effect.
[0070] In this step, the fusion network further processes the multi-scale feature map obtained from the backbone network. Typically, the fusion network will weight and merge or splice features from different scales, which can maintain different levels of information in the image. For example, features at the detail level can help identify the edges of the vertebral body, and macro-structure information helps understand the overall layout of the spine. Through such fusion, the obtained fused feature image contains more comprehensive spatial and semantic information, which can provide an accurate basis for subsequent positioning tasks.
[0071] Specifically, in some embodiments, the specific implementation of step S220 can refer to the following embodiments. The present embodiment is based on the spine image recognition method shown in the corresponding embodiment Figure 3 The details of step S220 in the spine image recognition method shown in the corresponding embodiment are described as follows. The spine image recognition method can include the following steps.
[0072] The multi-scale feature maps are fused through the FPN path and the PAN path to obtain a multi-scale fused image.
[0073] The CBAM module is used for channel attention and spatial attention enhancement on each of the multi-scale fusion images, to obtain a fusion feature image.
[0074] In the embodiment, the multi-scale feature maps are fused through the FPN path and the PAN path, the features of different scales are effectively integrated, and the fused features contain high-level semantic information and retain low-level spatial details. The problem that the traditional model cannot fully utilize different scale features when fusing features is solved, and the detection capability of the model for different size vertebrae is improved.
[0075] Meanwhile, the CBAM module is used for channel attention and spatial attention enhancement on the multi-scale fusion images, so that the model can pay more attention to the features of the vertebra region and suppress background noise. In particular, the sensitivity to the lesion area is improved, the problem of insufficient detection capability of the traditional model for the lesion area is solved, and the robustness and accuracy of the model are improved.
[0076] Specifically, on the FPN path, a high-level feature map (such as C5) is selected from the multi-scale feature maps output by the backbone network, and the size of the high-level feature map is doubled through an upsampling (such as Upsample) operation, so that the size of the high-level feature map is the same as that of a middle-level feature map (such as C4). The upsampling high-level feature map and the middle-level feature map are spliced (such as Concat), and then processed through a convolution layer (such as C2f module) to obtain a fused middle-level feature map (such as P4). The fused middle-level feature map (P4) is again subjected to an upsampling operation, so that the size of the middle-level feature map is the same as that of a low-level feature map (such as C3), and then the middle-level feature map is spliced with the low-level feature map, and processed through a convolution layer to obtain a fused low-level feature map (such as P3).
[0077] On the PAN path, the fused low-level feature map (P3) is subjected to a downsampling (such as Conv layer, step size is 2) operation to reduce the size of the low-level feature map by half, so that the size of the low-level feature map is the same as that of the middle-level feature map (P4). The downsampling low-level feature map and the middle-level feature map are spliced, and then processed through a convolution layer to obtain an optimized middle-level feature map. The optimized middle-level feature map is again subjected to a downsampling operation, so that the size of the middle-level feature map is the same as that of a high-level feature map (P5), and then the middle-level feature map is spliced with the high-level feature map, and processed through a convolution layer to obtain a final multi-scale fusion image (such as P3, P4, P5).
[0078] In the CBAM module, each multi-scale fusion image (such as P3, P4, P5) is subjected to channel attention enhancement and spatial attention enhancement.
[0079] The channel attention enhancement specifically comprises performing global average pooling and global maximum pooling on the multi-scale fusion image to obtain two different feature descriptors. The two feature descriptors are respectively processed through a shared multi-layer perception (MLP) to obtain two channel attention weight vectors. The two channel attention weight vectors are added and activated to obtain a final channel attention weight matrix Mc. The multi-scale fusion image is element-wise multiplied with the channel attention weight matrix Mc to obtain an image I' after channel attention enhancement.
[0080] The spatial attention enhancement specifically comprises performing average pooling and maximum pooling on the channel attention enhanced image I' in the channel dimension to obtain two different feature descriptors. The two feature descriptors are spliced and processed through a convolution layer to obtain a spatial attention weight matrix Ms. The channel attention enhanced image I' is element-wise multiplied with the spatial attention weight matrix Ms to obtain a final fusion feature image I''.
[0081] In S230, the fusion feature image is processed by the detection head for prediction and coordinate regression, and finally the position information of each vertebral body in the target spine image is output. The detection head accurately regresses the position of each vertebral body by integrating semantic information and spatial details, thereby achieving accurate key point positioning.
[0082] In this step, the image processed by the feature fusion network is input to the detection head, which usually includes a series of regression layers responsible for positioning each vertebral body in the image. The detection head uses the fused feature image to predict the position of each vertebral body. This process involves coordinate regression, where the model outputs the precise coordinates of each vertebral body based on the features learned by the neural network. Ultimately, the model will obtain the specific position information of each vertebral body in the image.
[0083] Specifically, for the P3, P4, and P5 feature maps output by the feature fusion network, for each scale feature map, further feature extraction is performed through a 3x3 Conv layer to output 256-channel features.
[0084] For bounding box prediction, 4 coordinate values (x, y, w, h) and a confidence value (1 value) can be output through a 1x1 Conv layer, corresponding to 24 types of vertebrae (each type is independently predicted).
[0085] For class prediction, 24 class probabilities (C2-C7, T1-T12, etc.) can be output through a 1x1 Conv layer.
[0086] For corner point regression, an independent branch can be added to output 8 values (4 corner points, x and y coordinates for each corner point) through a 1x1 Conv layer.
[0087] Finally, the prediction results of P3 (detecting smaller vertebrae such as cervical vertebrae), P4 (middle vertebrae such as thoracic vertebrae), and P5 (larger vertebrae such as lumbar vertebrae) are combined, the bounding boxes are filtered by non-maximum suppression (NMS), and the optimal prediction is retained.
[0088] In the above embodiment, the channel attention and spatial attention enhancement of the CBAM module is implemented by the following formula:
[0089]
[0090] wherein I represents an input feature map, I' represents a channel attention weighting result, I" represents an output feature map, M c represents a channel attention module, M s represents a spatial attention module, represents an element-wise multiplication. It should be noted that in S210, I represents a feature enhanced image, and I" represents a multi-scale feature map. In S220, I represents a multi-scale feature map, and I" represents a fused feature image.
[0091] In one aspect, the channel attention module is used to calculate the importance weight of each channel of the feature map, suppress irrelevant channels (such as features corresponding to the background), and enhance vertebrae structure related channels (such as edge and corner features), thereby solving the problem of feature redundancy in traditional models. On the other hand, the spatial attention module is used to generate spatial weights, so that the model focuses on the region where the vertebrae are located and ignores non-vertebral regions (such as muscles and ligaments), thereby improving the detection capability of small targets (such as cervical vertebrae) and lesion regions (such as compression fractures). At the same time, through the cascade operation of the formula, the adaptive optimization of the feature in the channel and spatial dimensions is realized, so that the features output by the backbone network and the feature fusion network are more pure and more discriminative, thereby ultimately improving the corner detection accuracy (error ≤ 2 pixels) and the positioning box confidence.
[0092] Specifically, for channel attention enhancement, GAP and GMP are performed on I respectively to obtain two 1x1xC vectors. Then, the two vectors are input into a shared MLP (containing a fully connected layer + ReLU activation) respectively, compressed to C / r dimensions (r is the reduction rate, r = 16), and then restored to C dimensions. Finally, the sum is added and activated by Sigmoid to obtain the channel attention weight Mc. The Mc and the input feature map I are multiplied element by element to complete the channel attention enhancement.
[0093] For spatial attention enhancement, first average pooling and maximum pooling are respectively performed on I' in the channel dimension to obtain two 1xHxW feature maps. The two feature maps are concatenated along the channel dimension to obtain a 2xHxW feature map. Through a 7x7 convolution layer (padding=3, keeping the size unchanged) and a Sigmoid activation, a spatial attention weight Ms (shape [B,1,H,W]) is generated. Ms is multiplied element-wise with the channel attention feature I' to complete the spatial attention enhancement.
[0094] In some embodiments of the present application, the specific steps of the training method of the key point recognition model described above include:
[0095] Obtain the spine image sample set, which contains a plurality of spine image samples, each of which is labeled with a corresponding position information label.
[0096] Input the spine image samples one by one into the key point recognition model to obtain the position information result.
[0097] According to the output position information result and the position information label, the parameters of the key point recognition model are updated until a predetermined end condition is reached, the training is ended, and a trained key point recognition model is obtained.
[0098] In embodiments of the present application, during training, a spine image sample set containing a plurality of spine image samples can be obtained first, each of which is at least one of a cervical vertebra sagittal image, a thoracic vertebra sagittal image, and a lumbar vertebra sagittal image, and is labeled with a corresponding position information label; then the plurality of spine image samples are divided into a training set, a validation set, and a test set according to a predetermined proportion, and then the parameters of the key point recognition model are adjusted and determined according to the spine image samples included in the training set, the validation set, and the test set, to obtain a trained key point recognition model.
[0099] It should be noted that the spine image samples need to cover normal, lesions (such as compression fractures, disc herniation), and patient data of different body types to ensure sample diversity. The training label above is marked by an artificial angle point according to the order of the vertebrae, and then a bounding rectangle is obtained by the coordinates of the angle points of each vertebra and the boundary is expanded as a vertebra positioning box. The boundary expansion value is generally 5-30, which can be adjusted according to actual conditions. The class identifier is labeled according to the actual vertebra name. Considering the particularity of the cervical vertebra C2 (in the shape of a triangle), 2 identical angle point coordinates can be used on the upper edge of the vertebra C2.
[0100] During model training, the spine image sample set can be divided into a training set, a validation set, and a test set, then trained based on the training set, validated based on the validation set, and tested based on the test set to obtain a trained key point recognition model.
[0101] Before training based on the training set, the spine image samples in the training set can be preprocessed. The preprocessing includes image size adjustment, normalization, data enhancement, and class encoding.
[0102] The specific method of data enhancement is to perform geometric transformation, brightness transformation, and lesion simulation on the data image. Among them, the geometric transformation includes horizontal flip, up-down translation, left-right translation, and small angle rotation. The brightness transformation includes contrast adjustment and Gaussian noise superposition, which enhances the adaptability of the model to image quality changes. The lesion simulation includes adding local gray scale changes to normal samples, which improves the sensitivity of the model to lesion characteristics. The above operations can better enrich the sample set and enhance the generalization ability of the model.
[0103] The probability of the above horizontal flip can be set to about 50%; the above up-down translation and left-right translation can be performed within a range of plus or minus 10 pixels and a range of plus or minus 20 pixels, respectively; and the above small angle rotation can be performed within a range of 15° clockwise or counterclockwise.
[0104] The adjustment factor of the above contrast adjustment can be set to 0.8-1.2; the mean of the above Gaussian noise superposition can be set to 0, and the variance can be set to 0.01.
[0105] The image size can then be adjusted to the input requirement size of the subsequent model. The adjustment method is to maintain the original width-height ratio of the image, and to reduce or enlarge the longer side to the desired size while uniformly filling the short side with 114 as the pixel value. The size of the adjusted image is 640*640, which can also be adjusted to 1024*1024 or 320*320. The smaller the size, the faster the inference speed of the model. To balance the time consumption and accuracy of the subsequent model inference, this embodiment takes 640*640 as an example for illustration.
[0106] After obtaining the enhanced training set, the key point recognition model can be trained based on the enhanced training set to update the parameters and weights in the network.
[0107] Specifically, the spine image samples in the training set are input into the key point recognition model to obtain the position information result output by the key point recognition model. The position information result and the position information label are compared to calculate the loss function. Then the loss function is minimized using the stochastic gradient descent method, and the parameters and weights in the key point recognition model are updated by back propagation until the loss function meets the predetermined condition, such as convergence or less than a predetermined threshold.
[0108] In some embodiments, the position information result includes target positioning frame information, target vertebra name and target corner point coordinates, and the position information label can include at least one of the positioning frame information label, the vertebra name label and the corner point coordinate label. When calculating the loss function, it can be obtained by comparing the target positioning frame information with the positioning frame information label and / or comparing the target corner point coordinates with the corner point coordinate label, or by comparing the target vertebra name with the vertebra name label.
[0109] Specifically, the positioning loss adopts CIOU loss for measuring the geometric overlap error between the predicted frame and the real frame. The classification loss adopts cross-entropy loss for optimizing the category prediction probability. The target confidence loss adopts cross-entropy loss for judging whether the predicted position exists the target. The corner point loss adopts SmoothL1 loss for minimizing the distance between the predicted corner point and the real corner point coordinates. Finally, the weighted sum (such as positioning loss x 0.05 + target confidence loss x 1.0 + classification loss x 0.5 + corner point loss x 1.5) balances the multi-task optimization. In the subsequent processing process, the AdamW optimizer can be considered, with an initial learning rate of 1e-4 and a strategy of decaying the learning rate by 0.1 times every 50 epochs.
[0110] After training, the key point recognition network with updated parameters based on the training set can be verified based on the verification set. Specifically, the key point recognition network is debugged based on the verification set data, and when the loss function meets the predetermined condition, the model parameters of this stage are output. If the loss function does not meet the predetermined condition, the learning rate and other hyperparameters are automatically adjusted, and the next round of network model training is performed. In some embodiments, the verification set loss decreases by less than 0.1% for 10 consecutive epochs, which is considered as model convergence.
[0111] When the loss function calculated on the verification set meets the predetermined condition, the parameters and weights can be retained, and then the retained parameters are tested based on the test set. Specifically, the test set data is input to make the image segmentation network with retained parameters and weights output the position information result and the model weight. According to the comparison of the model loss of multiple rounds and the corresponding weight, the model weight with the minimum loss is output, and the trained key point recognition model is determined. In some embodiments, the test set mAP (average precision) is greater than or equal to 95%, and the average error of the corner point coordinates is less than or equal to 2 pixels, which can be considered as meeting the accuracy standard.
[0112] After obtaining the trained key point recognition model, the key point recognition can be completed based on the key point recognition model.
[0113] In addition, for the data input in the verification set and the test set, data augmentation can not be performed.
[0114] In the above training method, the manually labeled corner points directly correspond to the corner point regression branch of the Head, and the positioning frame is generated from the corner points, ensuring that the label and the model output dimension are consistent, reducing intermediate conversion errors. The enhanced samples are further focused on effective features through the CBAM module, so that the model pays more attention to difficult samples (such as small vertebrae and lesion areas) during training, improving the convergence speed and accuracy. The positioning, classification and corner point regression share the bottom layer features (such as the vertebra structure features extracted by the Backbone), reducing redundant calculation, and at the same time realizing the complementarity between tasks through loss weight adjustment, for example, the accurate positioning of the corner point assists the optimization of the positioning frame.
[0115] In S300, the correction and adjustment of the position information includes positioning frame screening, positioning frame adjustment and coordinate value correction.
[0116] Specifically, in some embodiments, the specific implementation of step S300 can refer to Figure 6 . Figure 6 is according to Figure 2 The details of step S300 in the corresponding embodiment of the spinal image recognition method are described, wherein step S300 can include the following steps:
[0117] S310, screening the positioning frame according to the positioning frame information to obtain a target positioning frame.
[0118] S320, adjusting the vertebra name and corner point coordinates corresponding to each target positioning frame according to the target positioning frame.
[0119] In this embodiment, through the screening of the positioning frame, the positioning frame containing key information can be accurately identified from the complex image, improving the accuracy of image recognition. Adjusting the vertebra name and corner point coordinates in the target positioning frame can ensure that the obtained position information is more consistent with the actual anatomical structure, thereby improving the accuracy of the image. By correcting and adjusting the position information, noise and unnecessary interference in the image can be effectively eliminated, thereby improving the stability and robustness of the system in actual application, and the system can adapt to more complex spinal images.
[0120] In S310, in the spinal image, a plurality of possible positioning frames are obtained through a detection model. These positioning frames are potential key point regions in the image, which may contain various parts of the spine. By setting screening conditions (such as the size, position, confidence of the positioning frame, etc.), the real target positioning frame is screened out.
[0121] This step screens the positioning frame, so that the system can select the most suitable frame from a large number of candidate frames, ensuring that it contains the most critical spinal part in the image, effectively improving the recognition efficiency and accuracy.
[0122] Specifically, in some embodiments, the specific implementation of step S310 can refer to the following embodiments. Figure 6 The detailed description of step S310 in the spine image recognition method shown in the corresponding embodiment, in the spine image recognition method, the positioning frame information also includes a corresponding confidence level, and step S310 may include the following steps:
[0123] According to the confidence level corresponding to the positioning frame, a confidence score threshold is set to obtain a preliminary screening positioning frame.
[0124] The primary screening positioning frame is de-redundantly processed using a non-maximum suppression algorithm to obtain a target positioning frame.
[0125] In this embodiment, by setting a confidence threshold (such as 0.5), the model's uncertain predictions (such as background mistakenly detected as vertebrae) are directly eliminated, solving the problem of low-quality frames flooding in traditional methods. The NMS algorithm is used to eliminate duplicate detection frames of the same vertebra (such as false frames caused by similar features of adjacent vertebrae), ensuring that only one optimal frame is retained for each vertebra, and the number of positioning frames is reduced to 24, reducing the redundancy rate. Confidence is a comprehensive trust indicator of the model's prediction of positioning frames, categories, and corner points. After filtering, it can indirectly improve the accuracy of category identification and corner point coordinates (for example, the corner point error corresponding to the low-confidence frame is usually greater than 5 pixels, and the error after filtering is ≤2 pixels).
[0126] Specifically, a confidence threshold Tc is pre-set (recommended range 0.2-0.8, with 0.5 recommended for clinical scenarios). Threshold Tc can be dynamically adjusted based on data distribution. For example, if lesion data is feature-ambiguous, it can be appropriately lowered to 0.3. All localization boxes are traversed, and only those with confidence greater than threshold Tc are retained. The rest are marked as invalid and removed. For example, if 20 localization boxes are input with a confidence distribution of [0.1, 0.3, 0.5, 0.7, ...], 12 high-confidence boxes remain after filtering with a threshold of 0.5.
[0127] In some embodiments, the confidence score is output by the classification branch of the detection head (eg, Softmax probability), reflecting the model's certainty that the box contains the target vertebra.
[0128] Sort the initial screening positioning boxes by confidence from high to low, generating a list boxes = [b1, b2, ..., bn] (with b1 having the highest confidence). Initialize the target positioning box set target_boxes = []. Remove the first box bi from boxes and add it to target_boxes. Calculate the intersection over union (IoU) between bi and all remaining boxes using the following formula: IoU(bi,bj) = Area(bi∩bj) / Area(bi∪bj). Remove boxes from boxes whose IoU with bi is greater than Ti (NMS threshold, recommended 0.3-0.5), retaining boxes with no or low overlap. Repeat the above steps until boxes is empty.
[0129] The final target_boxes contains 24 positioning boxes (ideally), each box corresponds to a unique vertebra, and there is no overlap.
[0130] The confidence score integrates multiple dimensions of information, including the overlap (IoU) between the localization box and the ground-truth box, the class prediction probability, and the corner coordinate error. High-confidence boxes generally correspond to more accurate detection results. During the training phase, a loss function (such as the CIoU loss) is used to optimize the relationship between the localization box and the confidence score. This ensures that high-confidence boxes inherently have better positional accuracy, reducing post-processing workload.
[0131] In S320, after the target positioning frame is determined, the content in each frame is analyzed, and the name and corner coordinates of the vertebra in each positioning frame are adjusted accordingly in combination with the position information of the image. The image is positioned more accurately to adjust the coordinates of the vertebral corners to ensure that they are consistent with the anatomical structure in the actual image. This step is usually combined with deep learning technology to make adjustments by predicting the optimal position of the corners through the model. Then, based on the more accurate positioning, the corresponding cone name is obtained. For example, a positioning frame may represent the L1 vertebra, while another may represent the L2 vertebra. The system will automatically update the name of the vertebra based on the position of the positioning frame.
[0132] This adjustment process ensures that each target positioning frame accurately identifies the corresponding vertebral body, ensuring that every key element in the image is correctly identified. This adjustment significantly improves the accuracy of spinal image annotation and precisely adjusts corner coordinates to further optimize image recognition accuracy and reduce errors. This adjustment is crucial for subsequent spinal disease analysis and treatment planning.
[0133] Ultimately, the filtered and adjusted target positioning frame receives corrected position information, including updated vertebral names and precise corner coordinates. This corrected position information ensures that every key information point in the image is accurately located and labeled, thereby improving the overall performance of the image analysis system and enhancing the accuracy of the final diagnosis.
[0134] Specifically, in some embodiments, the specific implementation of step S320 can refer to the following embodiments. This embodiment is based on the spinal image recognition method shown in the corresponding embodiment Figure 6 The details of step S320 in the spinal image recognition method shown in the corresponding embodiment can include the following steps:
[0135] Determine the center point coordinates of each target positioning frame.
[0136] Adjust each target positioning frame according to the center point coordinates.
[0137] According to the adjusted target positioning frame, determine the corresponding vertebral body name and corner point coordinates.
[0138] In this embodiment, by calculating the center point coordinates of the positioning frame and sorting, a strict anatomical order is established, solving the problem of order confusion in traditional methods due to abnormal shape or lesions of the vertebrae, and improving the accuracy of the correspondence between the vertebral body name and the position. Based on the center point coordinates, the position of the positioning frame is fine-tuned to ensure that the spatial relationship of adjacent vertebrae conforms to the anatomical law (such as uniform intervertebral distance and natural curve arrangement), reduce the random error of model prediction, and improve the positioning accuracy of the positioning frame. The spatial information of the positioning frame is associated with the corner point coordinates, forcing the corner point coordinates to belong to the geometric area of the corresponding positioning frame, solving the problem of separation of corner points and positioning frames in traditional methods, and reducing the positioning error of corner points.
[0139] Specifically, for each target positioning frame, based on its corner point coordinates, the center point coordinates are calculated. All center point coordinates are stored in order of positioning frame index to form a center point set.
[0140] Arrange the positioning frames in ascending order of center point y coordinates (in the spinal image, from top to bottom, corresponding to cervical vertebrae, thoracic vertebrae, lumbar vertebrae, sacral vertebrae). Calculate the vertical distance between adjacent center points, detect abnormal distance (such as vertical distance greater than three times the average distance), and mark possible sorting errors or omissions.
[0141] For abnormal distance areas, the center point position is redistributed by linear interpolation, which can use the following formula:
[0142]
[0143] Where u i ′ is the adjusted y coordinate, and n is the number of frames.
[0144] Constrain the x coordinates of all center points near the center line of the spine. For example, by fitting a quadratic curve f(y) = ay 2 +by+c, x′ i =f(y′)i )
[0145] According to the adjusted center point C i ′, keep the width and height of the frame unchanged, and update the positioning frame coordinates.
[0146] According to the sorted positioning frame index i, the vertebral name is directly mapped (eg, i=1 corresponds to C2, i=2 corresponds to C3, and so on until i=24 corresponds to S1).
[0147] For each positioning box B i ′, the corresponding corner coordinates P j =(x j ,y j )(j=1,2,3,4) are constrained within the box. For details, please refer to the following formula:
[0148] x′ j =max(x′ i ,min(x j ,x i ′+w′ i ))
[0149] y′ j =max(y′ i ,min(y j ,y′ i +h′ i ))
[0150] where (x′ i ,y′ i ) is the coordinate of the upper left corner of the adjusted frame, w′ i ,h′ i To adjust the width and height of the rear frame.
[0151] Based on the adjusted geometric features of the positioning frame, the corner point positions are fine-tuned (for example, the corner points on the upper edge of the vertebra are constrained to be near the midpoint of the top edge of the frame) to further improve the corner point accuracy.
[0152] It should be noted here that this embodiment also requires coordinate correction and restoration, specifically, the real coordinates are calculated in reverse based on the scaling ratio (such as scaling the long side from the original size to 640) and the padding amount (the number of pixels filled on both sides of the short side) of the image size adjustment processing in step S100.
[0153] In S400 , after obtaining the identification frame of each vertebra, the intervertebral discs between the vertebrae can be located and the vertebrae can be renamed.
[0154] Specifically, in some embodiments, the specific implementation of step S400 can refer to the following embodiments. Figure 2Corresponding to the details of step S400 shown in the spinal image recognition method of the embodiment, the spinal image recognition method can include the following steps:
[0155] According to the corner point coordinates, a disc scan positioning line is determined to obtain a disc positioning result.
[0156] According to the cone name, a target cone is positioned.
[0157] Starting from the target cone, each vertebra is sequentially renamed to obtain a vertebra identifier.
[0158] In this embodiment, the scan positioning line is constructed based on the adjacent vertebra corner point coordinates, the disc positioning problem in three-dimensional space is converted into geometric calculation on a two-dimensional plane, and the positioning error is significantly reduced, especially for the positioning accuracy of cervical and lumbar discs. Renaming in anatomical order starting from the target cone avoids the identification disorder caused by missed or false detection, and the accuracy of the vertebra identifier is improved, that is, even in the case of 3 missed detections, 90% of the vertebrae can still be correctly identified. This embodiment does not depend on specific imaging modalities (such as MR / CT), but is based on the geometric relationship between the corner point coordinates and the cone name, and has strong universality.
[0159] Specifically, first, the corner points of each vertebra are divided into two groups, the two corner points on the upper edge of the vertebra are the upper group, and the two corner points on the lower edge of the vertebra are the lower group. Then, according to the midline between the lower group corner points of each vertebra and the upper group corner points of the next vertebra, the disc scan positioning line, i.e. the disc positioning result, is determined. In some embodiments, the positioning line can be extended to both ends by 10-15 pixels to ensure that the entire disc region is covered.
[0160] Then, a target cone with obvious features (such as C2 or S1, which has a unique shape and is not easily obscured) is identified in all vertebrae. The characteristics of C2 are that the ratio of the positioning box height to the width is in the range of 1.2-1.5 (cervical spine characteristics), the distance between the upper edge corner points is significantly smaller than the distance between the lower edge corner points (C2-specific trapezoidal structure), and it is located in the upper 1 / 3 region of the image (anatomy position constraint). If no C2 meeting the standard is found, the cone with the highest confidence (such as C3, C4, and S1) is selected as an alternative target cone.
[0161] For example, in some embodiments, if the category identification of the positioning box contains C2, the C2-based vertebral identification is renamed from top to bottom in the anatomical vertebral order (i.e., traversing the subsequent vertebrae in ascending order of the center point Y coordinate, and sequentially named as C2, C3, C4, …, S1). If the category identification of the rectangular box contains S1 and no C2, the S1-based vertebral identification is renamed from bottom to top in the anatomical vertebral order (i.e., traversing the subsequent vertebrae in descending order of the center point Y coordinate, and sequentially named as S1, L5, L4, …, C2). If the adjacent vertebral names are increasing (or decreasing) and the categories are continuous, the verification is successful.
[0162] If there is a category jump (such as C7 directly followed by T3), the missing categories (T1, T2) are inserted and marked as inferred. If a vertebral name is obviously inconsistent with the corner point feature (such as marked as L1 but the morphology is cervical vertebra), reclassify based on the corner point.
[0163] The device embodiment of the present application is introduced below, which can be used to execute the spinal image recognition method in the above-mentioned embodiments of the present application. For details not disclosed in the device embodiment of the present application, please refer to the above-mentioned embodiments of the spinal image recognition method of the present application.
[0164] Figure 7 A block diagram of a spinal image recognition device according to an embodiment of the present application is shown.
[0165] Referring to Figure 7 As shown in the figure, the spinal image recognition device 700 according to an embodiment of the present application includes a spine image acquisition module 710, a spine image input module 720, a spine image adjustment module 730, and a spine image identification module 740. Among them, the spine image acquisition module 710 is used to acquire a to-be-tested spine image, pre-process the to-be-tested spine image, and obtain a target spine image; the spine image input module 720 is used to input the target spine image into a key point recognition model, and obtain position information output by the key point recognition model for each vertebral body in the target spine image, the position information is marked on the target spine image, and at least includes positioning box information, vertebral body name and corner point coordinate; the spine image adjustment module 730 is used to correct and adjust the position information to obtain corrected position information; the spine image identification module 740 is used to perform intervertebral disc positioning and vertebral identification according to the corrected position information.
[0166] In some possible embodiments of the present application, the spine image acquisition module specifically comprises: a spine image acquisition submodule, configured to acquire a to-be-tested spine image, and determine a to-be-tested pixel value, the to-be-tested pixel value being a pixel value of each pixel point of the to-be-tested spine image; a data mapping processing submodule, configured to perform data mapping processing on the to-be-tested spine image according to the to-be-tested pixel value, to obtain an adjusted spine image; and an image size adjustment submodule, configured to perform size adjustment on the adjusted spine image, to obtain a target spine image.
[0167] In some possible embodiments of the present application, the key point identification model comprises a backbone network, a feature fusion network and a detection head; and the spine image input module specifically comprises: a backbone network submodule, configured to perform feature extraction and cross-stage feature fusion on the target spine image through the backbone network, to obtain a multi-scale feature map; a neck network submodule, configured to perform fusion feature processing on the multi-scale feature map through the feature fusion network, to obtain a fusion feature image, the fusion feature image containing semantic information and spatial details; and a head network submodule, configured to perform prediction processing and coordinate regression on the fusion feature image according to the semantic information and the spatial details through the detection head, to obtain position information corresponding to each vertebral body in the target spine image.
[0168] In some possible embodiments of the present application, the backbone network submodule specifically comprises: an image preliminary convolution unit, configured to perform initial convolution on the target spine image to obtain a preliminary convolution image; a CSPDarknet unit, configured to perform cross-stage feature enhancement on the preliminary convolution image through CSPDarknet, to obtain a feature-enhanced image; and a CBAM module unit, configured to perform channel attention and spatial attention enhancement on the feature-enhanced image through the CBAM module, to obtain a multi-scale feature map.
[0169] In some possible embodiments of the present application, the neck network submodule specifically comprises: a feature fusion unit, configured to perform feature fusion on the multi-scale feature map through an FPN path and a PAN path, to obtain a multi-scale fusion image; and an attention enhancement unit, configured to perform channel attention and spatial attention enhancement on each multi-scale fusion image through the CBAM module, to obtain a fusion feature image.
[0170] In some possible embodiments of the present application, the channel attention and spatial attention enhancement of the CBAM module is realized through the following formula:
[0171]
[0172] wherein I represents an input feature map, I' represents a channel attention weighting result, I" represents an output feature map, M represents a channel attention map, and F represents a spatial attention map. crepresents the channel attention module, M s represents the spatial attention module, Represents element-wise multiplication.
[0173] In some feasible embodiments of the present application, the spinal image recognition device further includes: a sample acquisition module for acquiring the spinal image sample set, wherein the spinal image sample set includes a plurality of spinal image samples, and each spinal image sample is marked with a corresponding position information label. A sample input module for inputting the spinal image samples one by one into the key point recognition model to obtain position information results. A parameter update module for updating the parameters of the key point recognition model according to the output position information results and the position information labels until a predetermined end condition is reached, thereby ending the training and obtaining a trained key point recognition model.
[0174] In some feasible embodiments of the present application, the spinal image adjustment module specifically includes: a positioning frame screening submodule, used to screen the positioning frame according to the positioning frame information to obtain the target positioning frame; a positioning frame adjustment submodule, used to adjust the vertebral name and corner point coordinates corresponding to each target positioning frame according to the target positioning frame.
[0175] In some feasible embodiments of the present application, the positioning frame information also includes a corresponding confidence level; the root positioning frame screening submodule specifically includes: a confidence score threshold limiting unit, which is used to perform confidence score threshold limiting according to the confidence level corresponding to the positioning frame to obtain a preliminary screening positioning frame; a non-maximum suppression algorithm unit, which is used to perform redundancy processing on the preliminary screening positioning frame through a non-maximum suppression algorithm to obtain a target positioning frame.
[0176] In some feasible embodiments of the present application, the positioning frame adjustment submodule specifically includes: a center point determination unit, used to determine the center point coordinates of each target positioning frame; a positioning frame adjustment unit, used to adjust each target positioning frame according to the center point coordinates; and a cone corner point determination unit, used to determine the corresponding vertebral name and corner point coordinates based on the adjusted target positioning frame.
[0177] In some feasible embodiments of the present application, the spinal image identification module specifically includes: an intervertebral disc positioning submodule, which is used to determine the intervertebral disc scanning positioning line according to the corner point coordinates to obtain the intervertebral disc positioning result; a target cone positioning submodule, which is used to locate the target cone according to the cone name; and a vertebra renaming submodule, which is used to rename each vertebra in sequence with the target cone as the starting point to obtain a vertebral identification.
[0178] In this embodiment, a modified key point recognition model (such as YOLOv8-Pose containing CBAM) is used, without relying on a segmentation model, directly outputting the positioning box, vertebral body name and corner coordinates, simplifying the process and improving the detection speed and robustness. The CBAM module enhances the model's capture of vertebral region features and suppresses background interference, especially for lesion images, and solves the problem of poor robustness of traditional automatic methods.
[0179] Figure 8 A structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application is shown.
[0180] It should be noted that, Figure 8 The computer system of the electronic device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present application.
[0181] As Figure 8 shown, the computer system includes a central processing unit (CPU) 1801, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1802 or loaded from a storage portion 1808 into a random access memory (RAM) 1803, such as performing the methods described in the above embodiments. In the RAM 1803, various programs and data required for system operation are also stored. The CPU 1801, the ROM 1802, and the RAM 1803 are connected to each other through a bus 1804. An input / output (I / O) interface 1805 is also connected to the bus 1804.
[0182] The following components are connected to the I / O interface 1805: an input portion 1806 including a keyboard, a mouse, and the like; an output portion 1807 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage portion 1808 including a hard disk, and the like; and a communication portion 1809 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication portion 1809 performs communication processing via a network such as the Internet. A drive 1810 is also connected to the I / O interface 1805 as necessary. A removable medium 1811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 1810 as necessary, so that a computer program read therefrom is installed in the storage portion 1808 as necessary.
[0183] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program in accordance with the embodiments of the present application. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer readable medium, the computer program containing computer programs for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network by the communication section 1809, and / or installed from the detachable medium 1811. When the computer program is executed by the central processing unit (CPU) 1801, various functions defined in the system of the present application are executed.
[0184] It should be noted that the computer readable medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable computer programs. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit programs for use by or in conjunction with an instruction execution system, device or apparatus. The computer programs contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination of the above.
[0185] The flow and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and
[0186] The units described in the embodiments of the present application can be implemented by software, or by hardware, or by a combination of software and hardware. The units described may
[0187] As another aspect, the present application also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs, which, when executed by the electronic device, enable the electronic device to implement the method described in the above embodiments.
[0188] The specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the method of the above embodiments. The specific implementation process can refer to the specific description of the above embodiments, which will not be repeated here. Figures 1 to 4 Figures 1 to 4 The specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the method of the above embodiments. The specific implementation process can refer to the specific description of the above embodiments, which will not be repeated here.
[0189] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into a plurality of modules or units.
[0190] Those skilled in the art can easily understand, through the above description of the embodiments, that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the methods according to the embodiments of the present application.
[0191] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. It is intended that the present application cover any and all variations of the present application that come within the scope of the claims and that the terms describe and of the specification be interpreted to cover such variations.
[0192] It should be understood that the present application is not limited to the precise construction that has been described above and illustrated in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the present application. The scope of the present application is limited only by the appended claims.
Claims
1. A spine image recognition method, characterized in that: The spine image recognition method comprises: Acquiring a spinal column image to be tested, and preprocessing the spinal column image to be tested to obtain a target spinal column image; Inputting the target spinal image into a key point recognition model to obtain position information output by the key point recognition model for each vertebra in the target spinal image, wherein the position information is marked on the target spinal image and includes at least positioning frame information, vertebral name, and corner point coordinates, wherein the key point recognition model includes a CBAM module; Correcting and adjusting the position information to obtain corrected position information; Intervertebral disc positioning and vertebra identification are performed based on the corrected position information.
2. The spine image recognition method according to claim 1, wherein: The acquiring of the spinal column image to be measured and preprocessing of the spinal column image to be measured to obtain the target spinal column image specifically includes: Acquire a spinal column image to be measured, and determine a pixel value to be measured, wherein the pixel value to be measured is a pixel value of each pixel point of the spinal column image to be measured; Performing data mapping processing on the spinal column image to be measured according to the pixel values to be measured to obtain an adjusted spinal column image; The adjusted spinal column image is resized to obtain a target spinal column image.
3. The spine image recognition method according to claim 1, wherein: The key point recognition model includes a backbone network, a feature fusion network, and a detection head; Inputting the target spinal column image into a key point recognition model to obtain position information output by the key point recognition model for each vertebra in the target spinal column image specifically includes: Through the backbone network, feature extraction and cross-stage feature fusion are performed on the target spine image to obtain a multi-scale feature map; Performing fusion feature processing on the multi-scale feature map through the feature fusion network to obtain a fused feature image, wherein the fused feature image contains semantic information and spatial details; The detection head performs prediction processing and coordinate regression on the fused feature image according to the semantic information and the spatial details to obtain position information corresponding to each vertebra in the target spinal column image.
4. The spine image recognition method according to claim 3, wherein: The backbone network is used to extract features and fuse cross-stage features on the target spine image to obtain a multi-scale feature map, specifically including: Performing initial convolution on the target spine image to obtain a preliminary convolution image; Performing cross-stage feature enhancement on the preliminary convolution image through CSPDarknet to obtain a feature enhanced image; The feature-enhanced image is subjected to channel attention and spatial attention enhancement through the CBAM module to obtain a multi-scale feature map.
5. The spine image recognition method according to claim 3, wherein: The step of performing fusion feature processing on the multi-scale feature map through the feature fusion network to obtain a fused feature image specifically includes: Performing feature fusion on the multi-scale feature map through the FPN path and the PAN path to obtain a multi-scale fused image; The CBAM module is used to perform channel attention and spatial attention enhancement on each of the multi-scale fused images to obtain a fused feature image.
6. The spine image recognition method according to claim 4 or 5, characterized in that: The channel attention and spatial attention enhancement of the CBAM module are achieved by the following formula: Among them, I represents the input feature map, I′ represents the channel attention weighted result, I″ represents the output feature map, and M c represents the channel attention module, M s represents the spatial attention module, Represents element-wise multiplication.
7. The spine image recognition method according to claim 1, wherein: The spine image recognition method further comprises: Acquire the spinal image sample set, wherein the spinal image sample set includes a plurality of spinal image samples, each spinal image sample is marked with a corresponding position information label; Inputting the spinal image samples one by one into the key point recognition model to obtain position information results; The parameters of the key point recognition model are updated according to the output position information result and the position information label until a predetermined end condition is reached, and the training is terminated to obtain a trained key point recognition model.
8. The spine image recognition method according to claim 1, wherein: Correcting and adjusting the position information to obtain corrected position information specifically includes: Filtering the positioning frame according to the positioning frame information to obtain the target positioning frame; According to the target positioning frames, the vertebral body names and corner point coordinates corresponding to the target positioning frames are adjusted.
9. The spine image recognition method according to claim 8, wherein: The positioning frame information also includes corresponding confidence level; The filtering of the positioning frame according to the positioning frame information to obtain the target positioning frame specifically includes: According to the confidence level corresponding to the positioning frame, a confidence score threshold is set to obtain a preliminary screening positioning frame; The primary screening positioning frame is de-redundantly processed using a non-maximum suppression algorithm to obtain a target positioning frame.
10. The spine image recognition method according to claim 8, wherein: The adjusting, according to the target positioning frames, the vertebral names and corner point coordinates corresponding to the target positioning frames specifically includes: Determine the center point coordinates of each target positioning frame; Adjusting each target positioning frame according to the center point coordinates; According to the adjusted target positioning frame, the corresponding vertebral body name and corner point coordinates are determined.
11. The spine image recognition method according to claim 1, wherein: Positioning the intervertebral disc and marking the vertebrae according to the corrected position information specifically includes: Determine the intervertebral disc scanning positioning line according to the corner point coordinates to obtain the intervertebral disc positioning result; According to the cone name, locate the target cone; Starting from the target cone, each vertebra is renamed in sequence to obtain the vertebral identification.
12. A spinal image recognition device, characterized in that: The spine image recognition device comprises: A spinal image acquisition module is used to acquire a spinal image to be tested, and pre-process the spinal image to be tested to obtain a target spinal image; a spinal image input module, configured to input the target spinal image into a key point recognition model, and obtain position information output by the key point recognition model for each vertebra in the target spinal image, wherein the position information is marked on the target spinal image and includes at least positioning frame information, vertebral name, and corner point coordinates; A spine image adjustment module, configured to correct and adjust the position information to obtain corrected position information; The spinal column image identification module is used to locate the intervertebral disc and identify the vertebrae according to the corrected position information.
13. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the spine image recognition method according to any one of claims 1 to 11 is implemented.
14. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the spine image recognition method according to any one of claims 1 to 11.
15. A computer program product comprising one or more computer programs, characterized in that When the one or more computer programs are executed by one or more processors, the steps of the spine image recognition method according to any one of claims 1 to 11 are implemented.
Citation Information
Patent Citations
Double-step spine X-ray film processing method based on single vertebral body segmentation
CN118918118A
Spine Cobb angle measuring method and device and electronic equipment
CN119693362A
Cited By
Processing method and system for spine image identification
CN121564450A
A processing method and system for spinal image labeling
CN121564450B
Craniocerebral image cutting method, craniocerebral image segmentation model training method and related equipment
CN121767282A
