Spine key point correction method and system
By using Linformer model and dual activation technology to correct the spinal key points in the prior art, the problem of inaccurate positioning of the spinal key points in the spinal X-ray image is solved, and higher accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202411695931.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-04-18
AI Technical Summary
The prior art When manually positioning and measuring spinal key points in spinal X-ray images, relying on the experience of a doctor, resulting in low accuracy of positioning and measurement results and being susceptible to image noise or artifacts.
A spinal key point correction method is adopted. By obtaining spinal X-ray images, the convolutional feature map is extracted and corrected using the Linformer model, the center point heat map, center point offset map and corner point offset map of the vertebral key points are calculated, and these images are jointly corrected by dual activation of the Linformer model, and the corrected vertebral key points position is finally calculated.
This method can reduce errors caused by image noise or artifacts, enhance the calculation accuracy of key point positions, improve the robustness and interpretability of the model, and provide a more reliable solution for spinal key point detection.
Smart Images
Figure CN119850487A_ABST
Abstract
Description
Technical Field
[0001] This document relates to the field of medical imaging technology, and in particular to a method and system for correcting key points of the spine. Background Art
[0002] X-ray images are an important form of spinal imaging. In recent years, the widespread application of computer vision has improved the efficiency of spinal imaging. Since spinal imaging requires manual positioning and measurement on the collected X-ray images, and the accuracy of the positioning and measurement results depends on the doctor's experience, a spinal key point correction system is needed to locate the key points of the spine in X-ray images. It can help doctors quickly adjust the position of the key points of the spine and automatically give more consistent key point positioning results, providing an accurate basis for the subsequent diagnosis, treatment and prevention of spinal diseases. Summary of the invention
[0003] One or more embodiments of this specification provide a method for correcting key points of the spine, including:
[0004] Acquire spinal X-ray images and read the X-ray images;
[0005] Extract the convolution feature map from the read X-ray image and correct the convolution feature map using the Linformer model;
[0006] Calculate the center point heat map, center point offset map and corner point offset map of the vertebral key points according to the corrected convolution feature map;
[0007] Using a dual activation Linformer model to jointly correct the center point offset map and the corner point offset map;
[0008] According to the corrected center point offset map and corner point offset map, combined with the center point heat map, the corrected vertebral key point positions are calculated.
[0009] Furthermore, the specific method of extracting the convolution feature map from the read X-ray image and correcting the convolution feature map using the Linformer model is as follows:
[0010] Extract convolutional feature maps from X-ray images using a high-resolution network;
[0011] The convolution feature map is divided into non-overlapping convolution feature windows in the image height direction, and the convolution feature map of each channel in each convolution feature window is flattened into a convolution feature vector, and the features in different channels remain unchanged.
[0012] Furthermore, the specific method for calculating the key point center point heat map, center point offset map and corner point offset map based on the corrected convolution feature map is as follows:
[0013] Three subtask branches are derived from the high-resolution network feature extraction module. Based on the subtask branches, the high-resolution network feature extraction module is used to extract the convolution feature map, and the convolution feature map is separated into a center point heat map, a center point offset map and a corner point offset map respectively; the subtask branches adopt a two-layer convolutional neural network.
[0014] Furthermore, the center point heat map is used to locate the vertebral center and distinguish the vertebral body from the background, and a Gaussian kernel is used to generate a center point heat map label; the center point offset map is used to obtain four corner points of the vertebral body relative to the center position, and the four corner points are used to represent the vertebral body.
[0015] Furthermore, the specific method of using the dual activation Linformer model to jointly correct the center point offset map and the corner point offset map is:
[0016] After extracting the center point of the vertebra from the center point heat map, extract the center point offset and corner point offset corresponding to the center point from the center point offset map and corner point offset map;
[0017] The center point offset and corner point offset of each vertebra in the spine are corrected by dual activation of the Linformer model, and the vertebra center point offset is added to the vertebra center point to obtain the corrected vertebra center point.
[0018] The corrected vertebral center point and the corrected corner point offset are added to obtain the corrected vertebral key point.
[0019] Furthermore, the corrected vertebral center point and the corrected corner point offset are added together to obtain the corrected vertebral key point. The specific method is:
[0020] According to the corrected center point offset map, corner point offset map and center point heat map, the corrected vertebral key point position is calculated: the vertebral key point is calculated according to the vertebral center predicted by the three subtask branches, the vertebral center point offset and the corner point offset relative to the vertebral center.
[0021] In the center point heat map, compare the heat value of the current pixel with the heat values of its eight neighboring points, find the point whose heat value is not less than the eight connected neighboring points, and take the K points with the largest heat values as the center points of the vertebra;
[0022] Take the center point offset corresponding to the vertebral center point from the center point offset map to obtain the vertebral center point position;
[0023] The corner point offset corresponding to the vertebral center point is taken out from the corner point offset map, and the position of the vertebral key point is obtained by combining the position of the vertebral center point.
[0024] Furthermore, the center point offset and corner point offset of each vertebra in the spine are corrected by dual activation of the Linformer model. The specific method is as follows:
[0025] The spine composed of vertebral key points is regarded as a continuous sequence, and the vertebra is regarded as a sampling point. The characteristics of each sampling point are the center point offset and corner point offset of the vertebra;
[0026] For the same dimension features, the Softmax function is used to normalize the features in the vertebral sequence dimension, and the Softmax function is used to normalize the features in different feature dimensions for the same vertebra;
[0027] The above two normalized features are element-wise multiplied, and after 1*1 convolution, they are multiplied with the input features of the Linformer model. The multiplied result is then added to the input features of the Linformer model to obtain the corrected center point offset and corner point offset of each vertebra on the spine.
[0028] One or more embodiments of this specification provide a spine key point correction system, including:
[0029] Image processing module: used to obtain spinal X-ray images and read the X-ray images;
[0030] The first correction module is used to extract the convolution feature map from the read X-ray image and correct the convolution feature map using the Linformer model;
[0031] The first calculation module is used to calculate the center point heat map, center point offset map and corner point offset map of the vertebral key points according to the corrected convolution feature map;
[0032] A second correction module: used for applying dual activation to the Linformer model to jointly correct the center point offset map and the corner point offset map;
[0033] The second calculation module is used to calculate the corrected vertebral key point positions according to the corrected center point offset map and corner point offset map in combination with the center point heat map.
[0034] One or more embodiments of the present specification provide an electronic device, including:
[0035] processor; and,
[0036] A memory arranged to store computer executable instructions, which, when executed, cause the processor to implement the steps of the above spinal key point correction method.
[0037] One or more embodiments of the present specification provide a storage medium for storing computer-executable instructions, wherein the computer-executable instructions implement the steps of the above-mentioned spine key point correction method when executed.
[0038] By adopting the embodiment of the present invention, through an efficient feature extraction and correction method, the center point offset map and the corner point offset map are jointly corrected through the Linformer model, and the center point heat map, the center point offset map and the corner point offset map are combined to reduce the errors caused by image noise or artifacts, enhance the calculation accuracy of the key point position, and enhance the robustness and interpretability of the model while improving the accuracy and efficiency, thereby providing a more reliable solution for spinal key point detection.
[0039] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0041] Figure 1 A flowchart of a spine key point correction method provided for one or more embodiments of this specification;
[0042] Figure 2 A schematic diagram of a spine key point correction model structure of a spine key point correction method provided in one or more embodiments of this specification;
[0043] Figure 3 A schematic diagram of the effects of a spine key point model before and after correction of a spine key point correction method provided in one or more embodiments of this specification;
[0044] Figure 4 A schematic diagram of the composition of a spinal key point correction system provided for one or more embodiments of this specification;
[0045] Figure 5 A schematic diagram of the structure of an electronic device provided for one or more embodiments of this specification. DETAILED DESCRIPTION
[0046] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.
[0047] Method Embodiment
[0048] According to an embodiment of the present invention, a spine key point correction method is provided. Figure 1 A flowchart of a spine key point correction method provided by one or more embodiments of this specification, Figure 2 A schematic diagram of a spine key point correction model structure of a spine key point correction method provided in one or more embodiments of this specification, such as Figure 1 , Figure 2 As shown, the spine key point correction method according to an embodiment of the present invention specifically includes:
[0049] S1. Obtain a spinal X-ray image and read the X-ray image.
[0050] Obtain spinal X-ray images from a medical image library, or use medical X-ray equipment to take spinal images, generate image files in DICOM (Digital Imaging and Communications in Medicine) format, read DICOM files, and obtain image data.
[0051] S2. Extract the convolutional feature map from the read X-ray image and correct the convolutional feature map using the Linformer model.
[0052] A convolutional feature map is extracted from an X-ray image using a high-resolution network; and a convolutional feature map is corrected using a Linformer model, which means that given a convolutional feature map corresponding to an X-ray image, the convolutional feature map is divided into non-overlapping convolutional feature windows in the image height direction, and the convolutional feature map of each channel in each convolutional feature window is flattened into a convolutional feature vector, and the features in different channels remain unchanged.
[0053] S3. Calculate the center point heat map, center point offset map and corner point offset map of the vertebral key points according to the corrected convolution feature map.
[0054] Three subtask branches are derived from the high-resolution network feature extraction module. Based on the subtask branches, the high-resolution network feature extraction module is used to extract the convolution feature map, and the convolution feature map is separated into a center point heat map, a center point offset map and a corner point offset map respectively; the subtask branches adopt a two-layer convolutional neural network.
[0055] The center point heat map is used to locate the center of the vertebra and distinguish the vertebra from the background. The center point heat map contains one channel. In order to make the trained key point estimation model output a heat map, a Gaussian kernel is used to generate the center point heat map label. The mean and standard deviation of each Gaussian distribution depend on the center and size of the vertebra, respectively. When multiple Gaussian distributions overlap, the value at the overlapping position of the center point heat map is the maximum value of multiple distributions at that point. The center point offset map is used to obtain the four corner points of the vertebra relative to the center position. The four corner points are used to represent the vertebra, and a two-dimensional coordinate is used to describe the corner point. Therefore, the corner point offset map has eight channels.
[0056] S4. Use the dual activation Linformer model to jointly correct the center point offset map and the corner point offset map.
[0057] After extracting the center point of the vertebra from the center point heat map, extract the center point offset and corner point offset corresponding to the center point from the center point offset map and corner point offset map;
[0058] The center point offset and corner point offset of each vertebra in the spine are corrected by dual activation of the Linformer model, and the vertebra center point offset is added to the vertebra center point to obtain the corrected vertebra center point.
[0059] Among them, the specific method of correcting the center point offset and corner point offset corresponding to each vertebra in the spine by dual activation of the Linformer model is as follows:
[0060] The spine composed of vertebral key points is regarded as a continuous sequence, and the vertebra is regarded as a sampling point. The features of each sampling point are the center point offset and corner point offset of the vertebra. The Softmax function is used to normalize the features of the same dimensional features in the vertebral sequence dimension, and the Softmax function is used to normalize the features of the same vertebra in different feature dimensions. The above two normalized features are multiplied element by element, multiplied with the input features of the Linformer model after 1*1 convolution, and then the multiplied result is added to the input features of the Linformer model to obtain the corrected center point offset and corner point offset of each vertebra on the spine.
[0061] Finally, the corrected vertebral center point and the corrected corner point offset are added to obtain the corrected vertebral key point. The specific method is:
[0062] According to the corrected center point offset map, corner point offset map and center point heat map, the corrected vertebral key point position is calculated: the vertebral key point is calculated according to the vertebral center predicted by the three subtask branches, the vertebral center point offset and the corner point offset relative to the vertebral center; in the center point heat map, the heat value of the current pixel is compared with the heat values of its eight neighboring points, and the points with heat values not less than the eight connected neighboring points are found, and the K points with the largest heat values are taken as the vertebral center points; the center point offset corresponding to the vertebral center point is taken out from the center point offset map to obtain the vertebral center point position; the corner point offset corresponding to the vertebral center point is taken out from the corner point offset map, and the vertebral key point position is obtained in combination with the vertebral center point position.
[0063] S5. Calculate the corrected vertebral key point positions according to the corrected center point offset map and corner point offset map in combination with the center point heat map.
[0064] The spine X-ray image is received as input through the above method, and the key points of each vertebra on the spine are automatically located using the spine key point correction method. When the key points corrected by the system are not satisfactory, the key points of each vertebra on the spine can also be manually corrected by dragging the key points with a mouse or touch.
[0065] The beneficial effects of the present invention are as follows:
[0066] The present invention adopts an efficient feature extraction and correction method, and jointly corrects the center point offset map and the corner point offset map through the Linformer model. Combining the center point heat map, the center point offset map and the corner point offset map can reduce the error caused by image noise or artifacts, enhance the calculation accuracy of the key point position, and enhance the robustness and interpretability of the model while improving the accuracy and efficiency, providing a more reliable solution for spinal key point detection.
[0067] System Example
[0068] According to an embodiment of the present invention, a spine key point correction system is provided. Figure 4 A schematic diagram of a spinal key point correction system provided in one or more embodiments of this specification, such as Figure 3 As shown, the spine key point correction system according to an embodiment of the present invention specifically includes:
[0069] Image processing module 40: used to obtain spinal X-ray images and read the X-ray images;
[0070] The first correction module 42 is used to extract a convolution feature map from the read X-ray image and correct the convolution feature map using a Linformer model;
[0071] The first calculation module 44 is used to calculate the center point heat map, the center point offset map and the corner point offset map of the vertebral key points according to the corrected convolution feature map;
[0072] A second correction module 46 is used for applying dual activation to the Linformer model to jointly correct the center point offset map and the corner point offset map;
[0073] The second calculation module 48 is used to calculate the corrected vertebral key point positions according to the corrected center point offset map and corner point offset map in combination with the center point heat map.
[0074] The embodiment of the present invention is a system embodiment corresponding to the above-mentioned method embodiment. The specific operations of each module can be understood by referring to the description of the method embodiment, which will not be repeated here.
[0075] Device Example 1
[0076] An embodiment of the present invention provides an electronic device, such as Figure 5 As shown, it includes: a memory 50, a processor 52, and a computer program stored in the memory 50 and executable on the processor 52. When the computer program is executed by the processor 52, the following method steps are implemented:
[0077] S1. Obtaining a spinal X-ray image and reading the X-ray image;
[0078] S2. Extract a convolutional feature map from the read X-ray image and correct the convolutional feature map using the Linformer model;
[0079] S3. Calculate the center point heat map, center point offset map and corner point offset map of the vertebral key points according to the corrected convolution feature map;
[0080] S4. Using a dual activation Linformer model to jointly correct the center point offset map and the corner point offset map;
[0081] S5. Calculate the corrected vertebral key point positions according to the corrected center point offset map and corner point offset map in combination with the center point heat map.
[0082] Device Example 2
[0083] An embodiment of the present invention provides a computer-readable storage medium, on which a program for implementing information transmission is stored. When the program is executed by the processor 52, the following method steps are implemented:
[0084] S1. Obtaining a spinal X-ray image and reading the X-ray image;
[0085] S2. Extract a convolutional feature map from the read X-ray image and correct the convolutional feature map using the Linformer model;
[0086] S3. Calculate the center point heat map, center point offset map and corner point offset map of the vertebral key points according to the corrected convolution feature map;
[0087] S4. Using a dual activation Linformer model to jointly correct the center point offset map and the corner point offset map;
[0088] S5. Calculate the corrected vertebral key point positions according to the corrected center point offset map and corner point offset map in combination with the center point heat map.
[0089] The computer-readable storage medium in this embodiment includes, but is not limited to, ROM, RAM, magnetic disk or optical disk, etc.
[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A spine key point correction method, characterized in that: include: Acquire spinal X-ray images and read the X-ray images; Extract the convolution feature map from the read X-ray image and correct the convolution feature map using the Linformer model; Calculate the center point heat map, center point offset map and corner point offset map of the vertebral key points according to the corrected convolution feature map; Using a dual activation Linformer model to jointly correct the center point offset map and the corner point offset map; According to the corrected center point offset map and corner point offset map, combined with the center point heat map, the corrected vertebral key point positions are calculated.
2. The method according to claim 1, characterized in that The specific method of extracting the convolution feature map from the read X-ray image and correcting the convolution feature map using the Linformer model is as follows: Extract convolutional feature maps from X-ray images using a high-resolution network; The convolution feature map is divided into non-overlapping convolution feature windows in the image height direction, and the convolution feature map of each channel in each convolution feature window is flattened into a convolution feature vector, and the features in different channels remain unchanged.
3. The method according to claim 1, characterized in that The specific method for calculating the key point center point heat map, center point offset map and corner point offset map based on the corrected convolution feature map is as follows: Three subtask branches are derived from the high-resolution network feature extraction module. Based on the subtask branches, the high-resolution network feature extraction module is used to extract the convolution feature map, and the convolution feature map is separated into a center point heat map, a center point offset map and a corner point offset map respectively; the subtask branches adopt a two-layer convolutional neural network.
4. The method according to claim 3, characterized in that The center point heat map is used to locate the vertebral center and distinguish the vertebral body from the background. The center point heat map label is generated using a Gaussian kernel. The center point offset map is used to obtain four corner points of the vertebral body relative to the center position, and the four corner points are used to represent the vertebral body.
5. The method according to claim 1, characterized in that The specific method of using the dual activation Linformer model to jointly correct the center point offset map and the corner point offset map is: After extracting the center point of the vertebra from the center point heat map, extract the center point offset and corner point offset corresponding to the center point from the center point offset map and corner point offset map; The center point offset and corner point offset of each vertebra in the spine are corrected by dual activation of the Linformer model, and the vertebra center point offset is added to the vertebra center point to obtain the corrected vertebra center point. The corrected vertebral center point and the corrected corner point offset are added to obtain the corrected vertebral key point.
6. The method according to claim 5, characterized in that The corrected vertebral center point and the corrected corner point offset are added to obtain the corrected vertebral key point. The specific method is: According to the corrected center point offset map, corner point offset map and center point heat map, the corrected vertebral key point position is calculated: the vertebral key point is calculated according to the vertebral center predicted by the three subtask branches, the vertebral center point offset and the corner point offset relative to the vertebral center. In the center point heat map, compare the heat value of the current pixel with the heat values of its eight neighboring points, find the point whose heat value is not less than the eight connected neighboring points, and take the K points with the largest heat values as the center points of the vertebra; Take the center point offset corresponding to the vertebral center point from the center point offset map to obtain the vertebral center point position; The corner point offset corresponding to the vertebral center point is taken out from the corner point offset map, and the position of the vertebral key point is obtained by combining the position of the vertebral center point.
7. The method according to claim 5, characterized in that The specific method of correcting the center point offset and corner point offset of each vertebra in the spine by dual activation of the Linformer model is as follows: The spine composed of vertebral key points is regarded as a continuous sequence, and the vertebra is regarded as a sampling point. The characteristics of each sampling point are the center point offset and corner point offset of the vertebra; For the same dimension features, the Softmax function is used to normalize the features in the vertebral sequence dimension, and the Softmax function is used to normalize the features in different feature dimensions for the same vertebra; The above two normalized features are element-wise multiplied, and after 1*1 convolution, they are multiplied with the input features of the Linformer model. The multiplied result is then added to the input features of the Linformer model to obtain the corrected center point offset and corner point offset of each vertebra on the spine.
8. A spine key point correction system, characterized in that: include: Image processing module: used to obtain spinal X-ray images and read the X-ray images; The first correction module is used to extract a convolution feature map from the read X-ray image and correct the convolution feature map using the Linformer model; The first calculation module is used to calculate the center point heat map, center point offset map and corner point offset map of the vertebral key points according to the corrected convolution feature map; A second correction module: used for applying dual activation to the Linformer model to jointly correct the center point offset map and the corner point offset map; The second calculation module is used to calculate the corrected vertebral key point positions according to the corrected center point offset map and corner point offset map in combination with the center point heat map.
9. An electronic device, characterized in that: include: processor; as well as, A memory arranged to store computer executable instructions, which, when executed, cause the processor to implement the steps of the spine key point correction method according to any one of claims 1 to 7.
10. A storage medium, characterized in that: Used to store computer executable instructions, which, when executed, implement the steps of the spine key point correction method as described in any one of claims 1 to 7.