A spine key point detection system

Abstract

The present invention relates to a spine key point detection system, comprising: a dual-plane vertebra detection network having a first input end, a second input end, a first output end, and a second output end, for obtaining a front boundary box of a vertebra to be detected based on a front image of the vertebra to be detected inputted from the first input end and outputting it at the first output end, and for obtaining a side boundary box of the vertebra to be detected based on a side image of the vertebra to be detected inputted from the second input end and outputting it at the second output end; a first single vertebra key point detection network connected to the first output end signal and used for obtaining the coordinates of key points presented in the front image of the vertebra to be detected; and a second single vertebra key point detection network connected to the second output end signal and used for obtaining the coordinates of key points presented in the side image of the vertebra to be detected: the present invention can more accurately predict the coordinates of key points of the vertebra to be detected.

Description

Technical Field

[0001] The present invention relates to the field of medicine and health technology, and in particular to a spine key point detection system. Background Art

[0002] Adolescent idiopathic scoliosis (AIS) is a type of lateral deviation and axial rotation of the spine that often occurs in adolescents during or around puberty. If not treated in time, it will harm the patient's health. Therefore, accurate assessment of adolescent idiopathic scoliosis is of great significance to clinicians.

[0003] In clinical practice, spinal morphological parameters such as Cobb, Lumbar Lordosis (LL) and Thoracic Kyphosis (TK) are often used to quantitatively evaluate patients' scoliosis. In order to estimate these parameters, doctors need to mark several key points of each vertebra on the patient's frontal and lateral spine X-rays, then connect the key points and calculate the angle of the connecting line. Manually marking key points is not only time-consuming and laborious, but also has intra-observer and inter-observer differences, so automatic frontal and lateral spine key point detection is very necessary.

[0004] Currently, many neural network-based spine key point detection methods have been proposed. These methods are divided into two categories: single-stage methods and two-stage methods. The single-stage method directly estimates the coordinates of all key points on the entire frontal and lateral spine X-ray film. Due to the lack of vertebral position constraints, the single-stage method is easily affected by the background. In addition, some scholars have proposed a two-stage method. The two-stage method is divided into two steps: (1) vertebra detection, generally using an object detector to obtain the vertebral bounding box; (2) key point detection based on a single vertebra, usually using a single vertebra key point detector to locate the key point within the vertebral bounding box. The frontal spine X-ray (AP X-ray) has a higher key point detection accuracy due to fewer organs and ribs occlusion and higher clarity. Unlike the frontal spine X-ray, the lateral spine X-ray (LAT X-ray) has more serious rib occlusion and vertebral image blur caused by other organs, resulting in the poor performance of the above two-stage method in lateral vertebra detection, which in turn affects the accuracy of lateral key point detection.

[0005] In order to improve the accuracy of frontal and side vertebra detection, some scholars have improved the accuracy of vertebra detection by fusing the features of the frontal and side images of the spine. However, this fusion is a global feature fusion performed on the entire spine image, lacking precise guidance on the features of a single vertebra or local features of the vertebra, so the detection of occluded or blurred vertebrae in the side image is limited. In addition, in the existing two-stage method of vertebra detection network, the features of the vertebrae candidate region (Region of Interest RoI) are generally obtained to perform detection independently, and this method does not consider the connection between vertebrae in the same view image. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the prior art and provide a spine key point detection system with high precision and high efficiency in acquiring the coordinates of vertebral key points.

[0007] The present invention provides a spine key point detection system, and its technical solution is as follows:

[0008] A spine key point detection system, comprising:

[0009] A dual-plane vertebra detection network is provided with a first input end, a second input end, a first output end, and a second output end, and is used to obtain a front boundary box of the vertebra under test according to a front image of the vertebra under test inputted at the first input end and output it at the first output end, and to obtain a side boundary box of the vertebra under test according to a side image of the vertebra under test inputted at the second input end and output it at the second output end;

[0010] a first single vertebra key point detection network in signal communication with the first output terminal and configured to obtain the coordinates of the key points presented in the frontal image of the vertebra being detected;

[0011] A second single vertebra key point detection network is in signal communication with the second output terminal and is used to obtain the coordinates of the key points presented in the lateral image of the vertebra being detected.

[0012] By adopting the above technical scheme, compared with the prior art, the technical scheme provided by the present invention application can at least bring the following beneficial effects: by establishing a dual-plane vertebra detection network, the front image and the side image of the vertebra to be measured can be used simultaneously to obtain more accurate vertebral boundary boxes corresponding to the front image and the side image, and then based on this, the key point coordinates of the front image of the vertebra to be measured can be obtained by using the first single vertebra key point detection network and the key point coordinates of the side image of the vertebra to be measured can be obtained by using the second single vertebra key point detection network, thereby avoiding the interference of occluded or blurred vertebrae in the side image, and more accurately predicting all the key points of the vertebra to be measured; at the same time, the connection between vertebrae in the same perspective image is also taken into account.

[0013] Preferably, the dual-plane vertebra detection network comprises:

[0014] A frontal vertebra detection network in signal communication with the first input terminal and the first output terminal, configured to obtain a frontal bounding box of the vertebra to be detected based on the frontal image of the vertebra to be detected inputted at the first input terminal and output the frontal bounding box at the first output terminal;

[0015] A lateral vertebra detection network in signal communication with the second input terminal and the second output terminal, configured to obtain a lateral boundary box of the vertebra to be detected based on the lateral image of the vertebra to be detected inputted at the second input terminal and output it at the second output terminal;

[0016] This is beneficial in that during the vertebral key point detection process, the front and side images of the spine of the same patient can be simultaneously input to obtain the corresponding vertebral bounding boxes of the front and side images.

[0017] Preferably, the frontal vertebra detection network includes:

[0018] A first backbone network and a regional candidate network module in signal communication with the first input terminal, for obtaining frontal candidates and frontal feature maps on the frontal image of the vertebra under test;

[0019] A first candidate feature extraction module for extracting front candidate region features from the front feature map according to the front candidate;

[0020] A front vertebra detection head in signal communication with the first output terminal for obtaining a front bounding box of the vertebra to be detected and outputting the front bounding box at the first output terminal;

[0021] This enables the collection of vertebral frontal bounding box information.

[0022] Preferably, the lateral vertebra detection network comprises:

[0023] A second backbone network and a region candidate network module in signal communication with the second input terminal, and configured to obtain a side candidate and a side feature map on the side image of the vertebra under test;

[0024] A second candidate feature extraction module in signal communication with the second backbone network and the region candidate network module, configured to extract side candidate region features from the side feature map according to the side candidates;

[0025] The epipolar module is respectively connected to the second candidate feature extraction module and the first candidate feature extraction module by signal, and is used to fuse the front candidate region features with the side candidate region features and enhance the side candidate region features;

[0026] A context enhancement module in signal communication with the second candidate feature extraction module, used to establish information interaction between each of the side candidate region features and its adjacent side candidate region features, so that each of the side candidate region features can fuse the information of the adjacent side candidate region features and enhance the context information of the side candidate region features;

[0027] A fusion module which is in signal communication with the epipolar module and the context enhancement module respectively, and is used to enhance the features of the side candidate region according to the output information of the epipolar module and the context enhancement module;

[0028] A lateral vertebra detection head which is respectively in signal communication with the fusion module and the second output terminal and is used to obtain the lateral boundary box of the vertebra to be detected and output at the second output terminal;

[0029] By setting up the epipolar module, the fusion of the features of the frontal and lateral vertebrae candidate regions is achieved, and the fused features are used to assist the detection of the lateral vertebrae; through the context enhancement module, the contextual relationship of the vertebrae in the lateral image is utilized, and the attention mechanism is used to fuse the features of the adjacent lateral candidate regions to enhance the features of the lateral candidate regions.

[0030] Preferably, the first candidate feature extraction module and the second candidate feature extraction module are both provided with a program to extract candidate region features using a RoI Align method to clarify candidate features and improve operating efficiency.

[0031] Preferably, the polar module includes:

[0032] An epipolar matching unit in signal communication with the first candidate feature extraction module and the second candidate feature extraction module, and configured to perform feature matching on the front candidate region features and the side candidate region features in combination with the front candidate and the side candidate;

[0033] A dual-plane feature fusion unit for fusing the output information of the epipolar matching unit to obtain fused candidate region features;

[0034] This helps to improve the accuracy of lateral vertebrae detection, provide more accurate vertebrae position constraints for the second-stage single vertebrae key point detection network, and improve the detection accuracy of key points.

[0035] Preferably, the dual-plane feature fusion unit uses a built-in program to perform a fusion operation on any side candidate region feature. The fusion process is run using the following formula:

[0036]

[0037] in is the fused feature, Representation and Matched frontal candidate region features, w m,j express for The weight of k represents the number of matched front candidate region features;

[0038] Among them, the for The weight w m,j The calculation method is as follows:

[0039]

[0040] w K and w Q are two learnable mapping matrices;

[0041] This is conducive to completing feature fusion and avoiding problems such as feature ambiguity to a certain extent.

[0042] Preferably, the context enhancement module calculates the attention weights of any two side candidate region features using an attention mechanism through a built-in program, so that each side candidate region feature can fuse the information of adjacent candidate region features and enhance the context information of the side candidate region features.

[0043] Preferably, the first single vertebra key point detection network and the second single vertebra key point detection network both include a plurality of vertebrae arranged along the running direction;

[0044] A prediction network for obtaining a front or side vertebral center heat map and a key point offset map of the measured vertebra;

[0045] A relative coordinate acquisition module for obtaining relative coordinates of key points in a vertebral image according to the vertebral center heat map and the key point offset map;

[0046] An absolute coordinate acquisition module for obtaining the coordinates of the key points in the image of the measured vertebrae;

[0047] Based on this, the key point coordinate information of the front and side images of the measured vertebra can be obtained.

[0048] Preferably, the prediction network includes:

[0049] An encoder for extracting image features;

[0050] A decoder for obtaining a center heat map and key point offsets of the vertebra under test;

[0051] Then, in the encoding stage, the image features are extracted and then the center heat map and key point offsets of all vertebrae are obtained through the decoder. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the operation of the spine key point detection system according to an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of key points of the spine on the front and side of the vertebrae being tested according to an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the operation of the polar module according to an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of epipolar geometry according to an embodiment of the present invention;

[0056] Figure 5 A schematic diagram of the program operation of the context enhancement module according to an embodiment of the present invention;

[0057] Figure 6 A schematic diagram of the VFLDNet network structure of an embodiment of the present invention;

[0058] Figure 7 It is a schematic diagram of the dual-plane vertebra detection network structure of the present invention;

[0059] Figure 8 Schematic diagram of the first or second single vertebra key point detection network structure of the present invention. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0061] An embodiment of the present invention provides a spine key point detection system, comprising:

[0062] A dual-plane vertebra detection network is provided with a first input end, a second input end, a first output end, and a second output end, and is used to obtain a front boundary frame of the vertebra under test according to a front image of the vertebra under test inputted at the first input end and output it at the first output end, and to obtain a side boundary frame of the vertebra under test according to a side image of the vertebra under test inputted at the second input end and output it at the second output end;

[0063] A first single vertebra key point detection network in signal communication with the first output terminal for acquiring the coordinates of the key points presented in the frontal image of the vertebra being tested;

[0064] A second single vertebra key point detection network is connected to the second output terminal signal and is used to obtain the coordinates of the key points presented in the side image of the vertebra being tested.

[0065] The dual-plane vertebra detection network of this embodiment includes:

[0066] A frontal vertebra detection network in signal communication with the first input terminal and the first output terminal, for obtaining a frontal bounding box of the vertebra to be detected based on the frontal image of the vertebra to be detected inputted at the first input terminal and outputting it at the first output terminal;

[0067] A lateral vertebra detection network is connected to the second input terminal and the second output terminal for obtaining a lateral boundary frame of the vertebra to be detected based on the lateral image of the vertebra to be detected inputted from the second input terminal and outputting it at the second output terminal.

[0068] The frontal vertebra detection network includes the following components arranged along the running direction:

[0069] A first backbone network and a regional candidate network module in signal communication with the first input terminal, for obtaining frontal candidates and frontal feature maps on the frontal image of the vertebra under test;

[0070] A first candidate feature extraction module for extracting front candidate region features from the front feature map according to the front candidate; the module is based on the RoI Align method;

[0071] The front vertebra detection head is connected to the first output terminal signal and is used to obtain the front boundary box of the vertebra to be detected and output it at the first output terminal.

[0072] The frontal spine image is input into the first backbone network and the region candidate network to obtain the frontal feature map and the frontal candidates, and then the RoI Align method is used to extract the candidate region features from the frontal feature map according to the frontal candidates.

[0073] In this embodiment, the lateral vertebra detection network includes:

[0074] A second backbone network and a region candidate network module in signal communication with the second input terminal, and used for obtaining a side candidate and a side feature map on the side image of the vertebra under test;

[0075] A second candidate feature extraction module in signal communication with the second backbone network and the region candidate network module, configured to extract side candidate region features from the side feature map according to the side candidates;

[0076] The epipolar module is respectively connected to the second candidate feature extraction module and the first candidate feature extraction module for fusing the front candidate region features with the side candidate region features and enhancing the side candidate region features;

[0077] A context enhancement module in signal communication with the second candidate feature extraction module, used to establish information interaction between each side candidate region feature and its adjacent side candidate region features, so that each side candidate region feature can fuse the information of the adjacent side candidate region features and enhance the context information of the side candidate region features;

[0078] A fusion module which is respectively connected to the epipolar module and the context enhancement module signals, and is used to enhance the features of the side candidate regions according to the output information of the epipolar module and the context enhancement module; the module fuses the original side candidate region features with the features enhanced by the epipolar module and the context enhancement module in an additive manner;

[0079] The lateral vertebra detection head is respectively connected to the fusion module and the second output terminal signal to obtain the lateral boundary frame of the vertebra to be detected and output at the second output terminal.

[0080] The second candidate feature extraction module is provided with a program to extract candidate region features using the RoI Align method.

[0081] Specifically, the pole module includes:

[0082] An epipolar matching unit in signal communication with the first candidate feature extraction module and the second candidate feature extraction module, and used for performing feature matching on the front candidate region features and the side candidate region features in combination with the front candidate and the side candidate;

[0083] A dual-plane feature fusion unit is used to fuse the output information of the epipolar matching unit to obtain the fused candidate region features.

[0084] The epipolar matching unit is based on the epipolar matching concept. Epipolar matching uses the position information of the candidate to use the epipolar geometry to achieve the matching of the front and side candidates, thereby achieving the matching of the front and side candidate area features. Figure 4 As shown in Figure 1, the same vertebra satisfies the epipolar geometric relationship in the frontal and side images. Assume that there is a vertebra m in three-dimensional space, its center point is P, and the projection points of P on the frontal and side images are p AP and p LAT .

[0085] Assume that p LAT It is known that according to epipolar geometry and p LAT , we can determine the epipolar line l( Figure 4 Indicated by the dashed line):

[0086] l=Fp LAT (1)

[0087] F=K' -T R[T] × K -1(2)

[0088] Among them, K is the internal parameter of the X-ray imaging device, R and T are the external parameters of the imaging device, and these parameters are calibrated in advance using a special calibration device. [·] × represents a skew symmetric matrix. According to epipolar geometry, p AP It must be on the epipolar line l:

[0089] lp AP =0 (3)

[0090] Therefore, we can establish p LAT A rough matching relationship with the points on line l. Similarly, in the process of vertebra detection, for any side candidate, the epipolar line corresponding to the center point of the candidate on the front can be obtained using its center point coordinates and epipolar geometry, and then a rough matching relationship between the side candidate and the front candidate with the center point on the epipolar line is established. Specifically, k matching relationships are selected based on the distance between the center point of the front candidate and the epipolar line, and then a matching relationship is established between each side candidate region feature and the k front candidate region features based on the correspondence between the candidate and the candidate region features.

[0091] The dual-plane feature fusion unit uses a weighted fusion method for dual-plane candidate region features based on the attention mechanism. This fusion method uses the attention mechanism to learn the weights of the matching front candidate region features and the corresponding side candidate region features, and then performs weighted fusion on the front and side candidate region features. The dual-plane feature fusion unit uses an internal program to perform weighted fusion on any side candidate region feature. The fusion process is run using the following formula:

[0092]

[0093] in is the fused feature, Representation and Matched frontal candidate region features, w m,j express for The weight of k represents the number of matched front candidate region features;

[0094] in, for The weight w m,j The calculation method is as follows:

[0095]

[0096] w K and w Q are two learnable mapping matrices.

[0097] In this embodiment, the context enhancement module uses the attention mechanism through a built-in program to calculate the attention weights of any two side candidate region features.

[0098] The context enhancement module uses the attention mechanism to establish information interaction between each side candidate region feature and its adjacent side candidate region features, so that each side candidate region feature can integrate the information of the adjacent candidate region features and enhance the context information of the side candidate region features. The proximity relationship between any two side candidate region features in position is measured by the intersection over union (IOU). Figure 5 As shown:

[0099] The N side candidate region features input in the figure are first converted (reshaped) into a one-dimensional vector, and then the attention mechanism is used to calculate the attention weights of any two side candidate region features. and The attention weight calculation formula is as follows:

[0100]

[0101] where w K and w Q are two learnable mapping matrices respectively.

[0102] at the same time, and Corresponding candidates and The IOU is calculated as follows:

[0103]

[0104] Then the attention weight is combined with the intersection-over-union ratio and normalized to obtain the weights of the two candidate region features.

[0105]

[0106] for The features obtained after interacting with the feature information of the adjacent candidate regions are:

[0107]

[0108] Where N represents the number of input side candidate region features. Convert and output.

[0109] Preferably, the first single vertebra key point detection network and the second single vertebra key point detection network both include:

[0110] A prediction network for obtaining a center map of a measured vertebra and a key point offset map;

[0111] A relative coordinate acquisition module for obtaining relative coordinates of key points in a vertebral image according to a vertebral center heat map and a key point offset map;

[0112] An absolute coordinate acquisition module is used to obtain the coordinates of key points in the image of the measured vertebra.

[0113] The prediction network includes the following settings along the running direction:

[0114] An encoder for extracting image features;

[0115] Decoder for obtaining the center point heat map and key point offset of the measured vertebra.

[0116] The prediction network used in this embodiment is a vertebra-focused landmark detection network (VFLDNet). Figure 6 shown.

[0117] VFLDNet adopts an encoder-decoder structure. The encoder first extracts image features, and then the decoder obtains the center heat map and key point offset of all vertebrae. In the encoding stage, the network uses ResNet34 as the backbone network. After the input vertebrae image passes through the backbone network, four feature maps {C1, C2, C3, C4} of different sizes are obtained, and their sizes are respectively In the decoding stage, four feature maps of different resolutions are cascaded and fused. Specifically, the lower-resolution feature map is first upsampled to the same size as the high-resolution feature using the bilinear interpolation method, and then the two feature maps are concatenated and fused using a 1×1 convolution. In this way, the four feature maps obtained in the encoding stage are fused to obtain three feature maps of different sizes {P1, P2, P3}, and then the P3 feature map is passed through the vertebral center heat map prediction head and the key point offset prediction head to obtain the predicted vertebral center prediction heat map and key point offset map.

[0118] The relative coordinate acquisition module obtains the relative coordinates of the key points in the vertebral image through the predicted vertebral center heat map and the key point offset map. In this embodiment, for the predicted vertebral center heat map, the module uses the argmax method to obtain the coordinates of the highest response value in the heat map. as the predicted value of the vertebral center and then extracted in the predicted keypoint offset map The predicted value at , and then the two are added to obtain the relative coordinates of the key point in the vertebral image.

[0119] The center heat map constructed in this embodiment is based on the following formula:

[0120]

[0121] in is the relative coordinate of the center of the i-th vertebra in its vertebral bounding box, calculated from the coordinates of the four key points on the i-th vertebra:

[0122]

[0123] σ is the Gaussian radius, set to 3, and n is the downsampling ratio of the output heat map compared to the input image, where n is 4. In addition, the key point offset is the vector from the center of the vertebra to the key point:

[0124]

[0125] Finally, the absolute coordinate acquisition module adds the relative coordinates of the predicted key point in the vertebral image to the coordinates of the upper left corner of the vertebral prediction bounding box to obtain the coordinates of the key point in the spine image.

[0126] In summary, this embodiment improves the detection accuracy of the side vertebrae and provides more accurate vertebra position constraints for the second-stage single vertebrae key point detection network, thereby improving the detection accuracy of the key points. This embodiment adopts several strategies to achieve high-precision estimation of the task: (1) Propose an antipodal module, which uses the antipodal module of the same vertebra in the front and side spine images to achieve the fusion of the front and side vertebrae candidate region features, and uses the fused features to assist the detection of the side vertebrae. (2) Propose a context enhancement module, which uses the contextual relationship of the vertebrae in the side image and uses the attention mechanism to fuse the adjacent side candidate region features to enhance the side candidate region features. (3) The key points are located by predicting the vertebrae center and the key point offset, which is more robust than directly predicting the key point coordinates. Specifically, due to the influence of organ occlusion, some key point features are relatively blurred, while the vertebrae center is clearer than the key point feature, so the vertebrae center is easier to predict, and the offset between the key point and the vertebrae center is relatively fixed, so its prediction value is also more accurate when the image is blurred. Therefore, a more robust prediction result can be obtained by adding the vertebrae center and the key point offset.

[0127] In summary, although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A spine key point detection system, characterized by: include: A dual-plane vertebra detection network is provided with a first input end, a second input end, a first output end, and a second output end, and is used to obtain a front boundary box of the vertebra under test according to a front image of the vertebra under test inputted at the first input end and output it at the first output end, and to obtain a side boundary box of the vertebra under test according to a side image of the vertebra under test inputted at the second input end and output it at the second output end; a first single vertebra key point detection network in signal communication with the first output terminal and configured to obtain the coordinates of the key points presented in the frontal image of the vertebra being detected; a second single vertebra key point detection network in signal communication with the second output terminal and configured to obtain the coordinates of the key points presented in the lateral image of the vertebra being detected; The dual-plane vertebra detection network includes: A frontal vertebra detection network in signal communication with the first input terminal and the first output terminal, configured to obtain a frontal bounding box of the vertebra to be detected based on the frontal image of the vertebra to be detected inputted at the first input terminal and output the frontal bounding box at the first output terminal; A lateral vertebra detection network in signal communication with the second input terminal and the second output terminal, configured to obtain a lateral boundary box of the vertebra to be detected based on the lateral image of the vertebra to be detected inputted at the second input terminal and output it at the second output terminal; The frontal vertebra detection network includes: A first backbone network and a regional candidate network module in signal communication with the first input terminal, for obtaining frontal candidates and frontal feature maps on the frontal image of the vertebra under test; A first candidate feature extraction module for extracting front candidate region features from the front feature map according to the front candidate; A front vertebra detection head in signal communication with the first output terminal for obtaining a front bounding box of the vertebra to be detected and outputting the front bounding box at the first output terminal; The lateral vertebra detection network includes: A second backbone network and a region candidate network module in signal communication with the second input terminal, and configured to obtain a side candidate and a side feature map on the side image of the vertebra under test; A second candidate feature extraction module in signal communication with the second backbone network and the region candidate network module, configured to extract side candidate region features from the side feature map according to the side candidates; The epipolar module is respectively connected to the second candidate feature extraction module and the first candidate feature extraction module by signal, and is used to fuse the front candidate region features with the side candidate region features and enhance the side candidate region features; A context enhancement module in signal communication with the second candidate feature extraction module, used to establish information interaction between each of the side candidate region features and its adjacent side candidate region features, so that each of the side candidate region features can fuse the information of the adjacent side candidate region features and enhance the context information of the side candidate region features; A fusion module which is in signal communication with the epipolar module and the context enhancement module respectively, and is used to enhance the features of the side candidate region according to the output information of the epipolar module and the context enhancement module; A lateral vertebra detection head which is respectively in signal communication with the fusion module and the second output terminal and is used to obtain the lateral boundary box of the vertebra to be detected and output at the second output terminal; The polar module includes: An epipolar matching unit in signal communication with the first candidate feature extraction module and the second candidate feature extraction module, and configured to perform feature matching on the front candidate region features and the side candidate region features in combination with the front candidate and the side candidate; A dual-plane feature fusion unit is used to fuse the output information of the epipolar matching unit to obtain fused candidate region features.

2. The spine key point detection system according to claim 1, characterized in that: The first candidate feature extraction module and the second candidate feature extraction module are both provided with a program, and a RoIAlign method is used to extract candidate region features.

3. The spine key point detection system according to claim 2 is characterized in that: The dual-plane feature fusion unit uses a built-in program to perform a feature fusion on any side candidate region. The fusion process is run using the following formula: in is the fused feature, Representation and Matched frontal candidate region features, w m，j express for The weight of k represents the number of matched front candidate region features; Among them, the for The weight w m，j The calculation method is as follows: w K and w Q are two learnable mapping matrices, dot represents the dot product operation, and k represents the number of matched positive candidate region features.

4. The spine key point detection system according to claim 3 is characterized in that: The context enhancement module calculates the attention weights of any two side candidate region features using the attention mechanism through a built-in program.

5. The spine key point detection system according to claim 4 is characterized in that: The first single vertebra key point detection network and the second single vertebra key point detection network both include a network arranged along the running direction; A prediction network for obtaining a center heat map and a key point offset map of the measured vertebra; A relative coordinate acquisition module for obtaining relative coordinates of key points in a vertebral image according to the vertebral center heat map and the key point offset map; An absolute coordinate acquisition module for obtaining the coordinates of the key point in the image of the measured vertebra.

6. The spine key point detection system according to claim 5, characterized in that: The prediction network includes: An encoder for extracting image features; A decoder for obtaining a center heat map and key point offsets of the measured vertebra.

Images