Dual-view multi-scheme fusion-assisted guidance method for pedicle screw placement in the spine

Abstract

The present invention belongs to the field of artificial intelligence and automated human spine surgery. The present invention discloses a dual-view multi-scheme fusion assisted guidance method for spinal pedicle screw placement, comprising the following steps: obtaining an anteroposterior and lateral images of the patient's spine by taking X-rays of the patient's spine; obtaining an anteroposterior segmentation model and a lateral segmentation model of the spine based on the anteroposterior and lateral images of the patient's spine; solving the insertion point, insertion direction and end point of the pedicle screw placement surgical probe based on the regional differences and positional relationships of the three key structures in the spine image through image algorithms such as edge detection, polygonal fitting and corner detection; inserting the probe from the insertion point along the insertion direction to the end point, and detecting and tracking the probe during the operation, thereby assisting doctors in achieving automated surgery.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and automated human spine surgery, and more specifically, to a dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement. Background Art

[0002] Pedicle screw placement is a crucial step in spinal surgery, restoring spinal alignment and maintaining spinal stability. For patients with spinal instability or fractures, pedicle screw placement can reduce spinal instability and maintain spinal stability. It can also correct scoliosis or kyphosis. For patients with degenerative spinal diseases, it can maintain spinal stability and promote bone fusion after surgery.

[0003] During clinical surgery, the patient needs to be under general anesthesia, and X-rays or other imaging technologies are used to obtain the patient's spinal bone map. Then, under the doctor's perspective and manual operation, the probe entry point, entry angle and end point are explored and punctured to complete the surgery.

[0004] However, with technological advancements, numerous spinal surgery robots have emerged. In 1995, Mazor Robotics developed the Spineassist robot, specifically for assisted spinal puncture procedures. However, the system was designed to be semi-active, requiring the surgeon to perform much of the work manually. In 2005, the MBARS robot was introduced for spinal surgery. It uses image-free scanning of the cartilage surface to create a three-dimensional model, then uses "tactile" feedback from force sensors to make probe insertion decisions. In 2019, a 5G remote orthopedic robot was introduced. This robot requires percutaneous placement of a patient tracer on the spinous process. It then uses X-ray 3D scanning to remotely plan the pedicle screw trajectory before inserting the screw. Traditional image fusion methods are then used to measure the deviation between the screw and the actual position. In 2020, Wang Yanan et al. used the Renaissance spinal surgery robot to repair thoracic and lumbar fractures. This required manual calculation and determination of the puncture point and angle in advance, and then uploading this data to the robot to complete the remainder of the procedure.

[0005] In addition, during surgery, the insertion point and angle of the probe must be calculated in advance, and corresponding algorithms have also been developed. The 1986 Roy-Camille method uses the intersection of the extended line through the articular process joint space and the bisector of the transverse process as the insertion point, with an angle of 0° with the sagittal plane. The 1984 Magerl method uses the intersection of the perpendicular line of the outer edge of the superior articular process and the bisector of the transverse process as the insertion point, with an angle of 10°-15° with the sagittal plane. The 1992 Weinstein method uses the intersection of the outer lower edge of the superior articular process and the midline of the transverse process as the insertion point, with an angle of 10°-15° with the sagittal plane. The 2000 AO method by Dang Gengcheng et al. uses the intersection of the tangent line of the outer edge of the superior articular process and the midline of the transverse process as the insertion point, with an angle of 5° with the sagittal plane. In 1988, Shan Yunguan et al. adopted the "cross" positioning method. For L1-L4, a vertical line was drawn through the midpoint of the posterior margin of the mastoid process of the superior articular process, and a horizontal line was drawn through the accessory process of the transverse process. The intersection of these two lines was the insertion point, with an angle of 5°-10° relative to the sagittal plane. For L5, the insertion point was the deepest intersection between the mastoid process of the superior articular process and the accessory process of the transverse process, with an angle of 10°-15° relative to the sagittal plane. These algorithms require the calculation and judgment of key structures such as the pedicles, transverse processes, superior articular processes, and spinous processes in the anteroposterior view of the spine.

[0006] Regarding spinal pedicle screw placement surgeries, one option is for the doctor to complete the entire process manually, which is time-consuming, labor-intensive, and lacks precision. This requires very high technical skills, physical strength, and surgical experience. Another option is to use a spinal surgical robot to complete the operation. However, these spinal surgical robots not only rely on sensors and specialized software, but also require the doctor to participate in about half of the work, such as image location judgment, puncture path planning, and manual parameter calculation and pre-set operations. This undoubtedly consumes a considerable amount of time during the operation.

[0007] As for the calculation algorithms of the probe insertion point and insertion angle during surgery, such as Roy-Camille, Magerl, Weinstein, AO, and the "cross" positioning method, they are all calculated and selected based on the key parts of the spine in the anteroposterior view. Not only are they not effectively combined with the lateral view, but the point selection is too fixed and lacks clinical flexibility.

[0008] In view of this, the present invention provides a dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement. Summary of the Invention

[0009] To overcome the challenges of existing technologies, this paper proposes a dual-view, multi-scheme fusion-assisted guidance method for pedicle screw placement. This method utilizes AI in machine vision to replace the doctor's visual work. The SwinUnet medical segmentation network is used to segment key areas of the anteroposterior and lateral X-ray images of the same spinal segment. For the anteroposterior view, the pedicles and transverse processes are segmented; for the lateral view, the pedicles and vertebral bodies are segmented.

[0010] According to one aspect of the present invention, a dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement is provided, comprising the following steps:

[0011] By taking X-rays of the patient's spine, the patient's spine anteroposterior and lateral images are obtained;

[0012] Based on the patient's spine anteroposterior and lateral images, the medical image segmentation network SwinUnet was used for training to obtain the spine anteroposterior image segmentation model and the spine lateral image segmentation model respectively;

[0013] Through the trained spine segmentation model, the spine anteroposterior and lateral images of different patients are predicted and the effective structural areas are segmented;

[0014] According to the differences and positional relationships of the effective structural areas in the spinal image, the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe are determined;

[0015] Insert the probe from the entry point to the end point along the entry direction.

[0016] As a preferred embodiment of the present invention, the steps of training the medical image segmentation network SwinUnet based on the anteroposterior and lateral images of the patient's spine to obtain the anteroposterior segmentation model and the lateral segmentation model of the spine, respectively, include the following steps:

[0017] The SwinUnet network model includes an encoder part, a decoder part, a bottleneck part and a skip connection part;

[0018] The encoder part extracts features through multiple sliding windows and SwinTransformer blocks, reduces the size of the feature map, increases the receptive field, and downsamples the spine image into a spine feature map to extract spine image feature information;

[0019] The decoder part adopts a structure similar to the encoder, and performs upsampling through patch expansion layers and SwinTransformer blocks to restore the extracted spine feature map to the same size as the input image and present the effective spine structure segmentation result;

[0020] The bottleneck part consists of two Swin Transformer blocks, which are used to extract and transfer fine-grained feature information for the last time;

[0021] The jump connection part is a bridge connecting the encoder and the decoder, which is used to transfer the coarse to fine multi-scale spine feature information extracted by the encoder part to the decoder part, retaining more effective spine structure detail information.

[0022] As a preferred embodiment of the present invention, the step of predicting the anteroposterior and lateral spinal images of different patients by using the trained spinal segmentation model and segmenting the effective structural area includes the following steps:

[0023] The trained SwinUnet segmentation model is used to predict the spine frontal view to obtain three effective structural regions: pedicle, transverse process and vertebral body, and the lateral view to obtain two effective structural regions: pedicle and vertebral body.

[0024] As a preferred embodiment of the present invention, the step of determining the insertion point, insertion direction, and termination point of the pedicle screw insertion surgical probe based on the differences and positional relationships of the effective structural areas in the spinal image comprises the following steps:

[0025] Perform edge detection, polygonal fitting, and corner point detection on the pedicle, transverse process, and vertebral body regions predicted from the frontal image. Determine the leftmost corner point a1 and the rightmost corner point a2 on the upper edge of the vertebral body. Connect a1 and a2 to obtain line L1, and calculate the slope k1 of L1.

[0026] Based on the predicted pedicle point set, a line L2 with a slope of k1 above the point set and dividing the total area of ​​the point set by 1 / 4 is solved. The right boundary of the point set is intersected at point b1. A line L3 with a slope of k1 dividing the total area of ​​the point set by 1 / 2 is solved. The left boundary of the point set is intersected at point b2.

[0027] Solve for a line L4 with slope k1 that runs below the point set and divides 1 / 3 of the total area of ​​the point set. L4 intersects the right boundary of the point set at point b3. Connect b1b2, b2b3, and b1b3 to form a triangle b1b2b3, where line segment b1b3 intersects L3 at point b4.

[0028] Take point c1 on line segment b1b2 at a distance of 1 / 5 from b1, take point c2 on line segment b2b4 at a distance of 1 / 5 from b2, and take point c3 on line segment b2b3 at a distance of 1 / 5 from b3;

[0029] Connect c1c2, c2c3, and c1c3 to form a triangle c1c2c3, which is the reasonable safety zone for the probe insertion point, where line segment c1c3 intersects line L3 at point c4;

[0030] Solve for the midpoint e of the line segment c2c4, which is the initial probe entry point. The value of e can range from c2 to c4. Any point can be selected as the probe entry point. The probe entry angle changes with the entry point. The formula is α=5+5|vector c2e| / |vector c2c4|.

[0031] As a preferred embodiment of the present invention, the step of determining the insertion point, insertion direction, and termination point of the pedicle screw insertion surgical probe based on the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps:

[0032] Perform edge detection, polygonal fitting, and corner point detection on the pedicle and vertebral body regions predicted from the lateral image, and determine the upper corner point m1 and the lower corner point m2 on the rightmost side of the pedicle. Connect m1 and m2 to obtain line L5, and calculate the slope k5 of L5.

[0033] According to the correspondence between the anteroposterior and lateral images of the spine on the pedicle, the midpoint p1 of the line m1m2 is taken as the insertion point; a line L6 passing through point p1 and with a slope of k6 = -(1 / k5) is solved;

[0034] Solve for the leftmost lower corner point m4 of the vertebral body, connect p1m4 to get line L7, and find the slope k7 of L7; find the angle bisector L8 between L6 and L7, with a slope k8;

[0035] Solve for the angle β between L6 and L8, which is the angle. The formula is: β = arctan(|k8-k6| / |1+k8*k6|), and the range is [arctan(|k6|), arctan(|k8-k6| / |1+k8*k6|)];

[0036] Take the corner point m5 on the rightmost side of the vertebral body and the corner point m6 on the leftmost side of the vertebral body on line L8, and take the midpoint p2 between m5 and m6 as the end point.

[0037] As a preferred embodiment of the present invention, the step of determining the insertion point, insertion direction, and termination point of the pedicle screw insertion surgical probe based on the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps:

[0038] During the operation, the probe position and needle tip coordinates are detected, and the lens is positioned according to the X-ray lens, giving priority to removing irrelevant surrounding elements;

[0039] Then, threshold segmentation is performed to distinguish different areas of the X-ray film based on the difference in grayscale values. The pixel values ​​below the probe area are changed to 0, and the reasonable pixel value of the probe area is retained as 255.

[0040] Then, corrosion expansion is performed to weaken the area outside the probe;

[0041] Then, the probe is extracted, the area belonging to the probe is retained, and the rest is removed;

[0042] Finally, corner detection is performed based on the retained probe part to obtain the probe tip coordinates.

[0043] As a preferred embodiment of the present invention, the step of determining the insertion point, insertion direction, and termination point of the pedicle screw insertion surgical probe based on the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps:

[0044] Clarify the scale, clarify the conversion relationship between the pixel distance in the X-ray film and the distance the robotic arm moves in the actual environment, calculate and record the coordinate z1 where the needle tip appears in the image for the first time,

[0045] Then control the robotic arm to move the probe, calculate and record the coordinate z2 where the needle tip appears in the image for the second time, and the pixel distance L between the two coordinates can be obtained.

[0046] The actual moving distance of the probe tip is obtained as D, and the actual distance represented by each unit pixel in the image is calculated as S = D / L;

[0047] It is necessary to clarify the scale conversion relationship between the anteroposterior and lateral views of the spine. According to the scale, the scale of the anteroposterior view is S1 and the scale of the lateral view is S2.

[0048] As a preferred embodiment of the present invention, the step of inserting the probe from the insertion point to the end point along the insertion direction comprises the following steps:

[0049] Predict the probe entry point e of the frontal view and calculate the entry angle α;

[0050] Predict the puncture point p1, end point p2 and puncture angle β of the lateral view;

[0051] The length S3 between p1 and p2 in the lateral view can be calculated based on the scale, and the probe insertion depth in the actual environment can be calculated as S4 = S3 / sin(α);

[0052] Adjust the probe to e in the frontal view, with an insertion angle of α, the lateral view angle of p1 and an insertion angle of β, and then insert the probe to a depth of S4. When the lateral view angle coincides with p2, the probe insertion task is completed.

[0053] The technical effects and advantages of the dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement of the present invention are as follows:

[0054] This invention effectively captures global contextual information from spinal feature maps based on a medical segmentation network. It employs a U-shaped network structure and introduces skip connections to fuse low-level spinal feature maps with high-level ones, promoting information interaction between multi-level features and improving segmentation accuracy in key spinal regions, resulting in more detailed and accurate segmentation. This reduces the time doctors spend on spinal X-rays and avoids negative consequences such as misdiagnosis.

[0055] Combining the fixedness of previous algorithms with the flexibility in clinical practice, this algorithm enables doctors to select safe and reasonable puncture information based on different surgical scenarios based on reasonable puncture information, ensuring the safe and smooth progress of the operation, reducing the number of X-ray radiation exposures to doctors and patients, and protecting the safety of doctors and patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flowchart of the multi-scheme fusion-assisted guidance method for dual-view spinal pedicle screw placement surgery based on AI image segmentation of the present invention;

[0057] Figure 2 This is a detailed flow chart of the dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement surgery based on AI image segmentation of the present invention;

[0058] Figure 3 This is the SwinUnet network model structure diagram of the present invention;

[0059] Figure 4 This is the result diagram of the spinal position predicted by the SwinUnet network model of the present invention;

[0060] Figure 5 This is the result diagram of the spinal lateral position predicted by the SwinUnet network model of the present invention;

[0061] Figure 6 Schematic diagram of post-processing of the prediction results of the spine anteroposterior view of the network model of the present invention;

[0062] Figure 7 Schematic diagram of post-processing of lateral spine image prediction results of the network model of the present invention;

[0063] Figure 8 This is a schematic diagram of the detection process of the spine anteroposterior imaging probe of the present invention;

[0064] Figure 9 Schematic diagram of the lateral spine imaging probe detection process of the present invention. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0066] To calculate the probe's entry point, angle, and endpoint during surgery, this paper explores and invents a new algorithm based on the integration of multiple methods, including Roy-Camille, Magerl, Weinstein, AO, and the "cross" positioning method. This method, after SwinUnet segmentation, combines the correlation between the frontal and lateral views, adding clinical flexibility to the fixed nature of the previous algorithm to calculate the entry point and angle, and provides a safety range. Based on the regional differences and positional relationships of the effective structures in the spinal image, the entry point, direction, and endpoint of the pedicle screw placement probe are determined, thereby achieving automated surgery.

[0067] For the surgical scenario of spinal pedicle screw placement, the main tasks are to solve the following two tasks: 1. Confirmation and positioning of the probe entry point and probe end point: Through AI image segmentation, the main areas of spinal puncture are determined, and then the probe entry point and probe end point are determined through these areas during the operation, that is, the probe entry direction is determined; 2. Probe detection and placement: Through AI image segmentation and the results of the solved puncture parameters, the probe insertion direction is adjusted, and the probe is moved to match the entry point to the end point.

[0068] Example 1

[0069] like Figure 1-2 As shown, the dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement described in this embodiment includes the following steps:

[0070] S1: X-rays are taken of the patient's spine to obtain an anteroposterior and lateral view of the patient's spine;

[0071] Specifically, the doctor takes X-rays of the patient's spine to obtain the patient's spine in the frontal and lateral views, and transmits the images of the two spinal areas of the patient to the server. The server receives the captured spinal images. During the operation, there will be multiple probe adjustments and insertion corrections, which will also be photographed and transmitted.

[0072] S2: Based on the patient's spine anteroposterior and lateral images, the medical image segmentation network SwinUnet is used for training to obtain the spine anteroposterior image segmentation model and the spine lateral image segmentation model respectively;

[0073] Specifically, in the present invention, the medical image segmentation network SwinUnet is used for training to obtain the steps of the spine front view segmentation model and the spine lateral view segmentation model respectively. The SwinUnet network model includes an encoder part, a decoder part, a bottleneck part and a jump connection part. The encoder part extracts features through multiple sliding windows and SwinTransformer blocks, reduces the size of the feature map, increases the receptive field, and downsamples the spine image to a spine feature map to extract spine image feature information; the decoder part adopts a structure similar to the encoder, upsamples through the patch expansion layer and the Swin Transformer block, restores the extracted spine feature map to the same size as the input image, and presents the spine effective structure segmentation result; the bottleneck part is composed of two Swin Transformer blocks for the last extraction and transmission of fine-grained feature information; the jump connection part is a bridge connecting the encoder and the encoder, for transmitting the coarse to fine multi-scale spine feature information extracted by the encoder part to the decoder part, helping the network to retain more spine effective structure detail information and improve the accuracy of the spine effective structure segmentation result. At the end, there will be a 1x1 convolution as the segmentation head, responsible for outputting the segmentation result. Specific models such as Figure 3 shown.

[0074] S3: Using the trained spine segmentation model, the spine anteroposterior and lateral images of different patients are predicted to segment the effective structural areas;

[0075] Specifically, the trained spinal segmentation model is used to predict the anteroposterior and lateral images of the spine of different patients. In the step of segmenting the effective structure, the trained SwinUnet segmentation model is used to predict the anteroposterior image of the spine to obtain three effective structure areas of pedicle, transverse process and vertebral body, and the lateral image is used to predict two effective structure areas of pedicle and vertebral body.

[0076] During the training phase of the spine segmentation model, the present invention produced a data set of 2,000 spine parts (including anteroposterior and lateral spine images), of which 1,600 images were used as training sets and 400 images were used as test sets. During the model prediction phase, the trained spine segmentation model was used to predict the anteroposterior and lateral spine images of different patients. In the step of segmenting the effective structure, the trained SwinUnet segmentation model was used to predict the anteroposterior spine images to obtain three effective structural regions, namely, pedicles, transverse processes, and vertebral bodies, and the lateral spine images were used to obtain two effective structural regions, namely, pedicles and vertebral bodies. The results are shown in Figure 2. Figure 4 and Figure 5 As shown, it will be used for subsequent processing to obtain the reasonable position and range of the probe entry point and end point.

[0077] S4: Determine the insertion point, insertion direction, and end point of the pedicle screw insertion probe based on the regional differences and positional relationships of the effective structures in the spinal image;

[0078] Specifically, if Figure 6 As shown, edge detection, polygonal fitting and corner point detection are performed on the three regions of pedicle, transverse process and vertebral body predicted by the orthogonal image, and the leftmost corner point a1 and the rightmost corner point a2 of the upper edge of the vertebral body are solved. Line L1 is obtained by connecting a1a2, and the slope k1 of L1 is calculated; according to the predicted pedicle point set, a line L2 with a slope of k1 and above the point set and dividing 1 / 4 of the total area of ​​the point set is solved, and the right boundary of L2 intersection point set is at point b1; a line L3 with a slope of k1 and dividing 1 / 2 of the total area of ​​the point set is solved, and the left boundary of L3 intersection point set is at point b2; a line L4 with a slope of k1 and below the point set and dividing 1 / 3 of the total area of ​​the point set is solved, and the right boundary of L4 intersection point set is at point b3; connect b1b2, b2b3, b1b 3 forms a triangle b1b2b3, where the line segment b1b3 intersects the line L3 at point b4; take point c1 on the line segment b1b2 at a distance of 1 / 5 from b1, take point c2 on the line segment b2b4 at a distance of 1 / 5 from b2, and take point c3 on the line segment b2b3 at a distance of 1 / 5 from b3; connect c1c2, c2c3, and c1c3 to form a triangle c1c2c3, which is the reasonable safe zone for the probe insertion point, where the line segment c1c3 intersects the line L3 at point c4; solve for the midpoint e of the line segment c2c4, which is the initial probe insertion point. The value range of e can be between b2 and b4. Any point can be selected as the probe insertion point. The probe insertion angle changes with the insertion point. The formula is α=5+5|vector c2e| / |vector c2c4|.

[0079] like Figure 7 As shown in the figure, edge detection, polygonal fitting and corner point detection are performed on the two regions of the pedicle and vertebral body predicted by the lateral image, and the upper corner point m1 and the lower corner point m2 of the pedicle on the right are solved. The line L5 is obtained by connecting m1m2, and the slope k5 of L5 is calculated; according to the correspondence between the anteroposterior image and the lateral image of the spine on the pedicle, the midpoint p1 of the line m1m2 is taken as the insertion point; a line L6 passing through point p1 and with a slope k6 = -(1 / k5) is solved; the lower corner point m4 of the vertebral body on the left is solved, and the line L6 is obtained by connecting p1m4. Line L7, and calculate the slope k7 of L7; calculate the angle bisector L8 between L6 and L7, with a slope k8; solve the angle β between L6 and L8, which is the angle, the formula is β=arctan(|k8-k6| / |1+k8*k6|), the range is [arctan(|k6|),arctan(|k8-k6| / |1+k8*k6|)]; take the corner point m5 on the rightmost side of the vertebral body and the corner point m6 on the leftmost side of the vertebral body on line L8, and take the midpoint p2 of m5m6 as the end point.

[0080] like Figure 8 and Figure 9 As shown in the figure, for the detection of the probe position and needle tip coordinates during the operation, the lens positioning will be performed according to the X-ray lens, and irrelevant elements around will be removed first; then threshold segmentation will be performed to distinguish according to the difference in grayscale values ​​of different areas in the X-ray film, and the pixel value below the probe area will be changed to 0 (white), and the reasonable pixel value of the probe area will be retained as 255 (black); then corrosion and expansion will be performed to weaken the area outside the probe; then probe extraction will be performed to retain the area belonging to the probe and remove the rest; finally, corner detection will be performed based on the retained probe part to obtain the needle tip coordinates of the probe.

[0081] To clarify the scale, on the one hand, it is necessary to clarify the conversion relationship between the pixel distance in the X-ray film and the distance moved by the robotic arm in the actual environment. First, calculate and record the coordinate z1 where the needle tip appears in the image for the first time, then control the robotic arm to move the probe, calculate and record the coordinate z2 where the needle tip appears in the image for the second time, and then the pixel distance L between the two coordinates can be obtained. If the probe tip moves a distance D in reality, the actual distance represented by each unit pixel in the image can be calculated as S = D / L; on the other hand, it is necessary to clarify the scale conversion relationship between the anteroposterior and lateral views of the spine. According to the previously clarified scale method, the scale of the anteroposterior view can be obtained as S1, and the scale of the lateral view can be obtained as S2.

[0082] S5: Insert the probe from the entry point to the end point along the entry direction.

[0083] Specifically, in the key link of probe insertion, first, predict the insertion point e of the probe in the frontal view and calculate the insertion angle α, then predict the insertion point p1, end point p2 and insertion angle β of the lateral view, and calculate the length S3 between p1 and p2 in the lateral view according to the scale, and then calculate the actual environment. The insertion depth of the probe is S4 = S3 / sin(α), and then adjust the probe to e, angle α in the frontal view, and the lateral view angle is p1, angle β, and then the insertion depth S4, when the lateral view angle coincides with p2, the probe insertion task is completed. In this process, the doctor can select these parameters within the given range or use the calculated default parameters. Multiple shots will be taken during the puncture process to ensure that the probe insertion parameters are accurate, the range is safe and reasonable, and it meets the surgical specifications and standards.

[0084] For the selection of medical segmentation networks for surgical planning, we chose the SwinUnet network to train our spinal segmentation model. Its key advantage is its pure transformer-based network construction, effectively capturing global contextual information from the spinal feature map. Its U-shaped network architecture and skip connections fuse low-level and high-level spinal feature maps, promoting information interaction between multi-level features and improving segmentation accuracy in key spinal regions, resulting in more detailed and accurate segmentation. This reduces the time doctors spend on spinal X-rays and avoids negative consequences such as misdiagnosis.

[0085] Regarding the selection of the insertion point, insertion angle, and end point, our algorithm, based on the integration of algorithms such as Roy-Camille, Magerl, Weinstein, AO, and the "cross" positioning method, calculates the default insertion point and point safety area in the anteroposterior view of the spine, and calculates the safe insertion angle according to the formula. Then, based on the spatial position relationship between the anteroposterior view and the lateral view, the safe insertion point, safe insertion angle, and end point of the lateral view are calculated. The final insertion point, insertion angle, and insertion depth of the spinal pedicle screw placement surgery are calculated in combination with the anteroposterior view and the lateral view. In this algorithm, the present invention combines the fixedness of previous algorithms with the flexibility in clinical practice, so that doctors can, based on the given reasonable puncture information, select safe and reasonable puncture information in the scenario according to different surgical scenarios, to ensure the safe and smooth progress of the operation, reduce the number of X-ray radiation exposures to doctors and patients, and protect the safety of doctors and patients.

[0086] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0087] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A dual-view multi-scheme fusion-assisted guidance method for pedicle screw placement in the spine, characterized by: The following steps are involved: By taking X-rays of the patient's spine, the patient's spine anteroposterior and lateral images are obtained; Based on the patient's spine anteroposterior and lateral images, the medical image segmentation network SwinUnet was used for training to obtain the spine anteroposterior image segmentation model and the spine lateral image segmentation model respectively; Using the trained spinal segmentation model, the spine anteroposterior images of different patients are predicted to obtain three effective structural regions: pedicles, transverse processes, and vertebral bodies. The lateral images are predicted to obtain two effective structural regions: pedicles and vertebral bodies. Based on the differences and positional relationships of effective structural areas in spinal images, the insertion point, insertion direction, and termination point of the pedicle screw placement surgical probe are determined, including the following steps: Perform edge detection, polygonal fitting, and corner point detection on the pedicle, transverse process, and vertebral body regions predicted from the frontal image. Determine the leftmost corner point a1 and the rightmost corner point a2 on the upper edge of the vertebral body. Connect a1 and a2 to obtain line L1, and calculate the slope k1 of L1. Based on the predicted pedicle point set, a line L2 with a slope of k1 above the point set and dividing the total area of ​​the point set by 1 / 4 is solved. The right boundary of the point set is intersected at point b1. A line L3 with a slope of k1 dividing the total area of ​​the point set by 1 / 2 is solved. The left boundary of the point set is intersected at point b2. Solve for a line L4 with slope k1 that is below the point set and divides 1 / 3 of the total area of ​​the point set. L4 intersects the right boundary of the point set at point b3. Connect b1b2, b2b3, and b1b3 to form triangle b1b2b3, where line segment b1b3 intersects line L3 at point b4; Take point c1 on line segment b1b2 at a distance of 1 / 5 from b1, take point c2 on line segment b2b4 at a distance of 1 / 5 from b2, and take point c3 on line segment b2b3 at a distance of 1 / 5 from b3; Connect c1c2, c2c3, and c1c3 to form a triangle c1c2c3, which is the reasonable safety zone for the probe insertion point, where line segment c1c3 intersects line L3 at point c4; Solve for the midpoint e of the line segment c2c4, which is the initial probe entry point. The value of e can range from c2 to c4. Any point can be selected as the probe entry point. The probe entry angle varies with the entry point. The formula is α=5+5|vector c2e| / |vector c2c4|; Perform edge detection, polygonal fitting, and corner point detection on the pedicle and vertebral body regions predicted from the lateral image, and determine the upper corner point m1 and the lower corner point m2 on the rightmost side of the pedicle. Connect m1 and m2 to obtain line L5, and calculate the slope k5 of L5. According to the correspondence between the anteroposterior and lateral images of the spine on the pedicle, the midpoint p1 of the line m1m2 is taken as the insertion point; a line L6 passing through point p1 and with a slope of k6=-(1 / k5) is solved; Solve for the leftmost lower corner point m4 of the vertebral body, connect p1m4 to get line L7, and find the slope k7 of L7; find the angle bisector L8 between L6 and L7, with a slope k8; Solve for the angle β between L6 and L8, which is the angle, and the formula is: , the range is ; Take the corner point m5 on the rightmost side of the vertebral body and the corner point m6 on the leftmost side of the vertebral body on line L8, and take the midpoint p2 between m5 and m6 as the end point.

2. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 1, characterized in that: The step of training the SwinUnet network model of the medical image segmentation network based on the anteroposterior and lateral images of the patient's spine to obtain the anteroposterior segmentation model and the lateral segmentation model of the spine, respectively, includes the following steps: The SwinUnet network model includes an encoder part, a decoder part, a bottleneck part and a skip connection part; The encoder part extracts features through multiple sliding windows and Swin Transformer blocks, reduces the size of the feature map, increases the receptive field, and downsamples the spine image into a spine feature map to extract spine image feature information; The decoder adopts a structure similar to that of the encoder, and performs upsampling through patch expansion layers and Swin Transformer blocks to restore the extracted spine feature map to the same size as the input image and present the effective spine structure segmentation result; The bottleneck part consists of two Swin Transformer blocks, which are used to extract and transfer fine-grained feature information for the last time; The jump connection part is a bridge connecting the encoder and the decoder, which is used to transfer the coarse to fine multi-scale spine feature information extracted by the encoder part to the decoder part, retaining more effective spine structure detail information.

3. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 2, characterized in that: The step of determining the insertion point, insertion direction, and end point of the pedicle screw placement surgical probe based on the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps: During the operation, the probe position and needle tip coordinates are detected, and the lens is positioned according to the X-ray lens, giving priority to removing irrelevant surrounding elements; Then, threshold segmentation is performed to distinguish different areas of the X-ray film based on the difference in grayscale values. The pixel values ​​below the probe area are changed to 0, and the reasonable pixel value of the probe area is retained as 255. Then, corrosion expansion is performed to weaken the area outside the probe; Then, the probe is extracted, the area belonging to the probe is retained, and the rest is removed; Finally, corner detection is performed based on the retained probe part to obtain the probe tip coordinates.

4. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 3, characterized in that: The step of determining the insertion point, insertion direction, and end point of the pedicle screw placement surgical probe based on the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps: Clarify the scale, clarify the conversion relationship between the pixel distance in the X-ray film and the distance the robotic arm moves in the actual environment, calculate and record the coordinate z1 where the needle tip appears in the image for the first time, Then control the robotic arm to move the probe, calculate and record the coordinate z2 where the needle tip appears in the image for the second time, and the pixel distance L between the two coordinates can be obtained. The actual moving distance of the probe tip is obtained as D, and the actual distance represented by each unit pixel in the image is calculated as S=D / L; It is necessary to clarify the scale conversion relationship between the anteroposterior and lateral views of the spine. According to the scale, the scale of the anteroposterior view is S1 and the scale of the lateral view is S2.

5. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 4, characterized in that: The step of inserting the probe from the insertion point to the end point along the insertion direction comprises the following steps: Predict the probe entry point e of the frontal view and calculate the entry angle α; Predict the puncture point p1, end point p2 and puncture angle β of the lateral view; The length S3 between p1 and p2 in the lateral view can be calculated based on the scale, and the insertion depth of the probe in the actual environment can be calculated as S4=S3 / sin(α).

Images