Multi-scheme fusion auxiliary guiding method for double-view spinal pedicle screw implantation

Abstract

The invention belongs to the field of artificial intelligence and human spine automatic surgery, and discloses a double-view spine pedicle screw implantation multi-scheme fusion auxiliary guiding method which comprises the following steps: shooting X-rays on a spine part of a patient to obtain a patient spine normal position map and a patient spine side position map; according to the spinal normal position map image and the spinal lateral position map image of the patient, a spinal normal position map segmentation model and a spinal lateral position map segmentation model are obtained; according to the regional difference and the position relation of the three key structures in the spine image, the puncture point, the puncture direction and the termination point of the pedicle screw implantation operation probe are solved through image algorithms such as edge detection, multilateral fitting and angular point detection; the probe is punctured from the puncture entering point to the termination point in the puncture entering direction, and the probe is detected and tracked in the operation process, so that doctors are assisted in achieving the automatic operation.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and automated human spine surgery, and more specifically, to a dual-viewing spinal pedicle screw placement multi-scheme fusion-assisted guidance method. Background Art

[0002] Pedicle screw placement is an important part of spinal surgery, which plays a role in restoring the sequence of the spine and maintaining the stability of the spine. For patients with spinal instability or spinal fractures, pedicle screw placement can play a role in resetting and maintaining spinal stability. For patients with scoliosis or kyphosis, correction can be performed. For spinal degenerative diseases, the stability of the spine can be maintained after surgery and bone fusion can be promoted.

[0003] During clinical surgery, the patient is required to be under general anesthesia, and X-rays or other imaging technologies are used to obtain the patient's spinal bone map. Then, under the doctor's perspective and manual operation, the probe's entry point, entry angle, and end point are probed and punctured to complete the surgery.

[0004] However, with the advancement of science and technology, many spinal surgery robots have been born. In 1995, Mazor Robotics developed the Spineassist robot, which is specifically used to complete spinal assisted puncture surgery, but the system is only designed to be semi-active, and a lot of work requires manual operation by doctors. In 2005, the MBARS robot was born and applied to spinal surgery. It uses image-free scanning of the cartilage surface to create a three-dimensional model, and then uses the feedback information "tactile" of the force sensor to complete the probe insertion decision. In 2019, the 5G remote orthopedic robot was born. It needs to place a patient tracer percutaneously on the spinous process of the spine, and then use X-ray 3D scanning to perform remote pedicle screw trajectory planning, and then insert the screw. The traditional image fusion method is used to measure the deviation between the screw and the actual position. In 2020, Wang Yanan and others used the Renaissance spinal surgery robot to perform thoracolumbar fracture repair surgery. It is necessary to manually calculate and determine the puncture point and angle in advance, and then upload the data to the robot to complete the rest of the operation.

[0005] In addition, during the operation, the insertion point and angle of the probe need to be calculated in advance, and the corresponding algorithms have also been studied. The Roy-Camille method in 1986 uses the intersection of the extended line through the articular process joint space and the bisector of the transverse process as the insertion point, and the angle with the sagittal plane is 0° as the insertion angle. The Magerl method in 1984 uses the intersection of the vertical line of the outer edge of the superior articular process and the bisector of the transverse process as the insertion point, and the angle with the sagittal plane is 10°-15°. Weinstein in 1992 uses the intersection of the outer lower edge of the superior articular process and the midline of the transverse process as the insertion point, and the angle with the sagittal plane is 10-15. The AO method of Dang Gengcheng et al. in 2000 uses the intersection of the tangent of the outer edge of the superior articular process and the midline of the transverse process as the insertion point, and the angle with the sagittal plane is 5°. In 1988, Shan Yunguan et al. adopted the "cross" positioning method. For L1-L4, a vertical line was drawn from the midpoint of the posterior edge of the mastoid process of the superior articular process, and a horizontal line was drawn on the accessory process of the transverse process. The intersection of the two lines was the insertion point, and the insertion angle was 5°-10° with the sagittal plane. For L5, the deepest intersection between the mastoid process of the superior articular process and the accessory process of the transverse process was the insertion point, and the insertion angle was 10°-15° with the sagittal plane. These algorithms need to be combined with key structures such as pedicles, transverse processes, superior articular processes, and spinous processes in the anteroposterior view of the spine to calculate and judge.

[0006] For the surgical plan of spinal pedicle screw placement, one scenario is that the doctor completes the entire surgical process manually, which is time-consuming and labor-intensive, and the accuracy is poor, which has very high requirements for the doctor's skills, physical strength and surgical experience. The other scenario is to use a spinal surgical robot to complete the surgery. However, these spinal surgical robots not only rely on sensors and previous special software, but also require doctors to participate in about half of the work, such as image site judgment, puncture path planning, manual calculation of parameters and pre-given operations. These undoubtedly consume a lot of time during the operation.

[0007] As for the calculation algorithms of the insertion point and angle of the probe during surgery, such as Roy-Camille, Magerl, Weinstein, AO, and the "cross" positioning method, they are all calculated and selected based on the key parts of the spine in the frontal view. Not only are they not effectively combined with the lateral view, but the point selection is too fixed and lacks clinical flexibility.

[0008] In view of this, the present invention provides a dual-view spinal pedicle screw placement multi-scheme fusion auxiliary guidance method. Summary of the invention

[0009] In order to overcome the problems in the prior art, the present invention proposes a dual-view multi-scheme fusion assisted guidance method for pedicle screw placement in the spine, which uses the application of AI in machine vision to replace the doctor's visual work. The SwinUnet medical segmentation network is used to segment the key parts of the frontal and lateral views of the same section of the spine taken by X-ray. For the frontal view, the pedicle and transverse process are selected for segmentation, and for the lateral view, the pedicle and vertebral body are selected for segmentation.

[0010] According to one aspect of the present invention, a dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement is provided, comprising the following steps:

[0011] By taking X-rays of the patient's spine, the patient's spine frontal and lateral images are obtained;

[0012] According to the patient's spine anteroposterior and lateral images, the medical image segmentation network SwinUnet is used for training to obtain the spine anteroposterior segmentation model and the spine lateral segmentation model respectively;

[0013] Through the trained spine segmentation model, the frontal and lateral images of the spine of different patients are predicted and the effective structure area is segmented;

[0014] According to the differences and positional relationships of the effective structural areas in the spinal image, the insertion point, insertion direction and end point of the surgical probe for pedicle screw placement are determined;

[0015] Insert the probe from the puncture point to the end point along the puncture direction.

[0016] As a preferred embodiment of the present invention, the steps of training the medical image segmentation network SwinUnet according to the anteroposterior and lateral images of the patient's spine to obtain the anteroposterior segmentation model and the lateral segmentation model of the spine, respectively, include the following steps:

[0017] The SwinUnet network model includes an encoder part, a decoder part, a bottleneck part and a skip connection part;

[0018] The encoder part extracts features through multiple sliding windows and SwinTransformer blocks, reduces the size of the feature map, increases the receptive field, and downsamples the spine image into a spine feature map to extract spine image feature information;

[0019] The decoder part adopts a structure similar to that of the encoder, and performs upsampling through patch expansion layers and SwinTransformer blocks to restore the extracted spine feature map to the same size as the input image and present the effective spine structure segmentation result;

[0020] The bottleneck part consists of two Swin Transformer blocks, which are used to extract and transfer fine-grained feature information for the last time;

[0021] The jump connection part is a bridge connecting the encoder and the decoder, which is used to transfer the coarse to fine multi-scale spine feature information extracted by the encoder part to the decoder part, and retain more effective spine structure detail information.

[0022] As a preferred embodiment of the present invention, the step of predicting the anteroposterior and lateral images of the spine of different patients by using the trained spine segmentation model to segment the effective structure area includes the following steps:

[0023] The trained SwinUnet segmentation model is used to predict the three effective structural regions of pedicle, transverse process and vertebral body in the frontal view of the spine, and the two effective structural regions of pedicle and vertebral body are predicted in the lateral view.

[0024] As a preferred embodiment of the present invention, the step of determining the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe according to the difference and positional relationship of the effective structural area in the spinal image comprises the following steps:

[0025] Perform edge detection, polygonal fitting and corner point detection on the three regions of pedicle, transverse process and vertebral body predicted by the frontal image, solve the leftmost corner point a1 and the rightmost corner point a2 on the upper edge of the vertebral body, connect a1a2 to get line L1, and calculate the slope k1 of L1;

[0026] According to the predicted pedicle point set, a line L2 with a slope of k1 and above the point set and dividing the total area of ​​the point set by 1 / 4 is solved, and the right boundary of the point set is intersected at point b1; a line L3 with a slope of k1 and dividing the total area of ​​the point set by 1 / 2 is solved, and the left boundary of the point set is intersected at point b2;

[0027] Solve for a line L4 with a slope of k1 that is below the point set and divides 1 / 3 of the total area of ​​the point set. L4 intersects the right boundary of the point set at point b3. Connect b1b2, b2b3, and b1b3 to form a triangle b1b2b3, where line segment b1b3 intersects line L3 at point b4.

[0028] Take point c1 at a distance of 1 / 5 from b1 on line segment b1b2, take point c2 at a distance of 1 / 5 from b2 on line segment b2b4, and take point c3 at a distance of 1 / 5 from b3 on line segment b2b3;

[0029] Connect c1c2, c2c3, c1c3 to form a triangle c1c2c3, which is the reasonable safety zone for the probe insertion point, where line segment c1c3 intersects line L3 at point c4;

[0030] The midpoint e of the line segment c2c4 is solved, which is the initial probe entry point. The value range of e can be between c2 and c4. Any point can be selected as the probe entry point. The probe entry angle changes with the entry point. The formula is α=5+5|vector c2e| / |vector c2c4|.

[0031] As a preferred embodiment of the present invention, the step of determining the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe according to the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps:

[0032] Perform edge detection, polygonal fitting and corner point detection on the two regions of the pedicle and vertebral body predicted by the lateral image, solve the upper corner point m1 and the lower corner point m2 on the rightmost side of the pedicle, connect m1m2 to get the line L5, and calculate the slope k5 of L5;

[0033] According to the correspondence between the anteroposterior and lateral images of the spine on the pedicle, the midpoint p1 of the line m1m2 is taken as the insertion point; a line L6 passing through the point p1 and having a slope of k6 = -(1 / k5) is solved;

[0034] Solve for the leftmost lower corner point m4 of the vertebral body, connect p1m4 to get line L7, and find the slope k7 of L7; find the angle bisector L8 between L6 and L7, with a slope k8;

[0035] Solve for the angle β between L6 and L8, which is the angle. The formula is: β = arctan (|k8-k6| / |1+k8*k6|), and the range is [arctan (|k6|), arctan (|k8-k6| / |1+k8*k6|)];

[0036] Take the corner point m5 on the rightmost side of the vertebral body and the corner point m6 on the leftmost side of the vertebral body on the line L8, and take the midpoint p2 of m5m6 as the end point.

[0037] As a preferred embodiment of the present invention, the step of determining the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe according to the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps:

[0038] During the operation, the position of the probe and the coordinates of the needle tip are detected. The lens is positioned according to the X-ray lens, and irrelevant surrounding elements are removed first;

[0039] Then, threshold segmentation is performed to distinguish different areas in the X-ray film according to the difference in grayscale values. The pixel values ​​below the probe area are changed to 0, and the reasonable pixel value of the probe area is retained as 255.

[0040] Then, corrosion expansion is performed to weaken the area outside the probe;

[0041] Then the probe extraction is performed, the area belonging to the probe is retained and the rest is removed;

[0042] Finally, corner point detection is performed based on the retained probe part to obtain the probe tip coordinates.

[0043] As a preferred embodiment of the present invention, the step of determining the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe according to the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps:

[0044] Clarify the scale, clarify the conversion relationship between the pixel distance in the X-ray film and the distance the robotic arm moves in the actual environment, calculate and record the coordinate z1 where the needle tip appears in the image for the first time,

[0045] Then the robot arm is controlled to move the probe, and the coordinate z2 where the needle tip appears for the second time in the image is calculated and recorded. The pixel distance L between the two coordinates can be obtained.

[0046] The actual moving distance of the probe tip is obtained as D, and the scale of the actual distance represented by each unit pixel in the image is calculated as S = D / L;

[0047] It is necessary to clarify the scale conversion relationship between the anteroposterior and lateral views of the spine, and obtain the scale of the anteroposterior view as S1 and the scale of the lateral view as S2 according to the scale.

[0048] As a preferred embodiment of the present invention, the step of inserting the probe from the insertion point along the insertion direction to the end point comprises the following steps:

[0049] Predict the puncture point e of the probe frontal image and calculate the puncture angle α;

[0050] Predict the puncture point p1, end point p2 and puncture angle β of the lateral view;

[0051] The length S3 between p1 and p2 in the lateral view can be calculated based on the scale, and the insertion depth of the probe in the actual environment can be calculated as S4 = S3 / sin(α);

[0052] Adjust the probe to e in the frontal view, with an insertion angle of α, the viewing angle of the lateral view to p1 and the insertion angle to β, then insert the probe to a depth of S4. When the viewing angle of the lateral view coincides with p2, the probe insertion task is completed.

[0053] Technical effects and advantages of the dual-view multi-scheme fusion-assisted guidance method for pedicle screw placement in the spine of the present invention:

[0054] The present invention effectively obtains global context information in the spinal feature map based on the medical segmentation network; it adopts a U-shaped network structure and introduces jump connections to fuse low-level spinal feature maps with high-level spinal feature maps, promote information interaction between multi-level features, improve the segmentation accuracy of key areas of the spine, and achieve more detailed and accurate segmentation of key areas. It saves doctors time for spinal X-ray diagnosis and avoids negative situations such as misdiagnosis.

[0055] Combining the fixedness of previous algorithms with the flexibility in clinical practice, this algorithm enables doctors to select safe and reasonable puncture information based on different surgical scenarios based on reasonable puncture information, ensuring the safe and smooth progress of the operation, reducing the number of X-ray radiation exposures to doctors and patients, and protecting the safety of doctors and patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flowchart of the multi-scheme fusion-assisted guidance method for dual-view spinal pedicle screw placement surgery based on AI image segmentation of the present invention;

[0057] Figure 2 It is a detailed flow chart of the multi-scheme fusion-assisted guidance method for dual-view spinal pedicle screw placement surgery based on AI image segmentation of the present invention;

[0058] Figure 3 It is the structural diagram of the SwinUnet network model of the present invention;

[0059] Figure 4 This is a graph of the spinal anteroposterior position predicted by the SwinUnet network model of the present invention;

[0060] Figure 5 This is a graph of the spinal lateral position prediction result of the SwinUnet network model of the present invention;

[0061] Figure 6 It is a schematic diagram of post-processing of the prediction results of the spine anteroposterior view of the network model of the present invention;

[0062] Figure 7 It is a schematic diagram of post-processing of the lateral spine image prediction results of the network model of the present invention;

[0063] Figure 8 It is a schematic diagram of the detection process of the spine anteroposterior imaging probe of the present invention;

[0064] Fig. 9 It is a schematic diagram of the detection process of the lateral spine image probe of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] For the calculation of the insertion point, insertion angle and end point of the probe during surgery, the present invention explores and invents a new calculation algorithm based on the integration of multiple methods such as Roy-Camille, Magerl, Weinstein, AO, and the "cross" positioning method. After SwinUnet segmentation, this method combines the correlation between the frontal image and the lateral image, adds the flexibility required in clinical practice to the fixedness of the previous algorithm, calculates the insertion point and insertion angle, and gives a safe range. According to the regional differences and positional relationships of the effective structures in the spinal image, the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe are determined, thereby realizing automated surgery.

[0067] For the surgical scenario of spinal pedicle screw placement, the following two tasks are mainly solved: 1. Confirmation and positioning of the probe entry point and probe end point: Through AI image segmentation, the main area of ​​spinal puncture is determined, and then the probe entry point and probe end point are determined through these areas during the operation, that is, the probe entry direction is determined; 2. Probe detection and placement: Through AI image segmentation and the results of the solved puncture parameters, the probe insertion direction is adjusted, and the probe is moved to match the entry point to the end point.

[0068] Example 1

[0069] like Figure 1-2 As shown, the dual-view spinal pedicle screw placement multi-scheme fusion auxiliary guidance method described in this embodiment includes the following steps:

[0070] S1: By taking X-rays of the patient's spine, the patient's spine anteroposterior and lateral images are obtained;

[0071] Specifically, the doctor takes X-rays of the patient's spine to obtain the patient's spine frontal and lateral images, and transmits the two spinal area images of the patient to the server. The server receives the captured spinal images. During the operation, there will be multiple probe adjustments and insertion corrections, which will also be photographed and transmitted.

[0072] S2: Based on the patient's spine anteroposterior and lateral images, the medical image segmentation network SwinUnet is used for training to obtain the spine anteroposterior segmentation model and the spine lateral segmentation model respectively;

[0073] Specifically, in the present invention, the medical image segmentation network SwinUnet is used for training, and in the steps of obtaining the spine frontal image segmentation model and the spine lateral image segmentation model respectively, the SwinUnet network model includes an encoder part, a decoder part, a bottleneck part and a jump connection part. The encoder part extracts features through multiple sliding windows and SwinTransformer blocks, reduces the size of the feature map, increases the receptive field, and downsamples the spine image to a spine feature map to extract the spine image feature information; the decoder part adopts a structure similar to the encoder, and upsamples through the patch expansion layer and the Swin Transformer block, and restores the extracted spine feature map to the same size as the input image, and presents the spine effective structure segmentation result; the bottleneck part is composed of two Swin Transformer blocks, which are used to extract and transmit fine-grained feature information for the last time; the jump connection part is a bridge connecting the encoder and the encoder, which is used to pass the coarse to fine multi-scale spine feature information extracted by the encoder part to the decoder part, helping the network to retain more spine effective structure detail information and improve the accuracy of the spine effective structure segmentation result. At the end, there will be a 1x1 convolution as a segmentation head responsible for outputting the segmentation result. Specific models such as Figure 3 shown.

[0074] S3: Use the trained spine segmentation model to predict the anteroposterior and lateral images of the spine of different patients and segment the effective structure area;

[0075] Specifically, the trained spinal segmentation model is used to predict the anteroposterior and lateral views of the spine of different patients. In the step of segmenting the effective structure, the trained SwinUnet segmentation model is used to predict the anteroposterior view of the spine to obtain three effective structural regions of pedicle, transverse process and vertebral body, and the lateral view is predicted to obtain two effective structural regions of pedicle and vertebral body.

[0076] In the training stage of the spine segmentation model, the present invention produced a data set of 2000 images of the spine (including the anteroposterior and lateral views of the spine), of which 1600 images were used as a training set and 400 images were used as a test set. In the model prediction stage, the trained spine segmentation model was used to predict the anteroposterior and lateral views of the spine of different patients. In the step of segmenting the effective structure, the trained SwinUnet segmentation model was used to predict the anteroposterior view of the spine to obtain three effective structural areas of pedicle, transverse process and vertebral body, and the lateral view was predicted to obtain two effective structural areas of pedicle and vertebral body. The results are as follows: Figure 4 and Figure 5 As shown, it will be used in the subsequent processing to obtain the reasonable position and range of the probe entry point and termination point.

[0077] S4: Determine the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe based on the regional differences and positional relationships of the effective structures in the spinal image;

[0078] Specifically, Figure 6 As shown, edge detection, polygonal fitting and corner point detection are performed on the three regions of pedicle, transverse process and vertebral body predicted by the frontal image, and the leftmost corner point a1 and the rightmost corner point a2 of the upper edge of the vertebral body are solved. Line L1 is obtained by connecting a1a2, and the slope k1 of L1 is calculated; according to the predicted pedicle point set, a line L2 with a slope of k1 and above the point set and dividing the total area of ​​the point set by 1 / 4 is solved, and the right boundary of L2 intersection point set is at point b1; a line L3 with a slope of k1 and dividing the total area of ​​the point set by 1 / 2 is solved, and the left boundary of L3 intersection point set is at point b2; a line L4 with a slope of k1 and below the point set and dividing the total area of ​​the point set by 1 / 3 is solved, and the right boundary of L4 intersection point set is at point b3; connect b1b2, b2b3, b1b 3 forms a triangle b1b2b3, where the line segment b1b3 intersects the line L3 at point b4; take the point c1 at a distance of 1 / 5 from b1 on the line segment b1b2, take the point c2 at a distance of 1 / 5 from b2 on the line segment b2b4, and take the point c3 at a distance of 1 / 5 from b3 on the line segment b2b3; connect c1c2, c2c3, and c1c3 to form a triangle c1c2c3, which is the reasonable safety zone of the probe insertion point, where the line segment c1c3 intersects the line L3 at point c4; solve the midpoint e of the line segment c2c4, which is the initial probe insertion point. The value range of e can be between b2 and b4. Any point can be selected as the probe insertion point. The probe insertion angle changes with the insertion point. The formula is α=5+5|vector c2e| / |vector c2c4|.

[0079] like Figure 7 As shown in FIG. 1 , the two regions of the pedicle and vertebral body predicted by the lateral image are subjected to edge detection, polygonal fitting and corner point detection, and the upper corner point m1 and the lower corner point m2 of the pedicle on the right are solved. The line L5 is obtained by connecting m1m2, and the slope k5 of L5 is calculated; according to the correspondence between the anteroposterior image and the lateral image of the spine on the pedicle, the midpoint p1 of the line m1m2 is taken as the puncture point; a line L6 passing through point p1 and with a slope k6 = -(1 / k5) is solved; the lower corner point m4 of the vertebral body on the left is solved, and p1m4 is connected to obtain Line L7, and find the slope k7 of L7; find the angle bisector L8 between L6 and L7, with a slope k8; solve the angle β between L6 and L8, which is the angle, and the formula is β=arctan(|k8-k6| / |1+k8*k6|), and the range is [arctan(|k6|),arctan(|k8-k6| / |1+k8*k6|)]; take the corner points m5 on the rightmost side of the vertebral body and m6 on the leftmost side of the vertebral body on line L8, and take the midpoint p2 of m5m6 as the end point.

[0080] like Figure 8 and Fig. 9 As shown in the figure, for the detection of the probe position and needle tip coordinates during the operation, the lens positioning will be performed according to the X-ray film lens, and the surrounding irrelevant elements will be removed first; then threshold segmentation will be performed to distinguish according to the difference in grayscale values ​​of different areas in the X-ray film, and the pixel value below the probe area will be changed to 0 (white), and the reasonable pixel value of the probe area will be retained as 255 (black); then corrosion expansion will be performed to weaken the area outside the probe; then probe extraction will be performed to retain the area belonging to the probe and remove the rest; finally, corner point detection will be performed based on the retained probe part to obtain the needle tip coordinates of the probe.

[0081] To clarify the scale, on the one hand, it is necessary to clarify the conversion relationship between the pixel distance in the X-ray film and the distance moved by the robotic arm in the actual environment. First, calculate and record the coordinate z1 where the needle tip appears in the image for the first time, then control the robotic arm to move the probe, calculate and record the coordinate z2 where the needle tip appears in the image for the second time, and then the pixel distance L between the two coordinates can be obtained. If the actual moving distance of the probe tip is D, the actual distance represented by each unit pixel in the image can be calculated to be S=D / L; on the other hand, it is necessary to clarify the scale conversion relationship between the anteroposterior and lateral views of the spine. According to the previously clarified scale method, the scale of the anteroposterior view can be obtained as S1, and the scale of the lateral view can be obtained as S2.

[0082] S5: Insert the probe from the puncture point along the puncture direction to the end point.

[0083] Specifically, in the key link of probe insertion, first, predict the probe insertion point e of the frontal view and calculate the insertion angle α, then predict the insertion point p1, end point p2 and insertion angle β of the lateral view, and calculate the length S3 between p1 and p2 in the lateral view according to the scale, and calculate the probe insertion depth S4=S3 / sin(α) in the actual environment, and then adjust the probe to e, angle α in the frontal view, p1, angle β in the lateral view, and then insert the probe to depth S4, and overlap with p2 in the lateral view, and the probe insertion task is completed. In this process, the doctor can select these parameters within the given range, or adopt the calculated default parameters. Multiple shots will be taken during the puncture process to ensure that the probe insertion parameters are accurate, the range is safe and reasonable, and it meets the surgical specifications and standards.

[0084] For the selection of medical segmentation network in surgical plan, we choose SwinUnet network to train our spinal segmentation model. Its main advantage is that it uses pure transformer to build the network, effectively obtaining the global context information in the spinal feature map; it adopts U-shaped network structure and introduces jump connection to fuse low-level spinal feature map with high-level spinal feature map, promote information interaction between multi-level features, improve the segmentation accuracy of key areas of the spine, and make the segmentation of key areas more detailed and accurate. It saves doctors time for diagnosing spinal X-rays and avoids negative situations such as misdiagnosis.

[0085] Regarding the selection of the puncture point, puncture angle, and end point, our algorithm, based on the integration of Roy-Camille, Magerl, Weinstein, AO, and "cross" positioning methods, calculates the default puncture point and point safety area in the spine anteroposterior view, and calculates the safe puncture angle according to the formula. Then, based on the spatial position relationship between the anteroposterior view and the lateral view, the safe puncture point, safe puncture angle, and end point of the lateral view are calculated. Combined with the anteroposterior view and the lateral view, the final puncture point, puncture angle, and puncture depth of the spinal pedicle screw placement surgery are calculated. In this algorithm, the present invention combines the fixedness of previous algorithms with the flexibility in clinical practice, so that doctors can select safe and reasonable puncture information in different surgical scenarios based on reasonable puncture information, ensure the safe and smooth progress of the operation, reduce the number of X-ray radiation to doctors and patients, and protect the safety of doctors and patients.

[0086] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0087] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A dual-view multi-scheme fusion-assisted guidance method for pedicle screw placement in the spine, characterized in that: The following steps are involved: By taking X-rays of the patient's spine, the patient's spine frontal and lateral images are obtained; According to the patient's spine anteroposterior and lateral images, the medical image segmentation network SwinUnet is used for training to obtain the spine anteroposterior segmentation model and the spine lateral segmentation model respectively; Through the trained spine segmentation model, the frontal and lateral images of the spine of different patients are predicted and the effective structure area is segmented; According to the differences and positional relationships of the effective structural areas in the spinal image, the insertion point, insertion direction and end point of the surgical probe for pedicle screw placement are determined; Insert the probe from the puncture point to the end point along the puncture direction.

2. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 1, characterized in that: The step of training the medical image segmentation network SwinUnet according to the anteroposterior and lateral images of the patient's spine to obtain the anteroposterior segmentation model and the lateral segmentation model of the spine respectively includes the following steps: The SwinUnet network model includes an encoder part, a decoder part, a bottleneck part and a skip connection part; The encoder part extracts features through multiple sliding windows and Swin Transformer blocks, reduces the size of the feature map, increases the receptive field, and downsamples the spine image into a spine feature map to extract spine image feature information; The decoder part adopts a structure similar to that of the encoder, and performs upsampling through patch expansion layers and Swin Transformer blocks to restore the extracted spine feature map to the same size as the input image and present the effective spine structure segmentation result; The bottleneck part consists of two Swin Transformer blocks, which are used to extract and transfer fine-grained feature information for the last time; The jump connection part is a bridge connecting the encoder and the decoder, which is used to transfer the coarse to fine multi-scale spine feature information extracted by the encoder part to the decoder part, and retain more effective spine structure detail information.

3. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 2, characterized in that: The step of predicting the anteroposterior and lateral images of the spine of different patients by using the trained spine segmentation model to segment out the effective structure area includes the following steps: The trained SwinUnet segmentation model is used to predict the three effective structural regions of pedicle, transverse process and vertebral body in the frontal view of the spine, and the two effective structural regions of pedicle and vertebral body are predicted in the lateral view.

4. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 3, characterized in that: The step of determining the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe according to the differences and positional relationships of the effective structural areas in the spinal column image comprises the following steps: Perform edge detection, polygonal fitting and corner point detection on the three regions of pedicle, transverse process and vertebral body predicted by the frontal image, solve the leftmost corner point a1 and the rightmost corner point a2 on the upper edge of the vertebral body, connect a1 and a2 to get line L1, and calculate the slope k1 of L1; According to the predicted pedicle point set, a line L2 with a slope of k1 and above the point set and dividing the total area of ​​the point set by 1 / 4 is solved, and the right boundary of the point set is intersected at point b1; a line L3 with a slope of k1 and dividing the total area of ​​the point set by 1 / 2 is solved, and the left boundary of the point set is intersected at point b2; Solve for a line L4 with a slope of k1 that is below the point set and divides the total area of ​​the point set by 1 / 3. L4 intersects the right boundary of the point set at point b3. Connect b1 b2, b2b3, and b1 b3 to form a triangle b1 b2b3, where line segment b1 b3 intersects line L3 at point b4. Take point c1 on line segment b1 b2 at a distance of 1 / 5 from b1, take point c2 on line segment b2b4 at a distance of 1 / 5 from b2, and take point c3 on line segment b2b3 at a distance of 1 / 5 from b3; Connect c1 c2, c2c3, c1 c3 to form a triangle c1 c2c3, which is the reasonable safety zone for the probe insertion point, where the line segment c1c3 intersects line L3 at point c4; The midpoint e of the line segment c2c4 is solved, which is the initial probe entry point. The value range of e can be between c2 and c4. Any point can be selected as the probe entry point. The probe entry angle changes with the entry point. The formula is α=5+5|vector c2e| / |vector c2c4|.

5. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 4, characterized in that: The step of determining the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe according to the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps: Perform edge detection, polygonal fitting and corner point detection on the two regions of the pedicle and vertebral body predicted by the lateral image, solve the upper corner point m1 and the lower corner point m2 on the rightmost side of the pedicle, connect m1m2 to get the line L5, and calculate the slope k5 of L5; According to the correspondence between the anteroposterior and lateral images of the spine on the pedicle, the midpoint p1 of the line m1m2 is taken as the insertion point; a line L6 passing through the point p1 and having a slope of k6 = -(1 / k5) is solved; Solve for the leftmost lower corner point m4 of the vertebral body, connect p1m4 to get line L7, and find the slope k7 of L7; find the angle bisector L8 between L6 and L7, with a slope k8; Solve for the angle β between L6 and L8, which is the angle. The formula is: β = arctan (|k8-k6| / |1+k8*k6|), and the range is [arctan (|k6|), arctan (|k8-k6| / |1+k8*k6|)]; Take the corner point m5 on the rightmost side of the vertebral body and the corner point m6 on the leftmost side of the vertebral body on line L8, and take the midpoint p2 of m5m6 as the end point.

6. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 5, characterized in that: The step of determining the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe according to the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps: During the operation, the position of the probe and the coordinates of the needle tip are detected. The lens is positioned according to the X-ray lens, and irrelevant elements around it are removed first; Then, threshold segmentation is performed to distinguish different areas in the X-ray film according to the difference in grayscale values. The pixel values ​​below the probe area are changed to 0, and the reasonable pixel value of the probe area is retained as 255. Then, corrosion expansion is performed to weaken the area outside the probe; Then the probe extraction is performed, the area belonging to the probe is retained and the rest is removed; Finally, corner point detection is performed based on the retained probe part to obtain the probe tip coordinates.

7. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 6, characterized in that: The step of determining the insertion point, insertion direction and end point of the pedicle screw insertion surgical probe according to the regional differences and positional relationships of the effective structures in the spinal image comprises the following steps: Clarify the scale, clarify the conversion relationship between the pixel distance in the X-ray film and the distance the robotic arm moves in the actual environment, calculate and record the coordinate z1 where the needle tip appears in the image for the first time, Then the robot arm is controlled to move the probe, and the coordinate z2 where the needle tip appears for the second time in the image is calculated and recorded. The pixel distance L between the two coordinates can be obtained. The actual moving distance of the probe tip is obtained as D, and the scale of the actual distance represented by each unit pixel in the image is calculated as S = D / L; It is necessary to clarify the scale conversion relationship between the anteroposterior and lateral views of the spine, and obtain the scale of the anteroposterior view as S1 and the scale of the lateral view as S2 according to the scale.

8. The dual-view multi-scheme fusion-assisted guidance method for spinal pedicle screw placement according to claim 7, characterized in that: The step of inserting the probe from the insertion point along the insertion direction to the end point comprises the following steps: Predict the puncture point e of the probe frontal image and calculate the puncture angle α; Predict the puncture point p1, end point p2 and puncture angle β of the lateral view; The length S3 between p1 and p2 in the lateral view can be calculated based on the scale, and the insertion depth of the probe in the actual environment can be calculated as S4 = S3 / sin(α); Adjust the probe to e in the frontal view, with an insertion angle of α, the viewing angle of the lateral view to p1 and the insertion angle to β, then insert the probe to a depth of S4. When the viewing angle of the lateral view coincides with p2, the probe insertion task is completed.

Images