Radiation-free navigation method and device for intraoperative robot

Abstract

The invention provides a non-radiative navigation method and device for an intraoperative robot. The method comprises the following steps: acquiring a spinal column three-dimensional model of an object; based on the spinal column three-dimensional model, determining spinal column operation planning information and guide plate planning information of all vertebral bodies to be provided with screws; a positioning pattern is planned on the guide plate; obtaining an intraoperative image after each vertebral body to be subjected to screw placement abuts against the guide plate, and performing registration on the spine three-dimensional model and the intraoperative vertebral body to be subjected to screw placement; and based on the registered three-dimensional model of the vertebral body to be subjected to screw placement and the spine, planning a surgical path of the surgical robot. According to the application, the guide plate of the vertebral body with the positioning pattern is planned, so that accurate positioning and navigation of the vertebral body are realized after the guide plate abuts against the vertebral body; therefore, on one hand, X-ray irradiation is not needed, and on the other hand, positioning precision is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image recognition, in particular to a radiation-free navigation method and device for an intraoperative robot. BACKGROUND

[0002] Spinal surgery has high risks and complex anatomical structures, and the damage to blood vessels and nerves adjacent to important blood vessels and nerves may cause paralysis or even death of patients, and the application of navigation can reduce the risk of surgery.

[0003] However, in the current navigation method, intraoperative CT scanning or frequent X-ray radiation damage is too large, and navigation without intraoperative CT scanning or frequent X-ray will produce a large error. SUMMARY

[0004] The problem solved by the present application is the large error and high radiation of current spinal surgery navigation.

[0005] To solve the above problems, the first aspect of the present application provides a radiation-free navigation method for an intraoperative robot, comprising:

[0006] acquiring a three-dimensional model of the spine of the object;

[0007] determining the spinal surgery planning information and the guide plate planning information of all to-be-nailing vertebrae based on the three-dimensional model of the spine; the guide plate is planned with a positioning pattern;

[0008] acquiring an intraoperative image of each to-be-nailing vertebra after abutting against the guide plate, and registering the three-dimensional model of the spine and the to-be-nailing vertebra in surgery;

[0009] planning a surgical path of the surgical robot based on the registered to-be-nailing vertebra and the three-dimensional model of the spine.

[0010] The second aspect of the present application provides a radiation-free navigation device for an intraoperative robot, comprising:

[0011] a three-dimensional construction module for acquiring a three-dimensional model of the spine of the object;

[0012] a guide plate planning module for determining the spinal surgery planning information and the guide plate planning information of all to-be-nailing vertebrae based on the three-dimensional model of the spine; the guide plate is planned with a positioning pattern;

[0013] a guide plate registration module for acquiring an intraoperative image of each to-be-nailing vertebra after abutting against the guide plate, and registering the three-dimensional model of the spine and the to-be-nailing vertebra in surgery;

[0014] a registration navigation module for planning a surgical path of the surgical robot based on the registered to-be-nailing vertebra and the three-dimensional model of the spine.

[0015] The third aspect of the present application provides an electronic device, comprising a memory and a processor;

[0016] The memory is configured to store a program;

[0017] The processor is coupled to the memory and configured to execute the program, so as to:

[0018] Obtain a three-dimensional model of a spine of a subject;

[0019] Based on the three-dimensional model of the spine, determine the spine surgery planning information and the guide plate planning information of all the to-be-nailing vertebrae; the guide plate is provided with a positioning pattern;

[0020] Obtain an intraoperative image of each to-be-nailing vertebrae after abutting against the guide plate, and register the three-dimensional model of the spine and the to-be-nailing vertebrae in surgery;

[0021] Based on the registered to-be-nailing vertebrae and the three-dimensional model of the spine, plan a surgical path of a surgical robot.

[0022] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the above-mentioned intraoperative robot radiation-free navigation method.

[0023] In the present application, the guide plate of the vertebrae with the positioning pattern is planned, so that accurate positioning and navigation of the vertebrae are realized after the guide plate abuts against the vertebrae; in this way, on the one hand, X-ray irradiation is not needed, and on the other hand, the positioning accuracy is greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 A flowchart of the intraoperative robot radiation-free navigation method according to the embodiment of the present application is shown;

[0025] Figure 2 An architecture diagram of the improved segmentation network of the intraoperative robot radiation-free navigation method according to the embodiment of the present application is shown;

[0026] Figure 3 An architecture diagram of the long connection fusion module of the intraoperative robot radiation-free navigation method according to the embodiment of the present application is shown;

[0027] Figure 4 An architecture diagram of the superposition fusion module of the intraoperative robot radiation-free navigation method according to the embodiment of the present application is shown;

[0028] Figure 5 A structure block diagram of the intraoperative robot radiation-free navigation device according to the embodiment of the present application is shown;

[0029] Figure 6 A structure block diagram of the electronic device according to the embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] In order to make the above objectives, characteristics and advantages of the present application more apparent, more comprehensible, the present application is described in detail below with reference to the drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided so that the present application can be more thoroughly understood and the scope of the present application can be accurately conveyed to those skilled in the art.

[0031] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in the present application should be understood as the usual meanings understood by those skilled in the art to which the present application belongs.

[0032] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in the present application should be understood as the usual meanings understood by those skilled in the art to which the present application belongs.

[0033] Spinal surgery is high-risk, and the spinal anatomy is complex, adjacent to important blood vessels and nerves, and damage to blood vessels and nerves can cause patients to be paralyzed or even die. The application of navigation can reduce the risk of surgery.

[0034] However, in the current navigation method, intraoperative CT scanning or frequent X-ray radiation damage is too great, and navigation without intraoperative CT scanning or frequent X-ray will produce a large error.

[0035] In view of the above problems, the present application provides a new navigation scheme based on a guide plate, which can greatly improve the navigation accuracy without X-ray by planning a guide plate and setting a positioning pattern.

[0036] The embodiment of the present application provides a radiation-free navigation method of an intraoperative robot, and the specific scheme of the method is shown in the method. Figures 1-4 The method can be performed by a radiation-free navigation device of an intraoperative robot, which can be integrated in a computer, a server, a computer, a server cluster, a data center or the like. As shown in Figure 1 It is a flowchart of a radiation-free navigation method of an intraoperative robot according to an embodiment of the present application; wherein the radiation-free navigation method of the intraoperative robot comprises:

[0037] S101, obtaining a three-dimensional model of the spine of the object;

[0038] S102, determining the spinal surgery planning information and the guide plate planning information of all to-be-nailing vertebral bodies based on the three-dimensional model of the spine; the guide plate is planned with a positioning pattern;

[0039] S103, obtaining an intraoperative image of each to-be-nailing vertebral body after abutting against the guide plate, and registering the three-dimensional model of the spine with the to-be-nailing vertebral body in the operation;

[0040] S104, based on the registered to-be-screwed vertebra and the three-dimensional model of the spine, planning a surgical path of the surgical robot.

[0041] In the present application, based on the registered to-be-screwed vertebra and the three-dimensional model of the spine, the to-be-screwed vertebra in the operation and the three-dimensional model of the spine before the operation and the position of the mechanical arm can be unified to the same coordinate system, and the surgical path of the surgical robot is planned.

[0042] In the present application, by planning the guide plate of the vertebra with the positioning pattern, the accurate positioning and navigation of the vertebra are realized after the guide plate and the vertebra abut. In this way, on the one hand, X-ray irradiation is not needed, and on the other hand, the positioning accuracy is greatly improved.

[0043] In the present application, based on the preoperative-intraoperative high-precision registration, the preoperative planning can be mapped to the spine in the real surgical scene, the mechanical arm is controlled to complete the automatic osteotomy operation, the automatic operation in the surgical process is realized, the human intervention is reduced, and thus the error caused by the human intervention is reduced, and the accuracy, stability and operation efficiency of the operation are improved.

[0044] In an embodiment, S103, acquiring an intraoperative image of each to-be-screwed vertebra after abutting against the guide plate, and registering the three-dimensional model of the spine and the to-be-screwed vertebra in the operation, comprises:

[0045] Acquiring an intraoperative image of a single to-be-screwed vertebra after abutting against the guide plate, the intraoperative image containing a positioning pattern on the guide plate;

[0046] Acquiring the three-dimensional model of the spine and the guide plate planning information of the to-be-screwed vertebra; the guide plate planning information containing the positioning pattern and the mutual positional relationship of the positioning pattern, the guide plate and the to-be-screwed vertebra;

[0047] Based on the intraoperative image and the guide plate planning information, determining a registration matrix of the intraoperative positioning pattern and the preoperative positioning pattern;

[0048] Based on the registration matrix and the mutual positional relationship of the positioning pattern, the guide plate and the to-be-screwed vertebra, determining the positional information of the guide plate and the to-be-screwed vertebra in the operation.

[0049] In the present application, for each to-be-screwed vertebra, a corresponding guide plate is planned; in specific use, after intraoperative preparation is completed, the feature recognition area of the vertebra is exposed, and then the fitting area of the guide plate is abutted against the feature recognition area; at this time, the relative position of the guide plate and the vertebra can be considered as the relative position in the preoperative planning; at this time, the intraoperative image is shot, and according to the positioning pattern in the intraoperative image and the positioning pattern in the preoperative planning, the registration matrix between the two positioning patterns can be determined; since the mutual positional relationship between the positioning pattern, the guide plate and the to-be-screwed vertebra is fixed and known in the preoperative planning, the coordinate system of the camera can be determined based on the shooting camera or the preset origin coordinate system (and the relationship between the shooting camera and the origin coordinate system), and the coordinates of the positioning pattern of the intraoperative image are the camera coordinate system; through the registration matrix, the positioning pattern in the preoperative planning can be mapped to the camera coordinate system and the origin coordinate system in the intraoperative (mapped twice through the camera), at this time, the positions of the guide plate and the to-be-screwed vertebra are also synchronously mapped, and the mapped positions are the position information of the guide plate and the to-be-screwed vertebra in the intraoperative.

[0050] In the present application, after the position information of the to-be-screwed vertebra is determined, the surgical robot can be controlled to screw according to the planned path, so that the screwing of the to-be-screwed vertebra is completed.

[0051] It should be noted that the screwing can be performed while maintaining the abutment of the guide plate and the vertebra, or the guide plate can be removed after the screwing (at this time, the intraoperative position of the vertebra does not change, and the movement of the vertebra caused by breathing is ignored).

[0052] In the present application, after the screwing of one to-be-screwed vertebra is completed, the guide plate is removed; the above steps are repeated to complete the screwing action of another to-be-screwed vertebra.

[0053] It should be noted that in a doctor's spinal surgery, there are two difficulties: one is that the intraoperative vertebra screwing requires a long time and a lot of effort, and the other is that after the intraoperative screwing, the positions of the remaining vertebrae will change significantly (the strong screwing will inevitably cause significant displacement of the remaining vertebrae after the screwing).

[0054] In the present application, by setting the guide plate and screwing one by one, on the one hand, the vertebra screwing action is completed by the robot, which can greatly speed up the screwing and greatly reduce the doctor's screwing energy consumption; on the other hand, by screwing one by one, the problem of significant deformation of the vertebra during screwing is avoided.

[0055] Preferably, in the present application, the surgical robot is controlled to screw while the screwing guide plate is abutted against the vertebra, so that the possible displacement of the vertebra can be monitored in real time, thereby eliminating the displacement of the vertebra caused by breathing and the possible displacement of the vertebra caused by the screwing action, and greatly increasing the screwing accuracy.

[0056] In an embodiment, the guide plate comprises a positioning plate and an identification plate,

[0057] The positioning plate is adapted to fit the corresponding vertebral body;

[0058] The identification plate is fixedly connected with the positioning plate and is adapted to identify the pose of the positioning plate.

[0059] In an embodiment, one side of the positioning plate relative to the identification plate is provided with a fitting surface 5 (the edge pointed by the arrow in the figure), which is adapted to fit the lamina.

[0060] In the present application, the positioning plate comprises a fitting part 3 and a curved part 4, the bottom surface of the fitting part is a fitting surface, which fits the fitting area of the vertebral body; the curved part is used to connect the two fitting parts and avoid the spinous process part of the spine.

[0061] In the present application, the fitting surface corresponds to the feature identification area of the vertebral body. The feature identification area / fitting part of the vertebral body is defined as follows: the lateral margin is 5mm outwardly extended from the junction of the vertebral body and the transverse process, the medial margin is 10mm inwardly extended from the junction of the lamina and the transverse process, and the upper and lower margins are the upper and lower margins of the lamina. The upper margin of the thoracic vertebra is 3mm below the upper margin of the lamina, and the lower margin is the lower margin of the lamina, because the lamina of the thoracic vertebra is in the shape of a tile.

[0062] In the present application, by setting the feature identification area, on the one hand, the soft tissue is easy to peel off, and on the other hand, the fitting area (the positioning plate) has uniqueness.

[0063] In an embodiment, the part where the positioning plate is combined with the identification plate is provided with a semicircular hollow 6, which is adapted to accommodate the finger for pressing.

[0064] In the present application, when the guide plate is fitted on the surface of the corresponding vertebral body, the doctor's finger can be inserted into the semicircular hollow to press the positioning plate, so as to avoid dislocation of the guide plate. In addition, by setting the semicircular hollow at this position, the doctor can press the positioning plate without blocking the identification plate above, thereby achieving the functions of pressing and not blocking at the same time.

[0065] In an embodiment, the surface of the semicircular hollow is provided with at least one pair of symmetrical protrusions 7, which are adapted to resist the finger when pressed.

[0066] In the present application, by setting the symmetrical protrusions, when the finger presses the surface of the semicircular hollow, the protrusions resist the finger. At this time, the finger can determine whether the pressing direction is in the center by sensing the symmetrical protrusions on both sides of the finger, so as to avoid instability or misplacement of the guide plate caused by deviation of the pressing.

[0067] In one embodiment, the part of the positioning plate combined with the identification plate is provided with a horizontal slot 8, and the identification plate is clamped at the bottom of the horizontal slot.

[0068] In one embodiment, the S102, based on the three-dimensional model of the spine, determines the spine surgery planning information and the guide plate planning information of all the to-be-nailing vertebral bodies, including:

[0069] Based on the three-dimensional model of the spine, the cortical bone screw planning of the vertebral body is determined;

[0070] Based on the cortical bone screw planning of the vertebral body, the spine surgery planning information and the guide plate planning information are determined.

[0071] In one embodiment, the S102, based on the three-dimensional model of the spine, determines the cortical bone screw planning of the vertebral body, including:

[0072] Obtain the statistical shape model and the cortical bone screw planning information;

[0073] Construct a registration matrix of the vertebral body and a plurality of statistical shape models corresponding to the serial number;

[0074] Based on the constructed registration matrix, select the statistical shape model closest to the vertebral body;

[0075] Based on the registration matrix of the closest statistical shape model, map the corresponding cortical bone screw planning information onto the vertebral body.

[0076] In one embodiment, based on the registration matrix of the closest statistical shape model, the corresponding cortical bone screw planning information is mapped onto the vertebral body, including:

[0077] Respectively calculate the registration matrix of each statistical shape model and the point cloud information of the vertebral body;

[0078] Obtain a quantitative evaluation index;

[0079] Based on the quantitative evaluation index, determine the closest statistical shape model;

[0080] Based on the registration matrix, map the closest statistical shape model to the point cloud information of the vertebral body.

[0081] In one embodiment, the S102, based on the three-dimensional model of the spine, determines the cortical bone screw planning of the vertebral body, including:

[0082] Obtain a plurality of spine CT images and segment the voxel data of the vertebral body corresponding to the serial number;

[0083] Convert the voxel data into point cloud data of the vertebral body;

[0084] Based on the point cloud data of the vertebral body, establish the statistical shape model of the vertebral body;

[0085] Cortical bone screw planning information is set on each statistical shape model.

[0086] In an implementation, the S101, acquiring a spine three-dimensional model of a subject, comprises:

[0087] Acquiring CT image data of the subject;

[0088] Inputting the CT image data into an improved segmentation network to obtain segmented spine image data;

[0089] Constructing a spine three-dimensional model based on the spine image data.

[0090] In an implementation, the S101, acquiring a spine three-dimensional model of a subject, comprises: Figure 2 As shown in the figure, the S101, acquiring a spine three-dimensional model of a subject, comprises:

[0091] Segmenting the CT image data to obtain a segmentation map;

[0092] Encoding the segmentation map based on a Swin Transformer to obtain a plurality of hierarchical encoded maps;

[0093] Inputting the plurality of hierarchical encoded maps into a long connection fusion module to obtain a plurality of hierarchical fusion maps;

[0094] Convoluting the encoded map at the bottommost layer to obtain a convolution map;

[0095] Decoding the convolution map and the plurality of hierarchical fusion maps based on the Swin Transformer to obtain a decoded map;

[0096] Reversibly splicing the decoded map to obtain the segmented spine image data.

[0097] In an implementation, the long connection fusion module has four hierarchical encoded maps as input and outputs three hierarchical fusion maps.

[0098] In an implementation, the S101, acquiring a spine three-dimensional model of a subject, comprises:

[0099] Linearly embedding the segmentation map to obtain an encoded map at a first hierarchical level;

[0100] Inputting the encoded map at the first hierarchical level into a Swin Transformer block to obtain a first feature map;

[0101] Patching the first feature to obtain an encoded map at a second hierarchical level;

[0102] Inputting the encoded map at the second hierarchical level into a Swin Transformer block to obtain a second feature map;

[0103] The second feature is patch-merged to obtain a third-level encoding map;

[0104] The third-level encoding map is input into a Swin Transformer block to obtain a third feature map;

[0105] The third feature is patch-merged to obtain a fourth-level encoding map.

[0106] In the present application, as shown in Figure 2 During encoding, the CT image data is converted into a sequence embedding, segmented into non-overlapping blocks of fixed size, and then the feature dimension is projected into one dimension through a linear embedding; the transformed image blocks pass through a series of SwinTransformer blocks and patch merging layers to generate multiple levels of feature representations. Among them, the patch merging layer is used for downsampling and dimension number muscle, and the Swin Transformer block is used for feature expression extraction.

[0107] In the present application, the change in the number of dimensions and the resolution is completed in the patch merging layer.

[0108] In one embodiment, as shown in Figure 3 The multiple levels of encoding maps are input into a long connection fusion module to obtain multiple levels of fusion maps, which includes:

[0109] The fourth-level encoding map is superimposed and fused with the third-level encoding map to obtain a third-level fusion map;

[0110] The third-level encoding map is superimposed and fused with the second-level encoding map to obtain a middle fusion map;

[0111] The third-level fusion map is superimposed and fused with the second-level encoding map and the middle fusion map to obtain a second-level fusion map;

[0112] The second-level encoding map is superimposed and fused with the first-level encoding map to obtain a front fusion map;

[0113] The middle fusion map is superimposed and fused with the first-level encoding map and the middle fusion map to obtain a rear fusion map;

[0114] The second-level fusion map is superimposed and fused with the first-level encoding map, the front fusion map, and the rear fusion map to obtain a first-level fusion map.

[0115] In the present application, the Swin Transformer block has great deficiencies in extracting local spatial features of images, so a CNN feature extraction is performed, then an alternating structure is constructed between encoding and decoding, and a long connection fusion module is connected.

[0116] The long connection fusion module fuses the extracted context features and the scale features of the multiple levels of the encoder through multiple fusion and dense skip connection, so as to compensate for the loss of spatial information caused by the down-sampling of the encoding.

[0117] In the present application, as shown in Figure 4 In the superimposed fusion, the input image of the previous level is up-sampled, and then spliced with multiple input images of the current level. After splicing, two 3x3 convolutions are performed to obtain the fusion result.

[0118] In the present application, as shown in Figure 4 In the long connection fusion module, in order to make up for the lack of Swin Transformer block in image feature extraction details, superimposed fusion is performed through convolution structure, so as to effectively reduce the semantic gap between encoding and decoding, and capture features of different levels and perception fields of different sizes, and obtain more rich multi-scale information. Further, the local spatial features and the overall global features of the image can be efficiently extracted, the deep and shallow semantics can be integrated, the splicing processing can be performed, and the loss of spatial information caused by down-sampling can be reduced.

[0119] In the present application, as shown in Figure 2 The decoding part is composed of a Swin Transformer block and a patch layer; the patch layer maps the features of adjacent dimensions to a larger size (2 times up-sampling) to realize up-sampling; at the same time, the Swin Transformer block is used for feature expression extraction. Finally, 4 times up-sampling is performed through the last patch layer, so as to unify the resolution with the input resolution; then a linear projection layer (inverse splicing) is used for feature mapping to perform segmentation prediction.

[0120] In one specific embodiment, after the superimposed fusion, the method further comprises:

[0121] Adaptively adjusting the output image of the superimposed fusion, and taking the adaptively adjusted output image as the output image of the original superimposed fusion.

[0122] In one specific embodiment, the adaptively adjusting the output image of the superimposed fusion comprises:

[0123] Dividing the output image into blocks to obtain independent blocks;

[0124] For each independent block, a first neighborhood block and a second neighborhood block of different intervals are obtained;

[0125] Based on the independent block and the first neighborhood block, a first feature block is generated;

[0126] Based on the independent block and the second neighborhood block, a second feature block is generated;

[0127] perform feature compression on the first feature block and the second feature block to obtain a compressed block;

[0128] traverse all independent blocks and generate an adjusted output image based on the obtained compressed blocks.

[0129] In the present application, the output image is divided into corresponding image blocks by a checkerboard, that is, the image blocks can be at the pixel level (that is, each pixel is an image block), or other levels, and the specific division is subject to the actual processing situation.

[0130] In the present application, the image is divided into blocks of the same size using a sliding window or a fixed step size.

[0131] Preferably, in the present application, each image block is 100-1000 pixels, so as to ensure the generation accuracy and reduce the amount of calculation, and more local area feature calculation is performed.

[0132] In the present application, an image block is selected as an independent block, and the adjacent image blocks above, below, left and right of the independent block are first neighborhood blocks; and the image blocks one block apart above, below, left and right of the independent block are second neighborhood blocks. The first neighborhood blocks and the second neighborhood blocks have different distances from the independent block.

[0133] In the present application, the neighborhood information of each independent block is extracted to capture local structures.

[0134] In the present application, the first feature block is generated, which is a local feature representation generated by using the independent block and the first neighborhood blocks thereof. Specifically, the independent block and the first neighborhood blocks are processed by a convolution layer and an attention layer to obtain the first feature block.

[0135] In the present application, the specific structure and specific parameters of the convolution layer and the attention layer can be obtained according to training data or determined according to actual conditions.

[0136] It should be noted that in the present application, the first neighborhood blocks are four, and the first feature blocks are multiple.

[0137] In the present application, the independent block and the first neighborhood blocks are processed by a convolution layer and an attention layer to obtain the first feature block, and the specific process is as follows: the independent block and the four neighborhood blocks are spliced together to form a multi-channel input, and a convolution layer is used to extract features from the spliced blocks; a self-attention mechanism or a channel attention mechanism is used to enhance important features, calculate attention weights, and weight the convolution layer output to enhance important features; the output of the attention layer is split into multiple feature blocks, and each feature block corresponds to the processing result of the independent block and at least one neighborhood block.

[0138] In this application, the second feature block is generated to generate a more extensive local feature representation using the independent block and its second neighborhood block, and the specific generation process is the same as that of the first feature block, except that the parameters of the convolutional layer and the attention layer are different.

[0139] In this application, the generated feature block is compressed into a more compact representation to reduce the amount of calculation and retain key information. Feature compression is performed using a pooling operation (such as max pooling or average pooling) or a fully connected layer.

[0140] In this way, by compression, a plurality of first feature blocks and second feature blocks are compressed into a compressed block corresponding to the size and position of the independent block, which is used to replace the independent block. All image blocks are replaced by compressed blocks to obtain an adjusted output image.

[0141] In this application, each image block of the output image is traversed to obtain the corresponding compressed block.

[0142] In this application, for image blocks / independent blocks near the edge, the first neighborhood block and the second neighborhood block are not complete, at which time the first neighborhood block and the second neighborhood block of the relative position are copied to complete. For example, the first neighborhood block on the upper side of the independent block does not exist, and the first neighborhood block on the lower side is copied as the block on the upper side for use.

[0143] In this application, by completing, the processing accuracy of the edge image block is greatly improved.

[0144] In this application, the similarity relationship between local regions is captured by the adaptive adjustment module to enhance the feature representation.

[0145] The embodiment of the application provides a kind of intraoperative robot radiation-free navigation device, for executing the kind of intraoperative robot radiation-free navigation method described in the above content of the application, the following detailed description of the kind of intraoperative robot radiation-free navigation device.

[0146] As shown in Figure 5 The kind of intraoperative robot radiation-free navigation device includes:

[0147] A three-dimensional construction module 101 is used to obtain a three-dimensional model of the spine of the object;

[0148] A guide plate planning module 102 is used to determine the spine surgery planning information and the guide plate planning information of all to-be-nailing vertebral bodies based on the three-dimensional model of the spine; the guide plate is planned with a positioning pattern;

[0149] A guide plate registration module 103 is used to obtain an intraoperative image of each to-be-nailing vertebral body abutting against the guide plate, and register the three-dimensional model of the spine with the to-be-nailing vertebral body in the operation;

[0150] a registration navigation module 104 for planning a surgical path of the surgical robot based on the registered to-be-pegged vertebra and the three-dimensional model of the spine.

[0151] In an embodiment, the guide plate registration module 103 is further configured to:

[0152] obtain an intraoperative image of the single to-be-pegged vertebra after abutting against the guide plate, the intraoperative image containing the positioning pattern on the guide plate; obtain the three-dimensional model of the spine and the guide plate planning information of the to-be-pegged vertebra; the guide plate planning information containing the positioning pattern and the mutual positional relationship of the positioning pattern with the guide plate and the to-be-pegged vertebra; determine a registration matrix of the positioning pattern in the intraoperative image and the positioning pattern before the operation based on the intraoperative image and the guide plate planning information; determine the positional information of the guide plate and the to-be-pegged vertebra in the operation based on the registration matrix and the mutual positional relationship of the positioning pattern with the guide plate and the to-be-pegged vertebra.

[0153] In an embodiment, the three-dimensional construction module 101 is further configured to:

[0154] obtain CT image data of the object; input the CT image data into the improved segmentation network to obtain segmented spinal image data; and construct a three-dimensional model of the spine based on the spinal image data.

[0155] In an embodiment, the three-dimensional construction module 101 is further configured to:

[0156] segment the CT image data to obtain a segmentation image; encode the segmentation image based on a Swin Transformer to obtain a plurality of levels of encoded images; input the plurality of levels of encoded images into a long connection fusion module to obtain a plurality of levels of fusion images; perform convolution on the encoded image at the bottommost layer to obtain a convolution image; decode the convolution image and the plurality of levels of fusion images based on a Swin Transformer to obtain a decoded image; and perform reverse splicing on the decoded image to obtain the segmented spinal image data.

[0157] In an embodiment, the long connection fusion module has four levels of input encoded images and three levels of output fusion images.

[0158] In an embodiment, the three-dimensional construction module 101 is further configured to:

[0159] The segmentation graph is linearly embedded to obtain a first-level encoding graph; the first-level encoding graph is input into a SwinTransformer block to obtain a first feature map; the first feature is patch-merged to obtain a second-level encoding graph; the second-level encoding graph is input into the SwinTransformer block to obtain a second feature map; the second feature is patch-merged to obtain a third-level encoding graph; the third-level encoding graph is input into the SwinTransformer block to obtain a third feature map; and the third feature is patch-merged to obtain a fourth-level encoding graph.

[0160] In an implementation manner, the three-dimensional construction module 101 is further configured to:

[0161] The fourth-level encoding graph is superimposed and fused with the third-level encoding graph to obtain a third-level fusion graph; the third-level encoding graph is superimposed and fused with the second-level encoding graph to obtain a middle fusion graph; the third-level fusion graph is superimposed and fused with the second-level encoding graph and the middle fusion graph to obtain a second-level fusion graph; the second-level encoding graph is superimposed and fused with the first-level encoding graph to obtain a front fusion graph; the middle fusion graph is superimposed and fused with the first-level encoding graph and the middle fusion graph to obtain a rear fusion graph; and the second-level fusion graph is superimposed and fused with the first-level encoding graph, the front fusion graph, and the rear fusion graph to obtain a first-level fusion graph.

[0162] The above-described embodiment of the intraoperative robot provides a radiation-free navigation device for an intraoperative robot, and the radiation-free navigation method for an intraoperative robot provided by the embodiment of the present application has the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0163] The above describes the internal functions and structures of the radiation-free navigation device for an intraoperative robot, as shown in Figure 6 The radiation-free navigation device for an intraoperative robot can be implemented as an electronic device in practice, which includes a memory 301 and a processor 303.

[0164] The memory 301 can be configured to store a program.

[0165] Additionally, the memory 301 can also be configured to store other various data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phonebook data, messages, pictures, videos, and the like. The memory 301 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic or optical disks.

[0166] The processor 303 is coupled to the memory 301 and configured to execute programs in the memory 301 for:

[0167] obtaining a three-dimensional model of a spine of a subject;

[0168] determining, based on the three-dimensional model of the spine, a surgical planning information of the spine and a guide plate planning information of all pedicle screws to be placed; the guide plate is planned with a positioning pattern thereon;

[0169] obtaining an intraoperative image of each pedicle screw to be placed against the guide plate, and registering the three-dimensional model of the spine and the pedicle screw to be placed in the intraoperative image;

[0170] planning a surgical path of a surgical robot based on the registered pedicle screw to be placed and the three-dimensional model of the spine.

[0171] In an embodiment, the processor 303 is further configured to:

[0172] obtaining an intraoperative image of each pedicle screw to be placed against the guide plate, and registering the three-dimensional model of the spine and the pedicle screw to be placed in the intraoperative image;

[0173] In an embodiment, the processor 303 is further configured to:

[0174] obtaining CT image data of the subject; inputting the CT image data into an improved segmentation network to obtain segmented spine image data; and constructing the three-dimensional model of the spine based on the segmented spine image data.

[0175] In an embodiment, the processor 303 is further configured to:

[0176] The CT image data is segmented to obtain a segmentation graph; the segmentation graph is encoded based on a Swin Transformer to obtain a plurality of levels of encoded graphs; the plurality of levels of encoded graphs are input into a long connection fusion module to obtain a plurality of levels of fusion graphs; the encoded graph at the bottommost layer is convolved to obtain a convolution graph; the convolution graph and the plurality of levels of fusion graphs are decoded based on the Swin Transformer to obtain a decoded graph; and the decoded graph is reversely spliced to obtain segmented spinal image data.

[0177] In an implementation, the encoded graphs input into the long connection fusion module are four levels, and the fusion graphs output are three levels.

[0178] In an implementation, the processor 303 is further configured to:

[0179] The segmentation graph is linearly embedded to obtain a first level of encoded graphs; the first level of encoded graphs is input into a Swin Transformer block to obtain a first feature map; the first feature is patch-merged to obtain a second level of encoded graphs; the second level of encoded graphs is input into the Swin Transformer block to obtain a second feature map; the second feature is patch-merged to obtain a third level of encoded graphs; the third level of encoded graphs is input into the Swin Transformer block to obtain a third feature map; and the third feature is patch-merged to obtain a fourth level of encoded graphs.

[0180] In an implementation, the processor 303 is further configured to:

[0181] The fourth level of encoded graphs and the third level of encoded graphs are superimposed and fused to obtain a third level of fusion graphs; the third level of encoded graphs and the second level of encoded graphs are superimposed and fused to obtain a middle fusion graph; the third level of fusion graphs and the second level of encoded graphs and the middle fusion graph are superimposed and fused to obtain a second level of fusion graphs; the second level of encoded graphs and the first level of encoded graphs are superimposed and fused to obtain a front fusion graph; the middle fusion graph and the first level of encoded graphs and the middle fusion graph are superimposed and fused to obtain a rear fusion graph; and the second level of fusion graphs and the first level of encoded graphs, the front fusion graph, and the rear fusion graph are superimposed and fused to obtain a first level of fusion graphs.

[0182] In the present application, Figure 6 only some components are shown schematically, and it does not mean that the electronic device only includes Figure 6 the components shown.

[0183] The electronic device provided by the embodiment has the same beneficial effects as the method adopted, run or implemented by the application program stored therein, based on the same inventive concept as the intraoperative robot radiation-free navigation method provided by the embodiment of the present application.

[0184] Those skilled in the art will appreciate that embodiments of the application can be readily used as a method, apparatus, or computer program product. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0185] The application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0186] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0188] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. The memory can include non-persistent memory, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or Flash memory, in a computer readable medium. The memory is an example of computer readable media.

[0189] The application also provides a computer readable storage medium corresponding to the radiation-free navigation method of an intraoperative robot provided by the foregoing embodiments, and a computer program (i.e., a program product) is stored on the computer readable storage medium, and when the computer program is run by a processor, the computer program will execute the radiation-free navigation method of an intraoperative robot provided by any of the foregoing embodiments.

[0190] The computer readable medium includes permanent and non-permanent, removable and non-removable media, and can be realized by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0191] According to the definition herein, the computer readable medium does not include transitory media such as modulated data signals and carriers.

[0192] The computer readable storage medium provided by the above embodiments of the application has the same beneficial effects as the method adopted, run or implemented by the application program stored therein, based on the same inventive concept as the radiation-free navigation method of an intraoperative robot provided by the embodiments of the application.

[0193] It should be noted that in the specification provided herein, a large number of specific details are explained. However, it can be understood that the embodiments of the application can be practiced without these specific details. In some examples, well-known structures and technologies are not shown in detail in order not to obscure the understanding of the specification.

[0194] It should also be noted that the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0195] The above merely provides an example of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the scope of claims of the present application.

Claims

1. A method for non-radiation navigation of an intraoperative robot, characterized in that, The method comprises the following steps: acquiring a three-dimensional model of a spine of a subject; determining, based on the three-dimensional model of the spine, surgical planning information of the spine and guide plate planning information of all to-be-screwed vertebrae; the guide plate is provided with a positioning pattern; acquiring an intraoperative image of each to-be-screwed vertebrae after abutting against the guide plate, and registering the three-dimensional model of the spine and the to-be-screwed vertebrae in the intraoperative image; based on the registered to-be-screwed vertebrae and the three-dimensional model of the spine, planning a surgical path of a surgical robot.

2. The method of claim 1, wherein, The method comprises the following steps: acquiring an intraoperative image of each to-be-screwed vertebrae after abutting against the guide plate, and registering the three-dimensional model of the spine and the to-be-screwed vertebrae in the intraoperative image; acquiring an intraoperative image of a single to-be-screwed vertebrae after abutting against the guide plate, wherein the intraoperative image contains the positioning pattern on the guide plate; acquiring the three-dimensional model of the spine and guide plate planning information of the to-be-screwed vertebrae; the guide plate planning information contains the positioning pattern and the mutual positional relationship between the positioning pattern and the guide plate and the to-be-screwed vertebrae; determining, based on the intraoperative image and the guide plate planning information, a registration matrix of the positioning pattern in the intraoperative image and the positioning pattern before the operation; 3. The method of radiation-free navigation of an intraoperative robot according to claim 1 or 2, characterized in that, based on the registration matrix and the mutual positional relationship between the positioning pattern and the guide plate and the to-be-screwed vertebrae, determining the positional information of the guide plate and the to-be-screwed vertebrae in the intraoperative image. The method comprises the following steps: acquiring CT image data of a subject; inputting the CT image data into an improved segmentation network to obtain segmented spinal image data; 4. The method of radiance-free navigation of an intraoperative robot according to claim 3, characterized in that, based on the spinal image data, constructing a three-dimensional model of the spine. The method comprises the following steps: segmenting the CT image data to obtain a segmentation image; encoding the segmentation image based on a Swin Transformer to obtain a plurality of levels of encoded images; inputting the plurality of levels of encoded images into a long connection fusion module to obtain a plurality of levels of fusion images; convolving the encoded image at the bottommost level to obtain a convolution image; decoding the convolution image and the plurality of levels of fusion images based on a Swin Transformer to obtain a decoded image; 5. The method of radiance-free navigation of an intraoperative robot according to claim 4, characterized in that, performing reverse splicing on the decoded image to obtain the segmented spinal image data.

6. The method of claim 4, wherein, The long connection fusion module has four levels of input encoded images and three levels of output fusion images. The method comprises the following steps: performing linear embedding on the segmentation image to obtain a first level of encoded images; inputting the first level of encoded images into a Swin Transformer block to obtain a first feature map; performing patch merging on the first feature to obtain a second level of encoded images; inputting the second level of encoded images into a Swin Transformer block to obtain a second feature map; performing patch merging on the second feature to obtain a third level of encoded images; inputting the third level of encoded images into a Swin Transformer block to obtain a third feature map; 7. The method of claim 4, wherein, performing patch merging on the third feature to obtain a fourth level of encoded images. The method comprises the following steps: superimposing and fusing the fourth level of encoded images and the third level of encoded images to obtain a third level of fusion images; Superimpose the third level of coding map and the second level of coding map to obtain a middle fusion map; Superimpose the third level of fusion map and the second level of coding map and the middle fusion map to obtain the second level of fusion map; Superimpose the second level of coding map and the first level of coding map to obtain a front fusion map; Superimpose the middle fusion map and the first level of coding map and the middle fusion map to obtain a rear fusion map; Superimpose the second level of fusion map and the first level of coding map, the front fusion map and the rear fusion map to obtain the first level of fusion map.

8. A device for the radiation-free navigation of an intraoperative robot, characterized in that It comprises: a three-dimensional construction module for obtaining a three-dimensional model of a spine of a subject; a guide plate planning module for determining, based on the three-dimensional model of the spine, surgical planning information of the spine and guide plate planning information of all to-be-nailing vertebrae; the guide plate is planned with a positioning pattern thereon; a guide plate registration module for obtaining an intraoperative image of each to-be-nailing vertebrae against the guide plate, and registering the three-dimensional model of the spine and the to-be-nailing vertebrae in surgery; a registration navigation module for planning a surgical path of a surgical robot based on the registered to-be-nailing vertebrae and the three-dimensional model of the spine.

9. An electronic device, comprising: It comprises: a memory and a processor; the memory for storing a program; the processor coupled to the memory for executing the program for: obtaining a three-dimensional model of a spine of a subject; determining, based on the three-dimensional model of the spine, surgical planning information of the spine and guide plate planning information of all to-be-nailing vertebrae; the guide plate is planned with a positioning pattern thereon; obtaining an intraoperative image of each to-be-nailing vertebrae against the guide plate, and registering the three-dimensional model of the spine and the to-be-nailing vertebrae in surgery; planning a surgical path of a surgical robot based on the registered to-be-nailing vertebrae and the three-dimensional model of the spine.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the radiation-free navigation method of the intraoperative robot according to any one of claims 1-7.

Images