Pedicle screw automatic planning apparatus, device, and computer program product

Abstract

The embodiment of the application is suitable for the field of computer-aided medical technology and image processing technology, and provides a pedicle screw automatic planning device, equipment and computer program product. The device comprises: a data unit configured to acquire spine image data, and extract pedicle outer layer point cloud data of a single vertebra to be planned from the spine image data; a prediction model configured to process the pedicle outer layer point cloud data, and output predicted coordinates, wherein the predicted coordinates comprise an entry point coordinate and an exit point coordinate; and a planning unit configured to plan an implantation position of a pedicle screw based on the entry point coordinate and the exit point coordinate. By using the device, the planning position of the pedicle screw can be quickly and accurately predicted based on the pedicle outer layer point cloud data by using a deep learning model, the amount of data to be processed in the planning process is reduced, and the planning efficiency is improved.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application belongs to the technical field of computer-aided medical technology and image processing technology, and particularly relates to a pedicle screw automatic planning device, equipment and computer program product. BACKGROUND

[0002] Pedicle screw internal fixation is one of the core technologies of spine surgery, which can be used for treating diseases such as fracture, deformity and degeneration. With the progress of medical imaging and robot-assisted navigation technology, precise and safe screw placement is increasingly recognized by doctors and patients.

[0003] The main process of robot-assisted navigation technology includes three-dimensional image scanning, intraoperative registration, intraoperative planning and intraoperative navigation. In the surgical process, if the operation time is prolonged, the risk of infection, the amount of bleeding and the incidence of postoperative complications will increase, and intraoperative planning usually takes a lot of time, resulting in an increase in operation time. In spine surgery, two screws need to be planned for each cone to be implanted, and the two screws need to pass through the position of the cone pedicle and cannot invade the spinal cord nerve of the spine. Therefore, the doctor needs to carefully observe and adjust the position and angle of each pedicle screw to find the optimal screw placement position.

[0004] In order to improve the efficiency of robot-assisted navigation surgery, more and more technologies are committed to solving the problem of automatic screw planning to improve the efficiency of intraoperative planning. However, robot-assisted navigation usually performs intraoperative planning based on computed tomography (CT) image data. The data volume of CT image data is large, and the processing process not only has high requirements for equipment, but also takes a long time to process a large amount of data, which is difficult to effectively improve the efficiency of intraoperative planning. SUMMARY

[0005] Therefore, the embodiment of the present application provides a pedicle screw automatic planning device, equipment and computer program product, which can quickly and accurately predict the planning position of the pedicle screw based on the outer layer point cloud data of the vertebral body, reduce the amount of data to be processed in the planning process, and improve the planning efficiency.

[0006] The first aspect of the embodiment of the present application provides a pedicle screw automatic planning device, comprising:

[0007] A data unit is configured to acquire spine image data, extract the outer layer point cloud data of a single vertebral body to be planned from the spine image data, and the spine image data is three-dimensional image data.

[0008] A prediction model is configured to process the outer layer point cloud data of the vertebral body and output predicted coordinates, wherein the predicted coordinates include entry point coordinates and exit point coordinates.

[0009] a planning unit, configured to plan an implantation position of a pedicle screw based on the entry point coordinate and the exit point coordinate;

[0010] The prediction model comprises a plurality of key point branches, and each key point branch is configured to predict a probability that each point in the outer layer point cloud data of the vertebral body belongs to an entry point or an exit point corresponding to the key point branch, and the probability is used as an attention weight when outputting the predicted coordinate to determine the entry point or the exit point.

[0011] Optionally, the prediction model is specifically configured to:

[0012] When the outer layer point cloud data of the vertebral body is received, each point in the outer layer point cloud data of the vertebral body is dimensioned up, and feature extraction is performed on each dimensioned-up point.

[0013] The extracted features are processed by a plurality of key point branches to predict a probability that each point belongs to an entry point or an exit point corresponding to the key point branch.

[0014] Optionally, the prediction model is further configured to:

[0015] Each point in the outer layer point cloud data of the vertebral body is dimensioned up respectively, a point cloud local feature of each dimensioned-up point is extracted, and a global feature is extracted from the point cloud local feature of each point.

[0016] After the global feature is extended to the same dimension as the point cloud local feature, the global feature and the point cloud local feature are fused to obtain a fused feature.

[0017] Optionally, the prediction model is further configured to:

[0018] The fused feature is processed by a plurality of key point branches to obtain a feature value of each point under each key point branch.

[0019] The feature value of each point under each key point branch is converted into a probability distribution to obtain a probability that each point belongs to an entry point or an exit point corresponding to the key point branch.

[0020] Optionally, the number of key point branches is four, the entry point coordinate comprises a first entry point coordinate and a second entry point coordinate, the exit point coordinate comprises a first exit point coordinate and a second exit point coordinate, and the four key point branches are respectively configured to predict the first entry point coordinate, the second entry point coordinate, the first exit point coordinate or the second exit point coordinate. The prediction model is further configured to:

[0021] Determine a key point type corresponding to each key point branch.

[0022] For any key point branch, the weighted sum of each point in the outer layer point cloud data of the vertebral body is obtained, and the key point coordinate value corresponding to the key point type is obtained, the key point coordinate value is the first entry point coordinate, the second entry point coordinate, the first exit point coordinate or the second exit point coordinate, and the weight in the weighted sum is the attention weight.

[0023] Optionally, the loss function of the prediction model is obtained by optimizing a mean square error function, and the loss function is used to update the model parameters of the prediction model by back propagation during model training; the optimization of the mean square error function comprises:

[0024] determining a point loss value in the model training process based on the mean square error function;

[0025] determining other loss values according to the requirements for placing the pedicle screw, the requirements for placing the pedicle screw including angle requirements, length requirements and geometric property requirements of two screws, the two screws including a first screw represented by the first entry point coordinate and the first exit point coordinate and a second screw represented by the second entry point coordinate and the second exit point coordinate; accordingly, the other loss values include one or more of angle loss value, length loss value and geometric consistency loss value;

[0026] constructing the loss function of the prediction model based on the point loss value and the other loss values.

[0027] Optionally, the construction of the loss function of the prediction model based on the point loss value and the other loss values comprises:

[0028] allocating weight values to the point loss value and the other loss values respectively;

[0029] weighting the corresponding point loss value and the other loss values according to the allocated weight values to obtain the loss function.

[0030] Optionally, the data unit is specifically configured to:

[0031] segmenting the spinal image data into voxel data of a single vertebral body;

[0032] extracting the outer layer point cloud data of the vertebral body from the voxel data of the single vertebral body to be planned.

[0033] The second aspect of the embodiment of the application provides a pedicle screw automatic planning method, comprising:

[0034] acquiring spinal image data, and extracting outer layer point cloud data of a single vertebral body to be planned from the spinal image data, the spinal image data being three-dimensional image data;

[0035] adopting a prediction model to process the outer-layer point cloud data of the vertebral body, and outputting predicted coordinates, the predicted coordinates including an entry point coordinate and an exit point coordinate;

[0036] based on the entry point coordinate and the exit point coordinate, planning an implantation position of a pedicle screw;

[0037] The prediction model includes a plurality of key point branches, each of the key point branches being used to predict a probability that each point in the outer-layer point cloud data of the vertebral body belongs to an entry point or an exit point corresponding to the key point branch, and the probability is used as an attention weight to determine the entry point or the exit point when the predicted coordinates are outputted.

[0038] A third aspect of the embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the computer device implements the method according to the second aspect.

[0039] A fourth aspect of the embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by a computer, the method according to the second aspect is implemented.

[0040] A fifth aspect of the embodiment of the present application provides a computer program product, including a computer program, when the computer program is executed, the method according to the second aspect is executed.

[0041] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0042] The embodiment of the present application aims at the problems of large data volume of spinal image data, long processing time and high requirements for processing equipment, by extracting the outer-layer point cloud data of the single vertebral body to be planned from the spinal image data, the outer-layer point cloud data of the single vertebral body can be used for coordinate prediction, thereby reducing the data volume to be processed in the planning process and improving the planning efficiency. In addition, by training the prediction model including a plurality of key point branches, each key point branch can be used to predict the position of each key point required in the implantation process. In addition, in the prediction process, the embodiment of the present application introduces attention weights in the model, so as to improve the accuracy of the model prediction, and the purpose of quickly and accurately predicting the planning position of the pedicle screw can be achieved. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0044] Figure 1 is a schematic diagram of an automatic pedicle screw planning device provided by an embodiment of the present application;

[0045] Figure 2 is a schematic diagram of an automatic pedicle screw planning method provided by an embodiment of the present application;

[0046] Figure 3 is an architecture schematic diagram of a prediction model provided by an embodiment of the present application;

[0047] Figure 4 is a schematic diagram of another automatic pedicle screw planning method provided by an embodiment of the present application;

[0048] Figure 5 is a schematic diagram of another automatic pedicle screw planning device provided by an embodiment of the present application;

[0049] Figure 6 is a schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the present application. However, persons skilled in the art will understand that embodiments of the present application can be practiced without these specific details. In other instances, well-known systems, devices, circuits, and methods have not been described in detail so as not to obscure the description of the present application.

[0051] As mentioned above, in order to improve the efficiency of robot-assisted navigation surgery, more and more technologies are devoted to solving the problem of automatic screw planning to improve the efficiency of intraoperative planning. For example, by improving the related algorithm, taking the CT image data of the spine as the algorithm input, and using the algorithm output to plan the positions of two screws, i.e., the positions of two entry points and two exit points. The related algorithm includes a traditional algorithm or a deep learning algorithm.

[0052] In traditional algorithms, the position of the pedicle can be determined according to special landmarks by detecting the geometric landmark points of the cone, so as to obtain the planned position of the screw; or the planned position of the screw can be obtained by registering the image data with the anatomical atlas; in addition, the best cross section of each cone can be found, and the screw placement planning is performed by using the cross section. Traditional algorithms have good recognition effect in common spinal cone morphology, but have poor generalization and are difficult to apply to special cases such as spinal scoliosis and physiological morphological lesions of the cone.

[0053] Deep learning algorithms can help improve the shortcomings of traditional algorithms. In the use of deep learning algorithms to process the screw placement planning of the pedicle screw, there are two processing processes: first, the CT image data is semantically segmented to obtain the voxel data of each cone, and second, the voxel data obtained by semantic segmentation is used as input data to identify each cone using a deep learning model. In related methods, a convolutional neural network model, such as the UNet model and its variants, is usually used to train the deep learning model, and the entry point and exit point of the screw planning position are predicted by the full connection layer regression prediction or Gaussian heat map prediction method. The training process of the deep learning model uses mean-square error (MSE) as the loss function to update the model parameters by back propagation. However, the above-mentioned use of deep learning algorithms to plan the screw placement position of the pedicle screw has the following problems:

[0054] 1) The voxel data of the spinal cone is large, requiring high-performance equipment, such as large memory devices, to process a large amount of voxel data. Moreover, predicting the screw entry point and exit point of multiple cones also requires a long time.

[0055] 2) In order to reduce the prediction time, the deep learning model often uses the full connection layer regression prediction method, but this regression method of calculating coordinate points lacks spatial generalization ability. On the contrary, the prediction results obtained by using Gaussian heat map for prediction are more accurate, but the calculation based on Gaussian heat map consumes a large amount of memory and time, which does not help to improve the efficiency of intraoperative planning.

[0056] 3) The training process of the deep learning model often uses MSE as the loss function to update the model parameters. From the perspective of model training, the MSE function only considers the exit point and entry point of the screw planning, and lacks consideration of the screw angle and length in the cone, making the deep learning model trained based on this loss function have low precision. The screw position predicted by using this model is quite different from the actual situation, and it is difficult to be directly applied in clinical practice.

[0057] To solve the above problems and improve the efficiency of automatic screw planning, the applicant has found, in the implementation of the method, that for pedicle screw planning, the bone mass inside the cone has no effect on the position of screw planning, and in the actual planning process, the use of the outermost vertebra voxel can also complete the planning, thereby significantly reducing the amount of data to be processed. In addition, the applicant has also improved the loss function used in the training process of the existing deep learning model, so that the deep learning model trained can better meet the clinical application. Therefore, the present application provides a pedicle screw automatic planning device, equipment and computer program product, which can quickly and accurately predict the planning position of the pedicle screw based on the processing of the outer layer point cloud data of the vertebra.

[0058] The technical solutions of the present application will be described below through specific embodiments.

[0059] Referring to Figure 1 , a schematic diagram of a pedicle screw automatic planning device provided by an embodiment of the present application is shown, which can specifically include a data unit 101, a prediction model 102 and a planning unit 103; wherein:

[0060] The data unit 101 is configured to obtain spine image data and extract the outer layer point cloud data of a single vertebra to be planned from the spine image data.

[0061] The prediction model 102 is configured to process the outer layer point cloud data of the vertebra and output predicted coordinates, wherein the predicted coordinates include entry point coordinates and exit point coordinates.

[0062] The planning unit 103 is configured to plan the implantation position of the pedicle screw based on the entry point coordinates and the exit point coordinates.

[0063] In the embodiments of the present application, the pedicle screw automatic planning device can be a computer device or composed of one or more components in the computer device, which can be used to process the spine image data and assist the doctor to complete the pedicle screw implantation planning. The above-mentioned computer device can be a desktop computer, a cloud server and the like, and the form of the planning device and the type of the computer device are not limited in the embodiments of the present application.

[0064] The above-mentioned spine image data can be three-dimensional image data, such as CT image data. Before the spine surgery, the CT device can be used to scan the spine region of the patient to obtain the CT image data. In some scenarios, the three-dimensional image data can also be magnetic resonance imaging (MRI) or other types of image data, and the type of the spine image data and the way of acquiring the spine image data are not limited in the embodiments of the present application.

[0065] The automatic pedicle screw planning device in the embodiments of the present application can include a data unit 101, a prediction model 102, and a planning unit 103. The data unit 101 and the planning unit 103 can be components of the device and can exist in the form of software modules or hardware modules. The prediction model 102 can be a deep learning model. After the prediction model 102 is trained, it can be configured in the device for use in automatic pedicle screw planning. Alternatively, the prediction model 102 can also be configured in other devices or apparatuses that can communicate with the above-mentioned device, for example, the prediction model 102 can be configured in a cloud server. In this way, when the prediction model 102 is needed for point cloud data processing, the device can communicate with other devices or apparatuses to call the prediction model 102. The embodiments of the present application do not limit the configuration form of the prediction model 102.

[0066] In the embodiments of the present application, the data unit 101 can be a unit for performing preliminary data processing on the spinal image data. Through the data unit 101, the spinal image data can be obtained, and the outer layer point cloud data of the single vertebra to be planned can be extracted from the spinal image data.

[0067] Exemplarily, the data unit 101 can be a unit in the device that can communicate with an image device, for example, the data unit 101 can communicate with a CT device, so that after the CT image data of the patient's spine is collected by the CT device, the spinal CT data can be obtained from the CT device.

[0068] From the perspective of pedicle screw planning, the internal bone of the vertebra has no effect on the position of screw planning, so the complete voxel data is not needed when planning the screw position. Based on the above reasons, after the data unit 101 obtains the spinal image data, the data unit 101 can further process the spinal image data to extract the outer layer point cloud data of the single vertebra to be planned. The above-mentioned single vertebra to be planned can be the vertebra that needs to be screwed in the spinal surgery. In a spinal surgery, the vertebra that needs to be screwed can include only one or multiple vertebrae. Therefore, when the data unit extracts the outer layer point cloud data of the single vertebra, one or more vertebrae that need to be screwed can be processed respectively to obtain the outer layer point cloud data of each vertebra that needs to be screwed.

[0069] In the embodiments of the present application, the data unit 101 can manually or automatically segment the spine image data into the voxel data of a single vertebral body. For example, the spine CT data of the whole patient can be segmented into the voxel data of a single vertebral body by manual segmentation or automatic segmentation (such as semantic segmentation). Then, the data unit 101 can extract the outer layer point cloud data of the vertebral body from the voxel data of the single vertebral body to be planned. The above process can be implemented by using a three-dimensional isosurface extraction method, and the embodiments of the present application are not limited thereto.

[0070] By processing the three-dimensional spine image data and extracting the outer layer point cloud data of a single vertebral body therefrom, the complete vertebral body voxel data can not be used, the data amount for subsequent processing is reduced, and the data processing efficiency is improved. At the same time, since the data amount required for processing in the prediction stage is greatly reduced, the requirement for device resources is also greatly reduced, for example, the present method can be implemented in a device with a smaller memory, greatly expanding the application range of the present method, and the time required for model prediction in the next stage can also be reduced.

[0071] In the embodiments of the present application, the prediction model 102 can process the outer layer point cloud data of a single vertebral body extracted by the data unit 101 and output the predicted coordinates of the screw placement position. It should be noted that when the number of vertebral bodies to be placed in the spine surgery is more than one, the prediction model 102 can process the outer layer point cloud data of each vertebral body respectively, so as to output the predicted coordinates in each vertebral body.

[0072] In the embodiments of the present application, the prediction model 102 described above can include a plurality of key point branches, each of which can be used to predict the probability that each point in the outer layer point cloud data of the vertebral body belongs to the entry point or the exit point corresponding to the key point branch. The probability can be used as an attention weight to determine the corresponding entry point coordinates or exit point coordinates when outputting the predicted coordinates.

[0073] Specifically, since each vertebral body to be placed needs to plan two screws, each screw includes an entry point and an exit point. Therefore, planning the placement positions of the two screws requires predicting the positions of the two entry points and the two exit points, respectively. For example, the two entry points can include a first entry point and a second entry point, and the two exit points can include a first exit point and a second exit point. Among them, the first entry point and the first exit point can be taken as a group, as the entry point and the exit point of one of the planned screws (for example, the first screw); the second entry point and the second exit point can be taken as another group, as the entry point and the exit point of the other screw (for example, the second screw). In this way, the number of key point branches included in the prediction model 102 can be four, and the four key point branches can be used to predict the first entry point, the second entry point, the first exit point and the second exit point, respectively.

[0074] In addition, in the embodiments of the present application, in the process of predicting the entry point and the exit point of the screw by using the key point branch, an attention mechanism is introduced, which can convert the probability output by each key point branch into an attention weight. The coordinates of each point output by the key point branch and the corresponding attention weight are combined to calculate the planning positions of the two screws, i.e., the coordinates of the entry point and the exit point of the first screw and the coordinates of the entry point and the exit point of the second screw.

[0075] The planning unit 103 can complete the planning of the placement position of the pedicle screw based on the predicted coordinates output by the prediction model 102.

[0076] In a possible implementation manner of the embodiments of the present application, the prediction model 102 can be improved based on a PointNet model for processing point clouds. In this way, the characteristics of point cloud feature extraction and the advantage of fast calculation of the PointNet model can be utilized, an attention mechanism is added for regression prediction on this basis, and a loss function suitable for pedicle screw planning is designed, so that the automatic planning of the pedicle screw placement position can be realized with better effect.

[0077] In the prior art, UNet and other medical image processing models are mainly used to train a deep learning model for pedicle screw planning. The UNet model is mainly used for processing voxel data, and the input of the model is voxel data. Although the UNet model has good local feature extraction capability and can retain spatial details through a skip connection, the use of the model has a high calculation cost, the input voxel data needs more device memory for processing, and the overall calculation time is long. Unlike the UNet model, the PointNet model is specially used for processing three-dimensional (3D) point cloud data. The input of the PointNet model is 3D point cloud data, the model can extract local features through hierarchical feature learning, has a low calculation cost, only needs to occupy a small amount of device memory for processing point cloud data, and has a short overall calculation time.

[0078] Therefore, the embodiments of the present application improve the PointNet model, train a deep learning model, i.e., the aforementioned prediction model 102, and apply it to the automatic planning of the pedicle screw placement position, which can significantly improve the planning efficiency.

[0079] In the following, the specific processing process of the prediction model 102 in the embodiments of the present application will be described in detail. Figure 1

[0080] ​In this embodiment, after receiving extravertebral point cloud data of a single vertebral body, the prediction model 102 can first increase the dimensionality of each point in the extravertebral point cloud data and then extract features for each point after dimensionality increase. Then, the prediction model 102 can process the extracted features using multiple keypoint branches to predict the probability that each point belongs to the in-point or out-point corresponding to that keypoint branch. The features extracted from the extravertebral point cloud data can include local point cloud features and global features.

[0081] Specifically, for prediction model 102, its input data is the extravertebral point cloud data of a single vertebral body. For example, the extravertebral point cloud data can be represented as: .in, This represents the set of point cloud data outside the vertebral body. N This represents the size of the point cloud, which is the data set mentioned above. The number of points included in This data set CCP N There are 3 points. The number 3 indicates the magnitude of the coordinate vector. The above input data can be received by the input layer of the prediction model 102. For the above input data, the output data of the prediction model 102 is the predicted coordinates of the two screws, that is, the first in-point coordinates and the first out-point coordinates of the first screw, and the second in-point coordinates and the second out-point coordinates of the second screw.

[0082] In this embodiment, the input layer of the prediction model 102 can be connected to a shared multilayer perceptron (MLP) layer. Each point in the input data shares weights after being transmitted to the shared MLP layer, which is equivalent to passing each point through a separate feedforward neural network.

[0083] For example, the prediction model 102 can upscale each point in the extrapyramidal point cloud data by using the shared MLP layer described above to process the point cloud data, passing it through multiple linear layers to map the three-dimensional data to higher-dimensional data. For example, the three-dimensional point cloud can be upscaled to 256 dimensions. Therefore, when performing feature extraction for each point after upscaling, the prediction model 102 can perform feature extraction on the higher-dimensional data. For example, it can extract the local features of each point from the 256-dimensional data. Based on this, global features are further extracted from the local features of each point. The process of extracting global features from the local features of the point cloud can be implemented using the max-pooling layer of the prediction model 102. The local and global features of the point cloud can be further fused and then transmitted to each keypoint branch for coordinate prediction.

[0084] In the embodiments of the present application, the global feature extracted from the local feature of the point cloud can have a dimension different from that of the local feature of the point cloud. Therefore, the prediction model 102 needs to expand the global feature to the same dimension as the local feature of the point cloud, and then fuse the global feature and the local feature to obtain the fused feature. The above feature fusion process can be realized by the feature fusion layer (Concat) of the prediction model 102.

[0085] As a specific example in the embodiments of the present application, the shared MLP layer of the prediction model 102 can include linear layers of (N, 64), (N, 128), (N, 256), etc. Each outer layer point cloud in the input model can be upgraded from three-dimensional mapping to 256-dimensional data after passing through the linear layers of (N, 64), (N, 128), (N, 256), etc. The extraction of the local feature of the point cloud can be performed on the 256-dimensional data, that is, the local feature (N, 256) of the point cloud is extracted from the 256-dimensional data after upgrading. Then, through the pooling processing of the max pooling layer, the global feature (1, 256) can be extracted from the local feature (N, 256).

[0086] Since the global feature (1, 256) and the local feature (N, 256) have different dimensions, they cannot be directly fused, and the prediction model 102 needs to expand the global feature (1, 256) to the same dimension as the local feature (N, 256), that is, expand the local feature (1, 256) to (N, 256), and then fuse it with the local feature (N, 256) to obtain the fused feature (N, 512).

[0087] The prediction model 102 can process the fused feature (N, 512) using multiple key point branches to obtain the feature value of each point under each key point branch.

[0088] In the embodiments of the present application, by converting the feature value of each point under each key point branch into a probability distribution, the probability that each point belongs to the entry point or the exit point corresponding to the key point branch can be obtained.

[0089] For example, the number of key point branches is 4, and the four key point branches can be represented as Screw-i (i = 1, 2, 3, 4), which are respectively used to predict the first entry point, the first exit point, the second entry point, and the second exit point. For example, the key point branch Screw-1 can be used to predict the first entry point, the key point branch Screw-2 can be used to predict the first exit point, the key point branch Screw-3 can be used to predict the second entry point, and the key point branch Screw-4 can be used to predict the second exit point.

[0090] In this way, after the fused feature (N, 512) is input into each key point branch, the corresponding feature value can be output by the key point branch, and by converting the feature value into a probability distribution, the probability of each point belonging to the entry point or exit point corresponding to each key point branch can be obtained.

[0091] For example, after the fused feature (N, 512) is input into the key point branch Screw-1, the feature value under the key point branch Screw-1 can be obtained. Since the key point branch Screw-1 is used to predict the first entry point, after the feature value is converted into a probability distribution, the probability of each point belonging to the first entry point corresponding to the key point branch Screw-1 can be obtained. Correspondingly, according to the feature values under other key point branches, such as the feature values under the key point branches Screw-2, Screw-3 and Screw-4, and after the probability distribution conversion, the probabilities of each point belonging to the first exit point, the second entry point and the second exit point can be obtained respectively. The above probabilities can be used as the attention weight of the corresponding key point branch in subsequent calculations.

[0092] In the embodiments of the present application, for any key point branch, the weighted sum of each point in the outer layer point cloud data of the vertebral body can be obtained to obtain the key point coordinate value of the corresponding key point type. The key point coordinate value is the first entry point coordinate, the second entry point coordinate, the first exit point coordinate or the second exit point coordinate. The weight in the weighted sum is the attention weight calculated above, and the key point type can be used to represent the type of key point predicted by the corresponding key point branch. For example, a key point branch is used to predict the first entry point coordinate, and the key point type of this branch can be the first entry point, and the corresponding calculated key point coordinate value is the first entry point coordinate.

[0093] In a possible implementation manner of the embodiments of the present application, in order to improve the accuracy of the prediction result of the deep learning model, the embodiments of the present application further provide a more targeted and more clinically applicable loss function (Loss Function) for updating the model parameters of the model in the model training process, so that the model training can converge faster, and the deep learning model trained can more accurately predict the entry point and exit point coordinates of the screw to be placed.

[0094] In the embodiments of the present application, the loss function can be obtained based on optimization of the mean square error function MSE.

[0095] For example, the mean square error function MSE can be expressed as:

[0096]

[0097] wherein, represents a predicted point, Points representing theoretical values.

[0098] Generally, the mean square error function MSE is mainly used to evaluate the distance between points, so the deep learning model taking the mean square error function MSE as the loss function only considers the distance between the input point and the output point. That is, the model trained by using the above mean square error function MSE as the loss function is aimed at the distance between the actual input point / output point and the predicted input point / output point during training, and does not directly consider factors such as the length of the screw. The screw formed by the input point and the output point predicted by the model may be quite different from the screw required in actual application. To solve this problem, the embodiments of the present application optimize the mean square error function MSE, can combine other factors in the screw placement process, and comprehensively construct a loss function to obtain a more accurate deep learning model.

[0099] For example, in the pedicle screw placement planning, not only the four key points need to be considered, but also the relative angle of the two screws of the cone is required in actual clinical application, and the included angle between the two screws and the central axis cannot be too large; the length of the screw is also required, and the two screws cannot be too long or too short; in addition, the geometric characteristics (such as symmetry and rationality of length) of the two screws of the same cone should also meet certain requirements. The embodiments of the present application optimize the mean square error function MSE, and consider not only the distance between points in the construction of the loss function and the model training, but also the above angle, length and geometric consistency requirements, thereby providing a loss function more suitable for clinical application.

[0100] Specifically, the point loss value in the model training process can be first determined based on the mean square error function, and other loss values can be determined according to the pedicle screw placement requirements. The two screws can include a first screw represented by the first input point coordinates and the first output point coordinates introduced in the foregoing, and a second screw represented by the second input point coordinates and the second output point coordinates. Correspondingly, the other loss values determined according to the placement requirements can include one or more of the angle loss value, the length loss value and the geometric consistency loss value. In this way, the loss function of the prediction model can be constructed based on the point loss value and the other loss values, so as to obtain a more accurate deep learning model more suitable for the pedicle screw placement requirements.

[0101] In the embodiments of the present application, the point loss value required for constructing the loss function is the same as the mean square error function MSE, and the point loss value may be represented as:

[0102]

[0103] The angle loss value may be represented as:

[0104]

[0105] wherein, , denotes the predicted angle of the screw 1, 2 to the central axis, , denotes the real angle of the screw 1, 2 to the central axis. The screw 1, 2 can be the first screw and the second screw in the aforementioned various embodiments.

[0106] length loss value may be expressed as:

[0107]

[0108] wherein, , denotes the predicted length of the screw 1, 2, , denotes the real length of the screw 1, 2. The screw 1, 2 can be the first screw and the second screw in the aforementioned various embodiments.

[0109] geometric consistency loss value may be expressed as:

[0110]

[0111] By using the geometric consistency loss value , it can be used to constrain the prediction result of the model to meet the geometric constraint condition of the real screw planning, so that in predicting the angle and length of the target, in addition to making the numerical value close, it can also ensure that the rationality of the geometric relationship is met.

[0112] In this way, the loss function constructed based on the point loss value and one or more of the angle loss value, the length loss value and the geometric consistency loss value can better meet the clinical needs of the pedicle screw placement process.

[0113] Specifically, the point loss value and the other loss values can be first assigned a weight value respectively, and then the corresponding point loss value and the other loss values are weighted according to the assigned weight value to construct the loss function. In this way, the weight value can be assigned to each loss value according to its influence on the screw planning. The weight value can be adjusted according to the surgical habits of different doctors. For example, if the doctor has a higher requirement for the angle of the screw planning, the weight value of the corresponding angle loss value can be increased, so that the model will pay more attention to the angle of the screw planning when training and predicting.

[0114] Exemplarily, the loss function can be constructed based on the point loss value, the angle loss value, the length loss value and the geometric consistency loss value, and the loss function can be expressed as:

[0115]

[0116] wherein, the weight values respectively represent weight values of point loss value, length loss value, angle loss value and geometric consistency loss value.

[0117] The embodiments of the present application can increase various indicators for evaluating screw planning, and can construct a loss function based on multiple loss values set according to the various indicators, so that the model obtained by training is more suitable for clinical application when predicting screw entry and exit points. In addition, the weight of each loss value can also be adjusted according to the operation habits of different doctors to meet the personalized planning needs of doctors.

[0118] In order to facilitate understanding, the automatic planning of pedicle screws provided by the embodiments of the present application will be introduced in combination with a complete example.

[0119] Referring to Figure 2 , a schematic diagram of an automatic pedicle screw planning method provided by the embodiments of the present application is shown, and the overall process of the method can be summarized as three parts, namely: data preparation, feature extraction and attention mechanism prediction. Next, in combination with Figure 2 , a detailed introduction will be made.

[0120] 1. In the data preparation stage:

[0121] In the robot-assisted navigation surgery, the overall spinal image data can be segmented into single vertebral voxel data based on the patient's spinal image data through manual or automatic segmentation. For example, as shown in Figure 2 , taking the CT image data of a patient as an example, the overall spinal CT image data can be segmented into single vertebral voxel data through semantic segmentation, and then the single vertebral voxel data is processed by point cloud. In the point cloud processing step, the single vertebral voxel data can be converted into point cloud data outside the vertebral body by using the method of three-dimensional isosurface extraction.

[0122] 2. In the feature extraction stage:

[0123] Based on the obtained single vertebral outer layer point cloud data of the vertebral body, each step of automatic planning of pedicle screws can be further performed. The embodiments of the present application can utilize a deep learning model to process the single vertebral outer layer point cloud data of the vertebral body, and output the corresponding predicted coordinates.

[0124] As shown in Figure 2As shown, for the obtained outer layer point cloud data of a single vertebral body, a deep learning model is called and model parameters are loaded, and the deep learning model can be the prediction model in the foregoing embodiments. First, the outer layer point cloud data of a single cone is input into the model for feature extraction. The embodiments of the present application can extract local features and global features of the point cloud through a multi-layer perceptron (MLP) and fuse the two, and then input the fused features into the key point branch, i.e., the four key point branches introduced in the foregoing embodiments, which are two in-point key point branches and two out-point key point branches.

[0125] As shown in Figure 3 , it is an architecture schematic diagram of a prediction model provided by the embodiments of the present application. The following describes how to build the model and how to use the model to perform feature extraction in combination with the architecture of the prediction model shown in Figure 3 .

[0126] 2.1 Model basis

[0127] The embodiments of the present application are improved based on the PointNet model for processing point clouds, and the characteristics of point cloud feature extraction and the advantage of fast calculation of the PointNet model are used to increase the attention mechanism regression prediction, and a loss function suitable for pedicle screw planning is designed, which can realize automatic planning with good effect.

[0128] Unlike the medical image model UNet commonly used in the prior art to process voxel data, PointNet is specially used to process 3D point cloud data.

[0129] UNet is a commonly used medical image model, and its input is voxel data. The model has good local feature extraction capability and can retain spatial details through skip connection, but the application of the UNet model has high calculation cost, the input voxel data needs more device memory for processing, and the calculation time is longer.

[0130] PointNet is specially used to process 3D point cloud data, and its input is 3D point cloud data. It can extract local features through hierarchical feature learning, has low calculation cost, and only needs to occupy a small amount of device memory for processing point cloud data, and the calculation time is shorter.

[0131] 2.2 Model design

[0132] The input data of the model can be represented as: ;

[0133] The output data of the model is: two pedicle screw planning positions, i.e., two in-point coordinates and two out-point coordinates.

[0134] Referring to Figure 3At the input layer of the model, the received data is the point cloud data of the outer layer of a single vertebral body, and the data processing process of the input layer can be represented as Input Layer(N, 3).

[0135] At the shared MLP layer, all points share weights, which is equivalent to passing each point through a separate feedforward neural network. Specifically, the input 3D point cloud can be mapped to 256 dimensions by Linear (N, 64), Linear (N, 128), Linear (N, 256), etc. linear layers, and the activation function is Relu, which is used to extract local features of the point cloud (N, 256).

[0136] At the max pooling layer, global features (1, 256) can be extracted from the local features of the point cloud (N, 256).

[0137] At the feature fusion layer (Concat), the global features (1, 256) can be first expanded to (N, 256), and then fused with the local features (N, 256) to obtain the fused features (N, 256).

[0138] Figure 3 Screw-i represents 4 key points, namely 2 entry points and 2 exit points; MLP-i represents an MLP branch of a certain key point, which is used to compress feature dimensions and improve calculation efficiency; Softmax-i represents converting the feature value of a certain MLP branch into a probability distribution to obtain an attention weight; Weighted Sum-i represents weighted sum of each point based on the attention weight of a certain MLP branch to obtain the predicted coordinate value of a certain key point:

[0139]

[0140] wherein, represents the coordinates of the predicted point, represents the attention weight calculated by Softmax, represents the nth point of the point cloud.

[0141] 2.3 Loss function design

[0142] The loss function is used to update the model parameters of the model during the model training process. According to the characteristics of the pedicle screw planning, the loss function is optimized based on the mean square error function MSE, wherein the MSE loss function can be represented as:

[0143]

[0144] wherein, denotes a predicted point, denotes a point of a theoretical value.

[0145] The mean square error function MSE is suitable for evaluating individual key points, but screw planning is not only 4 key points, in actual clinical, the relative angle of two screws of the cone is also required, the included angle of the two screws and the central axis cannot be too large; the length of the screw is also required, which cannot be too long or too short; the geometric characteristics (symmetry, rationality of length) of the two screws of the same cone are also required. Based on this, the embodiments of the present application can combine the point loss value, the angle loss value, the length loss value and the geometric consistency loss value to construct the loss function.

[0146] (a) Point loss value which can be expressed as (consistent with the MSE calculation):

[0147]

[0148] (b) Angle loss value which can be expressed as:

[0149]

[0150] wherein, , denote the predicted angles of the screws 1, 2 and the central axis, , denote the true angles of the screws 1, 2 and the central axis. The above-mentioned screws 1, 2 can be the first screw and the second screw in the foregoing various embodiments.

[0151] (c) Length loss value which can be expressed as:

[0152]

[0153] wherein, , denote the predicted lengths of the screws 1, 2, , denote the true lengths of the screws 1, 2. The above-mentioned screws 1, 2 can be the first screw and the second screw in the foregoing various embodiments.

[0154] (d) Geometric consistency loss value which can be expressed as:

[0155]

[0156] The geometric constraint condition of the true screw planning is used to constrain the prediction result of the model, and in predicting the angle and length of the target, in addition to the numerical proximity, the rationality of the geometric relationship is also required to be met.

[0157] (e) The loss function is constructed as follows:

[0158]

[0159] According to the influence of each loss value on the screw planning, a weight value is respectively given ,The weight values of the point loss value, the length loss value, the angle loss value and the geometric consistency loss value are respectively represented. The weight values may be adjusted according to the operation habits of different doctors, for example, if the doctor has high requirements for the angle of screw planning, the corresponding weight value can be increased, and the model will pay more attention to the angle of screw planning during training and prediction.

[0160] 3. In the attention mechanism prediction stage:

[0161] Referring back to Figure 2 , in the attention mechanism prediction stage, the deep learning model introduced in the foregoing can be used to complete the prediction of the screw position. The process can include the steps of obtaining attention weights by using a Softmax function for normalization processing, and obtaining the coordinates of each key point by weighted summation.

[0162] Based on the introduction of each of the foregoing embodiments, referring to Figure 4 , a schematic diagram of another automatic pedicle screw planning method provided by the embodiments of the present application is shown, which can specifically include the following steps:

[0163] S401, acquiring spine image data, extracting the outer point cloud data of a single vertebra to be planned from the spine image data, and the spine image data is three-dimensional image data.

[0164] S402, using a prediction model to process the outer point cloud data of the vertebra, and outputting predicted coordinates, the predicted coordinates including entry point coordinates and exit point coordinates.

[0165] S403, planning the implantation position of the pedicle screw based on the entry point coordinates and the exit point coordinates.

[0166] The prediction model can be the prediction model or deep learning model introduced in each of the foregoing embodiments, and the prediction model includes a plurality of key point branches, each of which is used to predict the probability that each point in the outer point cloud data of the vertebra belongs to the entry point or the exit point corresponding to the key point branch, and the probability is used as an attention weight to determine the entry point or the exit point when the predicted coordinates are output.

[0167] Figure 4The execution subject of the illustrated pedicle screw automatic planning method can be a computer device that performs the following steps by executing a computer program Figure 4 The illustrated steps can extract the outer point cloud data of a single vertebra to be planned from the spine image data after obtaining the spine image data, thereby reducing the amount of point cloud data to be processed subsequently. For the outer point cloud data of a single vertebra, the computer device can call a prediction model, use the outer point cloud data of the vertebra as input data of the prediction model, and use the prediction model to output predicted coordinates, i.e., the in-point coordinates and the out-point coordinates of two screws. The computer device can complete the planning of the screw placement positions of the two screws based on the in-point coordinates and the out-point coordinates output by the model.

[0168] The construction process of the prediction model in the embodiments of the present application and how to use the prediction model to predict the key point coordinates can be referred to the descriptions in the foregoing embodiments, which will not be repeated here.

[0169] It should be noted that the sequence numbers of the steps in the foregoing embodiments do not mean the execution sequence, and the execution sequence of the processes should be determined according to the functions and the inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0170] Referring to Figure 5 , another schematic diagram of a pedicle screw automatic planning device provided by the embodiments of the present application is shown, which can specifically include an acquisition module 501, a prediction module 502, and a planning module 503, wherein:

[0171] The acquisition module 501 is configured to acquire spine image data and extract the outer point cloud data of a single vertebra to be planned from the spine image data, wherein the spine image data is three-dimensional image data.

[0172] The prediction module 502 is configured to use a prediction model to process the outer point cloud data of the vertebra and output predicted coordinates, wherein the predicted coordinates include in-point coordinates and out-point coordinates.

[0173] The planning module 503 is configured to plan the placement positions of pedicle screws based on the in-point coordinates and the out-point coordinates.

[0174] The prediction model can be the prediction model or the deep learning model described in the foregoing embodiments, and the prediction model includes a plurality of key point branches, each of which is configured to predict the probability that each point in the outer point cloud data of the vertebra belongs to the in-point or the out-point corresponding to the key point branch, and the probability is used as an attention weight to determine the in-point or the out-point when the predicted coordinates are output.

[0175] The functions that can be achieved by the various modules of the pedicle screw automatic planning device provided by the embodiments of the present application are the same as thoseFigure 1 The devices corresponding to various embodiments introduced above can be mutually referred to, and will not be described again here.

[0176] Referring to Figure 6 , a schematic diagram of a computer device is shown. As shown in Figure 6 , the computer device 600 in the embodiment of the present application includes a processor 610, a memory 620, and a computer program 621 stored in the memory 620 and executable on the processor 610. The processor 610 implements the steps in the above-described various embodiments of the pedicle screw automatic planning method when executing the computer program 621, such as the steps S401-S403 shown in Figure 4 . Alternatively, the processor 610 implements the functions of the modules / units in the above-described various device embodiments when executing the computer program 621, such as the functions of the units 101-103 shown in Figure 1 , or Figure 5 the functions of the modules 501-503 shown in .

[0177] Illustratively, the computer program 621 can be divided into one or more modules / units, which are stored in the memory 620 and executed by the processor 610 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which can be used to describe the execution process of the computer program 621 in the computer device 600. For example, the computer program 621 can be divided into an acquisition module, a prediction module, and a planning module, and the specific functions of each module are as follows:

[0178] The acquisition module is configured to acquire spinal image data, and extract pedicle outer point cloud data of a single vertebra to be planned from the spinal image data, the spinal image data being three-dimensional image data.

[0179] The prediction module is configured to process the pedicle outer point cloud data using a prediction model, and output predicted coordinates, the predicted coordinates including entry point coordinates and exit point coordinates.

[0180] The planning module is configured to plan a pedicle screw implantation position based on the entry point coordinates and the exit point coordinates.

[0181] The computer device 600 can be a device capable of implementing the various steps in the above-described various method embodiments, or a device capable of implementing the functions of the various units or modules in the above-described various device embodiments. The computer device 600 can be a desktop computer, a cloud server, or the like. The computer device 600 can include, but is not limited to, a processor 610 and a memory 620. Those skilled in the art can understand that the computer device 600 can further include other components, and the components of the computer device 600 are not limited to the above.Figure 6 The computer device 600 is only an example of a computer device and does not serve to limit the computer device 600, and the computer device 600 can include more or fewer components than shown, or combine some components, or have different components, for example, the computer device 600 can also include an input / output device, a network access device, a bus, etc.

[0182] The processor 610 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0183] The memory 620 can be an internal storage unit of the computer device 600, for example, a hard disk or a memory of the computer device 600. The memory 620 can also be an external storage device of the computer device 600, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 620 can include both the internal storage unit and the external storage device of the computer device 600. The memory 620 is used to store the computer program 621 and other programs and data required by the computer device 600. The memory 620 can also be used to temporarily store data that has been output or will be output.

[0184] The embodiments of the present application also disclose a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the computer program implements the pedicle screw automatic planning method as described in the foregoing various embodiments, or implements the pedicle screw automatic planning method implemented by the foregoing various device embodiments.

[0185] The application further discloses a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and the computer program is executed by a computer to implement the pedicle screw automatic planning method in the foregoing various embodiments; or implement the pedicle screw automatic planning method implemented by the foregoing various device embodiments.

[0186] The application further discloses a computer program product, comprising a computer program, which, when running on a computer, causes the computer to execute the pedicle screw automatic planning method in the foregoing various embodiments; or implement the pedicle screw automatic planning method implemented by the foregoing various device embodiments.

[0187] The foregoing embodiments are only used to illustrate the technical solutions of the application, rather than limit the same. Although the application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and the modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in the protection scope of the application.

Claims

1. An automatic pedicle screw planning device, characterized in that, The method comprises the following steps: a data unit is configured to obtain spine image data, extract the outer layer point cloud data of a single vertebra to be planned from the spine image data, and the spine image data is three-dimensional image data; a prediction model is configured to process the outer layer point cloud data of the vertebra, and output predicted coordinates, wherein the predicted coordinates comprise an entry point coordinate and an exit point coordinate; a planning unit is configured to plan the implantation position of a pedicle screw based on the entry point coordinate and the exit point coordinate; wherein the prediction model comprises a plurality of key point branches, each of which is configured to predict the probability that each point in the outer layer point cloud data of the vertebra belongs to the entry point or the exit point corresponding to the key point branch, and the probability is used as an attention weight to determine the entry point or the exit point when the predicted coordinates are outputted; the number of key point branches is 4, the entry point coordinate comprises a first entry point coordinate and a second entry point coordinate, the exit point coordinate comprises a first exit point coordinate and a second exit point coordinate, and the four key point branches are respectively configured to predict the first entry point coordinate, the second entry point coordinate, the first exit point coordinate and the second exit point coordinate, and the prediction model is further configured to: determine the key point type corresponding to each key point branch; for any key point branch, the weighted sum of each point in the outer layer point cloud data of the vertebra is obtained to obtain the key point coordinate value of the key point type corresponding to the key point branch, and the key point coordinate value is the first entry point coordinate, the second entry point coordinate, the first exit point coordinate or the second exit point coordinate, and the weight in the weighted sum is the attention weight.

2. The apparatus of claim 1, wherein, The prediction model is specifically configured to: when receiving the outer layer point cloud data of the vertebra, dimensionally upgrading each point in the outer layer point cloud data of the vertebra, and performing feature extraction on each point after dimensionally upgrading; using a plurality of key point branches to process the extracted features to predict the probability that each point belongs to the entry point or the exit point corresponding to the key point branch.

3. The apparatus of claim 1 or 2, wherein, The prediction model is further configured to: respectively dimensionally upgrading each point in the outer layer point cloud data of the vertebra, extracting the point cloud local feature of each point after dimensionally upgrading, and extracting the global feature from the point cloud local feature of each point; after extending the global feature to the same dimension as the point cloud local feature, fusing the global feature and the point cloud local feature to obtain a fusion feature.

4. The apparatus of claim 3, wherein, The prediction model is further configured to: using a plurality of key point branches to process the fusion feature to obtain the feature value of each point under each key point branch; converting the feature value of each point under each key point branch into a probability distribution to obtain the probability that each point belongs to the entry point or the exit point corresponding to the key point branch.

5. The device of any one of claims 1 to 4, wherein, The loss function of the prediction model is obtained by optimizing the mean square error function, and the loss function is used to update the model parameters of the prediction model through back propagation during model training; the optimization of the mean square error function comprises: determining the point loss value in the model training process based on the mean square error function; determine other loss values according to the pedicle screw placement requirements, the pedicle screw placement requirements including angle requirements, length requirements and geometric characteristic requirements of two screws, the two screws including a first screw represented by the first entry point coordinate and the first exit point coordinate and a second screw represented by the second entry point coordinate and the second exit point coordinate; correspondingly, the other loss values include one or more of angle loss values, length loss values and geometric consistency loss values; construct a loss function of the prediction model based on the point loss values and the other loss values.

6. The apparatus of claim 5, wherein, The loss function of the prediction model is constructed based on the point loss values and the other loss values, including: assigning weight values to the point loss values and the other loss values respectively; weighting the corresponding point loss values and the other loss values according to the assigned weight values to obtain the loss function.

7. The apparatus of any one of claims 1 to 4 or 6, wherein, The data unit is specifically used for: segmenting the spine image data into voxel data of single vertebral bodies; extracting the outer layer point cloud data of the vertebral body from the voxel data of the single vertebral body to be planned.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the computer device implements the following method implemented by the pedicle screw automatic planning device according to any one of claims 1 to 7: obtaining spine image data, and extracting outer layer point cloud data of a single vertebral body to be planned from the spine image data, the spine image data being three-dimensional image data; processing the outer layer point cloud data of the vertebral body by using a prediction model to output prediction coordinates, the prediction coordinates including entry point coordinates and exit point coordinates; planning the placement position of a pedicle screw based on the entry point coordinates and the exit point coordinates; The prediction model includes a plurality of key point branches, each of which is used to predict the probability that each point in the outer layer point cloud data of the vertebral body belongs to the entry point or the exit point corresponding to the key point branch, and the probability is used as an attention weight when the prediction coordinates are outputted. The number of key point branches is 4, the entry point coordinates include first entry point coordinates and second entry point coordinates, the exit point coordinates include first exit point coordinates and second exit point coordinates, and the four key point branches are respectively used to predict the first entry point coordinates, the second entry point coordinates, the first exit point coordinates and the second exit point coordinates. When the computer program runs, the following method implemented by the pedicle screw automatic planning device according to any one of claims 1 to 7 is executed: ​ 9. A computer program product comprising a computer program, characterized in that, ​ Acquiring spine image data, extracting the outer layer point cloud data of a single vertebra to be planned from the spine image data, the spine image data being three-dimensional image data; Processing the outer layer point cloud data of the vertebra by using a prediction model, and outputting predicted coordinates, the predicted coordinates including entry point coordinates and exit point coordinates; Planning the implantation position of a pedicle screw based on the entry point coordinates and the exit point coordinates; The prediction model includes a plurality of key point branches, each of which is used to predict the probability of each point in the outer layer point cloud data of the vertebra belonging to the entry point or the exit point corresponding to the key point branch, and the probability is used as an attention weight to determine the entry point or the exit point when the predicted coordinates are outputted; The number of key point branches is 4, the entry point coordinates include first entry point coordinates and second entry point coordinates, the exit point coordinates include first exit point coordinates and second exit point coordinates, and the four key point branches are respectively used to predict the first entry point coordinates, the second entry point coordinates, the first exit point coordinates and the second exit point coordinates, and the processing of the outer layer point cloud data of the vertebra by using the prediction model and the output of the predicted coordinates include: Determining the key point type corresponding to each key point branch; For any key point branch, weighted summation is performed on each point in the outer layer point cloud data of the vertebra to obtain a key point coordinate value corresponding to the key point type, the key point coordinate value being the first entry point coordinates, the second entry point coordinates, the first exit point coordinates or the second exit point coordinates, and the weight in the weighted summation being the attention weight.

Images