Pedicle screw automatic planning device, equipment and computer program product

Abstract

The embodiment of the invention is suitable for the technical field of computer-aided medical treatment and image processing, and provides an automatic planning device and equipment for a pedicle screw and a computer program product. Extracting vertebral body outer layer point cloud data of a to-be-planned single vertebral body from the spine image data; the prediction model is used for processing the centrum outer layer point cloud data and outputting prediction coordinates, and the prediction coordinates comprise entry point coordinates and exit point coordinates; and the planning unit is used for planning the placement position of the pedicle screw based on the entry point coordinate and the exit point coordinate. By adopting the device, the planned position of the pedicle screw can be quickly and accurately predicted by using the deep learning model based on the point cloud data of the outer layer of the vertebral body, the data volume needing to be processed in the planning process is reduced, and the planning efficiency is improved.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application belongs to the technical field of computer-aided medical technology and image processing technology, and particularly relates to a pedicle screw automatic planning device, equipment and computer program product. BACKGROUND

[0002] Pedicle screw internal fixation is one of the core technologies of spine surgery, which can be used for treating diseases such as fracture, deformity and degeneration. With the progress of medical imaging and robot-assisted navigation technology, precise and safe screw placement is increasingly recognized by doctors and patients.

[0003] The main process of robot-assisted navigation technology includes three-dimensional image scanning, intraoperative registration, intraoperative planning and intraoperative navigation. In the surgical process, if the operation time is prolonged, the risk of infection, the amount of bleeding and the incidence of postoperative complications will increase, and intraoperative planning usually takes a lot of time, resulting in an increase in operation time. In spine surgery, two screws need to be planned for each cone to be implanted, and the two screws need to pass through the position of the cone pedicle and cannot invade the spinal cord nerve of the spine. Therefore, the doctor needs to carefully observe and adjust the position and angle of each pedicle screw to find the optimal screw placement position.

[0004] In order to improve the efficiency of robot-assisted navigation surgery, more and more technologies are committed to solving the problem of automatic screw planning to improve the efficiency of intraoperative planning. However, robot-assisted navigation usually performs intraoperative planning based on computed tomography (CT) image data. The data volume of CT image data is large, and the processing process not only has high requirements for equipment, but also takes a long time to process a large amount of data, which is difficult to effectively improve the efficiency of intraoperative planning. SUMMARY

[0005] Therefore, the embodiment of the present application provides a pedicle screw automatic planning device, equipment and computer program product, which can quickly and accurately predict the planning position of the pedicle screw based on the outer layer point cloud data of the vertebral body, reduce the amount of data to be processed in the planning process, and improve the planning efficiency.

[0006] The first aspect of the embodiment of the present application provides a pedicle screw automatic planning device, comprising: a data unit configured to obtain spine image data, extract outer layer point cloud data of a single vertebral body to be planned from the spine image data, and the spine image data is three-dimensional image data; a prediction model configured to process the outer layer point cloud data of the vertebral body and output predicted coordinates, wherein the predicted coordinates comprise an entry point coordinate and an exit point coordinate; a planning unit, configured to plan an implantation position of a pedicle screw based on the entry point coordinate and the exit point coordinate; The prediction model comprises a plurality of key point branches, and each key point branch is configured to predict a probability that each point in the outer layer point cloud data of the vertebral body belongs to an entry point or an exit point corresponding to the key point branch, and the probability is used as an attention weight when outputting the predicted coordinate.

[0007] Optionally, the prediction model is specifically configured to: When receiving the outer layer point cloud data of the vertebral body, dimensionality of each point in the outer layer point cloud data of the vertebral body is increased, and feature extraction is performed on each point after dimensionality increase. The extracted features are processed by a plurality of key point branches to predict a probability that each point belongs to an entry point or an exit point corresponding to the key point branch.

[0008] Optionally, the prediction model is further configured to: Each point in the outer layer point cloud data of the vertebral body is dimensionally increased respectively, a point cloud local feature of each point after dimensionality increase is extracted, and a global feature is extracted from the point cloud local feature of each point. After the global feature is extended to the same dimension as the point cloud local feature, the global feature and the point cloud local feature are fused to obtain a fusion feature.

[0009] Optionally, the prediction model is further configured to: The fusion feature is processed by a plurality of key point branches to obtain a feature value of each point under each key point branch. The feature value of each point under each key point branch is converted into a probability distribution to obtain a probability that each point belongs to an entry point or an exit point corresponding to the key point branch.

[0010] Optionally, the number of key point branches is four, the entry point coordinate comprises a first entry point coordinate and a second entry point coordinate, the exit point coordinate comprises a first exit point coordinate and a second exit point coordinate, and the four key point branches are respectively configured to predict the first entry point coordinate, the second entry point coordinate, the first exit point coordinate or the second exit point coordinate. The prediction model is further configured to: determine a key point type corresponding to each key point branch; For any key point branch, weighted summation is performed on each point in the outer layer point cloud data of the vertebral body to obtain a key point coordinate value corresponding to the key point type, and the key point coordinate value is the first entry point coordinate, the second entry point coordinate, the first exit point coordinate or the second exit point coordinate, and the weight in the weighted summation is the attention weight.

[0011] Optionally, the loss function of the prediction model is based on an optimization of a mean square error function, the loss function being used to update model parameters of the prediction model during model training by back propagation; the optimization of the mean square error function comprises: determining a point loss value in a model training process based on the mean square error function; determining other loss values based on the requirements for placing the pedicle screws, the requirements for placing the pedicle screws including angle requirements, length requirements and geometric characteristic requirements of two screws, the two screws including a first screw represented by the first entry point coordinates and the first exit point coordinates and a second screw represented by the second entry point coordinates and the second exit point coordinates; accordingly, the other loss values include one or more of an angle loss value, a length loss value and a geometric consistency loss value; constructing the loss function of the prediction model based on the point loss value and the other loss values.

[0012] Optionally, the construction of the loss function of the prediction model based on the point loss value and the other loss values comprises: allocating weight values to the point loss value and the other loss values respectively; weighting the corresponding point loss value and the other loss values according to the allocated weight values to obtain the loss function.

[0013] Optionally, the data unit is specifically configured to: segment the spinal image data into voxel data of a single vertebral body; extract the outer layer point cloud data of the vertebral body from the voxel data of the single vertebral body to be planned.

[0014] A second aspect of the embodiments of the present application provides a method for automatically planning a pedicle screw, comprising: acquiring spinal image data, and extracting outer layer point cloud data of a single vertebral body to be planned from the spinal image data, the spinal image data being three-dimensional image data; processing the outer layer point cloud data of the vertebral body by using a prediction model to output predicted coordinates, the predicted coordinates including entry point coordinates and exit point coordinates; planning a placement position of a pedicle screw based on the entry point coordinates and the exit point coordinates; wherein the prediction model comprises a plurality of key point branches, each of the key point branches being used to predict a probability that each point in the outer layer point cloud data of the vertebral body belongs to an entry point or an exit point corresponding to the key point branch, the probability being used as an attention weight to determine the entry point or the exit point when the predicted coordinates are outputted.

[0015] The third aspect of the embodiments of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the computer device implements the method of the second aspect.

[0016] The fourth aspect of the embodiments of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by a computer, the method of the second aspect is implemented.

[0017] The fifth aspect of the embodiments of the present application provides a computer program product, comprising a computer program, when the computer program is executed, the method of the second aspect is executed.

[0018] Compared with the prior art, the embodiments of the present application have the following beneficial effects: The embodiments of the present application are aimed at the problems of large data volume of spinal image data, long processing time and high requirements for processing equipment, by extracting the outer layer point cloud data of the single vertebra to be planned from the spinal image data, the outer layer point cloud data of the single vertebra can be used for coordinate prediction, thereby reducing the data volume to be processed in the planning process and improving the planning efficiency. In addition, by training a prediction model comprising a plurality of key point branches, the position of each key point required in the screw placement process can be predicted by each key point branch. In addition, in the prediction process, the present application introduces attention weights in the model to improve the accuracy of the model prediction, and the purpose of quickly and accurately predicting the pedicle screw planning position can be achieved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0020] Figure 1 is a schematic diagram of a pedicle screw automatic planning device provided by the embodiments of the present application; Figure 2 is a schematic diagram of a pedicle screw automatic planning method provided by the embodiments of the present application; Figure 3 is an architecture schematic diagram of a prediction model provided by the embodiments of the present application; Figure 4 is a schematic diagram of another pedicle screw automatic planning method provided by the embodiments of the present application; Figure 5 FIG. 7 is a schematic diagram of another pedicle screw automatic planning device provided by an embodiment of the present application; Figure 6 FIG. 8 is a schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0021] In the following description, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the present application. However, persons skilled in the art will understand that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0022] As mentioned above, in order to improve the efficiency of robot-assisted navigation surgery, more and more technologies are devoted to solving the problem of screw automatic planning to improve the efficiency of intraoperative planning. For example, by improving the related algorithm, taking the CT image data of the spine as the input of the algorithm, and using the output of the algorithm to plan the positions of two screws, i.e., the positions of two entry points and two exit points. The related algorithm includes a traditional algorithm or a deep learning algorithm.

[0023] In the traditional algorithm, the position of the pedicle can be determined according to the special landmark by detecting the geometric structure landmark of the cone, so as to obtain the planning position of the screw; or the planning position of the screw can be obtained by registering the image data with the anatomical atlas; in addition, the best cross section of each cone can be found, and the screw placement planning is performed by using the cross section. The traditional algorithm has good recognition effect in common spine cone morphology, but it is not general enough and is difficult to be applied to special cases such as scoliosis and physiological morphological lesions of the spine.

[0024] Deep learning algorithms can help improve the shortcomings of traditional algorithms. In the process of using deep learning algorithms to deal with the problem of screw placement planning of pedicle screws, there are two steps: first, the CT image data is semantically segmented to obtain the voxel data of each cone, and second, the voxel data obtained by semantic segmentation is used as input data to identify each cone by using a deep learning model. In related methods, a convolutional neural network model, such as a UNet model and its variants, is usually used to train the deep learning model, and the entry point and exit point of the screw planning position are predicted by the full connection layer regression prediction or the Gaussian heat map prediction method. The training process of the deep learning model uses mean-square error (MSE) as the loss function to update the model parameters in the reverse direction. However, the above-mentioned use of deep learning algorithms to plan the screw placement position of the pedicle screw has the following problems: 1) The voxel data of the spinal cone is large, and the device requirements are high, for example, a device with large memory is required to complete the processing of a large amount of voxel data. Moreover, it takes a long time to predict the screw entry and exit points of multiple cones.

[0025] 2) In order to reduce the prediction time, the deep learning model often uses a full connection layer regression prediction method, but this regression coordinate point calculation method lacks spatial generalization ability. On the contrary, the prediction result obtained by using the Gaussian heat map for prediction is more accurate, but the calculation based on the Gaussian heat map consumes a large amount of memory and time, which is not helpful to improve the efficiency of intraoperative planning.

[0026] 3) The training process of the deep learning model often uses MSE as the loss function to update the model parameters. From the perspective of model training, the MSE function only considers the exit and entry points of screw planning, lacks consideration of the screw angle and length in the cone, so that the deep learning model trained based on the loss function has low precision, and the screw position predicted by the model is greatly different from the actual situation, which is difficult to be directly applied in clinical application.

[0027] In order to solve the above problems and improve the efficiency of screw automatic planning, the applicant found that for vertebral body screw planning, the bone quality inside the cone has no effect on the position of screw planning, and the complete vertebral body voxel data is not needed in the actual planning process. The outermost voxel of the vertebral body can also complete the planning, thereby significantly reducing the amount of data to be processed. In addition, the applicant also improves the loss function used in the training process of the existing deep learning model, so that the deep learning model trained can be more consistent with clinical application. Therefore, the present application provides a pedicle screw automatic planning device, equipment and computer program product, which can quickly and accurately predict the planning position of the pedicle screw based on the processing of the outer layer point cloud data of the vertebral body by training a deep learning model.

[0028] The technical solutions of the present application will be described below through specific embodiments.

[0029] Referring to Figure 1 , a schematic diagram of a pedicle screw automatic planning device provided by an embodiment of the present application is shown, which can specifically include a data unit 101, a prediction model 102 and a planning unit 103; wherein: The data unit 101 is configured to obtain spinal image data, and extract the outer layer point cloud data of a single vertebral body to be planned from the spinal image data.

[0030] The prediction model 102 is configured to process the outer layer point cloud data of the vertebral body and output predicted coordinates, wherein the predicted coordinates include entry point coordinates and exit point coordinates.

[0031] The planning unit 103 is configured to plan the implantation position of the pedicle screw based on the entry point coordinate and the exit point coordinate.

[0032] In the embodiments of the present application, the pedicle screw automatic planning device can be a computer device or composed of one or more components in the computer device. The device can be used to process the spinal image data and assist the doctor to complete the pedicle screw implantation planning. The computer device can be a desktop computer, a cloud server, etc. The embodiments of the present application do not limit the form of the planning device and the type of the computer device.

[0033] The spinal image data can be three-dimensional image data, such as CT image data. Before the spinal surgery, the CT device can be used to scan the spinal region of the patient to obtain the CT image data. In some scenarios, the three-dimensional image data can also be magnetic resonance imaging (MRI) or other types of image data. The embodiments of the present application do not limit the type of the spinal image data and the way of collecting the spinal image data.

[0034] The pedicle screw automatic planning device in the embodiments of the present application can include a data unit 101, a prediction model 102 and a planning unit 103. The data unit 101 and the planning unit 103 can be components of the device and can exist in the form of software modules or hardware modules. The prediction model 102 can be a deep learning model. After the prediction model 102 is trained, it can be configured in the device for use in the automatic planning of the pedicle screw. Alternatively, the prediction model 102 can be configured in other devices or devices capable of communicating with the above-mentioned device, such as the cloud server. In this way, when the prediction model 102 is needed to process the point cloud data, the device can communicate with other devices or devices to call the prediction model 102. The embodiments of the present application do not limit the configuration form of the prediction model 102.

[0035] In the embodiments of the present application, the data unit 101 can be a unit for preliminary data processing of the spinal image data. Through the data unit 101, the spinal image data can be obtained, and the outer point cloud data of the single vertebra to be planned can be extracted from the spinal image data.

[0036] For example, the data unit 101 can be a unit in the device capable of communicating with the image device, such as the CT device. After the CT device is used to collect the CT image data of the patient's spine, the data unit 101 can obtain the CT data of the spine from the CT device.

[0037] From the perspective of pedicle screw planning, the internal bone of the vertebral body has no influence on the position of screw planning, and therefore, the complete voxel data is not required when planning the screw position. Based on the above reasons, the data unit 101 can further process the spinal image data to extract the outer layer point cloud data of the single vertebral body to be planned after the data unit 101 obtains the spinal image data. The single vertebral body to be planned can be the vertebral body that needs to be screwed in the spinal surgery. In a spinal surgery, the vertebral body that needs to be screwed can include one or more. Therefore, when the data unit extracts the outer layer point cloud data of the single vertebral body, the outer layer point cloud data of each vertebral body that needs to be screwed can be obtained by processing one or more vertebral bodies that need to be screwed respectively.

[0038] In the embodiment of the present application, the data unit 101 can manually or automatically segment the spinal image data into the voxel data of the single vertebral body. For example, the manual segmentation or automatic segmentation (such as semantic segmentation) can be used to segment the CT data of the whole spine of the patient into the voxel data of the single vertebral body. Then, the data unit 101 can extract the outer layer point cloud data of the vertebral body from the voxel data of the single vertebral body to be planned. The above process can be implemented by using the method of three-dimensional isosurface extraction, and the embodiment of the present application is not limited thereto.

[0039] By processing the three-dimensional spinal image data to extract the outer layer point cloud data of the single vertebral body, the complete vertebral body voxel data can not be used, the data amount of subsequent processing can be reduced, and the data processing efficiency can be improved. At the same time, since the data amount required for processing in the prediction stage is greatly reduced, the requirement for device resources is also greatly reduced, for example, the method can be implemented in a device with small memory, which greatly expands the application range of the method and can also reduce the time required for model prediction in the next stage.

[0040] In the embodiment of the present application, the prediction model 102 can process the outer layer point cloud data of the single vertebral body extracted by the data unit 101 to output the predicted coordinates of the screw placement position. It should be noted that when the number of vertebral bodies that need to be screwed in the spinal surgery is more than one, the prediction model 102 can process the outer layer point cloud data of each vertebral body respectively to output the predicted coordinates in each vertebral body.

[0041] In the embodiment of the present application, the prediction model 102 can include a plurality of key point branches, each key point branch can be used to predict the probability that each point in the outer layer point cloud data of the vertebral body belongs to the entry point or the exit point corresponding to the key point branch, and the probability can be used as an attention weight to determine the corresponding entry point coordinates or exit point coordinates when outputting the predicted coordinates.

[0042] Specifically, since each vertebral body that needs to be screwed needs to be planned two screws, each screw includes an entry point and an exit point. Therefore, planning the screw placement position of two screws also needs to predict the positions of two entry points and two exit points respectively. For example, the two entry points can include a first entry point and a second entry point, and the two exit points can include a first exit point and a second exit point. Among them, the first entry point and the first exit point can be taken as a group as the entry point and the exit point of one of the planned screws (for example, the first screw); the second entry point and the second exit point can be taken as another group as the entry point and the exit point of the other screw (for example, the second screw). In this way, the number of key point branches included in the prediction model 102 can be 4, and the 4 key point branches can be respectively used to predict the first entry point, the second entry point, the first exit point and the second exit point.

[0043] In addition, in the embodiments of the present application, in the process of predicting the entry point and the exit point of the screw by using the key point branch, an attention mechanism is also introduced, which can convert the probability output by each key point branch into an attention weight. The coordinates of each point output by the key point branch and the corresponding attention weight are combined to jointly calculate the planned positions of the two screws, that is, the coordinates of the entry point and the exit point of the first screw, and the coordinates of the entry point and the exit point of the second screw.

[0044] The planning unit 103 can complete the planning of the placement position of the pedicle screw based on the predicted coordinates output by the prediction model 102.

[0045] In a possible implementation manner of the embodiments of the present application, the prediction model 102 can be improved based on the PointNet model for processing point clouds. In this way, the characteristics of point cloud feature extraction and the advantage of fast calculation of the PointNet model can be utilized, the attention mechanism is added on this basis to regress and predict, and a loss function suitable for pedicle screw planning is designed, so as to realize the automatic planning of the placement position of the pedicle screw with better effect.

[0046] In the prior art, the UNet and other medical image processing models are mainly used to train the deep learning model for pedicle screw planning. The UNet model is mainly used to process voxel data, and the input of the model is voxel data. Although the UNet model has good local feature extraction capability and can retain spatial details through a skip connection, the use of the model has high calculation cost, the voxel data input needs more device memory for processing, and the overall calculation time is long. Unlike the UNet model, the PointNet model is specially used to process three-dimensional (3D) point cloud data. The input of the PointNet model is 3D point cloud data, the model can extract local features through hierarchical feature learning, has low calculation cost, only needs to occupy a small amount of device memory for processing point cloud data, and has short overall calculation time.

[0047] Therefore, the embodiment of the present application improves the PointNet model, trains a deep learning model, i.e., the aforementioned prediction model 102, and applies it to automatic planning of the pedicle screw placement position, which can significantly improve the planning efficiency.

[0048] Next, the specific processing process of the prediction model 102 in the above embodiment will be described in detail. Figure 1

[0049] In the embodiment of the present application, after receiving the outer-layer point cloud data of a single vertebral body, the prediction model 102 can first elevate the dimension of each point in the outer-layer point cloud data of the vertebral body, and then extract features for each elevated point. Then, the prediction model 102 can process the extracted features using multiple key point branches to predict the probability of each point belonging to the entry point or exit point corresponding to the key point branch. The features extracted from the outer-layer point cloud data of the vertebral body can include local and global point cloud features.

[0050] Specifically, for the prediction model 102, the input data is the outer-layer point cloud data of a single vertebral body. For example, the outer-layer point cloud data can be represented as: wherein, represents a set of outer-layer point cloud data of a vertebral body, N represents the size of the point cloud, i.e., the number of points included in the above data set represents the total number of points in the data set N The number 3 represents the size of the coordinate vector. The above input data can be received by the input layer of the prediction model 102. For the above input data, the data output by the prediction model 102 is the predicted coordinates of two screws, i.e., the first entry point coordinates and the first exit point coordinates of the first screw, and the second entry point coordinates and the second exit point coordinates of the second screw.

[0051] In the embodiment of the present application, the input layer of the prediction model 102 can be connected to a shared multilayer perceptron (MLP) layer (Shared MLP). Each point in the input data is transmitted to the shared MLP layer with shared weights, which is equivalent to passing each point through a separate feedforward neural network.

[0052] ​​​Exemplarily, the prediction model 102 can map and elevate the three-dimensional data to high-dimensional data by using the shared MLP layer to process the point cloud data and make the point cloud data pass through multiple linear layers (Linear) respectively, so as to map and elevate the three-dimensional data to high-dimensional data. For example, the three-dimensional point cloud is elevated to 256-dimensional data. Therefore, when the prediction model 102 performs feature extraction on each point after the elevation, the feature extraction can be performed on the high-dimensional data. For example, the point cloud local feature of each point is extracted from the 256-dimensional data. On this basis, the global feature is further extracted from the point cloud local feature of each point. The process of extracting the global feature from the point cloud local feature can be realized by the MaxPooling layer of the prediction model 102. The point cloud local feature and the global feature can be further fused and then transmitted to the key point branches for coordinate prediction.

[0053] In the embodiment of the present application, the global feature extracted from the point cloud local feature can have a different dimension from the point cloud local feature. Therefore, the prediction model 102 needs to expand the global feature to the same dimension as the point cloud local feature, and then fuse the global feature and the local feature to obtain the fused feature. The process of feature fusion can be realized by the Concat layer of the prediction model 102.

[0054] As a specific example in the embodiment of the present application, the shared MLP layer of the prediction model 102 can include linear layers such as (N, 64), (N, 128), (N, 256), and the like. Each outer layer point cloud in the input model can be mapped and elevated from three-dimensional data to 256-dimensional data after passing through the linear layers (N, 64), (N, 128), and (N, 256). The extraction of the point cloud local feature can be performed on the 256-dimensional data, that is, the point cloud local feature (N, 256) is extracted from the 256-dimensional data after the elevation. Then, through the pooling processing of the MaxPooling layer, the global feature (1, 256) can be extracted from the local feature (N, 256).

[0055] Since the global feature (1, 256) and the local feature (N, 256) have different dimensions, they cannot be directly fused, and the prediction model 102 needs to expand the global feature (1, 256) to the same dimension as the local feature (N, 256), that is, expand the local feature (1, 256) to (N, 256), and then fuse the local feature (N, 256) to obtain the fused feature (N, 512).

[0056] The prediction model 102 can process the fused feature (N, 512) by using multiple key point branches to obtain the feature value of each point under each key point branch.

[0057] In the embodiments of the present application, by converting the feature value of each point under each key point branch into a probability distribution, the probability of each point belonging to the corresponding entry point or exit point of the key point branch can be obtained.

[0058] For example, the number of key point branches is 4, and the four key point branches can be represented as Screw-i (i = 1, 2, 3, 4), which are respectively used to predict the first entry point, the first exit point, the second entry point and the second exit point. For example, the key point branch Screw-1 can be used to predict the first entry point, the key point branch Screw-2 can be used to predict the first exit point, the key point branch Screw-3 can be used to predict the second entry point, and the key point branch Screw-4 can be used to predict the second exit point.

[0059] In this way, after the fusion feature (N, 512) is sent into each key point branch, the corresponding feature value can be output by the key point branch. By converting the feature value into a probability distribution, the probability of each point belonging to the corresponding entry point or exit point of each key point branch can be obtained.

[0060] For example, after the fusion feature (N, 512) is sent into the key point branch Screw-1, the feature value under the key point branch Screw-1 can be obtained. Since the above-mentioned key point branch Screw-1 is used to predict the first entry point, after the feature value is converted into a probability distribution, the probability of each point belonging to the first entry point corresponding to the key point branch Screw-1 can be obtained. Correspondingly, according to the feature values under other key point branches, such as the feature values under the key point branches Screw-2, Screw-3 and Screw-4, and after the probability distribution conversion, the probability of each point belonging to the first exit point, the second entry point and the second exit point can be obtained respectively. The above-mentioned probability can be used as the attention weight of the corresponding key point branch in subsequent calculation.

[0061] In the embodiments of the present application, for any key point branch, the weighted sum of each point in the outer layer point cloud data of the vertebral body can be performed to obtain the key point coordinate value of the corresponding key point type. The above-mentioned key point coordinate value is the first entry point coordinate, the second entry point coordinate, the first exit point coordinate or the second exit point coordinate. The weight in the weighted sum is the attention weight calculated as described above, and the above-mentioned key point type can be used to represent the type of key point predicted by the corresponding key point branch. For example, a certain key point branch is used to predict the first entry point coordinate, and the key point type of the branch can be the first entry point, and the corresponding calculated key point coordinate value is the first entry point coordinate.

[0062] In a possible implementation of the embodiment of the application, in order to improve the accuracy of the prediction result of the deep learning model, the embodiment of the application further provides a loss function (Loss Function) that is more targeted and more suitable for clinical application, which is used to update the model parameters of the model in the model training process, so that the model training can converge faster, and the deep learning model obtained by training can more accurately predict the entry point and exit point coordinates of the screw to be implanted.

[0063] In the embodiment of the application, the loss function can be obtained by optimizing the mean square error function MSE.

[0064] For example, the mean square error function MSE can be represented as:

[0065] wherein, represents a predicted point, represents a point of a theoretical value.

[0066] Generally, the mean square error function MSE is mainly used to evaluate the distance between points, and therefore the deep learning model using the mean square error function MSE as the loss function only considers the distance between the entry point and the exit point. That is, the model trained by using the above-mentioned mean square error function MSE as the loss function is aimed at the distance between the actual entry point / exit point and the predicted entry point / exit point during training, and does not directly consider factors such as the length of the screw. The screw formed by the entry point and the exit point predicted by the model may be quite different from the screw required in actual application. In view of this problem, the embodiment of the application can combine other factors in the screw implanting process to comprehensively construct the loss function by optimizing the mean square error function MSE, so as to obtain a more accurate deep learning model.

[0067] For example, in the screw implanting planning of the pedicle screw, not only the four key points need to be considered, but also the relative angle of the two screws of the cone is required in actual clinical application, and the included angle between the two screws and the central axis cannot be too large; the length of the screw is also required, and the two screws cannot be too long or too short; in addition, the geometric characteristics (such as symmetry and rationality of length) of the two screws of the same cone should also meet certain requirements. The embodiment of the application optimizes the mean square error function MSE, considers not only the distance between points but also the above-mentioned angle, length and geometric consistency in the construction of the loss function and the model training, and provides a loss function that is more suitable for clinical application.

[0068] Specifically, the point loss value in the model training process can be determined based on the mean square error function first, and other loss values can be determined according to the pedicle screw placement requirements. The two screws can include a first screw represented by the first entry point coordinates and the first exit point coordinates and a second screw represented by the second entry point coordinates and the second exit point coordinates. Correspondingly, the other loss values determined according to the pedicle screw placement requirements can include one or more of the angle loss value, the length loss value, and the geometric consistency loss value. In this way, the loss function of the prediction model can be constructed based on the point loss value and the other loss values, so as to obtain a more accurate deep learning model that meets the pedicle screw placement requirements.

[0069] In the embodiments of the present application, the point loss value required for constructing the loss function is the same as the mean square error function MSE, and the point loss value may be represented as:

[0070] The angle loss value may be represented as:

[0071] wherein, , represents the angle of the predicted screw 1, 2 and the central axis, , represents the angle of the real screw 1, 2 and the central axis. The screws 1, 2 can be the first screw and the second screw in the foregoing embodiments.

[0072] The length loss value may be represented as:

[0073] wherein, , represents the length of the predicted screw 1, 2, , represents the length of the real screw 1, 2. The screws 1, 2 can be the first screw and the second screw in the foregoing embodiments.

[0074] The geometric consistency loss value may be represented as:

[0075] By using the geometric consistency loss value , the prediction result of the model can be constrained to meet the geometric constraint condition of the real screw planning, so that when the angle and the length of the prediction target are predicted, in addition to making the numerical value close, the rationality of meeting the geometric relationship can also be ensured.

[0076] In this way, the loss function constructed based on the point loss value and one or more of the angle loss value, the length loss value and the geometric consistency loss value can better meet the clinical needs of the pedicle screw placement process.

[0077] Specifically, the point loss value and the other loss values can be assigned weight values respectively, and then the corresponding point loss value and the other loss values can be weighted according to the assigned weight values to construct the loss function. In this way, the weight values can be assigned to each loss value according to the influence of the loss value on the screw planning. The weight values can be adjusted according to the surgical habits of different doctors. For example, if the doctor has a higher requirement for the angle of the screw planning, the weight value of the corresponding angle loss value can be increased, so that the model will pay more attention to the angle of the screw planning during training and prediction.

[0078] Exemplarily, the loss function can be constructed based on the point loss value, the angle loss value, the length loss value and the geometric consistency loss value, and the loss function can be represented as:

[0079] wherein the weight values respectively represent the weight values of the point loss value, the length loss value, the angle loss value and the geometric consistency loss value.

[0080] The embodiments of the present application can increase various indicators for evaluating the screw planning, and can construct a loss function based on multiple loss values set according to the various indicators, so that the model trained can better meet the clinical application when predicting the screw entry point and exit point. In addition, the weight of each loss value can also be adjusted according to the surgical habits of different doctors to meet the individual planning needs of doctors.

[0081] In order to facilitate understanding, the automatic pedicle screw planning provided by the embodiments of the present application will be introduced below in combination with a complete example.

[0082] Referring to Figure 2 , a schematic diagram of a method for automatic pedicle screw planning provided by the embodiments of the present application is shown, and the overall process of the method can be summarized as three parts, namely: data preparation, feature extraction and attention mechanism prediction. Next, in combination with Figure 2 , a detailed introduction will be made.

[0083] 1. In the data preparation stage: In the robot-assisted navigation surgery, the overall spinal image data can be segmented into single vertebral voxel data based on the patient's spinal image data through manual or automatic segmentation. For example, as shown in Figure 2As shown, taking a patient's CT image data as an example, semantic segmentation can divide the overall spinal CT image data into voxel data of individual vertebrae, and then point cloud processing is performed on the voxel data of individual vertebrae. In the point cloud processing step, the voxel data of individual vertebrae can be transformed into point cloud data of the outer layer of the vertebrae using the three-dimensional isosurface extraction method.

[0084] 2. In the feature extraction stage: Based on the obtained extravertebral point cloud data of a single vertebral body, the automatic planning steps for pedicle screw placement can be further executed. This embodiment of the application utilizes a deep learning model to process the extravertebral point cloud data of a single vertebral body and output the corresponding predicted coordinates.

[0085] like Figure 2 As shown, for the obtained outer point cloud data of a single vertebra, a deep learning model is invoked and model parameters are loaded. The deep learning model can be the prediction model in the aforementioned embodiments. First, the outer point cloud data of a single vertebra is input into the model for feature extraction. In this embodiment, a multilayer perceptron (MLP) can be used to extract local and global features of the point cloud and fuse them. The fused features are then input into keypoint branches, namely the four keypoint branches described in the aforementioned embodiments: two in-point keypoint branches and two out-point keypoint branches.

[0086] like Figure 3 The diagram shown is a schematic representation of the architecture of a prediction model provided in an embodiment of this application. The following is a detailed explanation... Figure 3 The architecture of the prediction model shown illustrates how to construct the model and how to use it for feature extraction.

[0087] 2.1 Model Basics This application's embodiments are based on the PointNet model for point cloud processing and improved upon it. By leveraging the point cloud feature extraction characteristics and fast computation speed of the PointNet model, attention mechanism regression prediction is added, and a loss function suitable for pedicle screw planning is designed, which can achieve automatic planning with better results.

[0088] Unlike existing technologies that commonly use medical image models such as UNet to process voxel data, PointNet is specifically designed for processing 3D point cloud data.

[0089] UNet is a commonly used medical image model that takes voxel data as input. This model has good local feature extraction capabilities and can preserve spatial details through skip connections. However, applying the UNet model has high computational costs, as the input voxel data requires more device memory for processing and the computation time is relatively long.

[0090] PointNet is specially used for processing 3D point cloud data, the input of which is 3D point cloud data, and local features can be extracted through hierarchical feature learning, the cost of calculation is low, the processing of point cloud data only needs to occupy a small amount of device memory, and the calculation time is shorter.

[0091] 2.2 Model design The input data of the model can be represented as: ; The output data of the model is: two pedicle screw planning positions, i.e. 2 entry point coordinates and 2 exit point coordinates.

[0092] Referring to Figure 3 , in the input layer of the model (Input Layer), the data received is the outer point cloud data of a single vertebral body, and the data processing process of the input layer can be represented as Input Layer(N, 3).

[0093] In the shared MLP layer (Shared MLP), all points share weights, which is equivalent to passing each point through a separate feedforward neural network. Specifically, the input 3D point cloud can be mapped to 256 dimensions through Linear (N, 64), Linear (N, 128), Linear (N, 256) and other linear layers, with the activation function being Relu, for extracting point cloud local features (N, 256).

[0094] In the max pooling layer (Max Pooling), global features (1, 256) can be extracted from point cloud local features (N, 256).

[0095] In the feature fusion layer (Concat), the global features (1, 256) can be first expanded to (N, 256), and then fused with the local features (N, 256) to obtain the fusion features (N, 256).

[0096] Figure 3 Screw-i in the formula represents 4 key points, which are 2 entry points and 2 exit points; MLP-i represents an MLP branch of a certain key point, which is used to compress feature dimensions and improve calculation efficiency; Softmax-i represents converting the feature value of a certain MLP branch into a probability distribution to obtain attention weights; Weighted Sum-i represents weighted summation with each point based on the attention weights of a certain MLP branch to obtain the predicted coordinate value of a certain key point:

[0097] wherein, represents the coordinates of the predicted point, represents the attention weights calculated by Softmax, the n-th point of the point cloud.

[0098] 2.3 Loss function design A loss function is used to update the model parameters of a model in the model training process. According to the characteristics of the pedicle screw planning, the embodiments of the present application optimize the loss function based on the mean square error function MSE, wherein the MSE loss function can be represented as:

[0099] wherein, represents a predicted point, represents a point of a theoretical value.

[0100] The mean square error function MSE is suitable for evaluating individual key points, but screw planning is not only 4 key points, and in actual clinical practice, the relative angle of the two screws of the cone is also required, and the included angle of the two screws and the central axis cannot be too large; the length of the screw is also required, which cannot be too long or too short; the geometric characteristics (symmetry, rationality of length) of the two screws of the same cone are also required. Based on this, the embodiments of the present application can combine the point loss value, the angle loss value, the length loss value and the geometric consistency loss value to construct the loss function.

[0101] (a) Point loss value which can be represented as (consistent with the MSE calculation):

[0102] (b) Angle loss value which can be represented as:

[0103] wherein, , represents the angle of the predicted screw 1, 2 and the central axis, , represents the angle of the true screw 1, 2 and the central axis. The above-mentioned screw 1, 2 can be the first screw and the second screw in the foregoing various embodiments.

[0104] (c) Length loss value which can be represented as:

[0105] wherein, , represents the length of the predicted screw 1, 2, , represents the length of the true screw 1, 2. The above-mentioned screw 1, 2 can be the first screw and the second screw in the foregoing various embodiments.

[0106] (d) geometric consistency loss value which can be expressed as:

[0107] The prediction result of the constraint model is consistent with the geometric constraint condition of the real screw planning. In predicting the angle and length of the target, in addition to numerical proximity, the rationality of the geometric relationship should also be met.

[0108] (e) The loss function is constructed as follows:

[0109] According to the influence of each loss value on the screw planning, a weight value is respectively given , respectively represent the weight values of the point loss value, the length loss value, the angle loss value and the geometric consistency loss value. The weight values can be adjusted according to the operation habits of different doctors. For example, if the doctor has high requirements for the angle of the screw planning, the corresponding weight value can be increased, and the model will pay more attention to the angle of the screw planning during training and prediction.

[0110] 3. In the attention mechanism prediction stage: Referring back to Figure 2 , in the attention mechanism prediction stage, the screw position can be predicted by using the deep learning model introduced above and the geometric attention mechanism. This process can include steps such as normalizing to obtain attention weights by using a Softmax function, and obtaining the coordinates of each key point by weighted summation.

[0111] Based on the introduction of the foregoing embodiments, referring to Figure 4 , a schematic diagram of another automatic pedicle screw planning method provided by the embodiments of the present application is shown, which can specifically include the following steps: S401, acquiring spine image data, extracting the outer point cloud data of a single vertebra to be planned from the spine image data, and the spine image data is three-dimensional image data.

[0112] S402, processing the outer point cloud data of the vertebra by using a prediction model, and outputting predicted coordinates, the predicted coordinates including entry point coordinates and exit point coordinates.

[0113] S403, planning the implantation position of the pedicle screw based on the entry point coordinates and the exit point coordinates.

[0114] The prediction model can be the prediction model or the deep learning model introduced in the foregoing embodiments, and the prediction model includes a plurality of key point branches, each of which is used to predict a probability that each point in the outer layer point cloud data of the vertebral body belongs to an entry point or an exit point corresponding to the key point branch, and the probability is used as an attention weight to determine the entry point or the exit point when the predicted coordinates are output.

[0115] Figure 4 The execution subject of the pedicle screw automatic planning method shown can be a computer device, which performs the following steps by executing a computer program. Figure 4 The steps shown can be used to extract the outer layer point cloud data of a single vertebral body to be planned from the spine image data after the spine image data is acquired, so as to reduce the amount of point cloud data to be processed subsequently. For the outer layer point cloud data of a single vertebral body, the computer device can call a prediction model, use the outer layer point cloud data of the vertebral body as input data of the prediction model, and output predicted coordinates, i.e., entry point coordinates and exit point coordinates of two screws, by using the prediction model. The computer device can complete the planning of the screw placement positions of the two screws based on the entry point coordinates and the exit point coordinates output by the model.

[0116] The construction process of the prediction model in the embodiments of the present application and how to use the prediction model to predict key point coordinates can be referred to the descriptions in the foregoing embodiments, and will not be described here.

[0117] It should be noted that the size of the serial number of each step in the foregoing embodiments does not mean the execution sequence, and the execution sequence of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0118] Referring to Figure 5 , another schematic diagram of a pedicle screw automatic planning device provided by the embodiments of the present application is shown, which can specifically include an acquisition module 501, a prediction module 502, and a planning module 503, wherein: The acquisition module 501 is configured to acquire spine image data, and extract outer layer point cloud data of a single vertebral body to be planned from the spine image data, wherein the spine image data is three-dimensional image data.

[0119] The prediction module 502 is configured to use a prediction model to process the outer layer point cloud data of the vertebral body, and output predicted coordinates, wherein the predicted coordinates include entry point coordinates and exit point coordinates.

[0120] The planning module 503 is configured to plan the placement positions of pedicle screws based on the entry point coordinates and the exit point coordinates.

[0121] The prediction model can be the prediction model or the deep learning model described in the foregoing embodiments, and the prediction model includes a plurality of key point branches, each of which is used to predict a probability that each point in the outer-layer point cloud data of the vertebral body belongs to an entry point or an exit point corresponding to the key point branch, and the probability is used as an attention weight to determine the entry point or the exit point when the predicted coordinates are output.

[0122] The functions of the various modules of the pedicle screw automatic planning device provided in the embodiments of the present application can be achieved by the functions of the various modules of the device provided in the embodiments of the present application. Figure 1 The device in the various embodiments introduced above is similar and can be mutually referred to, and will not be described again here.

[0123] Referring to Figure 6 , a schematic diagram of a computer device provided in an embodiment of the present application is shown. As shown in Figure 6 , the computer device 600 in the embodiment of the present application includes a processor 610, a memory 620, and a computer program 621 stored in the memory 620 and executable on the processor 610. The processor 610 implements the steps in the various embodiments of the pedicle screw automatic planning method when executing the computer program 621, for example, the steps S401 to S403 shown in Figure 4 . Alternatively, the processor 610 implements the functions of the modules / units in the various device embodiments when executing the computer program 621, for example, the functions of the units 101 to 103 shown in Figure 1 , or the functions of the modules 501 to 503 shown in Figure 5 .

[0124] For example, the computer program 621 can be divided into one or more modules / units, which are stored in the memory 620 and executed by the processor 610 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which can be used to describe the execution process of the computer program 621 in the computer device 600. For example, the computer program 621 can be divided into an acquisition module, a prediction module, and a planning module, and the specific functions of the modules are as follows: The acquisition module is configured to acquire spinal image data, extract outer-layer point cloud data of a single vertebral body to be planned from the spinal image data, and the spinal image data is three-dimensional image data.

[0125] The prediction module is configured to process the outer-layer point cloud data of the vertebral body by using a prediction model, and output predicted coordinates, the predicted coordinates including entry point coordinates and exit point coordinates.

[0126] A planning module is configured to plan an implantation position of the pedicle screw based on the entry point coordinate and the exit point coordinate.

[0127] The computer device 600 can be a device capable of implementing each step in the foregoing various method embodiments, or a device capable of implementing the functions of each unit or model in the foregoing various device embodiments. The computer device 600 can be a desktop computer, a cloud server, or the like. The computer device 600 can include, but is not limited to, a processor 610 and a memory 620. Those skilled in the art can understand that the computer device 600 can include more or fewer components, or combine certain components, or include different components, for example, the computer device 600 can also include an input / output device, a network access device, a bus, and the like. Figure 6 The computer device 600 is only an example and does not constitute a limitation on the computer device 600, and can include more or fewer components than shown, or combine certain components, or include different components, for example, the computer device 600 can also include an input / output device, a network access device, a bus, and the like.

[0128] The processor 610 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0129] The memory 620 can be an internal storage unit of the computer device 600, such as a hard disk or a memory of the computer device 600. The memory 620 can also be an external storage device of the computer device 600, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like. Further, the memory 620 can include both an internal storage unit and an external storage device of the computer device 600. The memory 620 is used to store the computer program 621 and other programs and data required by the computer device 600. The memory 620 can also be used to temporarily store data that has been output or will be output.

[0130] The application further discloses a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the computer program implements the pedicle screw automatic planning method in the foregoing various embodiments or the pedicle screw automatic planning method implemented by the foregoing various device embodiments.

[0131] The application further discloses a computer readable storage medium, which stores a computer program, and when the computer program is executed by a computer, the computer program implements the pedicle screw automatic planning method in the foregoing various embodiments or the pedicle screw automatic planning method implemented by the foregoing various device embodiments.

[0132] The application further discloses a computer program product, which comprises a computer program, and when the computer program is executed on a computer, the computer program causes the computer to execute the pedicle screw automatic planning method in the foregoing various embodiments or the pedicle screw automatic planning method implemented by the foregoing various device embodiments.

[0133] The foregoing embodiments are merely used to illustrate the technical solutions of the application, rather than limit the application. Although the application is described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced equivalently, and the modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in the protection scope of the application.

Claims

1. An automatic pedicle screw planning device, characterized in that, include: The data unit is used to acquire spinal imaging data and extract the outer layer point cloud data of the single vertebra to be planned from the spinal imaging data. The spinal imaging data is three-dimensional image data. A prediction model is used to process the extracorporeal point cloud data of the vertebral body and output predicted coordinates, which include in-point coordinates and out-point coordinates. The planning unit is used to plan the placement position of the pedicle screw based on the inlet coordinates and the outlet coordinates. The prediction model includes multiple key point branches, each of which is used to predict the probability that each point in the extravertebral point cloud data belongs to the in-point or out-point corresponding to the key point branch. The probability is used as an attention weight to determine the in-point or out-point when outputting the predicted coordinates.

2. The apparatus according to claim 1, characterized in that, The prediction model is specifically used for: When the extravertebral body point cloud data is received, the dimension of each point in the extravertebral body point cloud data is increased, and feature extraction is performed on each point after the dimension increase. The extracted features are processed using multiple keypoint branches to predict the probability that each point belongs to the in-point or out-point corresponding to the keypoint branch.

3. The apparatus according to claim 1 or 2, characterized in that, The prediction model is further used for: Each point in the extravertebral outer layer point cloud data is dimensionally increased, the local features of the point cloud of each point after dimensionality increase are extracted, and the global features are extracted from the local features of the point cloud of each point. After expanding the global features to the same dimension as the local features of the point cloud, the global features and the local features of the point cloud are fused to obtain the fused features.

4. The apparatus according to claim 3, characterized in that, The prediction model is further used for: The fused features are processed using multiple key point branches to obtain the feature values ​​of each point under each key point branch; The feature values ​​of each point under each of the key point branches are transformed into a probability distribution to obtain the probability that each point belongs to the entry point or exit point corresponding to the key point branch.

5. The apparatus according to any one of claims 1, 2, or 4, characterized in that, The number of keypoint branches is four. The in-point coordinates include first in-point coordinates and second in-point coordinates. The out-point coordinates include first out-point coordinates and second out-point coordinates. The four keypoint branches are respectively used to predict the first in-point coordinates, the second in-point coordinates, the first out-point coordinates, or the second out-point coordinates. The prediction model is further used for: Determine the key point type corresponding to each of the key point branches; For any of the key point branches, a weighted sum is performed on each point in the extravertebral point cloud data to obtain the key point coordinate values ​​corresponding to the key point type. The key point coordinate values ​​are the first in-point coordinates, the second in-point coordinates, the first out-point coordinates, or the second out-point coordinates. The weights in the weighted summation are the attention weights.

6. The apparatus according to claim 5, characterized in that, The loss function of the prediction model is obtained by optimizing the mean squared error function. This loss function is used to update the model parameters of the prediction model during backpropagation during model training. The optimization of the mean squared error function includes: The point loss value during model training is determined based on the mean square error function. Other loss values ​​are determined based on the placement requirements of the pedicle screws. The placement requirements of the pedicle screws include the angle requirements, length requirements, and geometric characteristic requirements of the two screws. The two screws include a first screw characterized by the first entry point coordinates and the first exit point coordinates, and a second screw characterized by the second entry point coordinates and the second exit point coordinates. Accordingly, the other loss values ​​include one or more of the following: angle loss value, length loss value, and geometric consistency loss value. The loss function of the prediction model is constructed based on the point loss value and the other loss values.

7. The apparatus according to claim 6, characterized in that, The loss function for constructing the prediction model based on the point loss value and the other loss values ​​includes: Assign weight values ​​to the point loss value and the other loss values ​​respectively; The loss function is constructed by weighting the corresponding point loss value and the other loss values ​​according to the assigned weight values.

8. The apparatus according to any one of claims 1, 2, 4, 6, or 7, characterized in that, The data unit is specifically used for: The spinal imaging data is segmented into voxel data of individual vertebrae; Extract the outer layer point cloud data of the vertebral body from the voxel data of the individual vertebra to be planned.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the computer device implements the following method implemented by the pedicle screw automatic planning device as described in any one of claims 1 to 8: Acquire spinal imaging data, and extract the outer layer point cloud data of the individual vertebra to be planned from the spinal imaging data. The spinal imaging data is three-dimensional image data. The extracorporeal point cloud data of the vertebral body is processed using a prediction model to output predicted coordinates, which include in-point coordinates and out-point coordinates. Based on the inlet coordinates and outlet coordinates, the placement position of the pedicle screw is planned; The prediction model includes multiple key point branches, each of which is used to predict the probability that each point in the extravertebral point cloud data belongs to the in-point or out-point corresponding to the key point branch. The probability is used as an attention weight to determine the in-point or out-point when outputting the predicted coordinates.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed, the following method implemented by the pedicle screw automatic planning device as described in any one of claims 1 to 8 is performed: Acquire spinal imaging data, and extract the outer layer point cloud data of the individual vertebra to be planned from the spinal imaging data. The spinal imaging data is three-dimensional image data. The extracorporeal point cloud data of the vertebral body is processed using a prediction model to output predicted coordinates, which include in-point coordinates and out-point coordinates. Based on the inlet coordinates and outlet coordinates, the placement position of the pedicle screw is planned; The prediction model includes multiple key point branches, each of which is used to predict the probability that each point in the extravertebral point cloud data belongs to the in-point or out-point corresponding to the key point branch. The probability is used as an attention weight to determine the in-point or out-point when outputting the predicted coordinates.

Images