Automatic spine screw placement method and device based on aggregation matching

Abstract

The invention provides an automatic spine screw placement method and device based on aggregation matching. The method comprises the steps that a spine historical image set is acquired; carrying out aggregation processing on the spine historical image set to obtain an aggregation model of a vertebral body; acquiring a spine medical image; inputting the spine medical image into a spine screw placement model to obtain a spine screw placement result; the spine screw placement model is obtained by training based on an aggregation model and a spine historical image set. According to the method, aggregation is carried out through the historical spine images, auxiliary screw placement is carried out through the aggregation model, dependence on the operative feeling of a doctor is reduced, screw placement parameters are determined only through the three-dimensional image, and secondary damage is avoided while precision is guaranteed.

Description

Technical Field

[0001] The present application relates to the field of medical image recognition technology, and in particular to a method and device for automatic spinal screw placement based on aggregate matching. Background Art

[0002] Spinal disorders refer to a variety of diseases and conditions that affect the structure and function of the spine. They can lead to a wide range of health problems and physical impairments, including motor dysfunction, neurological impairment, nerve root compression, spinal cord damage, organ compression, impaired cardiopulmonary function, and mental health issues. Screwing during spinal surgery is a common treatment for spinal disorders. It is primarily used for spinal deformity correction, spinal stabilization, and spinal fusion procedures.

[0003] However, current spinal screw placement mainly relies on the doctor's surgical intuition and the exploration of the screw channel with a pedicle probe to determine the screw placement parameters. This requires a high level of clinical experience from the doctor, makes it difficult to ensure accuracy, and is prone to causing secondary injuries. Summary of the Invention

[0004] The problem to be solved by this application is that the current method for determining the screw placement parameters is too dependent on the doctor's intuition.

[0005] To solve the above problems, the present application provides a first aspect of a method for automatic spinal screw placement based on aggregate matching, comprising:

[0006] Obtain a collection of historical spine images;

[0007] Aggregate the historical image collection of the spine to obtain an aggregated model of the vertebral body;

[0008] Acquire medical images of the spine;

[0009] The spinal medical image is input into a spinal nail placement model to obtain a spinal nail placement result; the spinal nail placement model is trained based on an aggregation model and a spinal historical image collection.

[0010] The second aspect of the present application provides a spinal automatic nail placement device based on polymer matching, which includes:

[0011] A collection acquisition module, which is used to acquire a collection of historical spinal images;

[0012] An aggregation processing module, which is used to perform aggregation processing on the historical image set of the spine to obtain an aggregated model of the vertebral body;

[0013] An image acquisition module, which is used to acquire spinal medical images;

[0014] The spinal nail placement module is used to input spinal medical images into a spinal nail placement model to obtain spinal nail placement results; the spinal nail placement model is trained based on an aggregation model and a set of spinal historical images.

[0015] A third aspect of the present application provides an electronic device, comprising: a memory and a processor;

[0016] The memory is used to store programs;

[0017] The processor, coupled to the memory, is configured to execute the program to:

[0018] Obtain a collection of historical spine images;

[0019] Aggregate the historical image collection of the spine to obtain an aggregated model of the vertebral body;

[0020] Acquire medical images of the spine;

[0021] The spinal medical image is input into a spinal nail placement model to obtain a spinal nail placement result; the spinal nail placement model is trained based on an aggregation model and a spinal historical image collection.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the above-mentioned method for automatic spinal screw placement based on aggregate matching.

[0023] In this application, historical spinal images are aggregated and the aggregated model is used to assist in nail placement, which not only reduces the reliance on the doctor's technical sense, but also determines the nail placement parameters only through three-dimensional images, thus ensuring accuracy while avoiding secondary injuries. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Flowchart of a method for automatic spinal screw placement based on aggregate matching according to an embodiment of the present application;

[0025] Figure 2 1. This is an architecture diagram of a spinal nail placement model according to an automatic spinal nail placement method based on aggregate matching according to an embodiment of the present application;

[0026] Figure 3 Schematic diagram of a classification structure of an automatic spinal screw placement method based on aggregate matching according to an embodiment of the present application;

[0027] Figure 4 1 is a structural block diagram of an automatic spinal nail placement device based on aggregate matching according to an embodiment of the present application;

[0028] Figure 5 2 is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] To make the above-mentioned objects, features, and advantages of the present application more clearly understood, specific embodiments of the present application are described in detail below with reference to the accompanying drawings. Although the accompanying drawings show exemplary embodiments of the present application, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0030] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art to which this application belongs.

[0031] To address the above issues, the present application provides a new automatic spinal nail placement solution based on aggregated matching, which can aggregate historical spinal images and assist in nail placement through an aggregated model, eliminating the problem that the current method for determining nail placement parameters is overly dependent on the doctor's technical sense.

[0032] The embodiment of the present application provides a method for automatically placing spinal nails based on aggregate matching. The specific scheme of the method is as follows: Figure 1-Figure 3 As shown, the method can be performed by an automatic spinal nail placement device based on aggregate matching, and the automatic spinal nail placement device based on aggregate matching can be integrated into electronic devices such as computers, servers, computers, server clusters, and data centers. Figure 1 , which is a flow chart of a method for automatic spinal nail placement based on aggregate matching according to one embodiment of the present application; wherein the method for automatic spinal nail placement based on aggregate matching includes:

[0033] S101, obtaining a collection of historical spine images;

[0034] In this application, the historical image of the spine is a CT / MRI image.

[0035] In this application, the spinal historical image collection includes medical images of subjects of different ages, different genders, different spinal abnormalities, and different spinal lesions, thereby improving the universality of vertebral aggregation by covering all possible vertebral morphologies.

[0036] S102, performing aggregation processing on the historical image set of the spine to obtain an aggregated model of the vertebral body;

[0037] In this application, the aggregation process is performed on each vertebra (L1-S1) of the spine, and multiple aggregated models of each vertebra are obtained through the aggregation process. The aggregated model is the standard model of the vertebra.

[0038] In the present application, an aggregated model of the vertebral body is obtained, and screw placement planning is performed on the aggregated model manually, thereby obtaining optimal screw placement planning information.

[0039] S103, acquiring a spinal medical image;

[0040] In this application, similarly, the spinal medical image is a CT / MRI image.

[0041] S104 , inputting the spinal medical image into a spinal nail placement model to obtain a spinal nail placement result; the spinal nail placement model is trained based on an aggregation model and a spinal historical image set.

[0042] In this application, historical spinal images are aggregated and the aggregated model is used to assist in nail placement, which not only reduces the reliance on the doctor's technical sense, but also determines the nail placement parameters only through three-dimensional images, thus ensuring accuracy while avoiding secondary injuries.

[0043] In this application, the aggregated model of the vertebral body is annotated with screw placement planning information, which may be manually annotated, so as to ensure that the annotated screw placement plan is the optimal plan.

[0044] In this application, by pre-planning the nail placement plan of the aggregate model and using the nail placement plan as a label of the aggregate model, the nail placement plan can be synchronously mapped when the aggregate model is mapped to the three-dimensional model of the spine to be nailed.

[0045] In this application, the screw placement planning information may include cortical screw planning and pedicle screw planning, thereby increasing the coverage and adaptability of the entire spinal column automatic screw placement.

[0046] In one embodiment, the step S102 of performing aggregation processing on the historical spinal image set to obtain an aggregated model of the vertebral body includes:

[0047] Segment the historical images of the spine to obtain the three-dimensional model data of the preset vertebrae;

[0048] Perform principal component analysis on the three-dimensional model data to obtain the corresponding dimensionality reduction data;

[0049] Perform cluster analysis on the dimensionality-reduced data to obtain multiple data clusters;

[0050] A secondary principal component analysis is performed in each data cluster to obtain the aggregation model of each data cluster.

[0051] In this application, segmentation processing is performed on historical spinal images to obtain 3D model data for each vertebra of the spine. Depending on the vertebra to be aggregated, 3D model data for different vertebrae can be selected. For example, if thoracic vertebra L1 is aggregated, after segmentation processing is performed on the historical spinal images, only the 3D model data for thoracic vertebra L1 corresponding to each historical spinal image is retained.

[0052] Preferably, after obtaining the three-dimensional model data of the preset vertebral body, data preprocessing is required:

[0053] Align the 3D model data (e.g., using Procrustes analysis) to eliminate the effects of rigid transformations (translation, rotation, scaling). Represent each model as a uniform shape vector (e.g., vertex coordinates) to obtain a shape matrix. Also, normalize the shape vectors: Center the shape matrix (subtract the mean shape vector). Normalize the shape vectors to ensure that features of different dimensions have equal weight.

[0054] In this application, principal component analysis is performed on the three-dimensional model data to obtain the corresponding dimensionality reduction data, including:

[0055] Compute the principal component analysis (PCA): Perform a PCA on the shape matrix to extract the principal component directions; retain the top k principal components (typically, select the number of principal components that achieve a cumulative variance explanation of at least 95%). This step projects the high-dimensional shape vectors into a lower-dimensional space. Obtain a low-dimensional embedding: Project each shape vector into the low-dimensional space of the PCA, obtaining a low-dimensional representation of each sample, i.e., reducing the dimensionality of the data.

[0056] In this application, cluster analysis is performed on the dimensionality-reduced data to obtain multiple data clusters, including:

[0057] A clustering algorithm (such as the K-Means algorithm) is run in the low-dimensional space after PCA dimensionality reduction to divide the reduced-dimensional data in the set into N clusters; the center point of each cluster can be regarded as the representative shape of the cluster.

[0058] Preferably, it is checked whether the samples within each cluster have a high similarity while the similarity between samples in different clusters is low, or the silhouette coefficient indicator is used to evaluate the clustering quality.

[0059] In this application, a secondary principal component analysis is performed in each data cluster to obtain an aggregation model for each data cluster, including:

[0060] For the samples in each cluster, recalculate its mean shape (the average shape of all samples in the cluster); perform principal component analysis again within the cluster, extract the first few principal components (such as the first 3 to 5 principal components), and build an aggregation model for each cluster.

[0061] In this application, aggregation processing is performed through principal component analysis-clustering-secondary principal component analysis. The differences between the three-dimensional model data are amplified by principal component analysis, and clustering is performed on this basis to greatly improve the clustering effect. In addition, the three-dimensional model data is classified through clustering, which facilitates secondary principal component analysis on this basis to achieve better aggregation.

[0062] In one embodiment, combined Figure 2 As shown, the step S104 of inputting the spinal medical image into the spinal nail placement model to obtain the spinal nail placement result includes:

[0063] The spinal medical image is input into the segmentation structure to obtain a 3D model of the vertebral body;

[0064] Input the vertebral three-dimensional model into the classification structure to obtain the vertebral category;

[0065] The vertebral category and aggregation model are fused to obtain the category model;

[0066] Input the category model and the vertebral three-dimensional model into the registration structure to obtain the registration matrix;

[0067] Based on the category model and registration matrix, the vertebral screw placement planning results are obtained.

[0068] Combine Figure 2 As shown, the spinal medical image is segmented to obtain a 3D model of the spinal vertebrae. This segmentation can segment the 3D models of all vertebrae in the spinal medical image. During specific use, only the 3D model of the vertebrae that need to be screwed is retained.

[0069] It should be noted that in this application, the spinal screw placement model can only be used to place screws in one vertebra at a time. Therefore, the spinal medical image is segmented to retain only the 3D model of the vertebra with a preset sequence number (e.g., thoracic vertebra L1). If screw placement is required for another vertebra, the spinal medical image is cyclically input into the spinal screw placement model to obtain the screw placement planning results for the other vertebra.

[0070] It should be noted that vertebrae with different serial numbers correspond to different classification structures. Therefore, when the spinal screw placement model is cyclically input, the parameters of the classification structure must be updated synchronously.

[0071] The vertebral three-dimensional model is input into the classification structure to obtain a vertebral category, which is the serial number of the aggregation model closest to the vertebral three-dimensional model.

[0072] In this application, the three-dimensional model of the historical image of the spine is annotated through the aggregated model, and the classification structure is trained with the annotated data. In this way, each time the three-dimensional model of the vertebral body is input into the classification structure, the aggregated model closest to the three-dimensional model of the vertebral body can be obtained.

[0073] It should be noted that in this application, the classification structure is only for one vertebra (eg, thoracic vertebra L1) at a time. Therefore, if multiple vertebrae are to be classified, the corresponding classification structure needs to be trained for each vertebra.

[0074] For example, a 3D model of thoracic vertebra L1 is obtained from historical spinal images and aggregated to obtain three clustered models. The thoracic vertebra L1 in the collection is labeled into different categories (three categories: cluster 1, cluster 2, and cluster 3) based on the clustered models it aggregates. In this way, after training the classification structure, the input of the classification structure is the 3D model, and the output is directly the category. The clustered model corresponding to this category is the clustered model that is closest to the input 3D model.

[0075] In this application, the vertebral three-dimensional model is adapted to the aggregation model through classification, thereby greatly increasing the actual processing speed and simultaneously reducing the amount of processed data (the classification result is directly given without the need for calculation and comparison of each aggregation model).

[0076] In this application, the classification structure outputs a category mask, and the corresponding aggregation model can be obtained by fusing or combining the category mask with the aggregation model of the arrangement setting.

[0077] In this way, the confirmation of the aggregation model is directly integrated into the deep convolutional model, thus achieving the effect of automatic processing.

[0078] In one embodiment, the vertebral body category is a mask of a clustered model that is closest to the vertebral body three-dimensional model.

[0079] In one embodiment, the segmentation structure, the classification structure, and the registration structure are obtained through independent training.

[0080] In this way, through independent training, the resource usage during training is greatly reduced, and the data depth of training is reduced, avoiding local iteration defects and overfitting problems.

[0081] In this application, the segmentation structure can be a 3D U-Net network, or a VoxNet model; the registration structure can be a Procrustes analysis network, a DeepICP network, an RPM-Net network, a PRNet network, etc.

[0082] In this application, the vertebral screw placement planning results are obtained based on the classification model and the registration matrix. The classification model is the closest aggregate model, on which the screw placement information is annotated; the registration matrix is the registration matrix between the aggregate model and the 3D vertebral model. The aggregate model can be mapped to the 3D vertebral model through the registration matrix. In this way, the annotated screw placement information can also be directly mapped to the 3D vertebral model through the registration matrix as the vertebral screw placement planning result.

[0083] In one embodiment, combined Figure 3 As shown, the vertebral body three-dimensional model is input into the classification structure to obtain the vertebral body categories, including:

[0084] Inputting the vertebral body three-dimensional model into the first branch to obtain a first branch diagram;

[0085] The vertebral three-dimensional model is input into the second branch, the third branch, the fourth branch, and the fifth branch respectively, and downsampling convolution processing, upsampling processing, and convolution layer processing are performed in sequence to obtain the corresponding second branch graph, third branch graph, fourth branch graph, and fifth branch graph; the step sizes of the downsampling convolution processing in the second branch, the third branch, the fourth branch, and the fifth branch are different;

[0086] Inputting the vertebral body three-dimensional model into the sixth branch to obtain the sixth branch map;

[0087] Performing feature fusion on the first branch graph, the second branch graph, the third branch graph, the fourth branch graph, and the fifth branch graph to obtain a fusion graph;

[0088] Merge the fusion graph and the sixth branch graph to obtain a merged graph;

[0089] The merged image is input into the classification layer to obtain the vertebral categories.

[0090] Combine Figure 3 As shown in the figure, from left to right are the first branch, the second branch, the third branch, the fourth branch, the fifth branch, and the sixth branch. In the first branch, the input vertebral 3D model is subjected to 1×1 convolution to obtain the first branch graph y1.

[0091] In the second branch, the input vertebral 3D model is processed by 5×5 convolution with a stride of 2, adaptive module processing, and 1×1 convolution upsampling, then merged with the first branch graph y1 and subjected to 3×3 convolution processing to obtain the second branch graph y2;

[0092] In the third branch, the input vertebral 3D model is processed by 9×9 convolution with a stride of 4, adaptive module processing, and 1×1 convolution upsampling, then merged with the second branch image y2 and subjected to 3×3 convolution processing to obtain the third branch image y3;

[0093] In the fourth branch, the input vertebral 3D model is processed by 17×17 convolution with a stride of 8, adaptive module processing, and 1×1 convolution upsampling, then merged with the third branch image y3 and subjected to 3×3 convolution processing to obtain the fourth branch image y4;

[0094] In the fifth branch, the input vertebral 3D model is processed by H×W convolution, adaptive module processing, and 1×1 convolution upsampling, then merged with the fourth branch image y4 and subjected to 3×3 convolution processing to obtain the fifth branch image y5;

[0095] In the sixth branch, the input vertebral 3D model is subjected to 1×1 convolution to obtain the sixth branch graph;

[0096] After feature fusion of the first branch graph y1, the second branch graph y2, the third branch graph y3, the fourth branch graph y4, and the fifth branch graph y5, a 1×1 convolution is performed to obtain a fused graph. The fused graph is added to the sixth branch graph and input into the classification layer to obtain the vertebral category.

[0097] In this application, feature fusion can be a merging or adding process to increase the fusion speed.

[0098] In this application, by setting up multiple branch structures, each branch captures different levels of features of the vertebral three-dimensional model through convolution kernels and strides of different scales, and finally performs feature fusion, thereby aggregating receptive fields of different scales and capturing features of different scales.

[0099] In this application, through layer-by-layer feature fusion (the outputs of the second to fifth branches are gradually merged with the feature maps of the previous stage to form a hierarchical feature fusion process), low-level features (such as edges and textures) are allowed to be gradually combined with high-level features (such as shapes and semantics), thereby generating a richer feature representation.

[0100] In this application, the feature fusion graph integrates the features of all branches, retaining local details and including global context information, which can improve the accurate positioning and classification of the model.

[0101] In this application, each of the second, third, fourth, and fifth branches performs downsampling convolution processing, then adaptive module processing, and finally convolution upsampling. In this way, the adaptive module dynamically adjusts the importance of features, further enhancing the ability to extract key features and improving the robustness of the model.

[0102] In one embodiment, the processing of the adaptation module includes:

[0103] Divide the input feature map into blocks to obtain independent blocks;

[0104] For each independent block, obtain a first neighboring block and a second neighboring block with different spacings;

[0105] generating a first feature block based on the independent block and the first neighborhood block;

[0106] generating a second feature block based on the independent block and the second neighborhood block;

[0107] Performing feature compression on the first feature block and the second feature block to obtain a compressed block;

[0108] Traverse all independent blocks and generate adaptive input feature maps based on the obtained compressed blocks.

[0109] In the present application, the input feature map is divided into blocks, that is, the input feature map is divided into corresponding image blocks through a checkerboard grid; wherein, the image block can be at the pixel level (that is, each pixel is an image block) or at other levels, and the specific division shall be based on the actual processing situation.

[0110] In this application, a sliding window or a fixed step size is used to divide the image into blocks of the same size.

[0111] It should be noted here that if the input feature map is a three-dimensional image, a plane (sagittal plane) is selected for chessboard division, and each grid is a strip grid with a lot of depth (the depth is the depth of the three-dimensional image), and the strip grid is an image block.

[0112] Preferably, in the present application, each image block is 100-1000 pixels, so that more feature calculations between local areas can be performed on the basis of ensuring generation accuracy and reducing the amount of calculation.

[0113] In this application, an image block is selected as an independent block. The adjacent image blocks above, below, to the left, and to the right of the independent block are the first neighboring blocks. The image blocks above, below, to the left, and to the right of the independent block, separated by one grid, are the second neighboring blocks. The first and second neighboring blocks have different spacing from the independent block.

[0114] In this application, the neighborhood information of each independent block is extracted to capture the local structure.

[0115] In the present application, the first feature block is generated to generate a local feature representation using an independent block and its first neighborhood block. Specifically, the independent block and the first neighborhood block are processed by a convolution layer and an attention layer to obtain the first feature block.

[0116] In this application, the specific structure and specific parameters of the convolutional layer and the attention layer can be obtained according to the training data or determined according to the actual situation.

[0117] It should be noted that, in this application, there are four first neighborhood blocks and multiple first feature blocks.

[0118] In this application, the independent block and the first neighborhood block are processed by the convolution layer and the attention layer to obtain the first feature block. The specific process is: the independent block and the four neighborhood blocks are spliced together to form a multi-channel input, and the convolution layer is used to extract features from the spliced block; the self-attention mechanism or the channel attention mechanism is used to enhance important features, calculate the attention weight, and weight the convolution layer output to enhance important features; the output of the attention layer is split into multiple feature blocks, each feature block corresponds to the processing results of the independent block and at least one neighborhood block.

[0119] In this application, a second feature block is generated to generate a broader local feature representation using an independent block and its second neighborhood block. The specific generation process is the same as that of the first feature block, except that the parameters of the convolution layer and the attention layer are different.

[0120] In this application, the generated feature blocks are compressed into a more compact representation to reduce the amount of computation and retain key information. Feature compression is performed using pooling operations (such as maximum pooling or average pooling) or fully connected layers.

[0121] In this way, through compression, the first feature blocks and the second feature blocks are compressed into a compressed block, which corresponds to the size and position of the independent block and is used to replace the independent block. All image blocks are replaced by the compressed block to obtain an adaptive input feature map.

[0122] In this application, each image block of the input feature map is traversed to obtain the corresponding compressed block.

[0123] In this application, for image blocks / independent blocks near edges, their first and second neighboring blocks are incomplete. In this case, they are completed by copying the first and second neighboring blocks at relatively similar positions. For example, if the first neighboring block above the independent block does not exist, the first neighboring block below is copied and used as the block above.

[0124] In this application, the processing accuracy of the adjacent image blocks is greatly improved by completion.

[0125] In this application, the similarity relationship between local regions is captured through an adaptive adjustment module, thereby enhancing feature representation.

[0126] In one embodiment, the training process of the classification structure is:

[0127] Based on the aggregated model of the vertebra, the 3D model of the vertebra is labeled by category;

[0128] The three-dimensional model of the vertebral body is input into the classification structure to obtain the predicted category;

[0129] Calculate the overall loss of the classification structure based on the predicted category and the labeled category;

[0130] The classification structure is iterated based on the overall loss until the loss converges.

[0131] In this application, a collection of historical spinal images is used to generate an aggregation model, and the collection of historical spinal images is also used to train a classification structure, thereby utilizing the homology of the data to greatly increase the classification accuracy of the classification structure.

[0132] In the present application, a data cluster corresponds to an aggregation model, and the category of the three-dimensional model of the vertebra corresponding to the dimensionality reduction data in the data cluster is the serial number of the aggregation model.

[0133] An embodiment of the present application provides an automatic spinal nail placement device based on aggregate matching, which is used to execute an automatic spinal nail placement method based on aggregate matching described above in the present application. The automatic spinal nail placement device based on aggregate matching is described in detail below.

[0134] like Figure 4 As shown, the automatic spinal nail placement device based on aggregation matching includes:

[0135] A collection acquisition module 101 is used to acquire a collection of historical spinal images;

[0136] An aggregation processing module 102 is used to perform aggregation processing on the historical image set of the spine to obtain an aggregated model of the vertebral body;

[0137] An image acquisition module 103 is used to acquire spinal medical images;

[0138] The spinal nail placement module 104 is used to input spinal medical images into a spinal nail placement model to obtain spinal nail placement results; the spinal nail placement model is trained based on an aggregation model and a set of spinal historical images.

[0139] In one embodiment, the aggregation processing module 102 is further configured to:

[0140] The historical images of the spine are segmented to obtain three-dimensional model data of the preset vertebrae; principal component analysis is performed on the three-dimensional model data to obtain the corresponding dimensionality reduction data; cluster analysis is performed on the dimensionality reduction data to obtain multiple data clusters; secondary principal component analysis is performed in each data cluster to obtain the aggregation model of each data cluster.

[0141] In one embodiment, the spinal nail placement module 104 is further configured to:

[0142] The spinal medical image is input into the segmentation structure to obtain a vertebral three-dimensional model; the vertebral three-dimensional model is input into the classification structure to obtain the vertebral category; the vertebral category and the aggregation model are fused to obtain a category model; the category model and the vertebral three-dimensional model are input into the registration structure to obtain a registration matrix; based on the category model and the registration matrix, the vertebral screw placement planning result is obtained.

[0143] In one embodiment, the vertebral body category is a mask of a clustered model that is closest to the vertebral body three-dimensional model.

[0144] In one embodiment, the segmentation structure, the classification structure, and the registration structure are obtained through independent training.

[0145] In one embodiment, the spinal nail placement module 104 is further configured to:

[0146] The three-dimensional model of the vertebra is input into the first branch to obtain the first branch graph; the three-dimensional model of the vertebra is input into the second branch, the third branch, the fourth branch, and the fifth branch respectively, and downsampling convolution processing, upsampling processing, and convolution layer processing are performed in sequence to obtain the corresponding second branch graph, third branch graph, fourth branch graph, and fifth branch graph; the step sizes of the downsampling convolution processing in the second branch, the third branch, the fourth branch, and the fifth branch are different; the three-dimensional model of the vertebra is input into the sixth branch to obtain the sixth branch graph; feature fusion is performed on the first branch graph, the second branch graph, the third branch graph, the fourth branch graph, and the fifth branch graph to obtain a fusion graph; the fusion graph and the sixth branch graph are merged to obtain a merged graph; the merged graph is input into the classification layer to obtain the vertebral category.

[0147] In one embodiment, the spinal nail placement module 104 is further configured to:

[0148] Based on the aggregation model of the vertebra, the three-dimensional model of the vertebra is labeled with categories; the three-dimensional model of the vertebra is input into the classification structure to obtain a predicted category; based on the predicted category and the labeled category, the overall loss of the classification structure is calculated; based on the overall loss, the classification structure is iterated until the loss converges.

[0149] The above-mentioned embodiment of the present application provides an automatic spinal nail placement device based on aggregate matching and an automatic spinal nail placement method based on aggregate matching provided in an embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0150] The above describes the internal functions and structure of an automatic spinal screw placement device based on aggregate matching, such as Figure 5 As shown, in practice, the automatic spinal nail placement device based on aggregate matching can be implemented as an electronic device, including: a memory 301 and a processor 303.

[0151] The memory 301 may be configured to store programs.

[0152] In addition, the memory 301 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0153] The memory 301 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0154] The processor 303 is coupled to the memory 301 and is configured to execute a program in the memory 301 to:

[0155] Obtain a collection of historical spine images;

[0156] Aggregate the historical image collection of the spine to obtain an aggregated model of the vertebral body;

[0157] Acquire medical images of the spine;

[0158] The spinal medical image is input into a spinal nail placement model to obtain a spinal nail placement result; the spinal nail placement model is trained based on an aggregation model and a spinal historical image collection.

[0159] In one embodiment, the processor 303 is further configured to:

[0160] The historical images of the spine are segmented to obtain three-dimensional model data of the preset vertebrae; principal component analysis is performed on the three-dimensional model data to obtain the corresponding dimensionality reduction data; cluster analysis is performed on the dimensionality reduction data to obtain multiple data clusters; secondary principal component analysis is performed in each data cluster to obtain the aggregation model of each data cluster.

[0161] In one embodiment, the processor 303 is further configured to:

[0162] The spinal medical image is input into the segmentation structure to obtain a vertebral three-dimensional model; the vertebral three-dimensional model is input into the classification structure to obtain the vertebral category; the vertebral category and the aggregation model are fused to obtain a category model; the category model and the vertebral three-dimensional model are input into the registration structure to obtain a registration matrix; based on the category model and the registration matrix, the vertebral screw placement planning result is obtained.

[0163] In one embodiment, the vertebral body category is a mask of a clustered model that is closest to the vertebral body three-dimensional model.

[0164] In one embodiment, the segmentation structure, the classification structure, and the registration structure are obtained through independent training.

[0165] In one embodiment, the processor 303 is further configured to:

[0166] The three-dimensional model of the vertebra is input into the first branch to obtain the first branch graph; the three-dimensional model of the vertebra is input into the second branch, the third branch, the fourth branch, and the fifth branch respectively, and downsampling convolution processing, upsampling processing, and convolution layer processing are performed in sequence to obtain the corresponding second branch graph, third branch graph, fourth branch graph, and fifth branch graph; the step sizes of the downsampling convolution processing in the second branch, the third branch, the fourth branch, and the fifth branch are different; the three-dimensional model of the vertebra is input into the sixth branch to obtain the sixth branch graph; feature fusion is performed on the first branch graph, the second branch graph, the third branch graph, the fourth branch graph, and the fifth branch graph to obtain a fusion graph; the fusion graph and the sixth branch graph are merged to obtain a merged graph; the merged graph is input into the classification layer to obtain the vertebral category.

[0167] In one embodiment, the processor 303 is further configured to:

[0168] Based on the aggregation model of the vertebra, the three-dimensional model of the vertebra is labeled with categories; the three-dimensional model of the vertebra is input into the classification structure to obtain a predicted category; based on the predicted category and the labeled category, the overall loss of the classification structure is calculated; based on the overall loss, the classification structure is iterated until the loss converges.

[0169] In this application, Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.

[0170] The electronic device provided in this embodiment is based on the same inventive concept as the method for automatic spinal screw placement based on aggregate matching provided in an embodiment of the present application, and has the same beneficial effects as the method adopted, run or implemented by the application stored therein.

[0171] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0172] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0173] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0174] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0175] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0176] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0177] The present application also provides a computer-readable storage medium corresponding to a method for automatic spinal nail placement based on aggregate matching provided in the aforementioned embodiment, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute a method for automatic spinal nail placement based on aggregate matching provided in any of the aforementioned embodiments.

[0178] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0179] The computer-readable storage medium provided in the above-mentioned embodiment of the present application and the automatic spinal screw placement method based on aggregate matching provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0180] It should be noted that, in the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0181] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0182] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for automatic spinal screw placement based on aggregate matching, characterized in that: include: Obtain a collection of historical spine images; Aggregate the historical image collection of the spine to obtain an aggregated model of the vertebral body; Acquire medical images of the spine; The spinal medical image is input into a spinal nail placement model to obtain a spinal nail placement result; the spinal nail placement model is trained based on an aggregation model and a spinal historical image collection.

2. The method for automatic spinal screw placement based on aggregate matching according to claim 1, characterized in that: The aggregating the historical spinal image set to obtain an aggregated model of the vertebral body includes: Segment the historical images of the spine to obtain the three-dimensional model data of the preset vertebrae; Perform principal component analysis on the three-dimensional model data to obtain the corresponding dimensionality reduction data; Perform cluster analysis on the dimensionality-reduced data to obtain multiple data clusters; A secondary principal component analysis is performed in each data cluster to obtain the aggregation model of each data cluster.

3. The method for automatic spinal screw placement based on aggregate matching according to claim 1 or 2, characterized in that: The step of inputting the spinal medical image into the spinal nail placement model to obtain the spinal nail placement result includes: The spinal medical image is input into the segmentation structure to obtain a 3D model of the vertebral body; Input the vertebral three-dimensional model into the classification structure to obtain the vertebral category; The vertebral category and aggregation model are fused to obtain the category model; Input the category model and the vertebral three-dimensional model into the registration structure to obtain the registration matrix; Based on the category model and registration matrix, the vertebral screw placement planning results are obtained.

4. The method for automatic spinal screw placement based on aggregate matching according to claim 3, characterized in that: The vertebral body category is a mask of the aggregation model that is closest to the vertebral body three-dimensional model.

5. The method for automatic spinal screw placement based on aggregate matching according to claim 3, characterized in that: The segmentation structure, the classification structure, and the registration structure are obtained through independent training.

6. The method for automatic spinal screw placement based on aggregate matching according to claim 3, characterized in that: The vertebral body three-dimensional model is input into the classification structure to obtain the vertebral body category, including: Inputting the vertebral three-dimensional model into the first branch to obtain a first branch diagram; The vertebral three-dimensional model is input into the second branch, the third branch, the fourth branch, and the fifth branch respectively, and downsampling convolution processing, upsampling processing, and convolution layer processing are performed in sequence to obtain the corresponding second branch graph, third branch graph, fourth branch graph, and fifth branch graph; the step sizes of the downsampling convolution processing in the second branch, the third branch, the fourth branch, and the fifth branch are different; Inputting the vertebral body three-dimensional model into the sixth branch to obtain the sixth branch map; Performing feature fusion on the first branch graph, the second branch graph, the third branch graph, the fourth branch graph, and the fifth branch graph to obtain a fusion graph; Merge the fusion graph and the sixth branch graph to obtain a merged graph; The merged image is input into the classification layer to obtain the vertebral categories.

7. The method for automatic spinal screw placement based on aggregate matching according to claim 3, characterized in that: The training process of the classification structure is: Based on the aggregated model of the vertebra, the 3D model of the vertebra is labeled by category; The three-dimensional model of the vertebral body is input into the classification structure to obtain the predicted category; Calculate the overall loss of the classification structure based on the predicted category and the labeled category; The classification structure is iterated based on the overall loss until the loss converges.

8. A spinal automatic nail placement device based on aggregate matching, characterized in that: include: A collection acquisition module, which is used to acquire a collection of historical spinal images; An aggregation processing module, which is used to perform aggregation processing on the historical image set of the spine to obtain an aggregated model of the vertebral body; An image acquisition module, which is used to acquire spinal medical images; The spinal nail placement module is used to input spinal medical images into a spinal nail placement model to obtain spinal nail placement results; the spinal nail placement model is trained based on an aggregation model and a set of spinal historical images.

9. An electronic device, characterized in that: include: memory and processor; The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program to: Obtain a collection of historical spine images; Aggregate the historical image collection of the spine to obtain an aggregated model of the vertebral body; Acquire medical images of the spine; The spinal medical image is input into a spinal nail placement model to obtain a spinal nail placement result; the spinal nail placement model is trained based on an aggregation model and a spinal historical image collection.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the automatic spinal screw placement method based on aggregate matching as described in any one of claims 1 to 7.

Images