Lumbar screw placement navigation positioning system and method based on 3D printing guide plate
By using convolutional neural network-based identification model and 3D printing technology designed in lumbar nailing surgery, the problem of traditional surgery relying on experience and line of sight disturbances of the guide position is solved, achieving higher insertion accuracy and surgical success rate.
Patent Information
- Application Number
- CN202510104190.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-23
- Publication Date
- 2025-05-06
AI Technical Summary
Traditional lumbar nailing surgery is highly dependent on the doctor's experience, and the guide position of the 3D printed guide may be disturbed by line of sight, resulting in reduced insertion accuracy.
The vertebral body recognition model and guide hole recognition model based on convolutional neural network are used to design 3D printing technology to correspond to the shape of the patient's vertebral body, including multiple guide holes, and the guide hole position is accurately positioned during the operation.
Reliance on physician experience reduces the risk of visual interference at the position of the guide hole, and improves the insertion accuracy of screws and the success rate of surgery.
Smart Images
Figure CN119924980A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical surgery navigation technology, and in particular to a lumbar screw placement navigation positioning system and method based on a 3D printed guide plate. Background Art
[0002] With the continuous advancement of medical technology, spinal surgery, especially lumbar screw placement, has become a common method for treating spinal diseases. Lumbar screw placement restores the stability and function of the spine by inserting screws into the vertebrae. It is widely used in the treatment of spinal injuries, degenerative diseases and deformity correction. The success of this operation is highly dependent on the precise placement of the screws. Any slight deviation may cause nerve and blood vessel damage, or even endanger the patient's life.
[0003] At present, traditional lumbar screw placement surgery mainly relies on preoperative imaging examinations to determine the position of the screws and is highly dependent on the doctor's experience.
[0004] However, the rapid development of 3D printing technology in recent years has brought new opportunities to the medical field. 3D printing can customize personalized medical tools and implants, such as spinal guides, according to the individual anatomical structure of the patient. Through this customized 3D printed guide, doctors can obtain more precise surgical assistance, thereby improving the success rate of surgery.
[0005] Nevertheless, during the operation, the position of the guide hole of the 3D printed guide plate may be affected by visual interference, making it difficult for the doctor to accurately see the position of the guide hole during the operation, thereby affecting the insertion accuracy of the screw. Summary of the invention
[0006] In order to solve the technical problem that traditional lumbar screw placement surgery is highly dependent on the doctor's experience, and the guide hole position of the 3D printed guide plate may be affected by visual interference, making it difficult for the doctor to accurately see the position of the guide hole during operation, thereby affecting the insertion accuracy of the screw, the present invention provides a lumbar screw placement navigation positioning system and method based on a 3D printed guide plate.
[0007] The technical solution provided by the embodiment of the present invention is as follows:
[0008] First aspect:
[0009] An embodiment of the present invention provides a lumbar screw placement navigation and positioning system based on a 3D printed guide plate, comprising:
[0010] A first acquisition module, used for acquiring a lumbar spine image;
[0011] A first preprocessing module, used for preprocessing the lumbar vertebra image;
[0012] The first building module is used to build a vertebral recognition model based on a convolutional neural network;
[0013] A first recognition module, used for inputting the preprocessed lumbar vertebra image into the vertebra recognition model to perform vertebra recognition, and determining the type and shape of each vertebra in the lumbar vertebra image;
[0014] A design module, for designing a 3D printing guide plate corresponding to the shape of each category of vertebrae in the lumbar vertebrae image by 3D printing technology, wherein the 3D printing guide plate includes a plurality of guide holes;
[0015] The second acquisition module is used to acquire the 3D printed guide plate image during the operation;
[0016] A second preprocessing module, used for preprocessing the 3D printing guide plate image;
[0017] The second building module is used to build a guide hole recognition model based on a convolutional neural network;
[0018] A second recognition module is used to input the preprocessed 3D printing guide plate image into the guide hole recognition model to perform guide hole recognition and determine the positions of each guide hole on the 3D printing guide plate;
[0019] The navigation and positioning module is used to perform navigation and positioning of lumbar screw placement surgery according to the positions of the guide holes in the 3D printed guide plate.
[0020] Second aspect:
[0021] An embodiment of the present invention provides a lumbar screw placement navigation and positioning method based on a 3D printed guide plate, comprising:
[0022] S1: Acquire lumbar spine images;
[0023] S2: preprocessing the lumbar vertebra image;
[0024] S3: Construct a vertebral recognition model based on convolutional neural network;
[0025] S4: inputting the preprocessed lumbar vertebrae image into the vertebrae recognition model to perform vertebrae recognition, and determining the type and shape of each vertebrae in the lumbar vertebrae image;
[0026] S5: Designing a 3D printing guide plate corresponding to the shape of each category of vertebrae in the lumbar vertebrae image by 3D printing technology, wherein the 3D printing guide plate includes a plurality of guide holes;
[0027] S6: Obtaining images of the 3D printed guide during surgery;
[0028] S7: preprocessing the 3D printing guide plate image;
[0029] S8: Construct a guide hole recognition model based on convolutional neural network;
[0030] S9: inputting the preprocessed 3D printing guide plate image into the guide hole recognition model to perform guide hole recognition, and determining the positions of each guide hole on the 3D printing guide plate;
[0031] S10: performing navigation positioning for lumbar screw placement surgery according to the positions of the guide holes in the 3D printed guide plate.
[0032] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0033] In the present invention, the preprocessed lumbar vertebrae image is input into the vertebral body recognition model for vertebral body recognition, the category and shape of each vertebra in the lumbar vertebrae image are determined, and a 3D printed guide plate corresponding to the shape of each category of vertebrae in the lumbar vertebrae image is designed through 3D printing technology. It is no longer highly dependent on the doctor's experience. The preprocessed 3D printed guide plate image is input into the guide hole recognition model for guide hole recognition, and the positions of each guide hole on the 3D printed guide plate are determined. This reduces the influence of line of sight interference on the guide hole position of the 3D printed guide plate, enables the doctor to accurately see the position of the guide hole during operation, and improves the insertion accuracy of the screw. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0035] Figure 1 A schematic structural diagram of a lumbar screw placement navigation and positioning system based on a 3D printed guide plate provided in an embodiment of the present invention;
[0036] Figure 2 A schematic flow chart of a lumbar screw placement navigation and positioning method based on a 3D printed guide plate provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The technical solution of the present invention is described below in conjunction with the accompanying drawings.
[0038] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.
[0039] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.
[0040] Reference Manual Attached Figure 1 , showing a structural schematic diagram of a lumbar screw placement navigation and positioning system based on a 3D printed guide plate provided in an embodiment of the present invention.
[0041] The embodiment of the present invention provides a lumbar screw placement navigation and positioning system 20 based on a 3D printed guide plate, comprising:
[0042] The first acquisition module 201 is used to acquire a lumbar spine image.
[0043] Specifically, a computerized tomography (CT) device is used to scan the patient's lumbar spine to obtain a lumbar spine CT image.
[0044] The first preprocessing module 202 is used to preprocess the lumbar vertebrae image.
[0045] In a possible implementation, the first preprocessing module 202 is specifically configured to:
[0046] The lumbar spine image is gray-scaled through gamma transformation.
[0047] It should be noted that gamma transform is an image processing technique used to adjust the brightness and contrast of an image. It changes the overall brightness perception of an image by performing a nonlinear transformation on the pixel values of the image. When using gamma transform, if the gamma value is greater than 1, the image will become darker. If the gamma value is less than 1, the image will become brighter. Gamma transform can make the details of the image more prominent, especially in the case of uneven brightness, which helps to improve the visual effect of the image. This method is often used for image enhancement, especially in medical imaging, photography, and display adjustment, to make the image more consistent with the human eye's perception of light changes.
[0048] Specifically, the grayscale processing of the lumbar vertebra image is performed according to the following formula:
[0049]
[0050] Among them, V out represents the grayscale value of the lumbar image after grayscale processing, A represents the proportional coefficient used to adjust the gamma transform intensity, V in represents the gray value of the lumbar spine image, and r represents the gamma value.
[0051] The lumbar vertebrae image after grayscale processing is denoised by median filtering.
[0052] It should be noted that median filtering is a commonly used image denoising technique that effectively removes noise from an image by replacing the value of each pixel in the image with the median of the values of its neighboring pixels. Unlike mean filtering, median filtering can better preserve the edge information of an image because it is not affected by extreme values (such as salt and pepper noise) during calculation. The specific process is that for each pixel, the pixel values within a certain range around it are taken, and these values are sorted by size, and the middle value is selected as the new value of the current pixel. Median filtering is widely used to remove noise from images while maintaining the clarity of image details and edges.
[0053] Specifically, according to the following formula, the lumbar vertebrae image after grayscale processing is denoised.
[0054] l median (x,y)=median[j(p,q)] (p,q)∈N(x,y)
[0055] Among them, l median (x,y) represents the pixel value of the pixel with horizontal coordinate x and vertical coordinate y in the lumbar vertebrae image after denoising, median represents the median, j(p,q) represents the pixel value of the pixel with horizontal coordinate p and vertical coordinate q in the neighborhood range, and N(x,y) represents the neighborhood range of the pixel with horizontal coordinate x and vertical coordinate y in the lumbar vertebrae image after grayscale processing.
[0056] In the present invention, grayscale processing is performed through gamma transformation, which can improve the contrast of the image, enhance the image details, and enhance the visibility of important areas (such as vertebrae) in particular under complex backgrounds. By using denoising techniques such as median filtering, the noise in the image (such as artifacts generated during the scanning process) can be effectively removed, reducing interference in subsequent image processing stages, making subsequent feature extraction more accurate. After preprocessing, important features in the image (such as the boundaries and shapes of vertebrae) are more prominent, and convolutional neural networks (CNNs) can learn and recognize these features more easily. This will make the model more accurate when performing vertebral recognition and reduce the possibility of misidentification.
[0057] The first construction module 203 is used to construct a vertebral body recognition model based on a convolutional neural network.
[0058] It should be noted that the convolutional neural network (CNN) is a deep learning model that is particularly good at processing visual data such as images and videos. CNN uses multiple convolutional layers, pooling layers, and fully connected layers to extract image features by simulating the structure and function of the biological visual system. In the convolutional layer, the model uses convolution kernels (filters) to perform convolution operations on the input image to extract local features (such as edges, textures, etc.). The pooling layer reduces the computational complexity by reducing the dimension of the feature map while retaining important spatial information. The fully connected layer comprehensively processes the extracted high-level features and finally performs classification or regression tasks. The advantage of CNN lies in its automated feature learning capabilities, which can effectively process and recognize complex patterns in images and is widely used in image recognition, object detection, speech recognition and other fields.
[0059] In one possible implementation, the vertebral body recognition model includes an input layer, multiple convolutional layers, multiple pooling layers, multiple random inactivation layers, multiple flattening layers, a feature fusion layer, a fully connected layer, and an output layer. The convolutional layers include a first convolutional layer, a second convolutional layer, and a third convolutional layer. The pooling layers include a first pooling layer, a second pooling layer, a third pooling layer, a fourth pooling layer, and a fifth pooling layer. The random inactivation layers include a first random inactivation layer, a second random inactivation layer, a third random inactivation layer, and a fourth random inactivation layer. The flattening layers include a first flattening layer, a second flattening layer, and a third flattening layer.
[0060] In the present invention, one of the biggest advantages of convolutional neural network (CNN) is its automatic feature learning ability. In traditional image processing methods, image features need to be manually extracted, while CNN automatically learns useful features in the image (such as the edge, shape, texture, etc. of the vertebra) through training without manual intervention. By using convolutional neural network (CNN) to construct a vertebra recognition model, the accuracy and efficiency of vertebra recognition can be greatly improved.
[0061] The first recognition module 204 is used to input the preprocessed lumbar vertebrae image into the vertebrae recognition model to perform vertebrae recognition and determine the type and shape of each vertebrae in the lumbar vertebrae image.
[0062] Specifically, first, the preprocessed lumbar vertebrae image is input into the vertebrae recognition model, which consists of multiple convolutional layers, pooling layers, random inactivation layers, etc., which are used to gradually extract the features of the image. The convolutional layer is responsible for extracting low-level features in the image, such as edges and textures, while the pooling layer helps the model better capture the global information of the image by reducing the dimension of the feature map. Then, the model further processes the extracted features through the fully connected layer, and finally classifies them through the SoftMax layer to determine the category of each vertebra in the image, which is usually represented by different vertebrae types (such as L1, L2, etc.). In addition, the shape recognition part is completed by the features extracted by the middle layer of the model. The model learns the geometric features of the vertebrae and then recognizes the specific shape of the vertebrae in the image. Finally, the model outputs the category and shape of each vertebra, thereby realizing vertebrae recognition and shape prediction of lumbar vertebrae images.
[0063] In the present invention, the model not only classifies the vertebrae, but also predicts the specific shape of each vertebra. By learning the geometric information in the image, the model can accurately identify the specific shape of the vertebrae and provide support for the design of 3D printed guides. The deep structure of the convolutional neural network enables the model to extract stable features from complex and noisy images, thereby enhancing the robustness under different image conditions.
[0064] In a possible implementation, the first identification module 204 is specifically configured to:
[0065] In the input layer, the preprocessed lumbar spine image is input.
[0066] In the first convolutional layer, feature extraction is performed on the preprocessed lumbar vertebrae image to obtain a first local feature map.
[0067] It should be noted that the first convolution layer uses 32 convolution kernels, each of which has a size of 5×5, and the activation function is the ReLU activation function.
[0068] In the first pooling layer, a pooling operation is performed on the first local feature map to obtain a first pooled feature map.
[0069] It should be noted that the pooling window size of the first pooling layer is 2×2, the step size is 2, and the maximum pooling operation is performed.
[0070] In the first random inactivation layer, some neurons in the first pooling feature map are randomly discarded to obtain the first random inactivation feature map.
[0071] It should be noted that the dropout rate of the first random dropout layer is 50%.
[0072] In the second convolutional layer, feature extraction is performed on the first random inactivation features to obtain a second local feature map.
[0073] It should be noted that the second convolution layer uses 64 convolution kernels, each of which has a size of 5×5, and the activation function is the ReLU activation function.
[0074] In the second pooling layer, a pooling operation is performed on the second local feature map to obtain a second pooled feature map.
[0075] It should be noted that the pooling window size of the second pooling layer is 2×2, the step size is 2, and the maximum pooling operation is performed.
[0076] In the second random inactivation layer, some neurons in the second pooling feature map are randomly discarded to obtain a second random inactivation feature map.
[0077] It should be noted that the dropout rate of the second random dropout layer is 50%.
[0078] In the third convolutional layer, feature extraction is performed on the second random inactivation feature map to obtain a third local feature map.
[0079] It should be noted that the third convolution layer uses 128 convolution kernels, each of which has a size of 5×5, and the activation function is the ReLU activation function.
[0080] In the third pooling layer, a pooling operation is performed on the third local feature map to obtain a third pooled feature map.
[0081] It should be noted that the pooling window size of the third pooling layer is 2×2, the step size is 2, and the maximum pooling operation is performed.
[0082] In the third random inactivation layer, some neurons in the third pooling feature map are randomly discarded to obtain the third random inactivation feature map.
[0083] It should be noted that the dropout rate of the third random dropout layer is 50%.
[0084] In the first flattening layer, the third random dropout feature map is flattened into a one-dimensional vector to obtain a first flattened vector.
[0085] In the fourth pooling layer, a pooling operation is performed on the first random dropout feature map to obtain a fourth pooling feature map.
[0086] It should be noted that the pooling window size of the fourth pooling layer is 4×4, the step size is 4, and the maximum pooling operation is performed.
[0087] In the second flattening layer, the fourth pooled feature map is flattened into a one-dimensional vector to obtain a second flattened vector.
[0088] In the fifth pooling layer, a pooling operation is performed on the second random dropout feature map to obtain a fifth pooling feature map.
[0089] It should be noted that the pooling window size of the fifth pooling layer is 2×2, the step size is 2, and the maximum pooling operation is performed.
[0090] In the third flattening layer, the fifth pooling feature map is flattened into a one-dimensional vector to obtain a third flattened vector.
[0091] In the feature fusion layer, the first flattened vector, the second flattened vector and the third flattened vector are feature fused to obtain a feature fusion vector.
[0092] In the fully connected layer, the feature fusion vector is used for feature recognition through the ReLU activation function.
[0093] It should be noted that the number of neurons in the fully connected layer is 256, and the activation function is the ReLU activation function.
[0094] In the fourth random dropout layer, some neurons of the output of the fully connected layer are randomly discarded.
[0095] It should be noted that the dropout rate of the fourth random dropout layer is 50%.
[0096] In the output layer, the output of the fourth random dropout layer is classified through the SoftMax activation function, and the category and shape of each vertebra in the preprocessed lumbar image are output.
[0097] In the present invention, the model gradually extracts image features and performs feature fusion through multiple convolutional layers, pooling layers, random inactivation layers and fully connected layers, effectively identifying the category (such as L1, L2, etc.) and shape of the vertebra. The convolutional layer is responsible for extracting low-level to high-level features in the image, the pooling layer reduces the computational complexity and retains important information, the random inactivation layer prevents overfitting, and the fully connected layer further processes and classifies the extracted features, and finally outputs the results through the SoftMax layer. This method can improve recognition accuracy, processing efficiency and generalization ability, and provide accurate auxiliary support for the diagnosis and surgical planning of lumbar diseases.
[0098] The design module 205 is used to design a 3D printing guide plate corresponding to the shape of each category of vertebrae in the lumbar vertebrae image by using 3D printing technology, and the 3D printing guide plate includes a plurality of guide holes.
[0099] In the present invention, the 3D printed guide can be accurately designed to fully match the vertebral shape based on the pre-processed and identified lumbar vertebrae image. The specific shape, size and position of each vertebra will be taken into account, thereby providing an accurate positioning tool for surgery. The doctor can directly place the screws accurately according to the guide holes on the guide, reducing errors and improving surgical accuracy.
[0100] The second acquisition module 206 is used to acquire the 3D printed guide plate image during the operation.
[0101] In a possible implementation, the 3D printing guide plate image includes a left image and a right image, and the second acquisition module 206 is specifically used to:
[0102] The left image and the right image of the 3D printed guide during the operation are obtained by setting the left camera and the right camera on the left and right sides above the operating table.
[0103] The first preprocessing module 207 is used to preprocess the 3D printing guide plate image.
[0104] In a possible implementation, the first preprocessing module 207 is specifically configured to:
[0105] The 3D printing guide plate image is grayscale processed through gamma transformation.
[0106] Specifically, the 3D printing guide plate image is gray-scaled according to the following formula:
[0107]
[0108] Among them, V′ out represents the grayscale value of the 3D printing guide image after grayscale processing, A represents the proportional coefficient used to adjust the gamma transform intensity, V′ in It represents the gray value of the 3D printing guide image, and r represents the gamma value.
[0109] The 3D printing guide plate image after grayscale processing is denoised by median filtering.
[0110] Specifically, according to the following formula, the 3D printing guide plate image after grayscale processing is denoised.
[0111] l' median (x,y)=median[j'(p,q)] (p,q)∈N'(x,y)
[0112] Among them, l' median (x,y) represents the pixel value of the pixel point with abscissa x and ordinate y in the 3D printing guide image after denoising, median represents the median, j'(p,q) represents the pixel value of the pixel point with abscissa p and ordinate q in the neighborhood, and N'(x,y) represents the neighborhood range of the pixel point with abscissa x and ordinate y in the 3D printing guide image after grayscale processing.
[0113] In the present invention, the 3D printing guide plate image is grayed by gamma transformation, so that the contrast of the image is enhanced. Especially in low-contrast images, the gamma transformation helps to highlight the edges of the guide holes and other key details and improve the visibility of image details. The noise in the image is removed by median filtering, which can effectively remove artifacts or stray signals that may be generated during the image acquisition process, making the image cleaner.
[0114] The second building module 208 is used to build a guide hole recognition model based on a convolutional neural network.
[0115] In the present invention, by constructing a guide hole recognition model based on a convolutional neural network, local features in the image can be recognized, human intervention can be avoided, and human errors in the feature extraction process can be reduced. This means that more accurate guide hole positioning can be achieved, especially for complex guide hole shapes or positions.
[0116] The second recognition module 209 is used to input the preprocessed 3D printing guide plate image into the guide hole recognition model to perform guide hole recognition and determine the positions of each guide hole on the 3D printing guide plate.
[0117] In the present invention, the guide hole recognition model constructed based on convolutional neural network (CNN) can automatically identify the position of the guide hole on the 3D printed guide plate. This makes the positioning of the guide hole very accurate, reducing the errors that may be generated during traditional manual measurement or operation. Especially in the case of complex guide plate design or small-sized guide holes, automated recognition technology can greatly improve accuracy. Through automated guide hole recognition, doctors can ensure the precise position of the guide hole, thereby accurately locating the insertion point of the screw or other medical device, reducing complications caused by positioning errors, such as accidental injury to blood vessels, nerves or other critical structures.
[0118] In a possible implementation manner, the second identification module 209 is specifically configured to:
[0119] According to the left image and the right image, a two-dimensional bounding box of each guide hole in the left image and the right image is determined by a guide hole recognition model.
[0120] Specifically, through the convolutional neural network, the preprocessed left and right images are first input into the model. The network extracts the key features of the guide holes, such as edges and shapes, through a series of convolutional layers, and uses the pooling layer to reduce the feature dimension to enhance the saliency of the features. Subsequently, the model comprehensively processes these features through the fully connected layer and outputs the two-dimensional bounding box of each guide hole in the image. These bounding boxes are represented by the coordinates of the upper left corner and the lower right corner, accurately locating the position of the guide hole in the image. Finally, the network determines the two-dimensional bounding boxes of all guide holes in the left and right images respectively, providing a basis for subsequent three-dimensional positioning.
[0121] In the present invention, CNN is used to automatically extract important features in the image, such as edges and shapes, which improves the accuracy and efficiency of guide hole recognition and reduces the need for manual intervention. At the same time, the model reduces the feature dimension through the pooling layer, enhances the saliency of the feature, ensures the accurate positioning of the bounding box, and provides reliable data support for subsequent three-dimensional positioning, thereby improving the accuracy and safety of the surgery.
[0122] According to the two-dimensional bounding box positions of the guide holes in the left image and the right image, similarity vectors between the guide holes in the left image and the right image are determined.
[0123] It should be noted that the similarity vector refers to the displacement vector connecting the corresponding guide hole positions in the left image and the right image. Specifically, the similarity vector is determined by calculating the displacement difference between the center points of the bounding boxes of each guide hole in the left image and the right image. These displacement vectors represent the relative movement from the left image to the corresponding guide hole position in the right image. The size and direction of the similarity vector reflect the relationship between the relative positions of the guide holes in the two images, which helps with image matching and guide hole positioning.
[0124] In the present invention, the similarity vector is determined by calculating the displacement difference between the center points of the two-dimensional bounding boxes of each guide hole in the left image and the right image, and the relative position relationship of the corresponding guide holes in the two images can be accurately captured. Through the size and direction of the displacement vector, the guide holes in the left image and the right image can be effectively matched, providing an accurate spatial reference for subsequent three-dimensional positioning. This method improves the accuracy and robustness of image matching, ensures the accuracy of guide hole positioning, and further optimizes the effect of the surgical navigation system.
[0125] In a possible implementation manner, the second identification module 209 is further configured to:
[0126] Connect the 2D bounding boxes of each guide hole in the left image with the 2D bounding boxes of each guide hole in the right image to obtain multiple displacement vectors:
[0127] D={d k =c i -c j ∣c i ∈B,c j ∈Q}
[0128] Where D represents the displacement vector set, d k represents the kth displacement vector, c i represents the 2D bounding box of the i-th guide hole in the left image, c j represents the two-dimensional bounding box of the j-th guide hole in the right image, B represents the set of two-dimensional bounding boxes of guide holes in the left image, and Q represents the set of two-dimensional bounding boxes of guide holes in the right image.
[0129] Calculate the angle and length of each displacement vector and construct a polar coordinate system.
[0130] In a possible implementation manner, the second identification module 209 is further configured to:
[0131] According to the following formula, calculate the angle and length of each vector and construct a polar coordinate system:
[0132]
[0133] Among them, α k represents the angle of the kth displacement vector, arctan2() represents the inverse tangent function, d k,y represents the kth displacement vector on the y-axis of the polar coordinate system, d k,x represents the kth displacement vector on the x-axis of the polar coordinate system, l k Represents the length of the kth displacement vector.
[0134] It should be noted that the polar coordinate system is a two-dimensional coordinate system used to represent points on a plane, and the position is defined by the distance from the origin and the angle with the reference direction. In the polar coordinate system, a point is represented by two parameters: the radial distance (i.e. the straight-line distance from the point to the origin) and the polar angle (i.e. the angle between the point and the reference direction (usually the x-axis)). This coordinate system is often used to deal with problems related to circular symmetry or angles, such as representing the direction or position of an object in image processing. Compared with the traditional rectangular coordinate system, the polar coordinate system can more intuitively express and calculate properties related to circles or angles in some applications.
[0135] In the polar coordinate system, the displacement vector within the preset radius is used as the similarity vector between each guide hole in the left image and the right image.
[0136] It should be noted that those skilled in the art can set the size of the preset radius according to actual needs, and the present invention is not limited here.
[0137] In the present invention, by calculating the displacement vector between the guide holes in the left image and the right image, the relative position relationship of the corresponding guide holes in the two images can be accurately captured. This is crucial to ensure the consistency of the guide holes in the two images, which helps to perform image matching and further three-dimensional positioning. The displacement vector is processed by the polar coordinate system. Compared with the rectangular coordinate system, the polar coordinate system is more suitable for processing calculations related to spatial directions such as rotation, angle and distance, making the guide hole position matching in the two images more reliable. By setting the radius and screening out the displacement vectors within the preset radius, the model can accurately judge the similarity between the guide holes, avoid interference with positioning due to distant or irrelevant displacement vectors, and thus achieve more accurate guide hole positioning.
[0138] According to the similarity vectors between the guide holes in the left image and the right image, the positions of the guide holes on the 3D printed guide plate are determined by triangulation.
[0139] Specifically, the process of determining the position of each guide hole on the 3D printed guide plate by triangulation based on the similarity vectors between each guide hole in the left image and the right image is to first calculate the similarity vectors of each pair of guide holes in the two images, representing the displacement difference of the guide holes in the two images, and then use triangulation to convert these two-dimensional similarity vectors into positions in three-dimensional space. The three-dimensional coordinates of the guide holes are calculated by combining the similarity vectors and the difference in viewing angles with the known relative positions of the left and right cameras. The specific operation is to expand the center point of the bounding box of each guide hole in the left and right images into a ray, representing the light emitted from each camera, and finally determine the position of the guide hole in three-dimensional space by calculating the intersection of these two rays, and accurately obtain the three-dimensional coordinates of each guide hole on the 3D printed guide plate, providing a reference for surgical navigation and positioning.
[0140] It should be noted that triangulation is a technique for determining the position of an object by measuring angles and known distances. This method is based on the principle of geometry. It measures the angles of an object from different observation points, and then uses the known baseline distance and angle information to calculate the specific position of the object through the relationship of triangles. In practical applications, triangulation is often used in fields such as geographic surveying, navigation positioning, and 3D reconstruction.
[0141] In a possible implementation manner, the second identification module 209 is further configured to:
[0142] Determine the positions of the guide holes on the 3D printed guide plate according to the following formula:
[0143]
[0144] in, represents the ray vector emitted by the left camera guide hole corresponding to the z-th similarity vector, Represents the scaling factor of the three-dimensional vector from the left camera to the guide hole position corresponding to the z-th similarity vector, Represents the three-dimensional vector from the left camera to the guide hole position corresponding to the z-th similarity vector, represents the ray vector emitted by the right camera guide hole corresponding to the z-th similarity vector, Represents the scaling factor of the three-dimensional vector from the right camera to the guide hole position corresponding to the z-th similarity vector, Represents the three-dimensional vector from the right camera to the guide hole position corresponding to the z-th similarity vector.
[0145] In the present invention, similar vectors in a two-dimensional image are converted into positions in three-dimensional space through triangulation, and the three-dimensional coordinates of the guide hole on the 3D printed guide plate can be accurately calculated. This provides high-precision positioning data for surgical navigation and reduces surgical risks caused by positioning errors. By calculating similar vectors in the left and right images and using triangulation combined with the relative positions of the cameras, data from different perspectives can be better fused. This data fusion provides a more comprehensive view of the guide hole position, which helps to improve the accuracy of positioning. Through precise three-dimensional positioning, doctors can perform surgery more accurately, avoid accidental injury to surrounding tissues and organs, and reduce the incidence of complications.
[0146] The navigation and positioning module 210 is used to perform navigation and positioning of lumbar screw placement surgery according to the positions of the guide holes in the 3D printed guide plate.
[0147] Specifically, the navigation positioning module 210 provides real-time surgical navigation support by accurately determining the three-dimensional position of each guide hole in the 3D printed guide. In actual operation, the module first relies on the three-dimensional coordinates of each guide hole calculated by triangulation in the early stage. These coordinates are analyzed and converted into three-dimensional spatial positions by analyzing the similar vectors of the guide hole positions in the left and right images. Then, the module uses these coordinate data in combination with the surgical navigation system to locate the exact position of the 3D printed guide on the patient's body in real time. Through precise positioning, the doctor can quickly and accurately complete the lumbar screw placement surgery according to the position of the guide hole, ensuring the correct insertion angle and depth of the screw.
[0148] In the present invention, the navigation positioning module 210 can accurately locate the position of each guide hole through the three-dimensional coordinate system. This accurate positioning helps the doctor insert the screw at the most appropriate angle and depth, avoiding complications caused by positioning errors. Real-time three-dimensional positioning can ensure the correct position of the guide plate in the patient's body, thereby minimizing the risk of positioning errors during surgery. For complex spinal surgeries, accurate navigation can avoid damage caused by inaccurate guide hole positioning.
[0149] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0150] In the present invention, the preprocessed lumbar vertebrae image is input into the vertebral body recognition model for vertebral body recognition, the category and shape of each vertebra in the lumbar vertebrae image are determined, and a 3D printed guide plate corresponding to the shape of each category of vertebrae in the lumbar vertebrae image is designed through 3D printing technology. It is no longer highly dependent on the doctor's experience. The preprocessed 3D printed guide plate image is input into the guide hole recognition model for guide hole recognition, and the positions of each guide hole on the 3D printed guide plate are determined. This reduces the influence of line of sight interference on the guide hole position of the 3D printed guide plate, enables the doctor to accurately see the position of the guide hole during operation, and improves the insertion accuracy of the screw.
[0151] Reference Manual Attached Figure 2 , showing a flow chart of a lumbar screw placement navigation positioning method based on a 3D printed guide provided by the present invention.
[0152] The embodiment of the present invention also provides a lumbar vertebrae screw placement navigation positioning method based on a 3D printed guide, which can be implemented by a lumbar vertebrae screw placement navigation positioning device based on a 3D printed guide, and the lumbar vertebrae screw placement navigation positioning device based on a 3D printed guide can be a terminal or a server. The processing flow of the lumbar vertebrae screw placement navigation positioning method based on a 3D printed guide can include the following steps:
[0153] S1: Acquire lumbar spine images.
[0154] S2: Preprocess the lumbar spine image.
[0155] In a possible implementation, S2 specifically includes sub-steps S201 and S202:
[0156] S201: grayscale processing is performed on the lumbar vertebra image through gamma transformation.
[0157] S202: De-noising the grayscale processed lumbar vertebrae image by median filtering.
[0158] S3: Construct a vertebral recognition model based on convolutional neural network.
[0159] In one possible implementation, the vertebral body recognition model includes an input layer, multiple convolutional layers, multiple pooling layers, multiple random inactivation layers, multiple flattening layers, a feature fusion layer, a fully connected layer, and an output layer. The convolutional layers include a first convolutional layer, a second convolutional layer, and a third convolutional layer. The pooling layers include a first pooling layer, a second pooling layer, a third pooling layer, a fourth pooling layer, and a fifth pooling layer. The random inactivation layers include a first random inactivation layer, a second random inactivation layer, a third random inactivation layer, and a fourth random inactivation layer. The flattening layers include a first flattening layer, a second flattening layer, and a third flattening layer.
[0160] S4: Input the preprocessed lumbar vertebrae image into the vertebrae recognition model for vertebrae recognition, and determine the type and shape of each vertebra in the lumbar vertebrae image.
[0161] In a possible implementation, S4 specifically includes sub-steps S401 to S419:
[0162] S401: In the input layer, the preprocessed lumbar spine image is input.
[0163] S402: In the first convolutional layer, feature extraction is performed on the preprocessed lumbar vertebrae image to obtain a first local feature map.
[0164] S403: In the first pooling layer, a pooling operation is performed on the first local feature map to obtain a first pooled feature map.
[0165] S404: In the first random inactivation layer, some neurons in the first pooling feature map are randomly discarded to obtain a first random inactivation feature map.
[0166] S405: In the second convolutional layer, feature extraction is performed on the first random inactivation feature to obtain a second local feature map.
[0167] S406: In the second pooling layer, a pooling operation is performed on the second local feature map to obtain a second pooled feature map.
[0168] S407: In the second random inactivation layer, some neurons in the second pooling feature map are randomly discarded to obtain a second random inactivation feature map.
[0169] S408: In the third convolutional layer, feature extraction is performed on the second random inactivation feature map to obtain a third local feature map.
[0170] S409: In the third pooling layer, a pooling operation is performed on the third local feature map to obtain a third pooled feature map.
[0171] S410: In the third random inactivation layer, some neurons in the third pooling feature map are randomly discarded to obtain a third random inactivation feature map.
[0172] S411: In the first flattening layer, the third random dropout feature map is flattened into a one-dimensional vector to obtain a first flattened vector.
[0173] S412: In the fourth pooling layer, a pooling operation is performed on the first random dropout feature map to obtain a fourth pooling feature map.
[0174] S413: In the second flattening layer, the fourth pooled feature map is flattened into a one-dimensional vector to obtain a second flattened vector.
[0175] S414: In the fifth pooling layer, a pooling operation is performed on the second random dropout feature map to obtain a fifth pooling feature map.
[0176] S415: In the third flattening layer, the fifth pooled feature map is flattened into a one-dimensional vector to obtain a third flattened vector.
[0177] S416: In the feature fusion layer, the first flattened vector, the second flattened vector, and the third flattened vector are subjected to feature fusion to obtain a feature fusion vector.
[0178] S417: In the fully connected layer, feature recognition is performed on the feature fusion vector through the ReLU activation function.
[0179] S418: In the fourth random dropout layer, some neurons of the output of the fully connected layer are randomly discarded.
[0180] S419: In the output layer, the output of the fourth random dropout layer is classified through the SoftMax activation function, and the category and shape of each vertebra in the preprocessed lumbar image are output.
[0181] S5: A 3D printing guide plate corresponding to the shape of each category of vertebrae in the lumbar image is designed by using 3D printing technology, and the 3D printing guide plate includes a plurality of guide holes.
[0182] S6: Obtain the 3D printed guide image during surgery.
[0183] In a possible implementation manner, the 3D printing guide plate image includes a left image and a right image, and S6 is specifically:
[0184] The left image and the right image of the 3D printed guide during the operation are obtained by setting the left camera and the right camera on the left and right sides above the operating table.
[0185] S7: Preprocessing the 3D printing guide image.
[0186] S8: Construct a guide hole recognition model based on convolutional neural network.
[0187] S9: Input the preprocessed 3D printing guide plate image into the guide hole recognition model to perform guide hole recognition, and determine the positions of each guide hole on the 3D printing guide plate.
[0188] In a possible implementation, S9 specifically includes sub-steps S901 to S903:
[0189] S901: According to the left image and the right image, a two-dimensional bounding box of each guide hole in the left image and the right image is determined by a guide hole recognition model.
[0190] S902: Determine similarity vectors between the guide holes in the left image and the right image according to the two-dimensional bounding box positions of the guide holes in the left image and the right image.
[0191] In a possible implementation, S902 specifically includes sub-steps S9021 to S9023:
[0192] S9021: Connect the two-dimensional bounding box of each guide hole in the left image with the two-dimensional bounding box of each guide hole in the right image to obtain multiple displacement vectors:
[0193] D={d k =c i -c j ∣c i ∈B,c j ∈Q}
[0194] Where D represents the displacement vector set, d k represents the kth displacement vector, c i represents the 2D bounding box of the i-th guide hole in the left image, c j represents the two-dimensional bounding box of the j-th guide hole in the right image, B represents the set of two-dimensional bounding boxes of guide holes in the left image, and Q represents the set of two-dimensional bounding boxes of guide holes in the right image.
[0195] S9022: Calculate the angle and length of each displacement vector and construct a polar coordinate system.
[0196] In a possible implementation manner, S9022 specifically includes:
[0197] According to the following formula, calculate the angle and length of each vector and construct a polar coordinate system:
[0198]
[0199] Among them, α k represents the angle of the kth displacement vector, arctan2() represents the inverse tangent function, d k,y represents the kth displacement vector on the y-axis of the polar coordinate system, d k,x represents the kth displacement vector on the x-axis of the polar coordinate system, l k represents the length of the kth displacement vector.
[0200] S9023: In the polar coordinate system, the displacement vector within the preset radius is used as the similarity vector between each guide hole in the left image and the right image.
[0201] S903: Determine the positions of the guide holes on the 3D printing guide plate by triangulation method according to the similarity vectors between the guide holes in the left image and the right image.
[0202] In a possible implementation manner, S903 specifically includes:
[0203] Determine the positions of the guide holes on the 3D printed guide plate according to the following formula:
[0204]
[0205] in, represents the ray vector emitted by the left camera guide hole corresponding to the z-th similarity vector, Represents the scaling factor of the three-dimensional vector from the left camera to the guide hole position corresponding to the z-th similarity vector, Represents the three-dimensional vector from the left camera to the guide hole position corresponding to the z-th similarity vector, represents the ray vector emitted by the right camera guide hole corresponding to the z-th similarity vector, Represents the scaling factor of the three-dimensional vector from the right camera to the guide hole position corresponding to the z-th similarity vector, Represents the three-dimensional vector from the right camera to the guide hole position corresponding to the z-th similarity vector.
[0206] S10: Navigation positioning of lumbar screw placement surgery is performed according to the positions of the guide holes in the 3D printed guide plate.
[0207] It should be noted that the lumbar screw placement navigation positioning method based on the 3D printed guide plate can be implemented by the above-mentioned lumbar screw placement navigation positioning system based on the 3D printed guide plate, and can achieve the same or similar technical effects. To avoid repetition, the present invention will not be described in detail.
[0208] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0209] In the present invention, the preprocessed lumbar vertebrae image is input into the vertebral body recognition model for vertebral body recognition, the category and shape of each vertebra in the lumbar vertebrae image are determined, and a 3D printed guide plate corresponding to the shape of each category of vertebrae in the lumbar vertebrae image is designed through 3D printing technology. It is no longer highly dependent on the doctor's experience. The preprocessed 3D printed guide plate image is input into the guide hole recognition model for guide hole recognition, and the positions of each guide hole on the 3D printed guide plate are determined. This reduces the influence of line of sight interference on the guide hole position of the 3D printed guide plate, enables the doctor to accurately see the position of the guide hole during operation, and improves the insertion accuracy of the screw.
[0210] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
[0211] There are a few points to note:
[0212] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.
[0213] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.
[0214] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.
[0215] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A lumbar screw placement navigation and positioning system based on a 3D printed guide plate, characterized in that: include: A first acquisition module, used for acquiring a lumbar spine image; A first preprocessing module, used for preprocessing the lumbar vertebra image; The first building module is used to build a vertebral recognition model based on a convolutional neural network; A first recognition module, used for inputting the preprocessed lumbar vertebra image into the vertebra recognition model to perform vertebra recognition, and determining the type and shape of each vertebra in the lumbar vertebra image; A design module, for designing a 3D printing guide plate corresponding to the shape of each category of vertebrae in the lumbar vertebrae image by 3D printing technology, wherein the 3D printing guide plate includes a plurality of guide holes; The second acquisition module is used to acquire the 3D printed guide plate image during the operation; A second preprocessing module, used for preprocessing the 3D printing guide plate image; The second building module is used to build a guide hole recognition model based on a convolutional neural network; A second recognition module is used to input the preprocessed 3D printing guide plate image into the guide hole recognition model to perform guide hole recognition and determine the positions of each guide hole on the 3D printing guide plate; The navigation and positioning module is used to perform navigation and positioning of lumbar screw placement surgery according to the positions of the guide holes in the 3D printed guide plate.
2. The lumbar screw placement navigation and positioning system based on 3D printed guide plate according to claim 1 is characterized in that: The first preprocessing module is specifically used for: Performing grayscale processing on the lumbar vertebra image through gamma transformation; The lumbar vertebrae image after grayscale processing is denoised by median filtering.
3. The lumbar screw placement navigation and positioning system based on 3D printed guide plate according to claim 1 is characterized in that: The vertebral body recognition model includes an input layer, multiple convolutional layers, multiple pooling layers, multiple random inactivation layers, multiple flattening layers, a feature fusion layer, a fully connected layer and an output layer. The convolutional layers include a first convolutional layer, a second convolutional layer and a third convolutional layer. The pooling layers include a first pooling layer, a second pooling layer, a third pooling layer, a fourth pooling layer and a fifth pooling layer. The random inactivation layers include a first random inactivation layer, a second random inactivation layer, a third random inactivation layer and a fourth random inactivation layer. The flattening layers include a first flattening layer, a second flattening layer and a third flattening layer.
4. The lumbar screw placement navigation and positioning system based on 3D printed guide plate according to claim 3 is characterized in that: The first identification module is specifically used for: In the input layer, the preprocessed lumbar vertebra image is input; In the first convolutional layer, feature extraction is performed on the preprocessed lumbar vertebrae image to obtain a first local feature map; In the first pooling layer, a pooling operation is performed on the first local feature map to obtain a first pooling feature map; In the first random dropout layer, some neurons in the first pooling feature map are randomly discarded to obtain a first random dropout feature map; In the second convolutional layer, feature extraction is performed on the first random inactivation feature to obtain a second local feature map; In the second pooling layer, a pooling operation is performed on the second local feature map to obtain a second pooling feature map; In the second random inactivation layer, some neurons in the second pooling feature map are randomly discarded to obtain a second random inactivation feature map; In the third convolutional layer, feature extraction is performed on the second random dropout feature map to obtain a third local feature map; In the third pooling layer, a pooling operation is performed on the third local feature map to obtain a third pooling feature map; In the third random inactivation layer, some neurons in the third pooling feature map are randomly discarded to obtain a third random inactivation feature map; In the first flattening layer, the third random dropout feature map is flattened into a one-dimensional vector to obtain a first flattened vector; In the fourth pooling layer, a pooling operation is performed on the first random dropout feature map to obtain a fourth pooling feature map; In the second flattening layer, the fourth pooling feature map is flattened into a one-dimensional vector to obtain a second flattened vector; In the fifth pooling layer, a pooling operation is performed on the second random dropout feature map to obtain a fifth pooling feature map; In the third flattening layer, the fifth pooling feature map is flattened into a one-dimensional vector to obtain a third flattened vector; In the feature fusion layer, the first flattened vector, the second flattened vector and the third flattened vector are subjected to feature fusion to obtain a feature fusion vector; In the fully connected layer, feature recognition is performed on the feature fusion vector through a ReLU activation function; In the fourth random dropout layer, randomly discarding some neurons of the output of the fully connected layer; In the output layer, the output of the fourth random inactivation layer is classified by a SoftMax activation function, and the category and shape of each vertebra in the preprocessed lumbar image are output.
5. The lumbar screw placement navigation and positioning system based on 3D printed guide plate according to claim 1, characterized in that: The 3D printing guide plate image includes a left image and a right image, and the second acquisition module is specifically used for: The left image and the right image of the 3D printed guide during the operation are obtained by setting the left camera and the right camera on the left and right sides above the operating table.
6. The lumbar screw placement navigation and positioning system based on 3D printed guide plate according to claim 5, characterized in that: The second identification module is specifically used for: According to the left image and the right image, determining a two-dimensional bounding box of each guide hole in the left image and the right image by using the guide hole recognition model; Determine a similarity vector between each guide hole in the left image and the right image according to a two-dimensional bounding box position of each guide hole in the left image and the right image; According to the similarity vectors between the guide holes in the left image and the right image, the positions of the guide holes on the 3D printing guide plate are determined by triangulation.
7. The lumbar screw placement navigation and positioning system based on 3D printed guide plate according to claim 6, characterized in that: The second identification module is further specifically used for: Connecting the two-dimensional bounding box of each guide hole in the left image with the two-dimensional bounding box of each guide hole in the right image to obtain a plurality of displacement vectors; Calculating the angle and length of each displacement vector to construct a polar coordinate system; In the polar coordinate system, the displacement vector within a preset radius is used as a similarity vector between each guide hole in the left image and the right image.
8. The lumbar screw placement navigation and positioning system based on 3D printed guide plate according to claim 7, characterized in that: The second identification module is further specifically used for: The angle and length of each of the vectors are calculated to construct a polar coordinate system.
9. The lumbar screw placement navigation and positioning system based on 3D printed guide plate according to claim 6, characterized in that: The second identification module is further specifically used for: Determine the positions of the guide holes on the 3D printing guide plate.
10. A lumbar screw placement navigation positioning method based on a 3D printed guide plate, characterized in that: include: S1: Acquire lumbar spine images; S2: preprocessing the lumbar vertebrae image; S3: Construct a vertebral recognition model based on convolutional neural network; S4: inputting the preprocessed lumbar vertebrae image into the vertebrae recognition model to perform vertebrae recognition, and determining the type and shape of each vertebrae in the lumbar vertebrae image; S5: Designing a 3D printing guide plate corresponding to the shape of each category of vertebrae in the lumbar vertebrae image by 3D printing technology, wherein the 3D printing guide plate includes a plurality of guide holes; S6: Obtaining images of the 3D printed guide during surgery; S7: preprocessing the 3D printing guide plate image; S8: Construct a guide hole recognition model based on convolutional neural network; S9: inputting the preprocessed 3D printing guide plate image into the guide hole recognition model to perform guide hole recognition, and determining the positions of each guide hole on the 3D printing guide plate; S10: performing navigation positioning for lumbar screw placement surgery according to the positions of the guide holes in the 3D printed guide plate.