A preoperative planning method and device for wound repair based on artificial intelligence

Abstract

The application provides a preoperative planning method and device for trauma repair based on artificial intelligence, the method comprising: acquiring a medical image of a single object; segmenting a bone in the medical image according to the medical image, and establishing a three-dimensional bone model; repairing a broken bone at a trauma position, and determining a rotation matrix of the broken bone; determining the size and setting position of an instrument template according to the repaired trauma position; displaying an image of the instrument template fitted with the bone, and adjusting and outputting the size and setting position of the instrument template according to the operation of an operator. In the application, a three-dimensional bone model is constructed through a preoperative medical image, so that the broken bone at the trauma fracture position is repaired, and the selection and setting of a fixed instrument after repair are determined. Through the preoperative planning, strong support can be provided for the reduction and fixation of trauma fractures.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, in particular to a preoperative planning method and device for trauma repair based on artificial intelligence. BACKGROUND

[0002] Fracture is a common disease in orthopedics. Fracture can be caused by external factors such as impact of external force, falling from a high place, etc. Such fracture caused by trauma is called traumatic fracture.

[0003] Preoperative planning of traumatic fracture can provide strong support in aspects of reduction and fixation, etc. However, there is currently a lack of specific solutions in this regard. SUMMARY

[0004] The problem solved by the present application is the lack of preoperative planning of traumatic fracture.

[0005] To solve the above problems, the first aspect of the present application provides a preoperative planning method for trauma repair based on artificial intelligence, comprising:

[0006] obtaining a medical image of a single object;

[0007] segmenting the bone in the medical image according to the medical image, and establishing a three-dimensional bone model;

[0008] repairing the broken bone at the trauma site and determining the rotation matrix of the broken bone;

[0009] determining the size and setting position of the instrument template according to the repaired trauma site;

[0010] displaying an image of the instrument template fitting the bone, and adjusting and outputting the size and setting position of the instrument template according to the operation of the operator.

[0011] The second aspect of the present application provides a preoperative planning device for trauma repair based on artificial intelligence, comprising:

[0012] an image acquisition module for acquiring a medical image of a single object;

[0013] a three-dimensional construction module for segmenting the bone in the medical image according to the medical image, and establishing a three-dimensional bone model;

[0014] a trauma repair module for repairing the broken bone at the trauma site and determining the rotation matrix of the broken bone;

[0015] a template determination module for determining the size and setting position of the instrument template according to the repaired trauma site;

[0016] The template display module is configured to display an image of the instrument template fitting the bone, and adjust and output the size and setting position of the instrument template according to the operation of the operator.

[0017] The third aspect of the present application provides an electronic device, comprising a memory and a processor;

[0018] The memory is configured to store a program.

[0019] The processor is coupled to the memory and configured to execute the program, so as to:

[0020] Obtain a medical image of a single object.

[0021] Segment the bone in the medical image according to the medical image, and establish a three-dimensional bone model.

[0022] Repair the broken bone at the trauma position, and determine the rotation matrix of the broken bone.

[0023] Determine the size and setting position of the instrument template according to the repaired trauma position.

[0024] Display an image of the instrument template fitting the bone, and adjust and output the size and setting position of the instrument template according to the operation of the operator.

[0025] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the above-mentioned preoperative planning method for trauma repair based on artificial intelligence.

[0026] In the present application, a three-dimensional bone model is constructed through a preoperative medical image, so as to repair the broken bone at the trauma fracture position, and determine the selection and setting of the fixed instrument after repair. Through the preoperative planning, strong support can be provided for the reduction and fixation of the trauma fracture. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 A flowchart of the preoperative planning method for trauma repair according to the embodiment of the present application;

[0028] Figure 2 A model architecture diagram of bone segmentation of the preoperative planning method for trauma repair according to the embodiment of the present application;

[0029] Figure 3 An architecture diagram of the hollow convolution of the preoperative planning method for trauma repair according to the embodiment of the present application;

[0030] Figure 4 An architecture diagram of global extraction of the preoperative planning method for trauma repair according to the embodiment of the present application;

[0031] Figure 5 This is a flowchart of a preoperative planning method for wound repair according to an embodiment of the present application;

[0032] Figure 6 Schematic diagram showing a method for preoperative planning of wound repair according to an embodiment of the present application;

[0033] Figure 7 1 is a structural block diagram of a preoperative planning device for wound repair according to an embodiment of the present application;

[0034] Figure 8 2 is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] To make the above-mentioned objects, features, and advantages of the present application more clearly understood, specific embodiments of the present application are described in detail below with reference to the accompanying drawings. Although the accompanying drawings show exemplary embodiments of the present application, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0036] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art to which this application belongs.

[0037] To address the above issues, the present application provides a new artificial intelligence-based preoperative planning solution for trauma repair, which can actively screen the chest X-rays of any subject through an artificial neural network chest screening model, thereby utilizing the universality of chest X-ray spectra to solve the current problem of a small osteoporosis screening range.

[0038] The embodiment of the present application provides a preoperative planning method for wound repair based on artificial intelligence, the specific scheme of the method is as follows: Figures 1-4 As shown, the method can be performed by an artificial intelligence-based preoperative planning device for wound repair, which can be integrated into electronic devices such as computers, servers, computers, server clusters, and data centers. Figure 1 , which is a flow chart of a preoperative planning method for wound repair based on artificial intelligence according to one embodiment of the present application; wherein the preoperative planning method for wound repair based on artificial intelligence includes:

[0039] S101, acquiring a medical image of a single object;

[0040] In this application, the medical image is a CT image, an X-ray image or an MRI image.

[0041] S102, segment the bone in the medical image according to the medical image, and establish a three-dimensional model of the bone;

[0042] S103, repair the broken bone at the trauma position, and determine the rotation matrix of the broken bone;

[0043] In this application, the trauma is an intertrochanteric fracture, and the trauma position is the proximal femur.

[0044] In this application, the rotation matrix of the broken bone means that one of the broken bones in the trauma will be accurately repaired in shape with the other broken bone through the rotation matrix.

[0045] S104, according to the repaired trauma position, determine the size and setting position of the instrument template;

[0046] In one embodiment, the instrument template includes a main nail module, a tail cap template, a helical blade template, and an interlocking screw template.

[0047] Main nail: usually refers to the main nail used for fracture fixation, designed to provide stability and support.

[0048] Tail cap: used to determine and manufacture the tail design of the nail, to ensure its compatibility with the bone or other structures.

[0049] Helical blade: designed for the shape and size of the helical blade, usually used in orthopedic surgery to ensure the accuracy and efficiency of cutting.

[0050] Interlocking screw: design for interlocking screw, to ensure better support and stability when fixing fractures.

[0051] S105, display the image of the instrument template fitting the bone, and adjust and output the size and setting position of the instrument template according to the operation of the operator.

[0052] In this application, through the preoperative medical image, a three-dimensional model of the bone is constructed, so that the broken bone at the fracture position is repaired, and the selection and setting of the fixed instrument after repair are determined. Through this preoperative planning, strong support can be provided for the reduction and fixation of the trauma fracture.

[0053] In this way, in the actual operation, the operator can fix the instrument after aligning the two ends of the broken bone according to the instrument template and setting position planned in the preoperative planning, so as to achieve accurate repair of the fracture trauma.

[0054] In one embodiment, in combination with Figure 2 As shown, segmenting the bone in the medical image according to the medical image includes:

[0055] Input the medical image into a feature extraction structure to obtain a low-level feature map and a high-level feature map;

[0056] Input the high-level feature map into an encoding structure to perform encoding processing to obtain an encoded feature map;

[0057] Input the low-level feature map and the encoded feature map into a decoding structure to obtain a skeleton segmentation result.

[0058] In an embodiment, in combination with Figure 2 As shown in the figure, inputting the high-level feature map into an encoding structure to perform encoding processing to obtain an encoded feature map comprises:

[0059] Inputting the high-level feature map into a plurality of branches arranged in parallel to obtain a plurality of branch feature maps corresponding thereto;

[0060] Performing global extraction and convolution processing in sequence on the plurality of branch feature maps to obtain a global feature map;

[0061] Performing splicing processing on the global feature map to obtain a spliced feature map;

[0062] The spliced feature map is the encoded feature map.

[0063] In an embodiment, in combination with Figure 2 As shown in the figure, inputting the low-level feature map and the encoded feature map into a decoding structure to obtain a skeleton segmentation result comprises:

[0064] Performing 1x1 convolution processing on the low-level feature map to obtain a convolution feature map;

[0065] Performing up-sampling processing on the encoded feature map to obtain an up-sampled feature map;

[0066] Performing feature fusion on the convolution feature map and the up-sampled feature map to obtain a fused feature map;

[0067] Performing convolution processing and up-sampling processing on the fused feature map to obtain a skeleton segmentation result.

[0068] In the present application, in combination with Figure 2As shown, it can be seen that the feature extraction structure is a hollow convolution, which outputs a low-level feature map to the decoding structure and outputs multiple high-level feature maps to the encoding structure; the multiple high-level feature maps form a corresponding number of branches in the encoding structure, wherein the first branch is spliced after 1x1 convolution, global extraction, multiple convolution processing, and 1x1 convolution; the second branch is spliced after 3x3 convolution, global extraction, multiple convolution processing, and 1x1 convolution; the third branch is spliced after 3x3 convolution, global extraction, multiple convolution processing, and splitting processing; the fourth branch is spliced after 3x3 convolution, global extraction, multiple convolution processing, and splitting processing; and the fifth branch is spliced after pooling processing, global extraction, multiple convolution processing, and 1x1 convolution processing.

[0069] Among them, the size of the high-level feature map input of the first branch to the fifth branch increases in turn, and the size of the feature map remains unchanged in the processing process of each branch, until the different sizes of the feature map are converted into the same size of the feature map through 1x1 convolution and splitting processing.

[0070] Among them, the feature map size of the third branch and the fourth branch is an integer multiple of the second branch; and the feature map size is split into multiple second branch sizes through splitting.

[0071] Among them, the feature map size of the first branch is smaller than that of the second branch, and the feature map size is adjusted to the feature map size of the second branch through 1x1 convolution.

[0072] Among them, the feature map size of the fifth branch is larger than that of the second branch, and the feature map size is adjusted to the feature map size of the second branch through 1x1 convolution.

[0073] In an embodiment, in combination with Figure 4 As shown, the process of global extraction is:

[0074] The input feature map is input into the 1x1 convolution layer arranged in parallel to obtain the first extraction map, the second extraction map and the third extraction map; wherein the first extraction map and the third extraction map are in the format of HWxN, and the second extraction map is in the format of NxHW.

[0075] The first extraction map and the second extraction map are multiplied to obtain a multiplication feature map.

[0076] The multiplication feature map is subjected to softmax processing to obtain a multiplication coefficient.

[0077] The multiplication coefficient is multiplied with the third extraction map to obtain a coefficient multiplication map.

[0078] The coefficient multiplication map and the input feature map are added to obtain an output feature map of global extraction.

[0079] In the present application, the input feature map is divided into three branches for 1x1 convolution processing, which adjusts the dimensions of the first branch and the third branch to the format of HWxN, and the dimension of the second branch to the format of NxHW through the convolution; then the first branch and the third branch are multiplied to obtain a feature map with a dimension of HWxHW, and a coefficient is obtained through softmax processing; the coefficient is multiplied with the third branch, and the multiplied feature map is added to the input feature map to obtain an output feature map.

[0080] In the present application, the dimensions of the first branch and the second branch are reversed through 1x1 convolution, so as to realize the multiplication between the feature maps; through the multiplication, the long-range dependency relationship is captured; through the setting of the coefficient, the dependency relationship is embedded in the feature map; and on this basis, the original input is added and residual connection is used, so as to embed the global extraction in the model without destroying the parameters.

[0081] In the present application, the long-range dependency relationship between features is captured through global extraction, so that the model can extract global features to make up for the defect that the current model mainly extracts local features.

[0082] In an embodiment, the activation function of the SoftMax operation is:

[0083]

[0084] where SoftMax1(c) is an activation function, c i , x j is an element in the input vector, and i and j are element serial numbers.

[0085] In the present application, SoftMax is a mathematical function, which is usually used to convert a set of arbitrary real numbers into real numbers representing a probability distribution. It is essentially a normalization function that can convert a set of arbitrary real values into probability values between [0, 1]. Because SoftMax converts them to values between 0 and 1, they can be interpreted as probabilities. If one of the inputs is very small or negative, SoftMax will convert it to a small probability, and if the input is very large, it will convert it to a large probability, but it will always remain between 0 and 1.

[0086] However, for the standard SoftMax function, since the input is mapped to 0 to 1, and the sum of all output values is 1, it means that even if some input values are very small, they will have a non-zero output value after processing by the SoftMax function. This will also cause the noise to be amplified, resulting in the final output result being more affected by noise.

[0087] In the present application, a 1 is added in the denominator of the SoftMax function; this change means that when the input value is very small, its output value can be closer to zero. This allows the corresponding output to tend to zero when there is no valuable information to add, thus greatly reducing unnecessary noise.

[0088] In an embodiment, in combination Figure 3 As shown, the medical image input feature extraction structure is input to obtain a low-level feature map and a high-level feature map, comprising:

[0089] The feature extraction structure comprises multiple branches, respectively performing dilated convolution on the input medical image; wherein the first branch performs five 3x3 convolutions on the medical image, and the five 3x3 convolutions are dilated convolutions with expansion rates of 1, 2, 3, 1, and 2 in sequence; the second branch performs four 3x3 convolutions on the medical image, and the four 3x3 convolutions are dilated convolutions with expansion rates of 1, 2, 5, and 1 in sequence; the third branch performs three 3x3 convolutions on the medical image, and the three 3x3 convolutions are dilated convolutions with expansion rates of 1, 2, and 4 in sequence; the fourth branch performs two 3x3 convolutions on the medical image, and the two 3x3 convolutions are dilated convolutions with expansion rates of 1 and 2 in sequence; the fifth branch performs one 3x3 convolution on the medical image, and the convolution is a dilated convolution with an expansion rate of 1.

[0090] The outputs of the five branches are the high-level feature map; the output of one selected from the third branch, the fourth branch, and the fifth branch is the low-level feature map.

[0091] In the present application, by setting five branches with gradually decreasing dilated convolution times, five feature maps with different sizes are generated, and the difference in image size is realized.

[0092] In the present application, by setting dilated convolution, the receptive field of different branches is increased under other unchanged conditions, and the comprehensive acquisition of multiple receptive fields is realized.

[0093] In the present application, by sequentially setting multiple dilated convolutions and arranging the expansion rates as described above, the entire elements within the receptive field can also be utilized under the condition of expanding the receptive field; thereby avoiding the loss of information caused by only utilizing part of the elements within the receptive field, and avoiding the loss of local correlation.

[0094] In the present application, by sequentially setting multiple dilated convolutions, the receptive field of different sizes is obtained through convolution operation, and multi-scale information can be acquired; and by continuously setting dilated convolutions, a context module is formed before and after the dilated convolution, thereby aggregating multi-scale context information.

[0095] In an embodiment, in combinationFigure 5 As shown, S103, the broken bone at the trauma site is repaired, and the rotation matrix of the broken bone is determined, comprising:

[0096] S301, fitting the proximal broken bone, determining the fitted femoral head center point and femoral neck center point, and determining the femoral head neck axis;

[0097] The proximal broken bone is identified, and the femoral head part (which may have defects) and the femoral neck part (which may have defects) are segmented out; the surface point cloud data of the complete part of the femoral head part is extracted, and the center point coordinate and radius of the sphere are obtained by using a spherical fitting algorithm (such as least squares method), and the center point coordinate is the femoral head center point; the surface point cloud data of a complete part of the femoral neck part is extracted, and the center axis of the cylinder is determined using a cylindrical fitting algorithm (such as RANSAC fitting based on point cloud), and a point on the center axis is selected as the femoral neck center point; connecting the femoral head center point and the femoral neck center point is the femoral head neck axis.

[0098] S302, sampling and fitting the distal broken bone to obtain the femoral shaft axis;

[0099] Uniformly select sampling points on the surface of the broken bone. These points can be obtained by the following methods: random sampling: randomly selecting a certain number of points from the surface of the broken bone. Uniform sampling: selecting sampling points on the surface of the broken bone according to certain rules (such as equal interval). Feature point sampling: selecting sampling points according to the features of the bone, such as edges or regions with large shape changes. Principal axis fitting: using principal component analysis (PCA): performing PCA analysis on the sampling points, and the principal axis is the first principal component direction. Curve fitting: such as the obvious bending of the femur, which can be fitted by a polynomial or a spline curve. Ellipse fitting: for some nonlinear cases, the least squares method can be used to fit the broken part of the femur to obtain the axis. Through the above comprehensive judgment, the femoral shaft axis is obtained.

[0100] S303, extracting the broken surface point set of the proximal broken bone and the broken surface point set of the distal broken bone;

[0101] The broken bone part can be identified first; then the RANSAC algorithm or the least squares method is used to fit the broken surface in the point cloud of the broken bone, if the broken surface is not a simple plane shape, the curved surface fitting method (such as quadratic surface fitting or B-spline fitting) can be used to obtain the broken surface; calculate the distance from all points of the identified broken bone part to the fitted plane, set a threshold, and collect all points within the threshold as the broken surface point set.

[0102] Thus, the fracture surface point set of the proximal broken bone and the fracture surface point set of the distal broken bone can be obtained respectively.

[0103] In S304, a rotation matrix of the femoral head is determined according to the fracture surface point set, the femoral shaft axis and the femoral head neck axis.

[0104] In the present application, the fracture surface point set at the broken bone, the femoral shaft axis and the femoral head neck axis are determined by analyzing and identifying the broken bone, and the rotation matrix of the femoral head is determined based on the same, so that the position repair of the broken bone trauma is realized.

[0105] In one embodiment, the determination of the rotation matrix of the femoral head according to the fracture surface point set, the femoral shaft axis and the femoral head neck axis comprises:

[0106] generating an initial rotation matrix of the femoral head;

[0107] calculating a neck-shaft angle according to the femoral shaft axis and the adjusted femoral head neck axis;

[0108] calculating an anteversion angle according to the adjusted femoral head neck axis and the reference plane;

[0109] iterating the rotation matrix until the neck-shaft angle and the anteversion angle meet a preset angle range;

[0110] calculating a point set distance according to the fracture surface point set of the femoral shaft and the fracture surface point set of the femoral head after iteration;

[0111] re-iterating the rotation matrix until the point set distance is minimum.

[0112] In the present application, the neck-shaft angle and the anteversion angle are first constrained in a preset angle range by adjusting the rotation matrix, and then the rotation matrix is fine-tuned under the premise of constraining the neck-shaft angle and the anteversion angle, so that the point set distance is minimum, at which time the fracture surface of the femoral shaft and the fracture surface of the femoral head should be fitted exactly, thereby realizing the position repair of the broken femur.

[0113] In the present application, the distance between the fracture surfaces of the femoral head and the femoral shaft is calculated to verify whether the splicing result meets the alignment splicing standard, and the neck-shaft angle and the anteversion angle are calculated to verify whether the splicing result meets the alignment splicing standard, so that the accurate alignment splicing and alignment splicing of the femur are realized, and accurate repair is realized.

[0114] In the present application, the neck-shaft angle refers to the included angle formed by the femoral shaft axis and the femoral head neck axis.

[0115] In the present application, the anteversion angle refers to the angle of the outward rotation of the femoral neck relative to the femoral shaft, i.e. the included angle formed by the projection of the femoral head neck axis and the frontal plane on the axial plane.

[0116] In this application, the standard position information of the femur anatomical pose can be obtained, and then the femoral head is set to the standard position of the femoral head, and the femoral shaft is set to the standard position of the femoral shaft (the femoral shaft can be directly located at the standard position through preliminary adjustment, so that only the femoral head is set during actual setting). The rotation matrix of the femoral head relative to the femoral shaft at this time is calculated as the initial matrix.

[0117] In this application, the femoral head-neck axis in the iteration process can be rotated and transformed according to the desired neck-shaft angle (such as 125° to 135°);

[0118] In this application, the point set distance is the Euclidean distance between the two fracture surface point sets; the ICP (Iterative Closest Point) algorithm can be used to find the minimum distance matching point pair set, and the rotation matrix is further optimized.

[0119] In this application, the corresponding function relationship between the rotation matrix and the point set distance can be calculated, and the rotation matrix that minimizes the point set distance can be found through iterative optimization (such as gradient descent or nonlinear optimization algorithm). In each iteration, the rotation matrix is updated, the new neck-shaft angle and anteversion angle are calculated, and the point set distance is compared until it converges to the minimum value.

[0120] In one embodiment, for repairing the broken bone at the trauma site, before determining the rotation matrix of the broken bone, further comprising identifying key points of the femur and the acetabulum, which can include:

[0121] Femoral head center: the geometric center of the spherical surface of the femoral head, which is part of the hip ball-and-socket joint.

[0122] Greater trochanter apex: the highest point of the greater trochanter.

[0123] Lesser trochanter apex: the highest point of the lesser trochanter.

[0124] Head-neck junction: the transition area between the femoral head and the femoral neck.

[0125] Midpoint of femoral shaft: the midpoint of the femoral shaft.

[0126] Acetabular center: the geometric center of the acetabulum.

[0127] Superior rim of acetabulum: the edge above the acetabulum.

[0128] Anterior Rim of Acetabulum: The edge of the acetabulum in the front.

[0129] Posterior Rim of Acetabulum: The edge of the acetabulum in the back.

[0130] Obturator Foramen: The anatomical structure located below the acetabulum.

[0131] Acetabular Fossa: The concave part of the acetabulum, the acetabular fossa.

[0132] In one embodiment, S105, the image of the instrument template fitting the bone is displayed, and the size and setting position of the instrument template are adjusted and output according to the operation of the operator:

[0133] As shown in Figure 6 , it is an image of the displayed prosthesis template (main nail module, tail cap template, spiral blade template and interlocking screw template) fitting the bone, which can be adjusted according to the operation of the operator.

[0134] In this application, in the image of the instrument template fitting the bone, the initial planning position of the main nail module, the tail cap template, the spiral blade template and the interlocking screw template is obtained as follows:

[0135] Main nail planning: Determine the optimal insertion path of the main nail between the proximal and distal fractures of the femur. Usually choose the opening position from the greater trochanter or the lower edge of the acetabulum. Determine the length and diameter of the main nail so that it can span the entire fracture area and provide sufficient support. Align the centerline of the main nail with the femoral shaft axis to ensure that the main nail can pass through the medullary cavity without damaging the bone anatomy.

[0136] Tail cap planning: The tail cap is usually installed at the tail end of the main nail and should be completely aligned with the main nail. Ensure that the position of the tail cap does not cause damage to soft tissues such as tendons or nerves. Select a tail cap with appropriate diameter and length according to the anatomical structure of the bone to avoid the tail cap protruding outside the bone or entering the joint cavity.

[0137] Spiral blade planning: Choose a suitable insertion path in the femoral head and neck area to ensure that the spiral blade can fully enter the femoral head and cover a larger bone area. The insertion angle should be consistent with the femoral neck axis to avoid damaging the articular surface of the femoral head. Select the length of the spiral blade according to the size of the femoral head and the length of the femoral neck. Usually, the head of the blade should be at least 5 mm away from the articular surface of the femoral head to avoid cutting the articular surface.

[0138] Positioning of the locking screw: One or more locking screws are installed at the distal end of the main nail. The screw position should be accurately aligned with the locking hole on the main nail.

[0139] The embodiment of the present application provides a preoperative planning device for trauma repair based on artificial intelligence, which is used for executing the preoperative planning method for trauma repair based on artificial intelligence described in the foregoing of the present application. The preoperative planning device for trauma repair based on artificial intelligence is described in detail below.

[0140] As shown in the figure, the preoperative planning device for trauma repair based on artificial intelligence comprises: Figure 7

[0141] An image acquisition module 101 is configured to acquire a medical image of a single object;

[0142] A three-dimensional construction module 102 is configured to segment a bone in the medical image according to the medical image, and establish a three-dimensional bone model;

[0143] A trauma repair module 103 is configured to repair a broken bone at a trauma position, and determine a rotation matrix of the broken bone;

[0144] A template determination module 104 is configured to determine a size and a setting position of an instrument template according to the repaired trauma position;

[0145] A template display module 105 is configured to display an image of the instrument template fitted with the bone, and adjust and output the size and the setting position of the instrument template according to an operation of an operator.

[0146] In an embodiment, the instrument template comprises a main nail module, a tail cap template, a helical blade template and a locking screw template.

[0147] In an embodiment, the three-dimensional construction module 102 is further configured to:

[0148] input the medical image into a feature extraction structure to obtain a low-level feature map and a high-level feature map; input the high-level feature map into an encoding structure for encoding processing to obtain an encoded feature map; and input the low-level feature map and the encoded feature map into a decoding structure to obtain a bone segmentation result.

[0149] In an embodiment, the three-dimensional construction module 102 is further configured to:

[0150] input the high-level feature map into a plurality of branches arranged in parallel to obtain a plurality of branch feature maps corresponding to the branches; perform global extraction and convolution processing on the plurality of branch feature maps in sequence to obtain a global feature map; and perform splicing processing on the global feature map to obtain a spliced feature map; the spliced feature map is the encoded feature map.

[0151] ​In an implementation, the three-dimensional construction module 102 is further configured to:

[0152] The low-level feature map is subjected to 1x1 convolution processing to obtain a convolution feature map; the encoded feature map is subjected to upsampling processing to obtain an upsampling feature map; the convolution feature map and the upsampling feature map are subjected to feature fusion to obtain a fusion feature map; and the fusion feature map is subjected to convolution processing and upsampling processing to obtain a bone segmentation result.

[0153] In an implementation, the trauma repair module 103 is further configured to:

[0154] The proximal broken bone is fitted to determine a fitted femoral head center point and a femoral neck center point, and a femoral head-neck axis is determined; the proximal broken bone is a femoral head, and the distal broken bone is a femoral shaft; the distal broken bone is sampled and fitted to obtain a femoral shaft axis; a set of fracture surface points of the proximal broken bone and a set of fracture surface points of the distal broken bone are extracted; and a rotation matrix of the femoral head is determined according to the sets of fracture surface points and the femoral shaft axis and the femoral head-neck axis.

[0155] In an implementation, the trauma repair module 103 is further configured to:

[0156] An initial rotation matrix of the femoral head is generated; a neck-shaft angle is calculated according to the femoral shaft axis and the adjusted femoral head-neck axis; an anteversion angle is calculated according to the adjusted femoral head-neck axis and a reference plane; the rotation matrix is iterated until the neck-shaft angle and the anteversion angle satisfy a preset angle range; and a set distance is calculated according to the set of fracture surface points of the femoral shaft and the set of fracture surface points of the femoral head after iteration; and the rotation matrix is re-iterated until the set distance is minimized.

[0157] The above-mentioned embodiments of the trauma repair preoperative planning device based on artificial intelligence provided by the present application have a corresponding relationship with the trauma repair preoperative planning method based on artificial intelligence provided by the embodiments of the present application, so the specific contents in the device have a corresponding relationship with the trauma repair preoperative planning method, and the specific contents can be referred to the records in the trauma repair preoperative planning method, which will not be described here in detail.

[0158] The above-mentioned embodiments of the trauma repair preoperative planning device based on artificial intelligence provided by the present application have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein, based on the same inventive concept as the trauma repair preoperative planning method based on artificial intelligence provided by the embodiments of the present application.

[0159] The above describes the internal functions and structures of the trauma repair preoperative planning device based on artificial intelligence, as shown in Figure 8 In practice, the trauma repair preoperative planning device based on artificial intelligence can be realized as an electronic device, which includes a memory 301 and a processor 303.

[0160] The memory 301 can be configured to store programs.

[0161] In addition, the memory 301 can also be configured to store other various data to support operations on the electronic device. Examples of these data include instructions for any application or method operating on the electronic device, contact data, phonebook data, messages, pictures, videos, etc.

[0162] The memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0163] The processor 303 is coupled to the memory 301 and is configured to execute programs in the memory 301 for:

[0164] obtaining a medical image of a single object;

[0165] segmenting a bone in the medical image according to the medical image and establishing a three-dimensional bone model;

[0166] repairing a broken bone at a trauma site and determining a rotation matrix of the broken bone;

[0167] determining a size and a setting position of an instrument template according to the repaired trauma site;

[0168] displaying an image of the instrument template fitting the bone and adjusting and outputting the size and the setting position of the instrument template according to an operation of an operator.

[0169] In an embodiment, the instrument template includes a main nail module, a tail cap template, a helical blade template and an interlocking screw template.

[0170] In an embodiment, the processor 303 is further configured to:

[0171] input the medical image into a feature extraction structure to obtain a low-level feature map and a high-level feature map; input the high-level feature map into an encoding structure for encoding processing to obtain an encoded feature map; and input the low-level feature map and the encoded feature map into a decoding structure to obtain a bone segmentation result.

[0172] In an embodiment, the processor 303 is further configured to:

[0173] The high-level feature map is input into a plurality of branches arranged in parallel to obtain a plurality of branch feature maps; the plurality of branch feature maps are respectively subjected to global extraction and convolution processing in sequence to obtain a global feature map; the global feature map is subjected to splicing processing to obtain a spliced feature map; the spliced feature map is the encoding feature map.

[0174] In an implementation, the processor 303 is further configured to:

[0175] The low-level feature map is subjected to 1x1 convolution processing to obtain a convolution feature map; the encoding feature map is subjected to up-sampling processing to obtain an up-sampled feature map; the convolution feature map and the up-sampled feature map are subjected to feature fusion to obtain a fusion feature map; the fusion feature map is subjected to convolution processing and up-sampling processing to obtain a bone segmentation result.

[0176] In an implementation, the processor 303 is further configured to:

[0177] The proximal broken bone is fitted to determine a fitted femoral head center point and a femoral neck center point, and a femoral head-neck axis; the proximal broken bone is a femoral head, and the distal broken bone is a femoral shaft; the distal broken bone is sampled and fitted to obtain a femoral shaft axis; a set of broken surface points of the proximal broken bone and a set of broken surface points of the distal broken bone are extracted; a rotation matrix of the femoral head is determined according to the sets of broken surface points and the femoral shaft axis and the femoral head-neck axis.

[0178] In an implementation, the processor 303 is further configured to:

[0179] An initial rotation matrix of the femoral head is generated; a neck-shaft angle is calculated according to the femoral shaft axis and the adjusted femoral head-neck axis; an anteversion angle is calculated according to the adjusted femoral head-neck axis and a reference plane; the rotation matrix is iterated until the neck-shaft angle and the anteversion angle meet a preset angle range; a set distance is calculated according to the set of broken surface points of the femoral shaft and the set of broken surface points of the femoral head after iteration; the rotation matrix is re-iterated until the set distance is minimum.

[0180] In the present application, the processor is further specifically configured to perform all processes and steps of the above-mentioned preoperative planning method for trauma repair based on artificial intelligence, and the specific content can be referred to the record in the preoperative planning method for trauma repair, which will not be described here.

[0181] In the present application, Figure 8 only some components are shown schematically, and it does not mean that the electronic device only includes Figure 8 the components shown.

[0182] The electronic device provided by the embodiment has the same beneficial effects as the method adopted, run or implemented by the application program stored in the electronic device.

[0183] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. In addition, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer usable program code.

[0184] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0185] These computer program instructions can also be stored in a computer readable memory that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including instruction apparatus, which implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0186] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0187] In a typical configuration, the computing device includes one or more processors (CPU), input / output interface, network interface and memory.

[0188] Memory can include non-persistent memory, Random Access Memory (RAM), and / or non-volatile memory, such as Read Only Memory (ROM) or Flash memory, in computer readable media. Memory is an example of computer readable media.

[0189] The application also provides a computer readable storage medium corresponding to the artificial intelligence-based preoperative planning method for wound repair provided by the foregoing embodiments, and a computer program (i.e., a program product) is stored on the computer readable storage medium. When the computer program is run by a processor, the artificial intelligence-based preoperative planning method provided by any of the foregoing embodiments is executed.

[0190] Computer readable media includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer readable media does not include transitory computer readable media, such as modulated data signals and carriers.

[0191] The computer readable storage medium provided by the above embodiments of the application has the same beneficial effects as the method adopted, run or implemented by the application program stored therein, based on the same inventive concept as the artificial intelligence-based preoperative planning method for wound repair provided by the embodiments of the application.

[0192] It should be noted that in the specification provided herein, a large number of specific details are explained. However, it can be understood that the embodiments of the application can be practiced without these specific details. In some examples, well-known structures and technologies are not shown in detail in order not to obscure the understanding of the specification.

[0193] It should also be noted that the terms "comprising," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0194] The above description is merely illustrative of the application, and not restrictive. Various modifications and changes can become apparent to those skilled in the art. Incorporating any modification, equivalent substitution, improvement, etc. within the spirit and principle of the application, shall be included in the scope of the claims of the application.

Claims

1. A preoperative planning method for trauma repair based on artificial intelligence, characterized in that: include: Acquire medical images of a single object; Segmenting the bones in the medical image according to the medical image and establishing a three-dimensional bone model; Repair the broken bone at the trauma location and determine the rotation matrix of the broken bone; Determine the size and location of the instrument template based on the location of the repaired wound; Display the image of the instrument template fitting the bone, and adjust and output the size and setting position of the instrument template according to the operator's operation; The repairing of the broken bone at the trauma location and determining the rotation matrix of the broken bone include: Fitting the proximal broken bone to determine the center point of the femoral head and the center point of the femoral neck, and determining the axis of the femoral head and neck; the proximal broken bone is the femoral head, and the distal broken bone is the femoral shaft; The distal fractured bones were sampled and fitted to obtain the femoral shaft axis; Extracting the fracture surface point set of the proximal fractured bone and the fracture surface point set of the distal fractured bone; Determine the rotation matrix of the femoral head based on the fracture surface point set, the femoral shaft axis, and the femoral head and neck axis. Determining the rotation matrix of the femoral head based on the fracture surface point set and the femoral shaft axis and the femoral head neck axis includes: Generate the initial rotation matrix of the femoral head; The neck-shaft angle was calculated based on the femoral shaft axis and the adjusted femoral head-neck axis; Calculate the anteversion angle based on the adjusted femoral head-neck axis and the reference plane; Iterate the rotation matrix until the neck-shaft angle and the anteversion angle meet a preset angle range; Calculate the point set distance based on the fracture surface point set of the femoral shaft and the fracture surface point set of the iterative femoral head; The rotation matrix is ​​iterated again until the distance between the point sets is minimized.

2. The artificial intelligence-based preoperative planning method for wound repair according to claim 1, characterized in that: The instrument template includes: a main nail template, a tail cap template, a spiral blade template and an interlocking screw template.

3. The artificial intelligence-based preoperative planning method for wound repair according to claim 1 or 2, characterized in that: Segmenting bones in the medical image according to the medical image includes: Inputting the medical image into a feature extraction structure to obtain a low-level feature map and a high-level feature map; Inputting the high-level feature map into a coding structure for coding processing to obtain a coded feature map; The low-level feature map and the encoded feature map are input into a decoding structure to obtain a skeleton segmentation result.

4. The artificial intelligence-based preoperative planning method for wound repair according to claim 3, characterized in that: The step of inputting the high-level feature map into a coding structure for coding to obtain a coded feature map comprises: Inputting the high-level feature map into multiple branches arranged in parallel to obtain corresponding multiple branch feature maps; Perform global extraction and sequential convolution processing on multiple branch feature maps to obtain a global feature map; Perform splicing processing on the global feature map to obtain a spliced ​​feature map; The splicing feature map is the encoding feature map.

5. The artificial intelligence-based preoperative planning method for wound repair according to claim 3, characterized in that: The step of inputting the low-level feature map and the encoded feature map into a decoding structure to obtain a skeleton segmentation result comprises: Perform 1×1 convolution on the low-level feature map to obtain the convolution feature map; Upsampling the encoded feature map to obtain an upsampled feature map; Perform feature fusion on the convolution feature map and the up-sampled feature map to obtain a fused feature map; The fused feature map is convolved and up-sampled to obtain the skeleton segmentation result.

6. An artificial intelligence-based preoperative planning device for wound repair, characterized in that: include: an image acquisition module for acquiring a medical image of a single object; a three-dimensional construction module, configured to segment bones in the medical image and establish a three-dimensional bone model based on the medical image; A trauma repair module is used to repair the broken bone at the trauma location and determine the rotation matrix of the broken bone; A template determination module, which is used to determine the size and setting position of the instrument template according to the location of the repaired wound; The template display module is used to display the image of the instrument template and the bone fitting, and adjust and output the size and setting position of the instrument template according to the operator's operation; The wound repair module is also used to: Fitting the proximal broken bone to determine the center point of the femoral head and the center point of the femoral neck, and determining the axis of the femoral head and neck; the proximal broken bone is the femoral head, and the distal broken bone is the femoral shaft; The distal fractured bones were sampled and fitted to obtain the femoral shaft axis; Extracting the fracture surface point set of the proximal fractured bone and the fracture surface point set of the distal fractured bone; Determine the rotation matrix of the femoral head based on the fracture surface point set, the femoral shaft axis, and the femoral head and neck axis. Determining the rotation matrix of the femoral head based on the fracture surface point set and the femoral shaft axis and the femoral head neck axis includes: Generate the initial rotation matrix of the femoral head; The neck-shaft angle was calculated based on the femoral shaft axis and the adjusted femoral head-neck axis; Calculate the anteversion angle based on the adjusted femoral head-neck axis and the reference plane; Iterate the rotation matrix until the neck-shaft angle and the anteversion angle meet a preset angle range; Calculate the point set distance based on the fracture surface point set of the femoral shaft and the fracture surface point set of the iterative femoral head; The rotation matrix is ​​iterated again until the distance between the point sets is minimized.

7. An electronic device, characterized in that: include: memory and processor; The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program to: Acquire medical images of a single object; Segmenting the bones in the medical image according to the medical image and establishing a three-dimensional bone model; Repair the broken bone at the trauma location and determine the rotation matrix of the broken bone; Determine the size and location of the instrument template based on the location of the repaired wound; Display the image of the instrument template fitting the bone, and adjust and output the size and setting position of the instrument template according to the operator's operation; The repairing of the broken bone at the trauma location and determining the rotation matrix of the broken bone include: Fitting the proximal broken bone to determine the center point of the femoral head and the center point of the femoral neck, and determining the axis of the femoral head and neck; the proximal broken bone is the femoral head, and the distal broken bone is the femoral shaft; The distal fractured bones were sampled and fitted to obtain the femoral shaft axis; Extracting the fracture surface point set of the proximal fractured bone and the fracture surface point set of the distal fractured bone; Determine the rotation matrix of the femoral head based on the fracture surface point set, the femoral shaft axis, and the femoral head and neck axis. Determining the rotation matrix of the femoral head based on the fracture surface point set and the femoral shaft axis and the femoral head neck axis includes: Generate the initial rotation matrix of the femoral head; The neck-shaft angle was calculated based on the femoral shaft axis and the adjusted femoral head-neck axis; Calculate the anteversion angle based on the adjusted femoral head-neck axis and the reference plane; Iterate the rotation matrix until the neck-shaft angle and the anteversion angle meet a preset angle range; Calculate the point set distance based on the fracture surface point set of the femoral shaft and the fracture surface point set of the iterative femoral head; The rotation matrix is ​​iterated again until the distance between the point sets is minimized.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the artificial intelligence-based preoperative planning method for wound repair as described in any one of claims 1 to 5.

Images