Wound repair preoperative planning method and device based on artificial intelligence

Abstract

The invention provides a wound repair preoperative planning method and device based on artificial intelligence. The method comprises the following steps: acquiring a medical image of a single object; according to the medical image, segmenting a skeleton in the medical image, and establishing a skeleton three-dimensional model; repairing the broken bone at the wound position, and determining a rotation matrix of the broken bone; according to the repaired wound position, the size and the setting position of an instrument template are determined; and displaying the fitting image of the instrument template and the skeleton, and adjusting and outputting the size and the setting position of the instrument template according to the operation of an operator. According to the method, the three-dimensional model of the bone is constructed through the preoperative medical image, so that the broken bone at the traumatic fracture position is repaired, selection and setting of the fixed instrument after repair are determined, and powerful support in the aspects of reduction and fixation can be provided for the traumatic fracture through the preoperative planning.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and more specifically, to a method and device for preoperative planning of trauma repair based on artificial intelligence. Background Art

[0002] Fractures are common orthopedic diseases. Fractures can be caused by traumatic factors, such as being hit by external forces, falling from a height, etc. These fractures caused by trauma are called traumatic fractures.

[0003] Preoperative planning for traumatic fractures can provide strong support for reduction and fixation of traumatic fractures, but there is currently a lack of specific plans in this regard. Summary of the invention

[0004] The problem addressed by this application is the lack of preoperative planning for traumatic fractures.

[0005] To solve the above problems, the first aspect of the present application provides a preoperative planning method for trauma repair based on artificial intelligence, comprising:

[0006] Acquire medical images of a single object;

[0007] According to the medical image, the bones in the medical image are segmented and a three-dimensional bone model is established;

[0008] Repair the broken bones at the trauma location and determine the rotation matrix of the broken bones;

[0009] Determine the size and location of the instrument template based on the location of the repaired wound;

[0010] The image of the instrument template fitting to the bone is displayed, and the size and setting position of the instrument template are adjusted and output according to the operator's operation.

[0011] The second aspect of the present application provides an artificial intelligence-based preoperative planning device for trauma repair, which comprises:

[0012] An image acquisition module, which is used to acquire a medical image of a single object;

[0013] A three-dimensional construction module, which is used to segment the bones in the medical image and establish a three-dimensional bone model according to the medical image;

[0014] A trauma repair module, which is used to repair the broken bones at the trauma location and determine the rotation matrix of the broken bones;

[0015] A template determination module, which is used to determine the size and setting position of the instrument template according to the location of the repaired wound;

[0016] The template display module is used to display the image of the instrument template fitting with the bone, and adjust and output the size and setting position of the instrument template according to the operator's operation.

[0017] A third aspect of the present application provides an electronic device, comprising: a memory and a processor;

[0018] The memory is used to store programs;

[0019] The processor, coupled to the memory, is configured to execute the program to:

[0020] Acquire medical images of a single object;

[0021] According to the medical image, the bones in the medical image are segmented and a three-dimensional bone model is established;

[0022] Repair the broken bones at the trauma location and determine the rotation matrix of the broken bones;

[0023] Determine the size and location of the instrument template based on the location of the repaired wound;

[0024] The image of the instrument template fitting to the bone is displayed, and the size and setting position of the instrument template are adjusted and output according to the operator's operation.

[0025] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the above-mentioned artificial intelligence-based preoperative planning method for trauma repair.

[0026] In this application, a three-dimensional model of the bone is constructed through preoperative medical images, so that the broken bones at the traumatic fracture site can be repaired, and the selection and setting of the fixation device after the repair can be determined. Through this preoperative planning, strong support can be provided for the reduction and fixation of the traumatic fracture. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flowchart of a preoperative planning method for wound repair according to an embodiment of the present application;

[0028] Figure 2 A model architecture diagram of bone segmentation in a preoperative planning method for wound repair according to an embodiment of the present application;

[0029] Figure 3 It is an architecture diagram of the dilated convolution of the preoperative planning method for wound repair according to an embodiment of the present application;

[0030] Figure 4 It is an architecture diagram of global extraction of the preoperative planning method for wound repair according to an embodiment of the present application;

[0031] Figure 5 A flowchart of a preoperative planning method for wound repair according to an embodiment of the present application;

[0032] Figure 6 It is a schematic diagram showing a method for preoperative planning of wound repair according to an embodiment of the present application;

[0033] Figure 7 is a structural block diagram of a preoperative planning device for wound repair according to an embodiment of the present application;

[0034] Figure 8 is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below in conjunction with the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0036] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0037] In response to the above problems, the present application provides a new artificial intelligence-based preoperative planning scheme for trauma repair, which can actively screen the chest X-rays of any subject through an artificial neural network chest screening model, thereby utilizing the universality of the chest X-ray spectrum to solve the current problem of a small osteoporosis screening range.

[0038] The present application embodiment provides a preoperative planning method for wound repair based on artificial intelligence, and the specific scheme of the method is as follows: Figure 1-Figure 4 As shown, the method can be performed by an artificial intelligence-based preoperative planning device for wound repair, which can be integrated into electronic devices such as computers, servers, computers, server clusters, and data centers. Figure 1 As shown, it is a flow chart of a preoperative planning method for wound repair based on artificial intelligence according to an embodiment of the present application; wherein the preoperative planning method for wound repair based on artificial intelligence includes:

[0039] S101, acquiring a medical image of a single object;

[0040] In the present application, the medical image is a CT image, an X-ray image or an MRI image.

[0041] S102, segmenting bones in the medical image according to the medical image, and establishing a three-dimensional bone model;

[0042] S103, repairing the broken bone at the trauma position and determining the rotation matrix of the broken bone;

[0043] In the present application, the trauma is an intertrochanteric fracture, and the trauma location is the proximal femur.

[0044] In the present application, the rotation matrix of a broken bone refers to the rotation matrix through which one of the broken bones in the injury can achieve accurate restoration of shape with the other broken bone.

[0045] S104, determining the size and setting position of the instrument template according to the repaired wound position;

[0046] In one embodiment, the instrument template includes: a main nail module, a tail cap template, a spiral blade template and a locking screw template.

[0047] Main nail: Usually refers to the main nail used for fracture fixation, designed to provide stability and support.

[0048] Tail Cap: Used to determine and manufacture the tail design of the nail to ensure its compatibility with bone or other structures.

[0049] Spiral Blade: Designed for the shape and size of a spiral blade, it is often used in orthopedic surgery to ensure accuracy and efficiency in cutting.

[0050] Locking Screws: Locking screws are designed to provide better support and stability when fixing fractures.

[0051] S105, displaying an image of the instrument template fitting with the bone, and adjusting and outputting the size and setting position of the instrument template according to the operator's operation.

[0052] In this application, a three-dimensional model of the bone is constructed through preoperative medical images, so that the broken bones at the traumatic fracture site can be repaired, and the selection and setting of the fixation device after the repair can be determined. Through this preoperative planning, strong support can be provided for the reduction and fixation of the traumatic fracture.

[0053] In this way, in the actual operation, the operator can align the two ends of the broken bone and then fix the instrument according to the instrument template and setting position planned before the operation to achieve accurate repair of the fracture trauma.

[0054] In one embodiment, in combination Figure 2 As shown, according to the medical image, the bones in the medical image are segmented, including:

[0055] Inputting the medical image into a feature extraction structure to obtain a low-level feature map and a high-level feature map;

[0056] Inputting the high-level feature map into a coding structure for coding processing to obtain a coded feature map;

[0057] The low-level feature map and the encoded feature map are input into a decoding structure to obtain a skeleton segmentation result.

[0058] In one embodiment, in combination Figure 2 As shown, the high-level feature map is input into the encoding structure for encoding processing to obtain an encoded feature map, including:

[0059] Inputting the high-level feature graph into multiple branches arranged in parallel to obtain corresponding multiple branch feature graphs;

[0060] Perform global extraction and sequential convolution processing on multiple branch feature maps to obtain a global feature map;

[0061] Perform splicing processing on the global feature map to obtain a spliced ​​feature map;

[0062] The splicing feature map is the encoding feature map.

[0063] In one embodiment, in combination Figure 2 As shown, the low-level feature map and the encoded feature map are input into a decoding structure to obtain a skeleton segmentation result, including:

[0064] Perform 1×1 convolution on the low-level feature map to obtain a convolution feature map;

[0065] Upsampling the encoded feature map to obtain an upsampled feature map;

[0066] Perform feature fusion on the convolution feature map and the up-sampled feature map to obtain a fused feature map;

[0067] The fused feature map is convolved and upsampled to obtain the bone segmentation result.

[0068] In this application, combined Figure 2As shown, it can be seen that the feature extraction structure is a hole convolution, which outputs a low-level feature map to the decoding structure and outputs multiple high-level feature maps to the encoding structure; multiple high-level feature maps form a corresponding number of branches in the encoding structure, wherein the first branch is spliced ​​after 1×1 convolution, global extraction, multiple convolution processing, and 1×1 convolution; the second branch is spliced ​​after 3×3 convolution, global extraction, multiple convolution processing, and 1×1 convolution; the third branch is spliced ​​after 3×3 convolution, global extraction, multiple convolution processing, and split processing; the fourth branch is spliced ​​after 3×3 convolution, global extraction, multiple convolution processing, and split processing; the fifth branch is spliced ​​after pooling processing, global extraction, multiple convolution processing, and 1×1 convolution processing.

[0069] Among them, the sizes of the high-level feature maps of the inputs of the first branch to the fifth branch increase successively. During the processing of each branch, the size of the feature map is kept unchanged until the feature maps of different sizes are converted into feature maps of the same size through 1×1 convolution and splitting processing.

[0070] Among them, the feature map sizes of the third branch and the fourth branch are integer multiples of the second branch; by splitting, they are split into multiple feature maps of the size of the second branch.

[0071] Among them, the feature map size of the first branch is smaller than that of the second branch, and it is adjusted to the feature map size of the second branch through 1×1 convolution.

[0072] Among them, the feature map size of the fifth branch is larger than that of the second branch, and it is adjusted to the feature map size of the second branch through 1×1 convolution.

[0073] In one embodiment, in combination Figure 4 As shown, the process of global extraction is:

[0074] Input the input feature map into a 1×1 convolution layer set in parallel to obtain a first extraction map, a second extraction map, and a third extraction map respectively; wherein the first extraction map and the third extraction map are in HW×N format, and the second extraction map is in N×HW format;

[0075] Multiply the first extracted image and the second extracted image to obtain a multiplied feature image;

[0076] Perform softmax-like processing on the multiplied feature map to obtain the multiplication coefficient;

[0077] Multiplying the multiplication coefficient with the third extraction map to obtain a coefficient multiplication map;

[0078] Add the coefficient multiplication map and the input feature map to obtain the globally extracted output feature map.

[0079] In the present application, the input feature map is divided into three branches for 1×1 convolution processing. The dimensions of the first branch and the third branch are adjusted to HW×N format, and the dimension of the second branch is N×HW format; then the first branch is multiplied by the third branch to obtain a feature map with a dimension of HW×HW, and a coefficient is obtained through softmax-like processing; the coefficient is multiplied with the third branch, and the multiplied feature map is added to the input feature map to obtain an output feature map.

[0080] In this application, the dimensions of the first branch and the second branch are reversed through 1×1 convolution, thereby realizing multiplication between feature maps; through this multiplication, long-range dependencies are captured; by setting coefficients, the dependencies are embedded in the feature maps; and on this basis, the original input is added and a residual connection is used to embed the global extraction into the model without destroying the parameters.

[0081] In this application, long-range dependencies between features are captured through global extraction, so that the model can extract global features, making up for the defect that the current model mainly extracts local features.

[0082] In one implementation, the activation function of the SoftMax-like operation is:

[0083]

[0084] Among them, SoftMax1(c) is the activation function, c i 、x j is the element in the input vector, i and j are the element numbers.

[0085] In this application, SoftMax is a mathematical function that is usually used to convert a set of arbitrary real numbers into real numbers representing probability distributions. It is essentially a normalization function that can convert a set of arbitrary real values ​​into probability values ​​between [0,1], because SoftMax converts them to values ​​between 0 and 1, so they can be interpreted as probabilities. If one of the inputs is small or negative, SoftMax turns it into a small probability, and if the input is large, it turns it into a large probability, but it will always remain between 0 and 1.

[0086] However, for the standard SoftMax function, since the input is mapped between 0 and 1, and the sum of all output values ​​is 1, this means that even if some input values ​​are very small, they will have a non-zero output value after being processed by the SoftMax function. This will also cause the noise to be amplified, resulting in the final output result being affected by greater noise.

[0087] In this application, a 1 is added to the denominator of the SoftMax-like function; this change means that when the input value is very small, their output value can be closer to zero. This allows the corresponding output to tend to zero when there is no valuable information to add, thus greatly reducing unnecessary noise.

[0088] In one embodiment, in combination Figure 3 As shown, the medical image is input into a feature extraction structure to obtain a low-level feature map and a high-level feature map, including:

[0089] The feature extraction structure comprises a plurality of branches, which respectively perform dilated convolution on the input medical image; wherein, the first branch performs five 3×3 convolutions on the medical image, and the five 3×3 convolutions are dilated convolutions with dilation rates of 1, 2, 3, 1, and 2, respectively; the second branch performs four 3×3 convolutions on the medical image, and the four 3×3 convolutions are dilated convolutions with dilation rates of 1, 2, 5, and 1, respectively; the third branch performs three 3×3 convolutions on the medical image, and the three 3×3 convolutions are dilated convolutions with dilation rates of 1, 2, and 4, respectively; the fourth branch performs two 3×3 convolutions on the medical image, and the two 3×3 convolutions are dilated convolutions with dilation rates of 1 and 2, respectively; the fifth branch performs one 3×3 convolution on the medical image, and the convolution is a dilated convolution with a dilation rate of 1.

[0090] Among them, the outputs of the five branches are the high-level feature maps; the output of one branch among the third branch, the fourth branch and the fifth branch is selected as the low-level feature map.

[0091] In the present application, five branches with gradually decreasing numbers of dilated convolutions are set to generate five feature maps of different sizes whose image sizes gradually increase from small to large, thereby achieving differentiation in image size.

[0092] In this application, by setting a dilated convolution, the receptive fields of different branches are increased while other conditions remain unchanged, thereby achieving comprehensive acquisition of multiple receptive fields.

[0093] In the present application, by sequentially setting a plurality of dilated convolutions and setting the arrangement of the dilation rates described above, all elements within the receptive field can be utilized when the receptive field is expanded; thereby avoiding information loss caused by utilizing only part of the elements within the receptive field and avoiding discarding local correlations.

[0094] In the present application, by setting multiple dilated convolutions in sequence, receptive fields of different sizes are obtained through the convolution operation, and multi-scale information can be obtained; and by continuously setting dilated convolutions, a context module is formed before and after the dilated convolution, thereby aggregating multi-scale context information.

[0095] In one embodiment, in combination Figure 5 As shown, the step S103, repairing the broken bone at the trauma position and determining the rotation matrix of the broken bone, includes:

[0096] S301, fitting the proximal broken bone, determining the fitted center point of the femoral head and the center point of the femoral neck, and determining the axis of the femoral head and neck; the proximal broken bone is the femoral head, and the distal broken bone is the femoral shaft;

[0097] The proximal fracture is identified and the femoral head part (which may have defects) and the femoral neck part (which may have defects) are segmented; the surface point cloud data of the complete part of the femoral head part is extracted, and the coordinates and radius of the center point of the sphere are obtained using a spherical fitting algorithm (such as the least squares method), and the coordinates of the center point are the center point of the femoral head; the surface point cloud data of a complete section of the femoral neck part is extracted, and the central axis of the cylinder is determined using a cylindrical fitting algorithm (such as RANSAC fitting based on point cloud), and a point on the central axis is selected as the center point of the femoral neck; the center point of the femoral head and the center point of the femoral neck are connected to obtain the axis line of the femoral head and neck.

[0098] S302, sampling and fitting the distal fractured bones to obtain the axis line of the femoral shaft;

[0099] Select sampling points evenly on the surface of the broken bone. These points can be obtained in the following ways: Random sampling: randomly select a certain number of points from the surface of the broken bone. Uniform sampling: select sampling points on the surface of the broken bone according to certain rules (such as equal spacing). Feature point sampling: select sampling points based on the characteristics of the bone, such as edges or areas with large shape changes. Principal axis fitting: use principal component analysis (PCA): perform PCA analysis on the sampling points, and the principal axis is the direction of the first principal component. Curve fitting: if the femur is obviously bent, a polynomial or spline curve can be used to fit the sampling points. Ellipse fitting: for some nonlinear situations, the least squares method can be used to fit an ellipse to the broken part of the femur to obtain the axis line. Through the above fitting, a comprehensive judgment is made to obtain the axis line of the femoral shaft.

[0100] S303, extracting a fracture surface point set of the proximal fractured bone and a fracture surface point set of the distal fractured bone;

[0101] The broken bone part can be identified first; then the fracture surface can be fitted in the point cloud of the broken bone using the RANSAC algorithm or the least squares method. If the fracture surface is not a simple plane shape, the fracture surface can be obtained using a surface fitting method (such as quadratic surface fitting or B-spline fitting); the distance from all points of the identified broken bone part to the fitting plane is calculated, a threshold is set, and all point sets within the threshold are taken as the fracture surface point set.

[0102] Thus, the fracture surface point set of the proximal broken bone and the fracture surface point set of the distal broken bone can be obtained respectively.

[0103] S304, determining the rotation matrix of the femoral head according to the fracture surface point set, the femoral shaft axis, and the femoral head and neck axis.

[0104] In the present application, the broken bone is analyzed and identified to determine the fracture surface point set at the broken bone and the axis line of the femoral shaft and the axis line of the femoral head and neck, and based on this, the rotation matrix of the femoral head is determined to achieve positional repair of the broken bone trauma.

[0105] In one embodiment, determining the rotation matrix of the femoral head according to the fracture surface point set and the femoral shaft axis and the femoral head and neck axis includes:

[0106] Generate the initial rotation matrix of the femoral head;

[0107] The neck-shaft angle was calculated based on the femoral shaft axis and the adjusted femoral head-neck axis;

[0108] The anteversion angle was calculated based on the adjusted femoral head and neck axis and the reference plane;

[0109] Iterate the rotation matrix until the neck-shaft angle and the forward tilt angle meet a preset angle range;

[0110] Calculate the point set distance according to the fracture surface point set of the femoral shaft and the fracture surface point set of the iterated femoral head;

[0111] The rotation matrix is ​​iterated again until the distance between the point sets is minimized.

[0112] In the present application, the neck-shaft angle and the anteversion angle are first constrained within a preset angle range by adjusting the rotation matrix; then, under the premise of constraining the neck-shaft angle and the anteversion angle, the rotation matrix is ​​fine-tuned to minimize the point set distance. At this time, the fracture surface of the femoral shaft and the fracture surface of the femoral head should fit exactly, thereby achieving the position repair of the femoral fracture.

[0113] In the present application, the distance between the fracture surfaces of the femoral head and the femoral shaft is calculated to verify whether the splicing result meets the alignment splicing standard; the neck-shaft angle and the anteversion angle are calculated to verify whether the splicing result meets the alignment splicing standard; thereby achieving accurate alignment splicing and alignment splicing of the femur and realizing accurate repair.

[0114] In the present application, the neck-shaft angle refers to the angle formed by the axis of the femoral shaft and the axis of the femoral head and neck.

[0115] In the present application, the anteversion angle refers to the angle of outward rotation of the femoral neck relative to the femoral shaft, that is, the angle formed by the projection of the femoral head and neck axis and the frontal plane on the axial plane.

[0116] In the present application, the standard position information of the anatomical posture of the femur can be obtained, and then by setting the femoral head to the standard position of the femoral head and the femoral shaft to the standard position of the femoral shaft (the femoral shaft can be directly located at the standard position through preliminary fine-tuning, so that only the femoral head is set during the actual setting), the rotation matrix of the femoral head relative to the femoral shaft is calculated as the initial matrix.

[0117] In the present application, the femoral head-neck axis in the iteration process can be rotated and transformed according to the desired neck-shaft angle (e.g., 125° to 135°);

[0118] In this application, the point set distance is the Euclidean distance between two fracture surface point sets. The ICP (Iterative Closest Point) algorithm can be used to find the point pair set with the minimum distance matching to further optimize the rotation matrix.

[0119] In the present application, the corresponding functional relationship between the rotation matrix and the point set distance can be calculated, and the rotation matrix that minimizes the point set distance can be found through iterative optimization (such as gradient descent or nonlinear optimization algorithm). In each iteration, the rotation matrix is ​​updated, the new neck-shaft angle and forward inclination angle are calculated, and the point set distances are compared until they converge to the minimum value.

[0120] In one embodiment, repairing the broken bone at the trauma location and determining the rotation matrix of the broken bone also includes identifying key points of the femur and acetabulum. These key points may include:

[0121] Femoral head center: The geometric center of the spherical surface of the femoral head, which is part of the ball and socket joint of the hip.

[0122] Greater Trochanter Apex: The highest point of the greater trochanter of the femur.

[0123] Lesser Trochanter Apex: The highest point of the lesser trochanter of the femur.

[0124] Head-Neck Junction: The transition area between the femoral head and the femoral neck.

[0125] Midpoint of Femoral Shaft: The middle position of the femoral shaft.

[0126] Acetabular Center: The geometric center of the acetabulum.

[0127] Superior Rim of Acetabulum: The upper edge of the acetabulum.

[0128] Anterior Rim of Acetabulum: The front edge of the acetabulum.

[0129] Posterior Rim of Acetabulum: The posterior edge of the acetabulum.

[0130] Obturator Foramen: The obturator foramen is the anatomical structure located below the acetabulum.

[0131] Acetabular Fossa: The concave portion of the acetabulum, the acetabular fossa.

[0132] In one embodiment, S105, an image of the instrument template and the bone fitting is displayed, and the size and setting position of the instrument template are adjusted and output according to the operator's operation:

[0133] like Figure 6 As shown, it is an image showing the prosthesis template (main nail module, tail cap template, spiral blade template and locking screw template) fitting with the bone, which can be adjusted according to the operator's operation.

[0134] In this application, in the image showing the fitting of the instrument template to the bone, the process of obtaining the initial planned positions of the main nail module, the tail cap template, the spiral blade template and the locking screw template is as follows:

[0135] Main nail planning: Determine the best insertion path for the main nail between the proximal and distal femoral fractures. Usually, the main nail is inserted through the opening of the greater trochanter or the lower edge of the acetabulum. Determine the length and diameter of the main nail so that it can span the entire fracture area and provide adequate support. Align the femoral axis: Align the centerline of the main nail with the femoral axis to ensure that the main nail can pass through the medullary cavity without destroying the bone anatomy.

[0136] Tail cap planning: The tail cap is usually installed at the tail end of the main nail and should be completely aligned with the main nail. Make sure that the position of the tail cap does not cause damage to soft tissues (such as tendons or nerves). Choose a tail cap with the appropriate diameter and length according to the anatomical structure of the bone to avoid the tail cap protruding from the bone or entering the joint cavity.

[0137] Planning of spiral blade: Choose a suitable insertion path in the femoral head and neck area to ensure that the spiral blade can fully enter the femoral head and cover a larger bone area. The insertion angle should be consistent with the axis of the femoral neck to avoid damaging the articular surface of the femoral head. Choose the length of the spiral blade according to the size of the femoral head and the length of the femoral neck. Usually, ensure that the blade head is at least 5 mm away from the articular surface of the femoral head to avoid cutting the articular surface.

[0138] Positioning of interlocking screws: Install one or more interlocking screws at the distal end of the main nail. The screw position should be precisely aligned with the locking hole on the main nail.

[0139] The embodiment of the present application provides an artificial intelligence-based preoperative planning device for trauma repair, which is used to execute the artificial intelligence-based preoperative planning method for trauma repair described in the above content of the present application. The artificial intelligence-based preoperative planning device for trauma repair is described in detail below.

[0140] like Figure 7 As shown, the artificial intelligence-based preoperative planning device for trauma repair comprises:

[0141] An image acquisition module 101, which is used to acquire a medical image of a single object;

[0142] A three-dimensional construction module 102, which is used to segment the bones in the medical image according to the medical image and establish a three-dimensional model of the bones;

[0143] A wound repair module 103 is used to repair the broken bone at the wound position and determine the rotation matrix of the broken bone;

[0144] A template determination module 104, which is used to determine the size and setting position of the instrument template according to the location of the repaired wound;

[0145] The template display module 105 is used to display the image of the instrument template and the skeleton fitting, and adjust and output the size and setting position of the instrument template according to the operator's operation.

[0146] In one embodiment, the instrument template includes: a main nail module, a tail cap template, a spiral blade template and a locking screw template.

[0147] In one embodiment, the three-dimensional construction module 102 is further used to:

[0148] The medical image is input into a feature extraction structure to obtain a low-level feature map and a high-level feature map; the high-level feature map is input into a coding structure for coding processing to obtain a coded feature map; the low-level feature map and the coded feature map are input into a decoding structure to obtain a bone segmentation result.

[0149] In one embodiment, the three-dimensional construction module 102 is further used to:

[0150] The high-level feature map is input into multiple branches arranged in parallel to obtain corresponding multiple branch feature maps; the multiple branch feature maps are globally extracted and sequentially convolved to obtain a global feature map; the global feature map is spliced ​​to obtain a spliced ​​feature map; the spliced ​​feature map is the encoded feature map.

[0151] In one embodiment, the three-dimensional construction module 102 is further used to:

[0152] The low-level feature map is subjected to 1×1 convolution processing to obtain a convolution feature map; the encoded feature map is upsampled to obtain an upsampled feature map; the convolution feature map and the upsampled feature map are subjected to feature fusion to obtain a fused feature map; the fused feature map is subjected to convolution processing and upsample processing to obtain the skeleton segmentation result.

[0153] In one embodiment, the wound repair module 103 is further configured to:

[0154] The proximal broken bone is fitted to determine the fitted center point of the femoral head and the center point of the femoral neck, and the axis line of the femoral head and neck; the proximal broken bone is the femoral head, and the distal broken bone is the femoral shaft; the distal broken bone is sampled and fitted to obtain the axis line of the femoral shaft; the fracture surface point set of the proximal broken bone and the fracture surface point set of the distal broken bone are extracted; according to the fracture surface point set and the axis line of the femoral shaft and the axis line of the femoral head and neck, the rotation matrix of the femoral head is determined.

[0155] In one embodiment, the wound repair module 103 is further configured to:

[0156] Generate an initial rotation matrix of the femoral head; calculate the neck-shaft angle based on the femoral shaft axis and the adjusted femoral head-neck axis; calculate the anteversion angle based on the adjusted femoral head-neck axis and the reference plane; iterate the rotation matrix until the neck-shaft angle and the anteversion angle meet a preset angle range; calculate the point set distance based on the fracture surface point set of the femoral shaft and the iterated fracture surface point set of the femoral head; re-iterate the rotation matrix until the point set distance is minimized.

[0157] The artificial intelligence-based preoperative planning device for trauma repair provided in the above-mentioned embodiment of the present application has a corresponding relationship with the artificial intelligence-based preoperative planning method for trauma repair provided in the embodiment of the present application. Therefore, the specific content in the device has a corresponding relationship with the preoperative planning method for trauma repair. The specific content can refer to the records in the preoperative planning method for trauma repair, and will not be repeated in this application.

[0158] The artificial intelligence-based preoperative planning device for trauma repair provided in the above-mentioned embodiments of the present application and the artificial intelligence-based preoperative planning method for trauma repair provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0159] The above describes the internal functions and structure of the artificial intelligence-based preoperative planning device for trauma repair, such as Figure 8 As shown, in practice, the artificial intelligence-based preoperative planning device for trauma repair can be implemented as an electronic device, including: a memory 301 and a processor 303.

[0160] The memory 301 may be configured to store programs.

[0161] In addition, the memory 301 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0162] The memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0163] The processor 303 is coupled to the memory 301 and is configured to execute a program in the memory 301 to:

[0164] Acquire medical images of a single object;

[0165] According to the medical image, the bones in the medical image are segmented and a three-dimensional bone model is established;

[0166] Repair the broken bones at the trauma location and determine the rotation matrix of the broken bones;

[0167] Determine the size and location of the instrument template based on the location of the repaired wound;

[0168] The image of the instrument template fitting to the bone is displayed, and the size and setting position of the instrument template are adjusted and output according to the operator's operation.

[0169] In one embodiment, the instrument template includes: a main nail module, a tail cap template, a spiral blade template and a locking screw template.

[0170] In one implementation, the processor 303 is further configured to:

[0171] The medical image is input into a feature extraction structure to obtain a low-level feature map and a high-level feature map; the high-level feature map is input into a coding structure for coding processing to obtain a coded feature map; the low-level feature map and the coded feature map are input into a decoding structure to obtain a bone segmentation result.

[0172] In one implementation, the processor 303 is further configured to:

[0173] The high-level feature map is input into multiple branches arranged in parallel to obtain corresponding multiple branch feature maps; the multiple branch feature maps are globally extracted and sequentially convolved to obtain a global feature map; the global feature map is spliced ​​to obtain a spliced ​​feature map; the spliced ​​feature map is the encoded feature map.

[0174] In one implementation, the processor 303 is further configured to:

[0175] The low-level feature map is subjected to 1×1 convolution processing to obtain a convolution feature map; the encoded feature map is upsampled to obtain an upsampled feature map; the convolution feature map and the upsampled feature map are subjected to feature fusion to obtain a fused feature map; the fused feature map is subjected to convolution processing and upsample processing to obtain the skeleton segmentation result.

[0176] In one implementation, the processor 303 is further configured to:

[0177] The proximal broken bone is fitted to determine the fitted center point of the femoral head and the center point of the femoral neck, and the axis line of the femoral head and neck; the proximal broken bone is the femoral head, and the distal broken bone is the femoral shaft; the distal broken bone is sampled and fitted to obtain the axis line of the femoral shaft; the fracture surface point set of the proximal broken bone and the fracture surface point set of the distal broken bone are extracted; according to the fracture surface point set and the axis line of the femoral shaft and the axis line of the femoral head and neck, the rotation matrix of the femoral head is determined.

[0178] In one implementation, the processor 303 is further configured to:

[0179] Generate an initial rotation matrix of the femoral head; calculate the neck-shaft angle based on the femoral shaft axis and the adjusted femoral head-neck axis; calculate the anteversion angle based on the adjusted femoral head-neck axis and the reference plane; iterate the rotation matrix until the neck-shaft angle and the anteversion angle meet a preset angle range; calculate the point set distance based on the fracture surface point set of the femoral shaft and the iterated fracture surface point set of the femoral head; re-iterate the rotation matrix until the point set distance is minimized.

[0180] In the present application, the processor is also specifically used to execute all the processes and steps of the above-mentioned artificial intelligence-based preoperative planning method for trauma repair. The specific content can be referred to the records in the preoperative planning method for trauma repair, and will not be repeated in this application.

[0181] In this application, Figure 8 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 8 Components shown.

[0182] The electronic device provided in this embodiment is based on the same inventive concept as the artificial intelligence-based preoperative planning method for trauma repair provided in the embodiment of the present application, and has the same beneficial effects as the method adopted, run or implemented by the application program stored in it.

[0183] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0184] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0185] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0186] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0187] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0188] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0189] The present application also provides a computer-readable storage medium corresponding to the artificial intelligence-based preoperative planning method for trauma repair provided in the aforementioned embodiment, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the artificial intelligence-based preoperative planning method for trauma repair provided in any of the aforementioned embodiments.

[0190] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0191] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the artificial intelligence-based preoperative planning method for trauma repair provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application program stored therein.

[0192] It should be noted that in the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0193] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0194] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A preoperative planning method for trauma repair based on artificial intelligence, characterized in that: include: Acquire medical images of a single object; According to the medical image, the bones in the medical image are segmented and a three-dimensional bone model is established; Repair the broken bones at the trauma location and determine the rotation matrix of the broken bones; Determine the size and location of the instrument template based on the location of the repaired wound; The image of the instrument template fitting the bone is displayed, and the size and setting position of the instrument template are adjusted and output according to the operator's operation.

2. The artificial intelligence-based preoperative planning method for trauma repair according to claim 1, characterized in that: The instrument template comprises: a main nail module, a tail cap template, a spiral blade template and an interlocking screw template.

3. The artificial intelligence-based preoperative planning method for wound repair according to claim 1 or 2, characterized in that: The step of segmenting bones in the medical image according to the medical image includes: Inputting the medical image into a feature extraction structure to obtain a low-level feature map and a high-level feature map; Inputting the high-level feature map into a coding structure for coding processing to obtain a coded feature map; The low-level feature map and the encoded feature map are input into a decoding structure to obtain a skeleton segmentation result.

4. The method for preoperative planning of trauma repair based on artificial intelligence according to claim 3, characterized in that: The step of inputting the high-level feature map into a coding structure for coding to obtain a coding feature map comprises: Inputting the high-level feature graph into multiple branches arranged in parallel to obtain corresponding multiple branch feature graphs; Perform global extraction and sequential convolution processing on multiple branch feature maps to obtain a global feature map; Perform splicing processing on the global feature map to obtain a spliced ​​feature map; The splicing feature map is the encoding feature map.

5. The artificial intelligence-based preoperative planning method for wound repair according to claim 3, characterized in that: The step of inputting the low-level feature map and the encoded feature map into a decoding structure to obtain a skeleton segmentation result comprises: Perform 1×1 convolution on the low-level feature map to obtain a convolution feature map; Upsampling the encoded feature map to obtain an upsampled feature map; Perform feature fusion on the convolution feature map and the up-sampled feature map to obtain a fused feature map; The fused feature map is convolved and upsampled to obtain the bone segmentation result.

6. The artificial intelligence-based preoperative planning method for wound repair according to claim 1 or 2, characterized in that: The step of repairing the broken bone at the trauma position and determining the rotation matrix of the broken bone includes: Fitting the proximal broken bone, determining the fitted center point of the femoral head and the center point of the femoral neck, and determining the axis of the femoral head and neck; the proximal broken bone is the femoral head, and the distal broken bone is the femoral shaft; The distal fractured bones were sampled and fitted to obtain the axis of the femoral shaft; Extract the fracture surface point set of the proximal fractured bone and the fracture surface point set of the distal fractured bone; The rotation matrix of the femoral head is determined based on the fracture surface point set, the femoral shaft axis, and the femoral head and neck axis.

7. The artificial intelligence-based preoperative planning method for wound repair according to claim 6, characterized in that: Determining the rotation matrix of the femoral head according to the fracture surface point set and the axis line of the femoral shaft and the axis line of the femoral head and neck includes: Generate the initial rotation matrix of the femoral head; The neck-shaft angle was calculated based on the femoral shaft axis and the adjusted femoral head-neck axis; The anteversion angle was calculated based on the adjusted femoral head and neck axis and the reference plane; Iterate the rotation matrix until the neck-shaft angle and the forward tilt angle meet a preset angle range; Calculate the point set distance according to the fracture surface point set of the femoral shaft and the fracture surface point set of the iterated femoral head; The rotation matrix is ​​iterated again until the distance between the point sets is minimized.

8. An artificial intelligence-based preoperative planning device for trauma repair, characterized in that: include: An image acquisition module, which is used to acquire a medical image of a single object; A three-dimensional construction module, which is used to segment the bones in the medical image and establish a three-dimensional bone model according to the medical image; A trauma repair module, which is used to repair the broken bones at the trauma location and determine the rotation matrix of the broken bones; A template determination module, which is used to determine the size and setting position of the instrument template according to the location of the repaired wound; The template display module is used to display the image of the instrument template fitting with the bone, and adjust and output the size and setting position of the instrument template according to the operator's operation.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program to: Acquire medical images of a single object; According to the medical image, the bones in the medical image are segmented and a three-dimensional bone model is established; Repair the broken bones at the trauma location and determine the rotation matrix of the broken bones; Determine the size and location of the instrument template based on the location of the repaired wound; The image of the instrument template fitting the bone is displayed, and the size and setting position of the instrument template are adjusted and output according to the operator's operation.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the artificial intelligence-based preoperative planning method for trauma repair as described in any one of claims 1-7.

Images