SCFE operation nail path design method based on generative adversarial network

Abstract

According to an original preoperative disease three-dimensional CT image of an SCFE patient, data labeling is carried out to obtain three-dimensional models of the femur, the femoral head epiphyseal edge, the femoral head and the subchondral bone of the patient, through labeling of a professional doctor, the starting point and the ending point of a nail channel corresponding to each three-dimensional model are obtained, and the three-dimensional model is obtained. A straight line connecting the starting point and the end point is converted into an actual point set in a three-dimensional space through an interpolation method, and a nail path design data set is obtained; according to the generative adversarial network basic framework and the residual network design, obtaining a residual generative adversarial network of SCFE operation nail path design, and putting the nail path design data set into the network for training to obtain a generative network capable of designing a nail path according to a three-dimensional bone model and an identification network capable of identifying whether the nail path meets a doctor design standard; and generating a nail path by using the generative network, fitting a straight line by using a least square method, and intercepting to obtain the SCFE surgical nail path. The method is of great significance in simplifying the operation process and improving the operation efficiency.

Description

Technical Field

[0001] The present invention relates to a method for designing a surgical nail track for SCFE surgery based on a generative adversarial network, belonging to the field of surgical technology optimization.

Background Art

[0002] Slipped capital femoral epiphysis (SCFE) is a common hip lesion in the adolescent population, and its standard treatment method is to implant screws into the femoral head to achieve bone fixation. When designing the nail track, doctors mainly rely on personal experience and intuition. There are significant differences in the design results among different operators, lacking an objective and unified standard. At the same time, manual or semi-automatic methods require repeated adjustment and verification, with low efficiency and difficult to meet the rapid decision-making needs of emergency or complex cases. In addition, due to the difficulty of comprehensively capturing the subtle differences in the individual anatomical structures of patients by traditional methods, the accurate positioning of the nail track path is insufficient, which may affect the surgical effect. Moreover, the existing technologies fail to make full use of intelligent algorithms and cannot customize the optimal nail track design plan for each patient, restricting the development of personalized treatment.

[0003] Therefore, how to design the nail track for SCFE surgery according to the patient's own situation before surgery is an urgent problem to be solved in this field.

Summary of the Invention

[0004] In view of this, the present invention provides a method for designing a surgical nail track for SCFE surgery based on a generative adversarial network to realize the automatic design of the nail track for SCFE patients' surgery.

[0005] An embodiment of the present invention provides a method for designing a surgical nail track for SCFE surgery based on a generative adversarial network, including:

[0006] Based on the original preoperative diseased three-dimensional CT images of SCFE patients, perform data annotation to obtain three-dimensional models of the femur, femoral head epiphysis edge, femoral head, and subchondral bone of all SCFE patients. Through professional doctor annotation on the three-dimensional models, obtain the starting point and ending point of the nail track corresponding to each three-dimensional model, connect the straight line between the starting point and the ending point, and use the interpolation method to convert this straight line into an actual point set in three-dimensional space.

[0007] Based on the three-dimensional models of the femur, femoral head epiphysis edge, femoral head, and subchondral bone of all SCFE patients and the point set corresponding to each three-dimensional model, obtain a nail track design data set.

[0008] Based on the basic framework of the adversarial generative network and the residual network, design a generative network, a discriminative network, and their hyperparameters, design a loss function, and obtain a residual generative adversarial network for SCFE surgical nail track design.

[0009] Put the nail track design dataset into the residual generative adversarial network for SCFE surgical nail track design to train a generative network capable of designing SCFE surgical nail tracks based on a three-dimensional bone model and a discriminative network capable of distinguishing whether an SCFE surgical nail track meets the doctor's design criteria. Use the generative network to generate nail tracks, fit a straight line using the least squares method, and perform truncation to obtain SCFE surgical nail tracks.

[0010] In the above method, based on the original preoperative diseased three-dimensional CT images of SCFE patients, data annotation is performed to obtain three-dimensional models of the femur, femoral head epiphyseal margin, femoral head, and subchondral bone of all SCFE patients. Through professional doctor annotation on the three-dimensional models, the starting and ending points of the nail track corresponding to each three-dimensional model are obtained. For the straight line connecting the starting and ending points, using the interpolation method, this straight line is transformed into an actual point set in three-dimensional space, including

[0011] Based on the original preoperative diseased three-dimensional CT images of SCFE patients, under the guidance of professional orthopedic doctors, the present invention labels the femur, femoral head epiphyseal margin, femoral head, and subchondral bone on the CT images with different labels to obtain the extracted three-dimensional model. The doctor manually calibrates the starting and ending points of the nail track according to the patient's anatomical features and the specific conditions of the lesion area. To ensure that the actual size of the nail track is consistent with the clinical requirements, the diameter of the nail track is selected and set to 7.3 mm according to the surgical standard. Using the interpolation method, each voxel on the nail track path is calculated based on the positions of the starting and ending points.

[0012] In the above method, based on the three-dimensional models of the femur, femoral head epiphyseal margin, femoral head, and subchondral bone of all SCFE patients and the point set corresponding to each three-dimensional model, a nail track design dataset is obtained, including

[0013] Based on the three-dimensional models of the femur, femoral head epiphyseal margin, femoral head, and subchondral bone of all SCFE patients and the point set corresponding to each three-dimensional model, the clicks are saved as three-dimensional image data in NIFTI format, with the same size as the patient's CT data, ensuring the unity and standardization of the annotation data; the nail track annotation data after scale alignment and interpolation processing is paired with the bone features segmented from the corresponding patient CT data, and the two together are used as the nail track design dataset for training the model to recognize and generate nail track paths that are more in line with anatomical features.

[0014] In the above method, based on the basic framework of the adversarial generative network and the residual network, a generative network, a discriminative network, and their hyperparameters are designed, and a loss function is designed to obtain the residual generative adversarial network for SCFE surgical nail track design, including

[0015] First, some concepts in this claim are elaborated: The basic framework of the adversarial generative network consists of two mutually competing networks, namely the generative network and the discriminative network. The generative network aims to generate realistic data samples, while the discriminative network attempts to distinguish between real data and generated data; the residual network refers to the residual block in the ResNet network model. This invention patent draws on its design idea, designs Residual Module I and Residual Module II, and introduces the idea of residuals into the construction of the generative network; finally, the generative network and the discriminative network designed in this paper are jointly called the residual generative adversarial network for SCFE surgical nail track design;

[0016] According to the conditional generative adversarial network training framework, design the algorithm pseudocode process, establish two mutually competing networks, the generative network and the discriminative network, and design specific neural network architecture parameters and update strategies;

[0017] First, some concepts in this claim are elaborated: The basic framework of the adversarial generative network consists of two mutually competing networks, namely the generative network and the discriminative network. The generative network aims to generate realistic data samples, while the discriminative network attempts to distinguish between real data and generated data; the residual network refers to the residual block in the ResNet network model. This invention patent draws on its design idea, designs Residual Module I and Residual Module II, and introduces the idea of residuals into the construction of the generative network; finally, the generative network and the discriminative network designed in this paper are jointly called the residual generative adversarial network for SCFE surgical nail track design;

[0018] According to the conditional generative adversarial network training framework, design the algorithm pseudocode process, establish two mutually competing networks, the generative network and the discriminative network, and design specific neural network architecture parameters and update strategies;

[0019] Train the adversarial generative network in a semi-supervised manner. The discriminative network uses the patient's three-dimensional bone model as the conditional processing branch. The input of the discriminative object processing branch is the nail track corresponding to the three-dimensional bone model, and the output is the evaluation score; the generative network uses the patient's three-dimensional bone model as the conditional processing branch. The input of the noise processing branch is randomly generated Gaussian noise, and the output is the optimal nail track parameters; in the design of the generative network and the discriminative network, there are a total of four modules that make up the network, namely the convolutional module, the transposed convolutional module, the residual module I that does not change the size, and the residual module II that reduces the size;

[0020] The convolutional module is used for feature extraction and consists of a convolutional layer, a BN layer, and an activation layer. Among them, the convolutional kernel size is 4, the stride is 2, and the activation layer uses the Leaky ReLU function;

[0021] The transposed convolutional module is used for image reconstruction and consists of a convolutional layer, a BN layer, and an activation layer. Among them, the convolutional kernel size is 4, the stride is 2, and the activation layer uses the ReLU function;

[0022] The residual module Ⅰ consists of two convolutional layers. The kernel size of both convolutional layers is 3, and the stride is 1 for both. The output of the module is

[0023] H Ι (x) = F(x) + x

[0024] where x is the input of the module, F(x) is the output of the main branch, and H Ι (x) is the expected output of the residual module Ⅰ;

[0025] The residual module Ⅱ reduces the output size to half of the input to achieve scale change. Its main branch consists of two convolutional layers. The kernel size of the first convolutional layer is 3 and the stride is 2. The kernel size of the second convolutional layer is 3 and the stride is 1. There is a convolutional layer in the residual branch with a kernel size of 1 and a stride of 2. The output of the module is the sum of the output of the main branch and the output of the residual branch. Its calculation formula is

[0026] H ΙΙ (x) = F(x) + F′(x)

[0027] where x is the input of the module, F(x) is the output of the main branch, F′(x) is the output of the residual branch, and H ΙΙ (x) is the output of the residual module Ⅱ

[0028] Both the residual module I and the residual module II use the BN layer for normalization between convolutional layers and use the LeakyReLU activation layer;

[0029] The main task of the generation network is to generate fake samples similar to the real data, so as to "deceive" the discriminator and make it unable to distinguish the difference between the generated samples and the real samples. The conditional processing network branch of the generation network takes three-dimensional bone features as conditional input, and the noise processing network branch takes Gaussian noise as input. The nail tracks designed by professional doctors corresponding to each three-dimensional bone feature are used as real samples to learn and summarize the distribution law of nail tracks on three-dimensional bone features. The outputs of the conditional processing network branch and the noise processing network branch are merged and used as the input of the merged processing branch, and the output is to generate the optimal three-dimensional nail track according to the three-dimensional bone feature conditions, making it as close as possible to the real sample in appearance and structure;

[0030] Specifically, the hyperparameters of the generation network are shown in Table 1; in the conditional processing branch, the two designed residual modules are used to extract the fine-grained information of the three-dimensional bone structure. First, the residual module Ⅱ is used to expand the number of channels and halve the output size to achieve effective compression and transmission of information. Subsequently, the residual module Ⅰ is used for further feature extraction to ensure the retention of feature information at each scale, so that the network can better capture the key features of the bone structure;

[0031] Table 1 Generation network structure and hyperparameters

[0032]

[0033]

[0034] In the noise processing branch, Gaussian noise is used as the starting point for generation; Gaussian noise is a common type of random noise, and its distribution follows the Gaussian distribution (i.e., the normal distribution), and its probability density function is controlled by the mean μ and the variance σ, as shown below

[0035]

[0036] where x represents the noise value;

[0037] Using Gaussian noise can provide a random and diverse starting point for the generation network, allowing the model to generate a variety of different samples, and is often used in the generation network of GAN; in the present invention, Gaussian noise is gradually upsampled from 1*1*1 to 32*32*32 through a transposed convolution module, and after being added to the processing result of the conditional processing branch, they are jointly input into the merging processing branch;

[0038] In the merging processing branch, the obtained result is deconvolved to a size of 256*256*256 through a transposed convolution module. In particular, to improve the influence of three-dimensional bone features on the generated nail path, the present invention particularly introduces the processing result of conditional processing before the convolutional layer in the merging branch, so that the generation network can better utilize the anatomical information in the conditional branch, ensuring that the generated nail path parameters match the anatomical features of the femoral head more accurately, thereby optimizing the surgical plan;

[0039] The main task of the discriminator is to classify a given sample and determine whether the sample is a real data sample or a fake sample generated by the generation network; the discriminator network designed in the present invention also uses three-dimensional bone features as the input of the conditional processing network branch, and uses the corresponding nail path data as the discriminant object processing network branch. The outputs of these two network branches are merged and used as the input of the merging processing network branch. By extracting features layer by layer, the discriminator network can distinguish the details and structural features of the discriminant object in the high-dimensional space. The final output layer judges whether the discriminated nail path is a nail path that meets the doctor's design standards and can be used in clinical surgery through binary classification;

[0040] The specific discriminator network hyperparameters are shown in Table 2. The conditional processing branch of the discriminator network is the same as that of the generator network. Two types of residual modules are combined with convolutional modules to extract information from three-dimensional bone features, and the three-dimensional bone features with a size of 256*256*256 are convolved to 32*32*32. This design of the conditional processing branch that is the same as the generator network enables the generator network and the discriminator network to maintain similar feature extraction strategies when processing three-dimensional bone feature information, thereby enhancing the synergy between the two and achieving more stable training.

[0041] The discriminator object processing branch of the discriminator network uses the aforementioned convolutional module, and its hyperparameters such as the convolutional kernel size, stride, and padding are consistent with the transposed convolutional layer in the generator network to ensure that the two have the same scale when extracting features from the staple track image. This design similarity can ensure that the discriminator network captures key information corresponding to the output of the generator network when analyzing the generated samples, which helps the entire model achieve closer cooperation in adversarial training, reach a better balance in the generation and discrimination processes, and improve the overall performance of the model.

[0042] In the merging processing branch of the discriminator network, the feature maps generated by the conditional processing branch and the discriminator object processing branch are superimposed, and the superimposed features are further refined and fused through multiple convolutional modules. Finally, the output is mapped to between 0 and 1 through the Sigmoid activation layer. This design can effectively integrate information from different branches and enhance the integrity of feature expression.

[0043] Table 2 Discriminator Network Structure and Hyperparameters

[0044]

[0045]

[0046] In the generative adversarial network, the selection of the loss function is crucial for model training and the quality of the finally generated images. Binary Cross-Entropy (BCE) is used as the loss function for the residual generative adversarial network, as shown below

[0047] BCELoss = -(ylog(p) + (1 - y)log(1 - p))

[0048] where y is the true label (1 represents true, 0 represents the generated image), and p is the probability given by the discriminator, that is, the discriminator's prediction value of whether the image is a real image.

[0049] In the above method, the nail track design dataset is put into the residual generative adversarial network for SCFE surgical nail track design for training to obtain a generative network capable of designing the SCFE surgical nail track according to the three-dimensional bone model and a discriminative network capable of identifying whether the SCFE surgical nail track meets the doctor's design criteria. The generative network is used to generate the nail track. After using the least squares method to fit a straight line and performing truncation, the SCFE surgical nail track is obtained, including

[0050] Put the nail track design dataset into the residual generative adversarial network for SCFE surgical nail track design for training, set the learning rates of the trained generative network and discriminative network, obtain a generative network capable of designing the SCFE surgical nail track according to the three-dimensional bone model and a discriminative network capable of identifying whether the SCFE surgical nail track meets the doctor's design criteria. Use the generative network to generate the nail track. By taking Gaussian noise as the input of the noise processing branch and the three-dimensional bone model of the SCFE patient as the input of the conditional processing branch, the nail track result generated by the network is obtained;

[0051] Since the generated nail track is not a perfect cylindrical line segment and there are certain subtle deviations, especially when encountering complex anatomical structures or blurred tissue boundaries, the generated result may deviate from the ideal straight line form. To further improve the accuracy and stability of the nail track, the least squares method is used to perform linear fitting on the generated nail track path;

[0052] The least squares method is a statistical optimization method. By minimizing the squared error between the generated data and the fitted straight line, a fitted line with the minimum error is obtained. This method can not only effectively remove the local irregular shapes and peripheral noises in the generated result, but also ensure the directionality and consistency of the nail track, thereby obtaining a smoother and more intuitive nail track trajectory. The fitted nail track path can be regarded as the learning effect of the generative adversarial network on the doctor's prior experience and can provide an accurate guiding reference for clinicians;

[0053] First, calculate the centroid of all the points generated by the generative network

[0054]

[0055] where N is the number of all points and i is used to enumerate different points;

[0056] Transfer all the points to the coordinate system with the centroid as the origin to obtain the offset vector

[0057]

[0058] Construct a lattice

[0059]

[0060] Perform singular value decomposition on A

[0061] A = UΣV T

[0062] The vector corresponding to the third column of matrix V is the direction vector (a, b, c) of the best - fitting line, and this vector represents the largest direction in the data; finally, the line equation is obtained

[0063]

[0064] where L(t) is a point on the line, t is a parameter, and (a, b, c) is the direction vector of the line is the centroid point through which the line passes;

[0065] After the line where the nail track is located is determined, regarding the selection of the starting point and the ending point of the nail track, the present invention intercepts according to the doctor's advice and the actual surgical situation. The interception is set as follows: the starting point of the nail track is located at the boundary of the femoral structure, and the ending point of the nail track is located 2 - 3 mm away from the subchondral bone within the femoral head; after such interception, the SCFE surgical nail track is obtained.

[0066] The technical solution provided by the embodiment of the present invention can, based on the preoperative three - dimensional CT images of SCFE patients, realize the personalized design of nail track parameters, effectively treat slipped capital femoral epiphysis in adolescents, and is of great significance for solving the problem of complex traditional surgical nail track design procedures and improving surgical efficiency.

Description of the Drawings

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, without creative and laborious efforts, other drawings can also be obtained based on these drawings.

[0068] Figure 1 is a schematic flowchart of a method for designing an SCFE surgical nail track based on a generative adversarial network provided by an embodiment of the present invention;

[0069] Figure 2 is a data annotation result diagram of an embodiment of the present invention. (a - c) are displays on different levels, and (d) is a three - dimensional display;

[0070] Figure 3 is a two - dimensional and three - dimensional display diagram of the nail track of an embodiment of the present invention;

[0071] Figure 4 is a matching diagram of the nail track and bone features of an embodiment of the present invention;

[0072] Figure 5 is a convolution module diagram of an embodiment of the present invention;

[0073] Figure 6 It is the deconvolution module diagram of an embodiment of the present invention;

[0074] Figure 7 It is the residual module without changing the size of an embodiment of the present invention;

[0075] Figure 8 It is the residual module with reduced size of an embodiment of the present invention;

[0076] Figure 9 It is the structure diagram of the generator network of an embodiment of the present invention;

[0077] Figure 10 It is the structure diagram of the discriminator network of an embodiment of the present invention;

[0078] Figure 11 It is the nail track result diagram generated by the network of an embodiment of the present invention;

[0079] Figure 12 It is the final nail track display diagram of an embodiment of the present invention.

Specific Embodiment

[0080] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0081] It should be clear that the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0082] The embodiment of the present invention provides a method for designing SCFE surgical nail tracks based on a generative adversarial network. Please refer to Figure 1 , which is a flow schematic diagram of a method for designing SCFE surgical nail tracks based on a generative adversarial network provided by an example of the present invention. The method includes the following steps:

[0083] Step 101: According to the original preoperative diseased three-dimensional CT images of SCFE patients, perform data annotation to obtain three-dimensional models of the femurs, femoral head epiphyseal margins, femoral heads, and subchondral bones of all SCFE patients. Through professional doctor annotation on the three-dimensional models, obtain the starting points and ending points of the nail tracks corresponding to each three-dimensional model, connect the straight line between the starting point and the ending point, and use the interpolation method to convert this straight line into an actual point set in three-dimensional space.

[0084] According to the original preoperative diseased three-dimensional CT images of SCFE patients, under the guidance of professional orthopedic doctors, the present invention labels the femurs, femoral head epiphyseal margins, femoral heads, and subchondral bones on the CT images with different labels, such as Figure 2As shown in (a-c), the three-dimensional model after extraction is obtained, such as Figure 2 shown in (d). According to the anatomical characteristics of the patient and the specific conditions of the lesion area, the doctor manually calibrates the starting point and the ending point of the nail path. To ensure that the actual size of the nail path is consistent with the clinical requirements, the diameter of the nail path is set to 7.3 mm according to the surgical standard. Using the interpolation method, each voxel on the nail path is calculated based on the positions of the starting point and the ending point, as Figure 3 shown, where (a, b, c) are the two-dimensional displays of the nail path, and (d) is the three-dimensional display of the nail path.

[0085] Step 102: Obtain the nail path design data set according to the three-dimensional models of the femur, femoral head epiphyseal margin, femoral head, and subchondral bone of all SCFE patients and the point sets corresponding to each three-dimensional model.

[0086] According to the three-dimensional models of the femur, femoral head epiphyseal margin, femoral head, and subchondral bone of all SCFE patients and the point sets corresponding to each three-dimensional model, the clicks are saved as three-dimensional image data in NIFTI format, and its size is the same as that of the patient's CT data, ensuring the unity and standardization of the labeled data; the nail path annotation data after scale alignment and interpolation processing is paired with the bone features segmented from the corresponding patient CT data, as Figure 4 shown ((a-d) are the displays on the slices). The two are used together as the nail path design data set for training the model to recognize and generate a nail path that more conforms to the anatomical characteristics.

[0087] Step 103: Design a generator network, a discriminator network, and their hyperparameters according to the basic framework of the generative adversarial network and the residual network, design a loss function, and obtain a residual generative adversarial network for SCFE surgical nail path design.

[0088] The basic framework of the generative adversarial network consists of two mutually adversarial networks, the generator network and the discriminator network. The generator network aims to generate realistic data samples, while the discriminator network attempts to distinguish between real data and generated data; the residual network refers to the residual block in the ResNet network model. This invention patent draws on its design idea, designs residual module I and residual module II, and introduces the idea of residuals into the construction of the generator network; finally, the generator network and the discriminator network designed in this paper are collectively called the residual generative adversarial network for SCFE surgical nail path design;

[0089] Based on the training framework of the conditional generative adversarial network, design the algorithm pseudocode process, establish two mutually adversarial networks, the generator network and the discriminator network, and design the specific neural network architecture parameters and update strategies;

[0090] Train the adversarial generation network in a semi-supervised manner. The discriminative network uses the patient's three-dimensional bone model as the conditional processing branch. The input of the discriminative object processing branch is the nail track corresponding to the three-dimensional bone model, and the output is the evaluation score. The generative network uses the patient's three-dimensional bone model as the conditional processing branch. The input of the noise processing branch is randomly generated Gaussian noise, and the output is the optimal nail track parameters. In the design of the network model, there are four modules that make up the nail track design network, namely the convolutional module, the transposed convolutional module, the residual module that does not change the size (hereinafter referred to as the residual module I), and the residual module that reduces the size (hereinafter referred to as the residual module II). The present invention will introduce them in turn later;

[0091] The convolutional module is used for feature extraction. As Figure 5 shown, it is composed of a convolutional layer, a BN layer, and an activation layer. Among them, the convolutional kernel size is 4, the stride is 2, and the activation layer uses Leaky ReLU (Leaky Rectified Linear Unit);

[0092] The transposed convolutional module is used for image reconstruction. It is composed of a convolutional layer, a BN layer, and an activation layer. As Figure 6 shown, the convolutional kernel size is 4, the stride is 2, and the activation layer uses the ReLU (Rectified Linear Unit) function;

[0093] The residual module I draws on the idea of ResNet and aims to extract local region features. As Figure 7 shown, it is composed of two convolutional layers. Both convolutional layers have a convolutional kernel size of 3 and a stride of 1. The input of the module is added to the output of the module. The calculation formula is as follows

[0094] H Ι (x) = F(x) + x

[0095] where x is the input of the module, F(x) is the output of the main branch, and H Ι (x) is the expected output of the residual module I;

[0096] The residual module II reduces the output size to half of the input to achieve scale change. As Figure 8 shown, its main branch is composed of two convolutional layers. The first convolutional layer has a convolutional kernel size of 3 and a stride of 2. The second convolutional layer has a convolutional kernel size of 3 and a stride of 1. There is a convolutional layer in the residual branch with a convolutional kernel size of 1 and a stride of 2. The output of the module is the sum of the output of the main branch and the output of the residual branch. The calculation formula is as follows

[0097] H ΙΙ (x) = F(x) + F′(x)

[0098] where x is the input of the module, F(x) is the output of the main branch, F′(x) is the output of the residual branch, and H ΙΙ (x) is the output of the residual module II

[0099] For the residual module without changing the size and the residual module with reduced size, BN layers are used for normalization between convolutional layers, and Leaky ReLU activation layers are used;

[0100] The main task of the generation network is to generate fake samples similar to real data, so as to "deceive" the discriminator and make it unable to distinguish the difference between the generated samples and the real samples. The generation network designed in the present invention is as Figure 9 shown. The conditional processing network branch of the generation network takes three-dimensional bone features as conditional input, and the noise processing network branch takes Gaussian noise as input. Taking the nail tracks designed by professional doctors corresponding to each three-dimensional bone feature as real samples, learning and summarizing the distribution law of nail tracks on three-dimensional bone features, the outputs of the conditional processing network branch and the noise processing network branch are merged as the input of the merged processing branch, and taking the generation of the optimal three-dimensional nail track according to the three-dimensional bone feature conditions as the output, so that it is as close as possible to the real sample in appearance and structure;

[0101] Specific generation network hyperparameters are shown in Table 1; in the conditional processing branch, fine-grained information of the three-dimensional bone structure is extracted through the two residual modules designed above. First, the residual module II is used to expand the number of channels and halve the output size, realizing effective compression and transmission of information. Subsequently, the residual module I is used for further feature extraction to ensure the retention of feature information through each scale, so that the network can better capture the key features of the bone structure;

[0102] Table 1 Generation network network structure and hyperparameters

[0103]

[0104]

[0105] In the noise processing branch, Gaussian noise is used as the starting point for generation; Gaussian noise is a common type of random noise, and its distribution follows a Gaussian distribution (i.e., a normal distribution), and its probability density function is controlled by the mean μ and the variance σ, as follows

[0106]

[0107] where x represents the noise value;

[0108] Using Gaussian noise can provide a random and diverse starting point for the generation network, allowing the model to generate a variety of different samples, and is often used in the generation network of GANs; in the present invention, Gaussian noise is gradually upsampled from 1*1*1 to 32*32*32 through a transposed convolution module, and after being added to the processing result of the conditional processing branch, they are jointly input into the merging processing branch;

[0109] In the merging processing branch, the obtained result is deconvolved to a size of 256*256*256 through a transposed convolution module. In particular, to enhance the influence of three-dimensional bone features on the generated nail tracks, the present invention particularly introduces the processing result of conditional processing before the convolutional layer of the merging branch, enabling the generation network to better utilize the anatomical information in the conditional branch, ensuring that the generated nail track parameters more accurately match the anatomical features of the femoral head, thereby optimizing the surgical plan;

[0110] The main task of the discriminator is to classify a given sample and determine whether the sample is a real data sample or a fake sample generated by the generation network; the discriminator network designed in the present invention is as Figure 10 shown. Similarly, the three-dimensional bone features are used as the input of the conditional processing network branch, and the corresponding nail track data is used as the discriminant object processing network branch. The outputs of these two network branches are merged and used as the input of the merging processing network branch. By extracting features layer by layer, the discriminator network can distinguish the details and structural features of the discriminant object in the high-dimensional space. The final output layer determines whether the discriminated nail track meets the doctor's design standards and can be used in clinical surgery through binary classification;

[0111] Specific discriminator network hyperparameters are shown in Table 2. The conditional processing branch of the discriminator network is the same as that of the generation network, using two types of residual modules combined with convolutional modules to extract information from the three-dimensional bone features, and convolving the three-dimensional bone features with a size of 256*256*256 to 32*32*32. This design of the conditional processing branch identical to that of the generation network enables the generation network and the discriminator network to maintain similar feature extraction strategies when processing three-dimensional bone feature information, thereby enhancing the synergistic effect between the two and achieving more stable training;

[0112] The discriminant object processing branch of the discriminator network uses the aforementioned convolutional module, and its hyperparameters such as the convolutional kernel size, stride, and padding are consistent with those of the transposed convolutional layer in the generation network to ensure that they have the same scale when extracting features from the top nail track images; this design similarity can ensure that the discriminator network captures key information corresponding to the output of the generation network when analyzing the generated samples, helps the entire model achieve closer cooperation in adversarial training, achieve a better balance in the generation and discrimination processes, and improve the overall performance of the model;

[0113] In the merging processing branch of the discrimination network, the feature maps generated by the conditional processing branch and the discrimination object processing branch are superimposed, and the superimposed features are further refined and fused through multiple convolutional modules. Finally, the output is mapped to between 0 and 1 through the Sigmoid activation layer. This design can effectively integrate information from different branches and enhance the integrity of feature expression;

[0114] Table 2 Discriminator Network Structure and Hyperparameters

[0115]

[0116]

[0117] In the generative adversarial network, the choice of the loss function is crucial for model training and the quality of the finally generated images. The present invention uses Binary Cross-Entropy (BCE) as the loss function of the residual generative adversarial network, as follows

[0118] BCELoss = -(ylog(p) + (1 - y)log(1 - p))

[0119] where y is the true label (1 represents true, 0 represents the generated image), and p is the probability given by the discriminator, that is, the discriminator's predicted value of whether the image is a real image;

[0120] Step 104: Put the nail track design data set into the residual generative adversarial network for SCFE surgical nail track design to train, obtain a generative network that can design SCFE surgical nail tracks according to the three-dimensional bone model, and a discrimination network that can identify whether the SCFE surgical nail tracks meet the doctor's design standards. Use the generative network to generate nail tracks, fit a straight line using the least squares method, and after intercepting, obtain the SCFE surgical nail tracks.

[0121] Put the nail track design data set into the residual generative adversarial network for SCFE surgical nail track design to train, set the learning rates of the trained generative network and discrimination network, obtain a generative network that can design SCFE surgical nail tracks according to the three-dimensional bone model, and a discrimination network that can identify whether the SCFE surgical nail tracks meet the doctor's design standards. Use the generative network to generate nail tracks. By taking Gaussian noise as the input of the noise processing branch and the three-dimensional bone model of the SCFE patient as the input of the conditional processing branch, obtain the nail track results generated by the network, as Figure 11 shown;

[0122] Since the generated nail tracks are not perfect cylindrical line segments and there are certain slight deviations. Especially when encountering complex anatomical structures or blurred tissue boundaries, the generated results may deviate from the ideal straight line shape. To further improve the accuracy and stability of the nail tracks, the least squares method is used to perform linear fitting on the generated nail track paths.

[0123] The least squares method is a statistical optimization method. By minimizing the squared error between the generated data and the fitted line, a fitted line with the minimum error is obtained. This method can not only effectively remove the local irregular shapes and peripheral noises in the generated results, but also ensure the directionality and consistency of the nail tracks, thus obtaining a smoother and more intuitive nail track trajectory. The fitted nail track path can be considered as the learning effect of the generative adversarial network on the doctor's prior experience, which can provide accurate guiding references for clinicians.

[0124] First, calculate the centroid of all the points generated by the generation network

[0125]

[0126] where N is the number of all points and i is used to enumerate different points.

[0127] Transfer all the points to a coordinate system with the centroid as the origin to obtain the offset vector

[0128]

[0129] Construct a dot matrix

[0130]

[0131] Perform singular value decomposition on A

[0132] A = UΣV T

[0133] The vector corresponding to the third column of matrix V is the direction vector (a, b, c) of the best-fitted line, and this vector represents the maximum direction in the data. Finally, obtain the line equation

[0134]

[0135] where L(t) is the point on the line, t is the parameter, (a, b, c) is the direction vector of the line, is the centroid point through which the line passes;

[0136] After the straight line where the nail track is located is determined, regarding the selection of the starting point and the ending point of the nail track, according to the doctor's advice and the actual surgical situation, the starting point of the nail track is located at the boundary of the femoral structure, and the ending point of the nail track is located 2-3 mm away from the subchondral bone within the femoral head. After such interception, the SCFE surgical nail track is obtained. The optimal nail track result is as Figure 12 shown.

[0137] The technical solution of the embodiment of the present invention has the following beneficial effects: According to the original preoperative three-dimensional CT images of SCFE patients, each image is divided into left and right sides by the median sagittal plane, and the images of the diseased side are retained to obtain the CT of the diseased side of SCFE patients. Subsequently, data annotation of the femur, the edge of the femoral head epiphysis, the femoral head, and the subchondral bone is performed and input into the nnU-Net medical image segmentation network for training and segmentation to obtain three-dimensional models of the femur, the edge of the femoral head epiphysis, the femoral head, and the subchondral bone of all SCFE patients, which can provide an anatomical reference for nail track design; through professional doctor annotation, the starting point and the ending point of the nail track corresponding to each three-dimensional model are obtained, and the straight line connecting the starting point and the ending point is converted into an actual point set in three-dimensional space by using the interpolation method. The point sets corresponding to each three-dimensional model are combined to obtain a nail track design data set, which can be used for network training; according to the basic framework of the generative adversarial network and the residual network, a generative network, a discriminative network, and their hyperparameters are designed, a loss function is designed to obtain a residual generative adversarial network for SCFE surgical nail track design, and the nail track design data set is put into the residual generative adversarial network for SCFE surgical nail track design for training to obtain a generative network capable of designing the SCFE surgical nail track according to the three-dimensional bone model and a discriminative network capable of discriminating whether the SCFE surgical nail track meets the doctor's design standard. The nail track is generated by using the generative network, the straight line is fitted by using the least square method, and after interception, the SCFE surgical nail track is obtained. This method is of great significance for simplifying the surgical process and improving surgical efficiency.

[0138] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the scope of protection of the present invention.

[0139] The content not detailedly described in the specification of the present invention belongs to the well-known technology of those skilled in the art.

Claims

1. A method for designing nail tracts for SCFE surgery based on generative adversarial networks, characterized in that: The method comprises: Based on the original preoperative 3D CT images of SCFE patients, data annotation was performed to obtain the 3D models of the femur, femoral epiphysis, femoral head and subchondral bone of all SCFE patients. Professional doctors annotated the 3D models to obtain the starting and end points of the nail tracts corresponding to each 3D model. The straight line connecting the starting and end points was converted into an actual point set in 3D space using the interpolation method. The screw tract design dataset was obtained based on the three-dimensional models of the femur, femoral epiphyseal edge, femoral head and subchondral bone of all SCFE patients and the point sets corresponding to each three-dimensional model; According to the basic framework of the generative adversarial network and the residual network, the generative network, the discriminative network and their hyperparameters are designed, and the loss function is designed to obtain the residual generative adversarial network for the design of the nail channel of SCFE surgery. The nail channel design dataset is put into the residual generative adversarial network for SCFE surgical nail channel design for training, and a generative network that can design SCFE surgical nail channels according to the three-dimensional bone model and a discriminative network that can identify whether the SCFE surgical nail channels meet the doctor's design standards are obtained. The generative network is used to generate the nail channel, and the least squares method is used to fit the straight line and intercept it to obtain the SCFE surgical nail channel.

2. The method for designing nail tracts for SCFE surgery based on generative adversarial networks according to claim 1, characterized in that: According to the original preoperative 3D CT images of SCFE patients, data annotation was performed to obtain the 3D models of the femur, femoral epiphysis, femoral head and subchondral bone of all SCFE patients. Professional doctors annotated the 3D models to obtain the starting and end points of the nail tracts corresponding to each 3D model. The straight line connecting the starting and end points was converted into an actual point set in 3D space using the interpolation method. The specific process is as follows: According to the original preoperative 3D CT images of SCFE patients, under the guidance of professional orthopedic surgeons, the femur, femoral epiphysis edge, femoral head and subchondral bone on the CT images are marked with different labels to obtain the extracted 3D model. The doctor manually calibrates the starting point and end point of the nail channel according to the patient's anatomical characteristics and the specific conditions of the lesion area. In order to ensure that the actual size of the nail channel is consistent with the clinical requirements, the diameter of the nail channel is set to 7.3 mm according to the surgical standards. The interpolation method is used to calculate each voxel on the nail channel path according to the positions of the starting point and the end point.

3. The method for designing nail tracts for SCFE surgery based on generative adversarial networks according to claim 1, characterized in that: Based on the three-dimensional models of the femur, femoral epiphysis, femoral head and subchondral bone of all SCFE patients and the point sets corresponding to each three-dimensional model, the screw track design data set was obtained. The specific process is as follows: Based on the 3D models of the femur, femoral epiphysis, femoral head and subchondral bone of all SCFE patients and the point sets corresponding to each 3D model, the clicks were saved as 3D image data in NIFTI format, and their size was the same as that of the patient's CT data to ensure the uniformity and standardization of the annotated data; the scale-aligned and interpolated nail channel annotation data were paired with the bone features segmented from the corresponding patient CT data, and the two together were used as a nail channel design data set to train the model to identify and generate nail channel paths that are more in line with anatomical characteristics.

4. The method for designing nail tracts for SCFE surgery based on generative adversarial networks according to claim 1, characterized in that: According to the basic framework of the generative adversarial network and the residual network, the generative network, the discriminative network and their hyperparameters are designed, and the loss function is designed to obtain the residual generative adversarial network for the design of the nail channel of SCFE surgery. The specific process is as follows: First, some concepts in the claims are explained: the basic framework of the adversarial generative network consists of two networks, the generative network and the discriminative network, which compete with each other. The generative network aims to generate realistic data samples, while the discriminative network attempts to distinguish between real data and generated data; the residual network refers to the residual block in the ResNet network model. The patent of this invention draws on its design ideas to design residual module I and residual module II, and introduces the idea of ​​residual into the construction of the generative network; finally, the generative network and the discriminative network designed in this paper are collectively called the residual generative adversarial network for SCFE surgical nail channel design; Based on the conditional generative adversarial network training framework, design the algorithm pseudocode process, establish two mutually competing networks, the generating network and the discriminating network, and design specific neural network architecture parameters and update strategies; The adversarial generative network is trained in a semi-supervised manner. The identification network uses the patient's three-dimensional bone model as a conditional processing branch. The input of the identification object processing branch is the nail channel corresponding to the three-dimensional bone model, and the output is the evaluation score. The generative network uses the patient's three-dimensional bone model as a conditional processing branch, and the input of the noise processing branch is randomly generated Gaussian noise, and the output is the optimal nail channel parameter. In the design of the generative network and the discriminative network, there are four modules that make up the network, namely, the convolution module, the deconvolution module, the residual module I that does not change the size, and the residual module II that reduces the size. The convolution module is used for feature extraction and consists of a convolution layer, a BN layer, and an activation layer. The convolution kernel size is 4, the step size is 2, and the activation layer uses the Leaky ReLU function. The deconvolution module is used for image reconstruction and consists of a convolution layer, a BN layer, and an activation layer. The convolution kernel size is 4, the step size is 2, and the activation layer uses the ReLU function. The residual module I consists of two convolutional layers, with the convolution kernel size of 3 and the step size of 1. The module output is H Ι (x)=F(x)+x Where x is the input of the module, F(x) is the main branch output, and H Ι (x) is the expected output of residual module I; The residual module II reduces the output size to half of the input to achieve scale change. Its main branch consists of two convolutional layers. The first convolutional layer has a convolution kernel size of 3 and a step size of 2. The second convolutional layer has a convolution kernel size of 3 and a step size of 1. There is a convolutional layer in the residual branch with a convolution kernel size of 1 and a step size of 2. The output of the module is the main branch output plus the residual branch output. The calculation formula is H ΙΙ (x)=F(x)+F′(x) Where x is the input of the module, F(x) is the output of the main branch, F′(x) is the output of the residual branch, and H ΙΙ (x) is the output of residual module II Both residual module I and residual module II use BN layer for normalization between convolutional layers and use LeakyReLU activation layer; The main task of the generative network is to generate false samples similar to real data, so as to "fool" the discriminator so that it cannot distinguish the difference between the generated samples and the real samples. The conditional processing network branch of the generative network takes the three-dimensional bone features as conditional input, and the noise processing network branch takes Gaussian noise as input. The nail channels designed by professional doctors corresponding to each three-dimensional bone feature are taken as real samples, and the distribution law of the nail channels on the three-dimensional bone features is learned and summarized. The outputs of the conditional processing network branch and the noise processing network branch are merged as the input of the merged processing branch, and the optimal three-dimensional nail channel generated according to the three-dimensional bone feature conditions is output, so that it is as close to the real sample as possible in appearance and structure; The specific hyperparameters of the generative network are shown in Table 1. In the conditional processing branch, the two residual modules designed above are used to extract the fine-grained information of the three-dimensional bone structure. First, the residual module II is used to expand the number of channels and halve the output size to achieve effective compression and transmission of information. Then, the residual module I is used for further feature extraction to ensure that the feature information at each scale is retained so that the network can better capture the key features of the bone structure. Table 1 Network structure and hyperparameters of generated network In the noise processing branch, Gaussian noise is used as the starting point for generation; Gaussian noise is a common type of random noise, and its distribution follows Gaussian distribution (i.e. normal distribution), and its probability density function is controlled by mean μ and variance σ, as shown below Where x represents the noise value; The use of Gaussian noise can provide a random and diverse starting point for the generative network, allowing the model to generate a variety of different samples, and is often used in the generative network of GAN. The present invention gradually upsamples the Gaussian noise from 1*1*1 to 32*32*32 through the deconvolution module, adds it to the processing result of the conditional processing branch, and then inputs it into the merging processing branch together. In the merging processing branch, the obtained result is deconvolved to a size of 256*256*256 through a deconvolution module. It is particularly important to mention that in order to improve the influence of the three-dimensional bone feature condition on the generated nail channel, the present invention specifically introduces the result of conditional processing before the convolution layer of the merging branch, so that the generating network can better utilize the anatomical information in the conditional branch, ensuring that the generated nail channel parameters more accurately match the anatomical features of the femoral head, thereby optimizing the surgical planning; The main task of the discriminator is to classify a given sample and determine whether the sample is a real data sample or a false sample generated by a generation network. The discrimination network designed by the present invention also uses the three-dimensional bone features as the input of the conditional processing network branch, and uses the corresponding nail channel data as the identification object processing network branch. The outputs of these two network branches are combined as the input of the combined processing network branch. By extracting features layer by layer, the discrimination network can identify the details and structural features of the identification object in a high-dimensional space. The final output layer determines whether the identified nail channel is a nail channel that meets the doctor's design standards and can be used for clinical surgery through binary classification. The specific hyperparameters of the discriminant network are shown in Table 2. The conditional processing branch of the discriminant network is the same as that of the generative network. Two residual modules are combined with the convolution module to extract information from the three-dimensional bone features, and the three-dimensional bone features with a size of 256*256*256 are convolved to 32*32*32. This conditional processing branch design, which is the same as that of the generative network, can enable the generative network and the discriminant network to maintain similar feature extraction strategies when processing three-dimensional bone feature information, thereby enhancing the synergy between the two and achieving more stable training. The identification object processing branch of the identification network uses the aforementioned convolution module, and its convolution kernel size, step size, and padding hyperparameters are consistent with the deconvolution layer in the generation network to ensure that the two have the same scale when extracting features from the top nail road images; this design similarity can ensure that the identification network captures the key information corresponding to the output of the generation network when analyzing the generated samples, which helps the entire model to achieve closer coordination in adversarial training, achieve a better balance between the generation and identification processes, and improve the overall performance of the model; In the merge processing branch of the identification network, the feature maps generated by the conditional processing branch and the identification object processing branch are superimposed, and the superimposed features are further refined and fused through multiple convolution modules. Finally, the output is mapped to the output between 0 and 1 through the Sigmoid activation layer. This design can effectively integrate information from different branches and enhance the integrity of feature expression; Table 2 Discriminator network structure and hyperparameters In the generative adversarial network, the choice of loss function is crucial to the model training and the quality of the final generated image; Binary Cross-Entropy (BCE) is used as the residual generative adversarial network loss function, as shown below BCELoss=-(ylog(p)+(1-y)log(1-p)) Among them, y is the true label (1 for real and 0 for generated image), and p is the probability given by the discriminator, that is, the discriminator’s prediction value on whether the image is a real image.

5. The method for designing nail tracts for SCFE surgery based on generative adversarial networks according to claim 1, characterized in that: The nail channel design dataset is put into the residual generative adversarial network for SCFE surgical nail channel design for training, and a generative network that can design SCFE surgical nail channels according to the three-dimensional bone model and a discriminative network that can identify whether the SCFE surgical nail channel meets the doctor's design standards are obtained. The generative network is used to generate the nail channel, and the least squares method is used to fit the straight line. After interception, the SCFE surgical nail channel is obtained. The specific process is as follows: The nail channel design dataset was put into the residual generative adversarial network for SCFE surgical nail channel design for training. The learning rate of the training generation network was set to 1e-4 and the learning rate of the identification network was set to 5e-5. A generation network capable of designing SCFE surgical nail channels according to the three-dimensional bone model and a identification network capable of identifying whether the SCFE surgical nail channels meet the doctor's design standards were obtained. The generation network was used to generate nail channels. Gaussian noise was input as the noise processing branch and the three-dimensional bone model of the SCFE patient was input as the conditional processing branch to obtain the nail channel results generated by the network. The least square method is used to perform straight line fitting on the generated nail path; First calculate the centroid of all points generated by the generator network Where N is the number of all points, and i is the enumeration of different points; Transfer all points to a coordinate system with the center of mass as the origin to get the offset vector Constructing lattice Perform singular value decomposition on A A=UΣV T The vector corresponding to the third column of the matrix V is the direction vector (a, b, c) of the best fitting line, which represents the largest direction in the data. Finally, the equation of the line is obtained. Where L(t) is the point on the line, t is the parameter, (a, b, c) is the direction vector of the line, is the centroid of the line; After the straight line where the nail channel is located is determined, the starting point and the end point of the nail channel are selected according to the doctor's advice and the actual surgical situation. The interception setting is: the starting point of the nail channel is located at the boundary of the femoral structure, and the end point of the nail channel is located in the femoral head 2-3mm away from the subchondral bone; after such interception, the SCFE surgical nail channel is obtained.

Images