A method for reconstructing a 3D model of pigtail wishbone using X-ray images

Abstract

The present invention provides a method for reconstructing a 3D model of a pigtail furcula using an X-ray image. A two-dimensional X-ray image of the pigtail furcula is collected, and a GAN network is used to generate missing features of the X-ray image of the input pigtail furcula. Then, the boundary features of the bone-flesh connection area of ​​the image are extracted by a Canny algorithm. The extracted edge features are fused with the image after feature generation to highlight the bone-flesh edge features of the image. Finally, the processed image is input into a neural network architecture of a 3D model of the pigtail furcula to reconstruct the 3D model of the pigtail furcula, thereby providing detailed spatial position and posture information of the bone, thereby helping a robotic arm to generate a tool path.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and three-dimensional model reconstruction, and in particular to a method for reconstructing a 3D model of a pigtail wishbone by using an X-ray image. Background Art

[0002] Pork is an indispensable delicacy on the Chinese table, and its tail forkbone is delicious. However, the removal of pig tail forkbone is completely manual, because its complex structure requires workers to have certain skills, and a production line needs to be equipped with multiple bone removal workers, which is inefficient and labor-intensive. Using an automated robotic arm for removal can save labor costs and improve the economic benefits of slaughterhouses. The construction of the tool path for removing pig tail forkbone using a robotic arm depends on a high-precision 3D model of the pig tail forkbone. However, the pig tail forkbone contains muscle, blood vessels, bones and other tissues. The bones are closely connected, with tiny details and joint connections. At the same time, due to the different body sizes of different pigs, their tail forkbones vary significantly. Reconstructing the 3D model based solely on X-ray images cannot restore the detailed features of the pig tail forkbone, and the insufficient reconstruction accuracy will affect the actual reflection of the model. Summary of the invention

[0003] In view of the problems in the prior art, the present invention provides a method for reconstructing a 3D model of a pigtail wishbone using X-ray images. The method aims to construct a 3D model of a pigtail wishbone by restoring and enhancing X-ray image features, which can provide detailed spatial position and posture information of the bone, thereby helping a robotic arm to generate a tool path.

[0004] A method for reconstructing a 3D model of a pigtail wishbone using an X-ray image comprises the following steps:

[0005] Step 1: Acquire a two-dimensional X-ray image of the pigtail furcula;

[0006] Step 2: Use the GAN network to repair the missing features of the collected image and obtain the repaired complete image;

[0007] Step 3: Extract the boundary features of the bone-flesh connection area from the completed image using the Canny algorithm, and fuse the extracted boundary features with the completed image to obtain a detail feature enhanced image;

[0008] Step 4: Building a neural network architecture for constructing a 3D model of the pigtail wishbone, preparing training data and training the neural network architecture according to the training strategy; the neural network architecture includes a representation network, a conversion module and a generation network, wherein the representation network is used to extract features of the input image, the conversion module is used to convert the two-dimensional features extracted by the representation network into three-dimensional features, and the three-dimensional features are restored to the pigtail wishbone 3D model through the generation network;

[0009] Step 5: Input the image with enhanced detail features into the trained neural network to construct a 3D model of the pigtail wishbone.

[0010] Further: it represents that the network uses several two-dimensional convolution residual blocks and attention mechanism modules to perform the feature extraction task of the pigtail wishbone two-dimensional X-ray image, and the two-dimensional convolution residual block includes a convolution layer, a batch normalization layer and a ReLU activation function with a specific structure; wherein, the feature extraction task is: the two-dimensional convolution residual block first uses a 4×4 convolution kernel to downsample with a step size of 2 to increase the number of feature map channels, and then performs a step size of 1 operation with a 3×3 convolution kernel, learns the residual features through shortcut connections, and adds an attention mechanism module after each two-dimensional convolution residual block, and outputs a 4096×4×4 feature representation after connecting five groups of two-dimensional convolution residual blocks and attention mechanism modules in series.

[0011] Further: The conversion module is used to fuse the global information of the two-dimensional feature map through a 1×1 convolution layer and a ReLU activation function and adjust the number of channels, reshape the features into a three-dimensional form of 2048×2×4×4, and then integrate the three-dimensional feature information through a 1×1×1 three-dimensional deconvolution layer, and then realize the feature space conversion after a ReLU activation function to obtain three-dimensional features.

[0012] Further: the generation network includes several three-dimensional deconvolution blocks, three-dimensional convolution blocks and two-dimensional convolution blocks, the three-dimensional deconvolution block contains multiple upsampling, batch normalization, and ReLU activation operations, the three-dimensional convolution block and the two-dimensional convolution block are used for output transformation, and the output size is a three-dimensional pigtail wishbone 3D model.

[0013] Further: Step 4 specifically includes the following steps:

[0014] Step 4.1: Define the input as a multi-angle 2D projection sequence , output as 3D image , seeking mapping make ;

[0015] Step 4.2: Build a neural network model consisting of a representation network R, a conversion module T, and a generation network G

[0016] Step 4.3: Preprocess the pigtail wishbone X-ray image, including adjusting the 2D projection image to And the overall image is adjusted to , D is the depth related parameter;

[0017] Step 4.4: Prepare training data and define the mean square error loss function , select Adam optimizer and train the neural network model according to the training strategy; among them, is the sample size, Predict images for the model, It is a 3D label;

[0018] Step 4.5: Jump to step 4.2 and execute step 5 when the mean square error loss value is close to stable.

[0019] Further: Step 4.3 specifically includes the following steps:

[0020] Step 4.3.1: Adjust the data sample size, unify the 2D projection image to 128×128, and adjust the overall image to the size according to the individual imaging depth differences of the pigtail wishbones , The depth-related parameters are determined based on actual measurements; the bilinear interpolation algorithm is used for scaling operations;

[0021] Step 4.3.2: Scale and normalize the 2D projection and the overall image, and map the pixel intensity values ​​to the interval [0,1]. The specific operation is to find the minimum and maximum values ​​of the image intensity and use the formula Convert each pixel intensity value;

[0022] Step 4.3.3: Calculate the statistical mean of the pixel intensity of the training data and standard deviation , perform on the input image Operation, so that its statistical distribution is close to the standard Gaussian distribution.

[0023] Further: Step 4.4 specifically includes the following steps:

[0024] Step 4.4.1: Take the pre-processed 2D projection image as input and stack it into 3D volume data ,in, is the number of two-dimensional projections; at the same time, the three-dimensional model data of the pigtail wishbone corresponding to the three-dimensional volume data is used as the true label ;

[0025] Step 4.4.2: Use the mean square error loss function to measure the neural network model's prediction of the 3D image With the real label image The difference between them is used to adjust the parameters of the neural network model by minimizing the mean square error loss value;

[0026] Step 4.4.3: Use the Adam optimizer to update the parameters of the neural network model, calculate the gradient through the back-propagation algorithm, and adjust the parameters based on the gradient information.

[0027] Further: Step 3 specifically includes the following steps:

[0028] Step 3.1: Use Gaussian filtering to smooth the completed image to obtain a smooth image :

[0029]

[0030]

[0031] in, Indicates that the completed image is at coordinates The pixel value at is the coordinate position relative to the center pixel, is the standard deviation of the Gaussian function; is the radius of the Gaussian kernel;

[0032] Step 3.2: Calculate the gradient of the smoothed image in the horizontal and vertical directions using the Sobel operator:

[0033] Horizontal direction:

[0034]

[0035]

[0036] Vertical direction:

[0037]

[0038]

[0039] in, Represents the smoothed image in coordinates The pixel value at are the coordinates in the operator matrix, is the horizontal gradient, is the vertical gradient;

[0040] Step 3.3: Calculate the gradient magnitude and gradient direction of each pixel through the horizontal gradient and vertical gradient:

[0041] Gradient Amplitude:

[0042]

[0043] Gradient direction:

[0044]

[0045] in, Indicated in coordinates The gradient amplitude at , i.e., the edge strength, Indicated in coordinates The gradient direction at , that is, the edge direction;

[0046] Step 3.4: Thin the pixels along the gradient direction, that is, compare the gradient magnitudes of adjacent pixels and retain the pixels with the local maximum gradient magnitude;

[0047] Step 3.5: Set the threshold and , distinguish strong edges, weak edges and non-edges, that is, according to the gradient amplitude calculated in step 3.3 ,when When , the pixel is considered to be a strong edge. When the pixel is considered to be a weak edge, The pixel is considered as a non-edge; the threshold is selected based on the image gradient magnitude histogram;

[0048] Step 3.6: Determine the final bone-flesh edge image by connecting the weak edge and the strong edge, and finally obtain a clear bone-flesh edge image;

[0049] Step 3.7: Fuse the extracted bone and flesh edge image with the completed image to obtain a detail feature enhanced image :

[0050]

[0051] in, is the weighting coefficient , Indicates that the completed image is at coordinates The pixel value at Represents the pixel value of the edge image at coordinates.

[0052] Further: The GAN network includes a generator network and a discriminator network. The generator network is used to receive the feature representation or random noise vector of the pigtail fork X-ray image, and learn the mapping relationship from the input to the repaired image through multi-layer convolution and deconvolution operations to generate a repaired image with repair features; the discriminator network is used to distinguish between the real pigtail fork bone X-ray image and the repaired image generated by the generator, and optimize the output of the generator network through adversarial training with the generator network.

[0053] The beneficial effects of the present invention are as follows: a GAN network is used to generate missing features of an input X-ray image of a pigtail furcula, and then the boundary features of the bone-flesh connection area of ​​the image are extracted by a Canny algorithm, the extracted edge features are fused with the image after feature generation, the bone-flesh edge features of the image are highlighted, and finally the processed image is input into the neural network architecture of a pigtail furcula 3D model to reconstruct the pigtail furcula 3D model, thereby being able to provide detailed spatial position and posture information of the bone, thereby helping the robotic arm to generate a tool path. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of the present invention;

[0055] Figure 2 A diagram of the neural network architecture for the pigtail wishbone 3D model. DETAILED DESCRIPTION

[0056] The present invention is described in detail below in conjunction with the accompanying drawings. The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention. The directional terms such as left, middle, right, top, and bottom in the embodiments of the present invention are only relative concepts or are based on the normal use state of the product, and should not be considered as restrictive.

[0057] A method for reconstructing a 3D model of a pigtail wishbone using X-ray images, such as Figure 1 As shown, the following steps are included:

[0058] Step 1: Acquire a two-dimensional X-ray image of the pigtail furcula;

[0059] Step 2: Use the GAN network to repair the missing features of the collected image and obtain the repaired complete image;

[0060] The GAN network includes a generator network, a discriminator network, a perceptual and structural loss module (PerceptualLoss), an integrated pre-trained feature extraction network (such as the VGG network), a pixel difference-based structural loss calculation component, and a detail enhancement module;

[0061] The generator network is used to receive the feature representation or random noise vector of the pigtail fork X-ray image, and learn the mapping relationship from the input to the repaired image through multi-layer convolution and deconvolution operations to generate a repaired image with repair features; the generator network initializes the features according to the input type, and then extracts and transforms the features through a series of convolution layers; the first few layers have a kernel size of 3×3, a stride of 1 or 2, and an increasing number of filters to capture local to high-level semantic features; batch normalization technology and ReLU activation function are used between convolution layers, the former stabilizes training and accelerates convergence, and the latter introduces nonlinear enhancement of feature expression; the network contains a deconvolution layer for upsampling, first using a deconvolution layer with a kernel size of 4 and a stride of 2 to initially expand the size, and then using a deconvolution layer with a kernel size of 3 and a stride of 1 to fine-tune the repaired image to be close to the resolution and quality of the original image; the last layer outputs the repaired pigtail fork X-ray image to ensure that it matches the format and dimension of the input image to complete feature restoration;

[0062] The discriminator network is used to distinguish between the real pigtail wishbone X-ray image and the restored image generated by the generator, and the output of the generator network is optimized through adversarial training with the generator network; the discriminator network consists of multiple convolutional layers. The first few layers use convolution kernels with a kernel size of 4×4, a number of filters of 64, and a step size of 2 to quickly extract low-level features and reduce resolution. The number of filters in subsequent layers gradually increases (such as 128 and 256), the kernel size remains 4×4, and the step size is adjusted as needed to enhance the ability to distinguish image details and overall structural features; batch normalization technology is used between convolutional layers to stabilize training, and LeakyReLU is used as the activation function to avoid gradient disappearance and ensure training effectiveness and stability; finally, the probability value is output through the fully connected layer to judge the authenticity of the input image, and guide the generator to generate more realistic restored images in adversarial training;

[0063] The perceptual and structural loss module (PerceptualLoss), which integrates pre-trained feature extraction networks (such as VGG networks) and pixel-based structural loss calculation components, is used to measure the difference between the restored image and the real image at the feature level and pixel level, and guide the generator network to generate high-quality restored images;

[0064] The detail enhancement module further enhances the image details such as the bone-flesh connection area based on the image repaired by the generator to improve the overall image quality and feature clarity;

[0065] Step 3: Extract the boundary features of the bone-flesh connection area from the completed image using the Canny algorithm, and fuse the extracted boundary features with the completed image to obtain a detail feature enhanced image;

[0066] Step 3.1: Use Gaussian filtering to smooth the completed image to remove noise and obtain a smooth image:

[0067]

[0068]

[0069]

[0070] in, Indicates that the completed image is at coordinates The pixel value at is the coordinate position relative to the center pixel, is the standard deviation of the Gaussian function; is the radius of the Gaussian kernel;

[0071] Step 3.2: Calculate the gradient of the smoothed image in the horizontal and vertical directions using the Sobel operator:

[0072] Horizontal direction:

[0073]

[0074]

[0075] Vertical direction:

[0076]

[0077]

[0078] in, Represents the smoothed image in coordinates The pixel value at are the coordinates in the operator matrix, is the horizontal gradient, is the vertical gradient;

[0079] Step 3.3: Calculate the gradient magnitude and gradient direction of each pixel through the horizontal gradient and vertical gradient:

[0080] Gradient Amplitude:

[0081]

[0082] Gradient direction:

[0083]

[0084] in, Indicated in coordinates The gradient amplitude at , i.e., the edge strength, Indicated in coordinates The gradient direction at , that is, the edge direction;

[0085] Step 3.4: Thin the pixels along the gradient direction, that is, compare the gradient magnitudes of adjacent pixels, retain the pixel with the local maximum gradient magnitude, and thin each pixel to remove those pixels in the image that are not at the bone and flesh edges;

[0086] Step 3.5: Set the threshold and , distinguish strong edges, weak edges and non-edges, that is, according to the gradient amplitude calculated in step 3.3 ,when When , the pixel is considered to be a strong edge. When the pixel is considered to be a weak edge, The pixel is considered as a non-edge; the threshold is selected based on the image gradient magnitude histogram;

[0087] Step 3.6: Determine the final bone-flesh edge image by connecting the weak edge and the strong edge, and finally obtain a clear bone-flesh edge image;

[0088] Step 3.7: Fuse the extracted bone and flesh edge image with the completed image to obtain a detail feature enhanced image :

[0089]

[0090] in, is the weighting coefficient , Indicates that the completed image is at coordinates The pixel value at Indicates the edge image at coordinates The pixel value at ;

[0091] Step 4: Build a neural network architecture for constructing a 3D model of the pigtail wishbone, prepare training data, and train the neural network architecture according to the training strategy; Figure 2 As shown, the neural network architecture includes a representation network, a conversion module and a generation network, wherein the representation network is used to extract features of the input image, the conversion module is used to convert the two-dimensional features extracted by the representation network into three-dimensional features, and the three-dimensional features are restored to a pigtail wishbone 3D model through the generation network;

[0092] The representation network uses several two-dimensional convolution residual blocks and attention mechanism modules to perform the feature extraction task of the two-dimensional X-ray image of the pigtail wishbone. The two-dimensional convolution residual block includes a convolution layer with a specific structure, a batch normalization layer and a ReLU activation function; wherein, the feature extraction task is: the two-dimensional convolution residual block first uses a 4×4 convolution kernel with a step size of 2 to downsample, increase the number of feature map channels, and then uses a 3×3 convolution kernel with a step size of 1 to learn the residual features through shortcut connections, and adds an attention mechanism module after each two-dimensional convolution residual block, and connects five groups of two-dimensional convolution residuals in series. The block and attention mechanism module output a 4096×4×4 feature representation; in the representation network, the two-dimensional convolution residual block based on the residual learning concept and the attention mechanism module CBAM are introduced. Through the synergy of the convolution layer, batch normalization layer and ReLU activation function with a specific structure, the rich features of the image are effectively extracted and the gradient disappearance problem is alleviated; the attention mechanism module helps the network to focus on important channel information and spatial area information, providing sufficient and high-quality feature information for the reconstruction of the three-dimensional model, so that the final generated three-dimensional model can more accurately present the pigtail wishbone structure;

[0093] The conversion module is used to fuse the global information of the two-dimensional feature map through a 1×1 convolution layer and a ReLU activation function and adjust the number of channels, reshape the features into a three-dimensional form of 2048×2×4×4, and then integrate the three-dimensional feature information through a 1×1×1 three-dimensional deconvolution layer, and then realize the feature space conversion after a ReLU activation function to obtain three-dimensional features; the conversion module is mainly responsible for connecting the representation network and the generation network to achieve the goal of feature space conversion; through unique convolution and reshaping operations, the conversion module successfully converts the feature space dimension, integrates the three-dimensional feature information, provides appropriate input for the generation network, promotes knowledge transfer, and helps to improve the accuracy and stability of three-dimensional model generation;

[0094] The generation network includes several three-dimensional deconvolution blocks, three-dimensional convolution blocks and two-dimensional convolution blocks. The three-dimensional deconvolution block contains multiple upsampling, batch normalization, and ReLU activation operations. The three-dimensional convolution block and the two-dimensional convolution block are used for output transformation and output of a three-dimensional pigtail wishbone 3D model. The generation network is based on the three-dimensional deconvolution block. Through multiple upsampling, batch normalization, ReLU activation and specially designed output transformation module operations, it gradually restores the image resolution and finely adjusts the features to output a high-quality three-dimensional model. The optimized setting of the output transformation module ensures the accuracy of the model output, so that the generated pigtail wishbone 3D model has significant advantages in terms of accuracy and detail presentation, which provides strong support for the research and practical application of the pigtail wishbone, improves the work efficiency and the quality of results in related fields, and has obvious progress in terms of model accuracy and efficiency compared with the existing technology.

[0095] Step 4 specifically includes the following steps:

[0096] Step 4.1: Convert the problem of reconstructing 3D images from 2D projections of the pigtail wishbone into a deep learning framework, defining the input as a sequence of multi-angle 2D projections , output as 3D image , seeking mapping make ;

[0097] Step 4.2: Construct a neural network model including a representation network R, a transformation module T, and a generation network G. Through residual learning, convolution, deconvolution and other operations, feature extraction, dimension conversion and 3D model generation are realized, such as 3D model ; Construct an overall encoder-decoder framework, where the encoder learns the feature representation of 2D projections, the transformation module connects the encoder and decoder and performs feature dimension conversion, and the decoder generates 3D images based on the learned features;

[0098] Step 4.3: Preprocess the pigtail wishbone X-ray image, including adjusting the 2D projection image to And the overall image is adjusted to , D is the depth related parameter;

[0099] Step 4.3.1: Adjust the data sample size, unify the 2D projection image to 128×128, and adjust the overall image to the size according to the individual imaging depth differences of the pigtail wishbones , The depth-related parameters are determined based on actual measurements; a bilinear interpolation algorithm is used for scaling operations, which takes into account memory and computational efficiency while retaining key information (such as bone contours and joint positions), ensuring that the input network image format is unified and convenient for feature learning;

[0100] Step 4.3.2: Scale and normalize the 2D projection and the overall image, and map the pixel intensity values ​​to the interval [0,1]. The specific operation is to find the minimum and maximum values ​​of the image intensity and use the formula Convert each pixel intensity value;

[0101] Step 4.3.3: Calculate the statistical mean of the pixel intensity of the training data and standard deviation , perform on the input image Operation, so that its statistical distribution is close to the standard Gaussian distribution;

[0102] Step 4.4: Prepare training data and define the mean square error loss function , select Adam optimizer, Adam optimizer parameters , train the neural network model according to the training strategy; among them, is the sample size, Predict images for the model, It is a 3D label;

[0103] Step 4.4.1: Take the pre-processed 2D projection image as input and stack it into 3D volume data ,in, is the number of two-dimensional projections to simulate the multi-view imaging effect; at the same time, the three-dimensional model data of the pigtail wishbone corresponding to the three-dimensional volume data is used as the true label ;

[0104] Step 4.4.2: Use the mean square error loss function to measure the neural network model's prediction of the 3D image With the real label image The difference between the two is calculated by minimizing the mean square error loss value, adjusting the parameters of the neural network model so that the neural network model can learn the accurate mapping relationship and generate a reconstruction result close to the reality.

[0105] Step 4.4.3: Use the Adam optimizer to update the parameters of the neural network model. The Adam optimizer combines adaptive learning rate and momentum technology; set the learning rate ; Calculate the gradient through the back propagation algorithm and adjust the parameters based on the gradient information;

[0106] Step 4.5: Jump to step 4.2 and execute step 5 when the mean square error loss value is close to stable;

[0107] Step 5: Input the detail feature enhanced image into the trained neural network to construct a 3D model of the pigtail wishbone.

[0108] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for reconstructing a 3D model of a pigtail wishbone using X-ray images, characterized in that: The following steps are involved: Step 1: Acquire a two-dimensional X-ray image of the pigtail furcula; Step 2: Use the GAN network to repair the missing features of the collected image and obtain the repaired complete image; Step 3: Extract the boundary features of the bone-flesh connection area from the completed image using the Canny algorithm, and fuse the extracted boundary features with the completed image to obtain a detail feature enhanced image; Step 4: Build a neural network architecture for constructing a pigtail wishbone 3D model, prepare training data and train the neural network architecture according to the training strategy; the neural network architecture includes a representation network, a conversion module and a generation network, wherein the representation network is used to extract features of the input image, the conversion module is used to convert the two-dimensional features extracted by the representation network into three-dimensional features, and the three-dimensional features are restored to the pigtail wishbone 3D model through the generation network; specifically, the following steps are included: Step 4.1: Define the input as a multi-angle 2D projection sequence , output as 3D image , seeking mapping make ; Step 4.2: Construct a neural network model including a representation network R, a transformation module T and a generation network G; Step 4.3: Preprocessing the pigtail wishbone X-ray image, including adjusting the 2D projection image to 128×128 and the overall image to D×128×128, where D is a depth-related parameter; Step 4.4: Prepare training data and define the mean square error loss function , select Adam optimizer and train the neural network model according to the training strategy; among them, is the sample size, Predict images for the model, It is a 3D label; Step 4.5: Jump to step 4.2 and execute step 5 when the mean square error loss value is close to stable; Step 5: Input the detail feature enhanced image into the trained neural network to construct a 3D model of the pigtail wishbone.

2. The method for reconstructing a 3D model of a pigtail wishbone using an X-ray image according to claim 1, characterized in that: It indicates that the network uses several 2D convolution residual blocks and attention mechanism modules to perform the feature extraction task of the pigtail wishbone 2D X-ray image. The 2D convolution residual block includes a convolution layer with a specific structure, a batch normalization layer and a ReLU activation function. Among them, the feature extraction task is: the two-dimensional convolution residual block first uses a 4×4 convolution kernel with a step size of 2 for downsampling to increase the number of feature map channels, and then uses a 3×3 convolution kernel for a step size of 1 operation. The residual features are learned through shortcut connections, and an attention mechanism module is added after each two-dimensional convolution residual block. After connecting five groups of two-dimensional convolution residual blocks and attention mechanism modules in series, a 4096×4×4 feature representation is output.

3. The method for reconstructing a 3D model of a pigtail wishbone using an X-ray image according to claim 1, characterized in that: The conversion module is used to fuse the global information of the two-dimensional feature map through a 1×1 convolution layer and a ReLU activation function and adjust the number of channels, reshape the features into a three-dimensional form of 2048×2×4×4, and then integrate the three-dimensional feature information through a 1×1×1 three-dimensional deconvolution layer, and then realize the feature space conversion after the ReLU activation function to obtain three-dimensional features.

4. The method for reconstructing a 3D model of a pigtail wishbone using an X-ray image according to claim 1, characterized in that: The generation network includes several three-dimensional deconvolution blocks, three-dimensional convolution blocks and two-dimensional convolution blocks. The three-dimensional deconvolution block contains multiple upsampling, batch normalization, and ReLU activation operations. The three-dimensional convolution block and the two-dimensional convolution block are used to output transformation and output a three-dimensional pigtail wishbone 3D model.

5. The method for reconstructing a 3D model of a pigtail wishbone using an X-ray image according to claim 1, characterized in that: Step 4.3 specifically includes the following steps: Step 4.3.1: Adjust the data sample size, unify the 2D projection image to 128×128, and adjust the overall image to the size according to the individual imaging depth differences of the pigtail wishbones , The depth-related parameters are determined based on actual measurements; the bilinear interpolation algorithm is used for scaling operations; Step 4.3.2: Scale and normalize the 2D projection and the overall image, and map the pixel intensity values ​​to the interval [0,1]. The specific operation is to find the minimum and maximum values ​​of the image intensity and use the formula Convert each pixel intensity value; Step 4.3.3: Calculate the statistical mean of the pixel intensity of the training data and standard deviation , perform on the input image Operation, so that its statistical distribution is close to the standard Gaussian distribution.

6. The method for reconstructing a 3D model of a pigtail wishbone using an X-ray image according to claim 1, characterized in that: Step 4.4 specifically includes the following steps: Step 4.4.1: Take the pre-processed 2D projection image as input and stack it into 3D volume data ,in, is the number of two-dimensional projections; at the same time, the three-dimensional model data of the pigtail wishbone corresponding to the three-dimensional volume data is used as the true label ; Step 4.4.2: Use the mean square error loss function to measure the neural network model's prediction of the 3D image With the real label image The difference between them is used to adjust the parameters of the neural network model by minimizing the mean square error loss value; Step 4.4.3: Use the Adam optimizer to update the parameters of the neural network model, calculate the gradient through the back-propagation algorithm, and adjust the parameters based on the gradient information.

7. The method for reconstructing a 3D model of a pigtail wishbone using an X-ray image according to claim 1, characterized in that: Step 3 specifically includes the following steps: Step 3.1: Use Gaussian filtering to smooth the completed image to obtain a smooth image : ； ； in, Indicates that the completed image is at coordinates The pixel value at is the coordinate position relative to the center pixel, is the standard deviation of the Gaussian function; is the radius of the Gaussian kernel; Step 3.2: Calculate the gradient of the smoothed image in the horizontal and vertical directions using the Sobel operator: Horizontal direction: ； ； Vertical direction: ； ； in, Represents the smoothed image in coordinates The pixel value at are the coordinates in the operator matrix, is the horizontal gradient, is the vertical gradient; Step 3.3: Calculate the gradient magnitude and gradient direction of each pixel through the horizontal gradient and vertical gradient: Gradient Magnitude: ； Gradient direction: ； in, Indicated in coordinates The gradient amplitude at , i.e., the edge strength, Indicated in coordinates The gradient direction at , that is, the edge direction; Step 3.4: Thin the pixels along the gradient direction, that is, compare the gradient magnitudes of adjacent pixels and retain the pixels with the local maximum gradient magnitude; Step 3.5: Set the threshold and , distinguish strong edges, weak edges and non-edges, that is, according to the gradient amplitude calculated in step 3.3 ,when When , the pixel is considered to be a strong edge. When the pixel is considered as a weak edge, The pixel is considered as a non-edge; the threshold is selected based on the image gradient magnitude histogram; Step 3.6: Determine the final bone-flesh edge image by connecting the weak edge and the strong edge, and finally obtain a clear bone-flesh edge image; Step 3.7: Fuse the extracted bone and flesh edge image with the completed image to obtain a detail feature enhanced image : ； in, is the weighting coefficient , Indicates that the completed image is at coordinates The pixel value at Indicates the edge image at coordinates The pixel value at .

8. The method for reconstructing a 3D model of a pigtail wishbone using an X-ray image according to claim 1, characterized in that: The GAN network includes a generator network and a discriminator network; the generator network is used to receive the feature representation or random noise vector of the pigtail fork X-ray image, and learn the mapping relationship from the input to the repaired image through multi-layer convolution and deconvolution operations to generate a repaired image with repair features; the discriminator network is used to distinguish between the real pigtail fork bone X-ray image and the repaired image generated by the generator, and optimize the output of the generator network through adversarial training with the generator network.

Images