Pelvic image smoothing method and device based on generative adversarial network, medium, program product and terminal

Abstract

The invention provides a pelvis image smoothing method and device based on a generative adversarial network, a medium, a program product and a terminal, and the method comprises the steps: obtaining an original image of a pelvis region, and carrying out the processing of the original image through an image segmentation algorithm, so as to define the boundary features of a ruler-shaped pelvis. A segmentation result is input into an initial generator, a preliminarily smooth pelvic image is generated, and zigzag and non-uniform boundaries are reduced. And inputting the preliminary smooth image into a discriminator for quality evaluation, carrying out joint training on the generator and the discriminator according to a discrimination result, and optimizing the performance of the generator. And finally, combining an optimization generator with a ruler-shaped pelvis segmentation result to generate a smooth pelvis region image with rich details. The problems of image boundary sawtooth and unsmoothness caused by insufficient sampling frequency are solved, the precision of medical image analysis is improved, the image quality is improved, and a reliable basis is provided for clinical diagnosis.

Description

Technical Field

[0001] The present application relates to the field of intelligent medicine, and in particular to a pelvic image smoothing method, device, medium, program product and terminal based on a generative adversarial network. Background Art

[0002] In the field of medical image processing, especially in image analysis of the pelvic region, image quality directly impacts clinical diagnosis and treatment outcomes. While modern imaging technology has made significant advances, enabling the generation of high-resolution images, practical applications still face numerous challenges, particularly in the representation of ulnar structures, which often exhibit jagged and uneven boundaries. Ulnar structures typically refer to morphological features of specific anatomical regions, and their accuracy and smoothness are crucial for subsequent analysis and diagnosis. When the image sampling frequency is insufficient, the imaging device cannot fully capture subtle anatomical variations in the pelvic region. This deficiency can lead to significant jagged boundaries and discontinuities during image reconstruction, resulting in noticeable protrusions or sharp angles, reducing the recognizability of ulnar structures and potentially misleading clinical diagnosis, affecting the physician's assessment of lesions. Furthermore, excessively low sampling frequencies can result in rough transitions between adjacent pixels, creating noticeable edge defects that blur or misdelineate the morphology of critical structures, further reducing overall image quality and introducing errors in subsequent image analysis, seriously impacting pathological interpretation.

[0003] Currently, the following existing technologies exist to address the problem of uneven boundaries of ruler-like structures. Edge detection-based algorithms: for example, Canny edge detection and Sobel operator, to enhance the boundary features of images. However, edge detection-based algorithms are prone to produce false edges when processing complex structures, thereby introducing noise. Image restoration technology achieves a certain degree of image smoothing through bilateral filtering and anisotropic diffusion. However, when trying to remove the aliasing problem, it often leads to blurred details and loss of edge information, thus affecting the true representation of the structure. Erosion, dilation, and opening operations based on mathematical morphology are also used to smooth image edges. However, when processing highly complex and diverse ruler-like structures, the performance of these methods is often not ideal and may not be able to fully restore the true boundary morphology. Although these existing technologies have improved the smoothness of image boundaries to a certain extent, there are still problems such as the inability to effectively solve jagged boundaries, loss of details, and inability to restore true boundaries. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of the present application is to provide a pelvic image smoothing method, device, medium, program product and terminal based on a generative adversarial network, which is used to solve the problems existing in the existing ruler-like structure smoothing technology, such as the inability to effectively eliminate jagged boundaries, resulting in loss of details and inability to restore the true boundaries.

[0005] To achieve the above-mentioned objectives and other related objectives, the first aspect of the present application provides a pelvic image smoothing method based on a generative adversarial network, the method comprising: acquiring a pelvic region image; segmenting the pelvic region image based on an image segmentation algorithm to generate a ulnar pelvic segmentation result; inputting the ulnar pelvic segmentation result into an initial generator to generate an initial smoothed pelvic image; inputting the initial smoothed pelvic image into an initial discriminator to generate an initial discrimination result; based on the initial smoothed pelvic image and the initial discrimination result, jointly training the initial generator and the initial discriminator to generate an optimized generator; inputting the ulnar pelvic segmentation result into the optimized generator to generate a smoothed pelvic region image.

[0006] In some embodiments of the first aspect of the present application, the process of inputting the ulnar pelvis segmentation result into the initial generator to generate the ulnar pelvis segmentation result includes: inputting the ulnar pelvis segmentation result into a first encoder to generate a first feature; inputting the first feature into a second encoder to generate a second feature; inputting the second feature into a third encoder to generate a third feature; performing a feature fusion operation on the first feature, the second feature and the third feature to generate a comprehensive feature vector; inputting the comprehensive feature vector into a first decoder to generate a fourth feature; inputting the fourth feature and the third feature into a second decoder to generate a fifth feature; and inputting the fifth feature and the second feature into a third decoder to generate the ulnar pelvis segmentation result.

[0007] In some embodiments of the first aspect of the present application, the following operations are also performed before performing feature fusion operations on the first feature, the second feature, and the third feature: performing a global average pooling operation on the first feature to generate a weight vector for each channel; inputting the weight vector of each channel into a fully connected network to generate a normalized weight for each channel; multiplying the normalized weight of each channel by the first feature channel by channel to generate the attention-optimized first feature; performing a global average pooling operation on the second feature to generate a weight vector for each channel; inputting the weight vector of each channel into a fully connected network to generate a normalized weight for each channel; multiplying the normalized weight of each channel by the second feature channel by channel to generate the attention-optimized second feature; performing a global average pooling operation on the third feature to generate a weight vector for each channel; inputting the weight vector of each channel into a fully connected network to generate a normalized weight for each channel; multiplying the normalized weight of each channel by the third feature channel by channel to generate the attention-optimized third feature.

[0008] In some embodiments of the first aspect of the present application, the process of inputting the initial smoothed pelvic image into the initial discriminator to generate the authenticity of the corresponding pelvic segmentation result includes: inputting the initial smoothed pelvic image into the first convolutional layer in sequence to generate a sixth feature; inputting the sixth feature into the second convolutional layer to generate a seventh feature; inputting the seventh feature into the third convolutional layer to generate an eighth feature; and inputting the eighth feature into the activation function to generate the authenticity of the pelvic segmentation result.

[0009] In some embodiments of the first aspect of the present application, the process of inputting the seventh feature into the third convolutional layer to generate the eighth feature includes: inputting the seventh feature into the third convolutional layer to generate an initial feature vector; performing a feature graph flattening operation on the initial feature vector to generate a node feature matrix; calculating the attention weights between each team of nodes in the node feature matrix to obtain attention weights; performing a multi-head attention calculation operation based on the node feature matrix and the attention weights to generate a multi-head attention matrix; and adjusting the multi-head attention matrix to a vector with the same dimension as the initial feature vector to generate the eighth feature.

[0010] In some embodiments of the first aspect of the present application, the process of jointly training the initial generator and the initial discriminator includes: setting the weights in the initial generator to fixed preset parameters; inputting the initial smoothed pelvis image and the ideal pelvic smoothed image into the initial discriminator to generate a predicted discrimination result; calculating a first loss function based on the predicted discrimination result and the initial discrimination result; updating the weights of the initial discriminator based on the first loss function to generate an optimized discriminator; based on the optimized discriminator, inputting the ulnar pelvis segmentation result into the initial discriminator to generate an updated pelvic smoothing result; calculating a second loss function and a third loss function of the updated pelvic smoothing result; based on the second loss function and the third loss function, updating the weights of the initial discriminator through a backpropagation algorithm to generate the optimized generator; performing alternating optimization operations on the optimized discriminator and the optimized generator until the second loss function and the third loss function converge.

[0011] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a pelvic image smoothing device based on an adversarial generative network, comprising: an image acquisition module: used to acquire a pelvic area image; an adversarial network module: used to segment the pelvic area image based on an image segmentation algorithm to generate a ulnar pelvic segmentation result; inputting the ulnar pelvic segmentation result into an initial generator to generate an initial smoothed pelvic image; inputting the initial smoothed pelvic image into an initial discriminator to generate an initial discrimination result; based on the initial smoothed pelvic image and the initial discrimination result, jointly training the initial generator and the initial discriminator to generate an optimized generator; and a pelvic smoothing module: used to input the ulnar pelvic segmentation result into the optimized generator to generate a smooth pelvic area image.

[0012] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pelvic image smoothing method based on the generative adversarial network.

[0013] To achieve the above-mentioned objectives and other related objectives, the fourth aspect of the present application provides a computer program product, which includes a computer program code. When the computer program code is run on a computer, the computer implements the pelvic image smoothing method based on the adversarial generative network.

[0014] To achieve the above-mentioned purpose and other related purposes, the fifth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the pelvic image smoothing method based on the adversarial generative network.

[0015] As described above, the pelvic image smoothing method, device, medium, program product and terminal based on the adversarial generative network of the present application have the following beneficial effects: by comprehensively optimizing the pelvic region image processing process, the image quality and recognizability of the ulnar pelvic structure are significantly improved. First, the initial segmentation is performed based on the advanced image segmentation algorithm, making the outline of the ulnar pelvis clearer, laying the foundation for subsequent processing. On this basis, the generator is used to generate the initial smoothed pelvic image, which effectively reduces the problems of jagged and uneven boundaries and enhances the overall smoothness of the image. After the evaluation of the initial discriminator and the joint training of the generator and the discriminator, the performance of the generator is further optimized, enabling it to generate smoother and well-detailed pelvic region images. This not only improves the accuracy of medical image processing, but also provides a more reliable image basis for clinical diagnosis, helps doctors make accurate judgments, and promotes the technological progress of medical image analysis in the pelvic region. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flow chart of an embodiment of a pelvic image smoothing method based on a generative adversarial network is shown in the present application.

[0017] Figure 2 A schematic diagram of a ruler-shaped pelvic region in an embodiment of a pelvic image smoothing method based on a generative adversarial network of the present application is shown.

[0018] Figure 3 A flow chart of another embodiment of the pelvic image smoothing method based on a generative adversarial network of the present application is shown.

[0019] Figure 4 A schematic diagram of the structure of the generator in an embodiment of the pelvic image smoothing method based on the adversarial generative network of the present application is shown.

[0020] Figure 5 A schematic diagram of the structure of the discriminator in an embodiment of the pelvic image smoothing method based on the adversarial generative network of the present application is shown.

[0021] Figure 6 A schematic diagram of the optimized pelvic area in an embodiment of the pelvic image smoothing method based on a generative adversarial network of the present application is shown.

[0022] Figure 7 A structural diagram of an embodiment of a pelvic image smoothing device based on a generative adversarial network is shown in the present application.

[0023] Figure 8 A structural diagram of an embodiment of a pelvic image smoothing terminal based on a generative adversarial network of the present application is shown. DETAILED DESCRIPTION

[0024] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0025] Before further explaining the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations:

[0026] <1> Adversarial network: Adversarial network is a deep learning architecture consisting of two models: the generator and the discriminator. The generator is responsible for generating data samples, while the discriminator is used to evaluate the authenticity of these samples. The two are continuously optimized through a mutually adversarial training process to achieve the goal of improving the quality of generated samples.

[0027] <2> Image segmentation algorithm: Image segmentation algorithm is a technology used to divide an image into multiple different regions. The purpose is to separate the object or region of interest from the background. It is often used in computer vision tasks such as semantic segmentation and instance segmentation for subsequent analysis and processing.

[0028] <3> Generator: The generator is a component of an adversarial network whose main function is to generate samples similar to real data based on the input. The generator proposes realistic data examples by learning the characteristics of the target distribution to confuse the discriminator.

[0029] <4> Discriminator: The discriminator is another key component in the adversarial network, responsible for determining whether the input sample is from the real dataset or generated by the generator. The effectiveness of the discriminator directly affects the optimization and training process of the generator.

[0030] <5> Encoder: The encoder is the part of a deep learning model that converts input data into a low-dimensional feature representation. The encoder extracts important features through operations such as convolution, providing information for subsequent decoding or classification tasks.

[0031] <6> Decoder: A decoder is a component in a deep learning model that converts low-dimensional feature representations into high-dimensional outputs. It is typically used for tasks such as image reconstruction and speech synthesis. The decoder ensures the quality and consistency of the output by gradually generating or recovering data.

[0032] <7> Spatial attention mechanism: The spatial attention mechanism is a method to enhance the model's ability to pay attention to input features. By applying weight distribution on the feature map, it highlights important areas and suppresses irrelevant areas, thereby improving the model's ability to capture important information.

[0033] <8> Feature map flattening: This operation converts a multidimensional feature map into a one-dimensional vector, commonly used in neural networks to facilitate processing in subsequent layers. Flattening preserves feature information and simplifies the data structure to accommodate the input requirements of fully connected layers.

[0034] <9> Node feature matrix: The node feature matrix is ​​a matrix used to represent the node features in the graph neural network. Each row corresponds to the feature vector of a node, which facilitates modeling and analysis of graph structure data.

[0035] <10> Multi-head attention matrix: The multi-head attention matrix is ​​an extension of the self-attention mechanism. By computing attention in parallel on multiple subspaces, it forms multiple attention weight matrices, enhancing the model's ability to capture information in multiple aspects and improving feature expression.

[0036] To facilitate understanding of the embodiments of this application, first Figure 1 Detailed description. Figure 1 The following is a flow chart of a pelvic image smoothing method based on a generative adversarial network according to an embodiment of the present invention. The pelvic image smoothing method based on a generative adversarial network according to this embodiment mainly includes the following steps:

[0037] Step S11: Acquire a pelvic region image.

[0038] In one embodiment of the present invention, a pelvic region image is generated using medical imaging technologies such as CT or MRI to display images of the human pelvic region. Pelvic region images are widely used in medical diagnosis, treatment planning, and surgical navigation. By displaying the pelvic bony structure and surrounding tissues, they help physicians identify lesions and anatomical features. Pelvic region images typically display structures such as the sacrum, hip bone, pubic bone, and pelvic cavity.

[0039] However, taking the pelvic region image as an example, when the segmentation result appears as Figure 2 When the pelvic region is imaged, the boundaries often appear jagged or uneven, with noticeable sharp angles or protrusions. This phenomenon is often closely related to insufficient sampling frequency. The sampling frequency determines the imaging system's ability to capture image detail. When the sampling frequency is too low, subtle changes in the pelvic region cannot be accurately captured, and problems such as boundary discontinuity and jaggedness may occur during image reconstruction. Insufficient frequency can also lead to less than smooth transitions between adjacent pixels, resulting in protrusions or sharp angles at the image edges, ultimately producing an uneven pelvic region. This not only reduces the overall image quality but may also affect subsequent analysis and diagnostic results.

[0040] Step S12: Segmenting the pelvic region image based on an image segmentation algorithm to generate a ulnar pelvic segmentation result.

[0041] In one embodiment of the present invention, methods for segmenting the pelvic region image based on an image segmentation algorithm include, but are not limited to, grayscale threshold segmentation, edge detection segmentation, convolutional neural network segmentation, and U-net segmentation. Grayscale threshold segmentation divides the image region into foreground and background using a grayscale threshold; edge detection segmentation utilizes edge detection operators (such as Sobel and Canny) to identify image boundaries and perform region segmentation based on the boundaries; convolutional neural network segmentation utilizes a multi-layer convolutional neural network to extract features and perform segmentation; and U-net segmentation utilizes an encoder-decoder structure to achieve high-resolution segmentation.

[0042] It should be noted that the reason why the present invention performs a segmentation operation before smoothing the pelvic structure is that the CT image contains the pelvic structure, surrounding tissues (such as muscle, fat, soft tissue) and background noise. If the original image without segmentation is directly smoothed, it will cause interference from the background and irrelevant areas, and the characteristics of the non-pelvic area (such as muscle tissue or air part) are significantly different from the anatomical characteristics of the bones, which will cause the smoothing effect of the mixed processing on the bone structure boundary to be distorted. In the present invention, the smoothing process is limited to the image pixels of the pelvic structure, which reduces the amount of calculation and improves the processing speed; at the same time, the processing of the target area can avoid the waste of resources on the background and useless areas, thereby improving the operating efficiency.

[0043] In one embodiment of the present invention, the subsequent processing flow of the pelvic region image is as follows: Figure 3 As shown, the severely fragmented pelvic segmentation result is input into the generator to generate a predicted smooth segmentation result. The predicted smooth segmentation result and the smooth segmented pelvis result are then input into the discriminator to determine whether the prediction is true, completing the joint training of the generator and discriminator. The generator and discriminator will be described in detail below.

[0044] Step S13: inputting the ulnar pelvis segmentation result into an initial generator to generate an initial smoothed pelvis image.

[0045] In one embodiment of the present invention, the process of inputting the ulnar pelvis segmentation result into the initial generator to generate the ulnar pelvis segmentation result includes: inputting the ulnar pelvis segmentation result into a first encoder to generate a first feature; inputting the first feature into a second encoder to generate a second feature; inputting the second feature into a third encoder to generate a third feature; performing a feature fusion operation on the first feature, the second feature and the third feature to generate a comprehensive feature vector; inputting the comprehensive feature vector into a first decoder to generate a fourth feature; inputting the fourth feature and the third feature into a second decoder to generate a fifth feature; and inputting the fifth feature and the second feature into a third decoder to generate the ulnar pelvis segmentation result.

[0046] In this embodiment, the generator structure is as follows Figure 4 As shown in the figure, the algorithm consists of a decoding stage, a feature fusion stage, and an encoding stage, focusing on achieving a smooth ruler-like structure. First, in the feature extraction stage, the preprocessed pelvic region image is input into the first encoder, where a series of convolution operations extract the first feature. This feature is then fed into the second encoder to generate the second feature, and then into the third encoder to generate the third feature. The core goal of this stage is to deeply extract key feature information within the image, providing basic data for the subsequent smoothing process.

[0047] Furthermore, during the decoding phase, the integrated feature vector is fed into the first decoder to generate the fourth feature, thereby restoring the image to a higher resolution. The primary goal of this phase is to enhance the smoothness of the ruler-like structure by reproducing detailed details. The generated fourth feature is then skip-connected with the third feature from the third encoder and fed into the second decoder to generate the fifth feature, ensuring full utilization of global information during reconstruction. Simultaneously, the fifth feature is skip-connected with the second feature from the second encoder and fed into the third decoder, ultimately smoothing the ruler-like structure.

[0048] In one embodiment of the present invention, the following operations are also performed before performing the feature fusion operation on the first feature, the second feature and the third feature: performing a global average pooling operation on the first feature to generate a weight vector for each channel; inputting the weight vector of each channel into the fully connected network to generate a normalized weight for each channel; multiplying the normalized weight of each channel by the first feature channel by channel to generate the first feature after attention optimization; performing a global average pooling operation on the second feature to generate a weight vector for each channel; inputting the weight vector of each channel into the fully connected network to generate a normalized weight for each channel; multiplying the normalized weight of each channel by the second feature channel by channel to generate the second feature after attention optimization; performing a global average pooling operation on the third feature to generate a weight vector for each channel; inputting the weight vector of each channel into the fully connected network to generate a normalized weight for each channel; multiplying the normalized weight of each channel by the third feature channel by channel to generate the third feature after attention optimization.

[0049] In this embodiment, during the attention optimization stage, attention optimization operations are performed on the first, second, and third features, respectively, with the goal of focusing on key areas of the ruler-like structure, such as edges and jagged structures, to achieve more efficient smoothing. This stage utilizes an attention mechanism to dynamically adjust the degree of attention paid to different areas by calculating attention weights in spatial and channel dimensions, particularly strengthening attention to irregular and non-smooth features. This optimization strategy not only highlights local areas that require smoothing, but also effectively suppresses interference from background areas and other irrelevant features. In addition, the application of the attention mechanism helps to adjust the weight relationship between different features, so that important features occupy a more prominent position in subsequent processing, ensuring that the optimized feature data can fully and accurately reflect the details of the ruler-like structure.

[0050] It is worth noting that the attention-optimized first, second, and third features are fused, hierarchically integrating feature information from different encoding levels to ultimately generate a comprehensive feature vector. The main purpose of this feature fusion is to combine the advantages of global and local information, preserving the important structural information contained in deep features while taking into account the detailed representation of shallow features, providing richer and more accurate input for the subsequent smoothing process in the decoding stage.

[0051] Step S14: inputting the initial smoothed pelvis image into an initial discriminator to generate an initial discrimination result.

[0052] In one embodiment of the present invention, the process of inputting the initial smoothed pelvis image into the initial discriminator to generate the authenticity of the corresponding pelvic segmentation result includes: inputting the initial smoothed pelvis image into the first convolutional layer in sequence to generate the sixth feature; inputting the sixth feature into the second convolutional layer to generate the seventh feature; inputting the seventh feature into the third convolutional layer to generate the eighth feature; and inputting the eighth feature into the activation function to generate the authenticity of the pelvic segmentation result.

[0053] In this embodiment, the structure of the discriminator is as follows: Figure 5 As shown, the process of inputting the initial smoothed pelvic image into the initial discriminator includes the following steps: the initial smoothed pelvic image is sequentially input into the first convolutional layer, where low-level features are extracted through convolution operations to generate the sixth feature. This feature mainly contains edge and basic texture information, providing a basis for subsequent processing. Next, the sixth feature is passed to the second convolutional layer, where a higher-level seventh feature is generated, which further captures the complex structural information in the image. Subsequently, the seventh feature enters the third convolutional layer, where further features are extracted to generate the eighth feature, which realizes the abstract processing of deeper image information. Finally, the eighth feature is input into the activation function, which performs a nonlinear transformation and ultimately generates an evaluation result related to the smoothness of the scale structure. Through the above training process, the discriminator can efficiently analyze and evaluate the features of the initial smoothed pelvic image.

[0054] In this embodiment, the activation functions include, but are not limited to, ReLU, Sigmoid, Tanh, Leaky ReLU, and Softmax. The ReLU function corrects input values ​​less than zero to zero; the Sigmoid and Tanh functions map input values ​​to a range between 0 and 1. Selecting the appropriate activation function is crucial in the discriminator's feature extraction process, and should be flexibly applied based on specific task requirements.

[0055] In one embodiment of the present invention, the process of inputting the seventh feature into the third convolutional layer to generate the eighth feature includes: inputting the seventh feature into the third convolutional layer to generate an initial feature vector; performing a feature graph flattening operation on the initial feature vector to generate a node feature matrix; calculating the attention weights between each team of nodes in the node feature matrix to obtain attention weights; performing a multi-head attention calculation operation based on the node feature matrix and the attention weights to generate a multi-head attention matrix; and adjusting the multi-head attention matrix to a vector with the same dimension as the initial feature vector to generate the eighth feature.

[0056] In one embodiment of the present invention, the process of inputting the seventh feature into the third convolutional layer includes the following steps: First, the seventh feature is passed to the third convolutional layer to generate an initial feature vector. Exemplarily, the dimension of the initial feature vector is B×C×H×W, where B represents the batch size, C represents the number of channels, H represents the height of the feature map, and W represents the width of the feature map. Then, a feature map flattening operation is performed on the initial feature vector to generate a node feature matrix whose shape is B×N×C, where N=H×W represents the total number of nodes in the spatial dimension. Subsequently, the attention weights between each pair of nodes in the node feature matrix are calculated, and an attention weight matrix is ​​generated by dot product or other similarity measurement methods. Based on the node feature matrix and the attention weights, a multi-head attention calculation operation is performed to obtain feature representations of multiple heads and merge them into a multi-head attention matrix. Finally, the multi-head attention matrix is ​​adjusted to a vector consistent with the number of channels of the initial feature vector, thereby generating the eighth feature for subsequent processing.

[0057] Furthermore, the process of generating the attention weight matrix includes: calculating the similarity between two nodes by the cosine similarity in Formula 1 or the dot product form in Formula 2. Among them, F in Formula 1 and Formula 2 i and F j Represent the feature vectors of node i and node j respectively. By calculating the similarity between all nodes, an attention weight matrix is ​​generated, whose shape is N×N, where the value of the (i, j) position represents the attention weight of node i to node j.

[0058]

[0059] similarity(i,j)=F i ·F j (Formula 2)

[0060] Furthermore, the multi-head attention calculation operation to obtain the feature representations of multiple heads and merge them into a multi-head attention matrix includes: for each attention head, the attention weight is calculated according to the aforementioned steps, and the input node feature matrix is ​​weighted summed using these weights to generate the feature representation corresponding to each attention head. Next, the feature representations of all attention heads are spliced ​​along the feature dimension to generate an intermediate feature representation with a higher dimension, which is used to represent the ability to integrate the attention information of different heads. Subsequently, a linear transformation is used to project the spliced ​​intermediate feature representation to the required output dimension to obtain the final multi-head attention matrix. In this process, the merged feature matrix can effectively capture the complex relationship between the input features, thereby enhancing the performance of the model in feature learning and subsequent tasks.

[0061] Step S15: Based on the initial smoothed pelvis image and the initial discrimination result, the initial generator and the initial discriminator are jointly trained to generate an optimized generator.

[0062] In one embodiment of the present invention, the process of jointly training the initial generator and the initial discriminator includes: setting the weights in the initial generator to fixed preset parameters; inputting the initial smoothed pelvis image and the ideal pelvic smoothed image into the initial discriminator to generate a predicted discrimination result; calculating a first loss function based on the predicted discrimination result and the initial discrimination result; updating the weights of the initial discriminator based on the first loss function to generate an optimized discriminator; based on the optimized discriminator, inputting the ulnar pelvis segmentation result into the initial discriminator to generate an updated pelvic smoothing result; calculating a second loss function and a third loss function of the updated pelvic smoothing result; based on the second loss function and the third loss function, updating the weights of the initial discriminator through a backpropagation algorithm to generate the optimized generator; performing alternating optimization operations on the optimized discriminator and the optimized generator until the second loss function and the third loss function converge.

[0063] In this embodiment, the weights in the initial generator are set to fixed preset parameters to ensure that the generator structure remains stable and avoid interference caused by frequent parameter updates in the early stages of joint training. This allows the discriminator to focus on learning how to evaluate the quality of the generator output, thereby improving the discriminator's performance. For example, the generator weights are initialized to a standard normal distribution.

[0064] In this embodiment, an initial smoothed pelvic image (the image generated by the generator) and an ideal smoothed pelvic image (the actual target image) are input into the initial discriminator to generate a predicted discrimination result. The input format requires that the two images have the same size and number of channels. The discriminator generates a corresponding predicted discrimination result based on the input image and outputs a probability value between 0 and 1 to indicate the likelihood that the input image is a real image. By using neural network calculations, a Sigmoid activation function is used in the final layer of the discriminator's output, so that all results can be interpreted as probabilities.

[0065] Furthermore, in calculating the first loss function based on the predicted discrimination result and the initial discrimination result, the first loss function is an adversarial loss function as shown in Formula 1. As shown in Formula 3, Denotes the adversarial loss function, which is the objective function used to optimize the discriminator. The discriminator D is responsible for determining whether the input data is a real sample or a synthetic sample generated by the generator F. In the formula, x represents the real data sample. Expected value The calculation is based on the real data distribution x. The task of the generator F is to generate synthetic data similar to the real data x. For the input real sample x, the expression

[0066] Represents a modification of x. Specifically, Is a mask value that acts on x to selectively process the data and form a new sample. The output of the discriminator D for the real data x is log(D(x)), which is the logarithm of the authenticity probability of the real sample. At the same time, The output of the discriminator for the generated fake data shows the logarithm of the probability that the generated sample is considered unreal. The overall purpose of this formula is to improve the performance of the generator and discriminator by maximizing the probability that the real sample is correctly identified while minimizing the probability that the generated sample is identified as real.

[0067]

[0068] For the updated pelvic smoothing result, the second loss function and the third loss function are calculated. The second loss function is shown in Formula 4. Where, Represents the reconstruction loss, which is used to measure the difference between the generated image and the real image; Represents a mask used to selectively process certain areas of the input image, where the value indicates which pixels need to be retained and which need to be ignored; x represents the real image sample of the input; F represents the generator, which is responsible for generating synthetic images similar to the real sample; represents the inverse operation of the overlay mask, which converts the selective area in the mask into the area that needs to be operated on by the generator; ⊙ represents the Hadamard Product, that is, element-wise multiplication. This operation is used to perform element-wise multiplication of the image positioned on the mask (or its inverse) with the other image, thereby achieving conditional processing. Represents the difference between the real image x and the image generated by the generator. The generator inputs the masked real image to generate a new synthetic image, which is then compared with the original image.

[0069]

[0070] Furthermore, the third loss function is shown in Formula 5, represents the reconstruction loss, Represents resistance to loss, represents the overall loss function, that is, the optimization goal of the model during training; rec Represents the weight coefficient of reconstruction loss, which is used to control the weight of reconstruction loss in the overall loss; λadv Represents the weight coefficient of the adversarial loss, which is used to control the weight of the adversarial loss in the overall loss.

[0071]

[0072] In this embodiment, in joint training, weight updates are achieved by calculating the gradient of the loss function. First, a second loss function needs to be calculated to measure the difference between the discriminator's prediction of the input image and the actual result. Then, the backpropagation algorithm is used to calculate the gradient of the loss function relative to the discriminator weight. Through the chain rule, the gradient is passed from the output layer to the input layer, and the contribution of the weight of each layer to the loss is calculated. Then, the calculated gradient is multiplied by the preset learning rate to determine the amplitude of the weight update, and the amplitude is subtracted from the current weight to obtain the new weight. In addition, for the weight update of the generator, similar calculations are performed using the second loss function and the third loss function, and the gradient of the generator weight is obtained by backpropagation to perform weight update.

[0073] In this embodiment, the process of performing alternating optimization operations on the optimized discriminator and the optimized generator includes: in each round of iteration, first fixing the weights of the generator and focusing on updating the weights of the discriminator. In this way, the discriminator can be optimized based on the current generator to improve its ability to distinguish between real samples and synthetic samples. After completing the update of the discriminator, the weights of the discriminator are fixed and the focus is on updating the weights of the generator. This ensures that the generator is optimized under the feedback of the existing discriminator, thereby improving the authenticity of the generated samples. This alternating optimization strategy enables the generator and the discriminator to gradually improve their respective performance in mutual competition, ultimately achieving more realistic generated images and stronger discrimination capabilities.

[0074] In this embodiment, the criteria for judging whether the training process has converged generally include monitoring the changes in the second loss function and the third loss function over multiple iteration cycles. When the magnitude of the change in these loss functions is lower than a preset threshold (for example, the change is less than 0.01), the model can be considered to have stabilized. Secondly, observe whether the value of the loss function remains stable within a certain range. If the value of the loss function remains near a certain level during multiple iterations and there is no obvious downward trend, it means that the performance of the model is relatively stable. In addition, a maximum number of training rounds can be set. Even if the loss function does not reach the convergence condition, it can be chosen to stop training after reaching this round. Through these criteria, the training status of the model can be effectively judged to decide whether to continue or terminate the training process.

[0075] Step S16: inputting the ulnar pelvis segmentation result into the optimization generator to generate a smooth pelvic region image.

[0076] Figure 6The smoothed pelvic segmentation results of this example are shown. The trained generator of the generative adversarial network is used to process the stepped pelvis results to optimize and improve the quality of the pelvic image. By learning from a large number of high-quality pelvic images, the generator has mastered the complex morphological and structural features. Using the optimal weights obtained through training, it can effectively transform the input stepped pelvic image into a smooth and continuous visual image. This processing not only significantly improves the visual quality of the image but also provides a clearer and more accurate image foundation for subsequent medical analysis and diagnosis, ensuring its important value and role in clinical applications.

[0077] In the embodiments of this application, terms such as "first" and "second" are used to distinguish between identical or similar items with substantially the same function or effect. For example, the first feature and the second feature are used solely to distinguish between different features and do not define their order. Those skilled in the art will understand that terms such as "first" and "second" do not define the quantity or order of execution, and that terms such as "first" and "second" do not necessarily define differences.

[0078] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" represent examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0079] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0080] Figure 7 Schematic block diagram of a pelvic image smoothing method and apparatus 700 based on a generative adversarial network provided in an embodiment of the present application. Figure 7 As shown, the device includes an image acquisition module 701, an adversarial network module 702 and a pelvic smoothing module 703.

[0081] Image acquisition module 701: used to acquire pelvic area images.

[0082] Adversarial network module 702: used to segment the pelvic region image based on an image segmentation algorithm to generate a ulnar pelvic segmentation result; input the ulnar pelvic segmentation result into an initial generator to generate an initial smooth pelvic image; input the initial smooth pelvic image into an initial discriminator to generate an initial discrimination result; based on the initial smooth pelvic image and the initial discrimination result, jointly train the initial generator and the initial discriminator to generate an optimized generator.

[0083] Pelvic smoothing module 703: used to input the ulnar pelvic segmentation result into the optimization generator to generate a smooth pelvic area image.

[0084] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0085] It should also be understood that the division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present application may be integrated into a single processor, or may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0086] Figure 8 : is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Figure 8 As shown, the electronic terminal includes: at least one processor 801, a memory 802, at least one network interface 803 and a user interface 805. The various components in the device are coupled together via a bus system 804. It is understood that the bus system 804 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 804 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 8 Various buses are labeled as bus systems.

[0087] The user interface 805 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0088] It will be appreciated that the memory 802 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM) or a programmable read-only memory (PROM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0089] The memory 802 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 800. Examples of such data include: any executable program for operating on the electronic terminal 800, such as an operating system 8021 and an application 8022; the operating system 8021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 8022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The pelvic image smoothing method based on the adversarial generative network provided in the embodiment of the present invention can be included in the application 8022.

[0090] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 801. Processor 801 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 801 or by software instructions. The above processor 801 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 801 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor 801 can be a microprocessor or any conventional processor. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory. The processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0091] In an exemplary embodiment, the electronic terminal 800 may be configured to execute the aforementioned method using one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs).

[0092] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: computer program code, which, when running on a computer, enables the computer to execute the pelvic image smoothing method based on the adversarial generative network of any embodiment shown in the embodiments above.

[0093] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable storage medium, which stores a program code. When the program code is run on a computer, the computer executes the pelvic image smoothing method based on the adversarial generative network of any embodiment shown in the embodiments above.

[0094] As used in this specification, the terms "component," "module," "system," and the like are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0095] Those skilled in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0096] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0098] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0099] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0100] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (program) are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. Available media can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0101] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0102] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0103] In summary, the present application provides a pelvic image smoothing method, device, medium, program product and terminal based on a generative adversarial network. The present invention provides a method for improving the smoothing effect of pelvic images by obtaining the original image of the pelvic region and processing it using an image segmentation algorithm to clarify the boundary features of the ulnar pelvis. The segmentation result is input into the initial generator to generate a preliminary smoothed pelvic image, reducing jagged and uneven boundaries. The preliminary smoothed image is input into the discriminator for quality assessment, and the generator and discriminator are jointly trained based on the discrimination results to optimize the generator performance. Finally, the optimized generator is combined with the ulnar pelvic segmentation result to generate a smooth and detailed pelvic region image. The problem of jagged and uneven image boundaries caused by insufficient sampling frequency is solved, the accuracy of medical image analysis is improved, the image quality is improved, and a reliable basis for clinical diagnosis is provided. Therefore, the present application effectively overcomes the various shortcomings of the prior art and has high industrial utilization value.

[0104] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A pelvic image smoothing method based on a generative adversarial network, characterized in that: The method comprises: Obtaining images of the pelvic region; Segmenting the pelvic region image based on an image segmentation algorithm to generate a ulnar pelvic segmentation result; Inputting the ulnar pelvis segmentation result into an initial generator to generate an initial smooth pelvis image; Inputting the initial smoothed pelvic image into an initial discriminator to generate an initial discriminant result; Based on the initial smoothed pelvic image and the initial discrimination result, jointly training the initial generator and the initial discriminator to generate an optimized generator; The ulnar pelvis segmentation result is input into the optimization generator to generate a smooth pelvic region image.

2. The pelvic image smoothing method based on a generative adversarial network according to claim 1, characterized in that: The process of inputting the ulnar pelvis segmentation result into the initial generator to generate the ulnar pelvis segmentation result includes: Inputting the ulnar pelvis segmentation result into a first encoder to generate a first feature; inputting the first feature into a second encoder to generate a second feature; inputting the second feature into a third encoder to generate a third feature; Performing a feature fusion operation on the first feature, the second feature and the third feature to generate a comprehensive feature vector; The comprehensive feature vector is input into the first decoder to generate a fourth feature; the fourth feature and the third feature are input into the second decoder to generate a fifth feature; the fifth feature and the second feature are input into the third decoder to generate a ulnar pelvis segmentation result.

3. The pelvic image smoothing method based on a generative adversarial network according to claim 2, characterized in that: Before performing the feature fusion operation on the first feature, the second feature and the third feature, the following operation is also performed: Performing a global average pooling operation on the first feature to generate a weight vector for each channel; inputting the weight vector for each channel into a fully connected network to generate a normalized weight for each channel; multiplying the normalized weight for each channel by the first feature channel by channel to generate an attention-optimized first feature; Performing a global average pooling operation on the second feature to generate a weight vector for each channel; inputting the weight vector for each channel into a fully connected network to generate a normalized weight for each channel; multiplying the normalized weight for each channel by the second feature channel by channel to generate an attention-optimized second feature; Perform a global average pooling operation on the third feature to generate a weight vector for each channel; input the weight vector for each channel into a fully connected network to generate a normalized weight for each channel; multiply the normalized weight for each channel by the third feature channel by channel to generate an attention-optimized third feature.

4. The pelvic image smoothing method based on a generative adversarial network according to claim 1, characterized in that: The process of inputting the initial smoothed pelvis image into the initial discriminator to generate the authenticity of the corresponding pelvis segmentation result includes: The initial smoothed pelvic image is sequentially input into the first convolutional layer to generate the sixth feature; the sixth feature is input into the second convolutional layer to generate the seventh feature; the seventh feature is input into the third convolutional layer to generate the eighth feature; the eighth feature is input into the activation function to generate the authenticity of the pelvic segmentation result.

5. The pelvic image smoothing method based on a generative adversarial network according to claim 4, characterized in that: The process of inputting the seventh feature into the third convolutional layer to generate the eighth feature includes: Inputting the seventh feature into the third convolutional layer to generate an initial feature vector; Performing a feature map flattening operation on the initial feature vector to generate a node feature matrix; Calculating the attention weights between each team of nodes in the node feature matrix to obtain attention weights; Based on the node feature matrix and the attention weight, performing a multi-head attention calculation operation to generate a multi-head attention matrix; The multi-head attention matrix is ​​adjusted to a vector having the same dimension as the initial feature vector to generate an eighth feature.

6. The pelvic image smoothing method based on a generative adversarial network according to claim 1, characterized in that: The process of jointly training the initial generator and the initial discriminator includes: The weights in the initial generator are set to fixed preset parameters; the initial smoothed pelvic image and the ideal pelvic smoothed image are input into the initial discriminator to generate a predicted discriminant result; a first loss function is calculated according to the predicted discriminant result and the initial discriminant result; according to the first loss function, the weight of the initial discriminator is updated to generate an optimized discriminator; Based on the optimized discriminator, the ulnar pelvis segmentation result is input into the initial discriminator to generate an updated pelvic smoothing result; a second loss function and a third loss function of the updated pelvic smoothing result are calculated; based on the second loss function and the third loss function, the initial discriminator is weighted updated by a back propagation algorithm to generate the optimized generator; Alternating optimization operations are performed on the optimization discriminator and the optimization generator until the second loss function and the third loss function converge.

7. A pelvic image smoothing device based on a generative adversarial network, characterized in that: include: Image acquisition module: used to acquire pelvic area images; Adversarial network module: used for segmenting the pelvic region image based on an image segmentation algorithm to generate a ulnar pelvic segmentation result; inputting the ulnar pelvic segmentation result into an initial generator to generate an initial smooth pelvic image; Inputting the initial smoothed pelvic image into an initial discriminator to generate an initial discriminant result; Based on the initial smoothed pelvic image and the initial discrimination result, jointly training the initial generator and the initial discriminator to generate an optimized generator; Pelvic smoothing module: used for inputting the ulnar pelvic segmentation result into the optimization generator to generate a smooth pelvic area image.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the pelvic image smoothing method based on the adversarial generative network described in any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that The computer program product includes computer program codes, and when the computer program codes are executed on a computer, the computer is enabled to implement the pelvic image smoothing method based on a generative adversarial network as claimed in any one of claims 1 to 6.

10. An electronic terminal comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the pelvic image smoothing method based on the adversarial generative network according to any one of claims 1 to 6.

Images