Bone CT image segmentation method, device, computer equipment and storage medium

Abstract

The present invention discloses a bone CT image segmentation method, apparatus, computer equipment, and storage medium. The method comprises: obtaining a training image and its true mask; extracting a first feature map from the true mask, decoding the first feature map to obtain a generated mask, and discriminating between the true mask and the generated mask; constructing a first loss function based on the discrimination result, and updating parameters using the first loss function to obtain a pre-trained model; inputting the training image into the pre-trained model to extract a second feature map, generating a training image segmentation result for the second feature map, and constructing an initial segmentation model; calculating the mutual information between the first feature map and the second feature map to construct a second loss function, updating the parameters of the initial segmentation model, and obtaining a bone CT image segmentation model for image segmentation. The present invention learns image features through pre-training, and uses the mutual information to construct a loss function to update the model during image segmentation training, so that the model features are separated from each other, thereby improving the accuracy and stability of the model segmentation prediction.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a bone CT image segmentation method, device, computer equipment and storage medium. Background Art

[0002] Orthopedic diseases refer to a range of conditions affecting the human skeletal system, including fractures, osteoporosis, arthritis, and bone tumors. The likelihood of developing orthopedic diseases increases with age, and medical imaging plays a crucial role in the diagnosis and treatment of orthopedic diseases. Computed tomography (CT) imaging technology is a commonly used and important tool in orthopedic disease diagnosis. Clinically, CT imaging is widely used in a variety of orthopedic treatment scenarios, such as fracture diagnosis and assessment, and bone tumor diagnosis. This imaging modality provides high-resolution, three-dimensional images of bone and tissue structures, helping physicians accurately locate lesions and assess their extent and severity, thereby providing patients with more accurate diagnoses and treatment plans. Currently, all orthopedic disease diagnosis and treatment planning applications rely heavily on successful bone localization in CT images. Bone segmentation in bone CT images is a crucial processing step in CT imaging technology. However, traditional bone segmentation is performed manually or semi-automatically, which is inefficient and tedious, making it impractical in clinical practice. Therefore, high-quality and fast automatic bone segmentation in bone CT images is crucial for orthopedic disease analysis and treatment planning.

[0003] In recent years, due to the rapid development of deep learning technology, deep learning-based medical image segmentation methods have become popular. Deep learning-based segmentation methods enable networks to learn deep-level image features, no longer relying solely on the strength of bones in the image. This method has achieved excellent results, surpassing traditional segmentation methods in both speed and accuracy. However, existing deep learning-based segmentation methods still have some common drawbacks. For example, one method uses a graph cut segmentation method. After given a starting point, a spatial enhancement filter is used to enhance the bone image boundary, and then the traditional graph cut segmentation method is used to improve segmentation accuracy. The drawbacks of this method are: high image quality requirements and slow image processing speed. Another method uses a cascade method to cascade image segmentation networks at multiple scales. The cascade method and depth are calculated based on the image. Then, a corresponding segmentation training method is used to combine two-dimensional and three-dimensional segmentation methods to segment three-dimensional images in slice and whole forms. The drawbacks of this method are: the network is large and complex, the overall interpretability is poor, the training time is long, and it consumes a lot of resources. Another method incorporates distance weighting into the training of the image segmentation network. The target bone is first separated from the overall image, and then the edge weight image is obtained using the voxel calculation method. The edge weight image is used to deepen the model's attention to the edge to achieve the effect of fine segmentation of the fracture edge. The disadvantage of this method is that the edge weight needs to be calculated before segmentation, the mask accuracy is high, and the trained model has poor generalization and can only achieve high segmentation accuracy for bones in a certain part.

[0004] Therefore, how to provide a bone CT image segmentation method that can ensure segmentation accuracy, lower training difficulty and higher generalization is a problem to be solved by those skilled in the art. Summary of the Invention

[0005] Embodiments of the present invention provide a bone CT image segmentation method, apparatus, computer device, and storage medium, which aim to ensure segmentation quality while reducing the training difficulty of bone CT image segmentation and improving the generalization of bone CT image segmentation, thereby improving the accuracy and stability of model segmentation prediction.

[0006] In a first aspect, an embodiment of the present invention provides a bone CT image segmentation method, comprising:

[0007] Obtain bone CT training images and their true masks;

[0008] Inputting the true mask of the bone CT training image into an encoder for feature extraction to obtain a first feature map, decoding the first feature map through a decoder to obtain a corresponding generated mask, and then using a discriminator to distinguish the true mask from the generated mask;

[0009] Constructing a first loss function based on the discrimination result of the discriminator, and updating the parameters of the encoder, decoder and discriminator by using the first loss function to construct a pre-trained model;

[0010] Inputting the bone CT training image into the pre-trained model, extracting a second feature map from the bone CT training image using an encoder in the pre-trained model, and then generating a training image segmentation result from the second feature map using a decoder in the pre-trained model, thereby constructing an initial segmentation model;

[0011] Calculating the mutual information between the first feature map and the second feature map, constructing a second loss function based on the mutual information, and updating the parameters of the initial segmentation model using the second loss function to construct a bone CT image segmentation model;

[0012] The bone CT image segmentation model is used to perform image segmentation processing on the designated bone CT image.

[0013] In a second aspect, an embodiment of the present invention provides a bone CT image segmentation device, comprising:

[0014] An image acquisition unit, used for acquiring bone CT training images and their true masks;

[0015] a mask processing unit, configured to input the true mask of the bone CT training image into an encoder for feature extraction to obtain a first feature map, decode the first feature map through a decoder to obtain a corresponding generated mask, and then use a discriminator to distinguish the true mask from the generated mask;

[0016] A first parameter updating unit is configured to construct a first loss function based on a discrimination result of the discriminator, and to update parameters of the encoder, decoder, and discriminator using the first loss function to construct a pre-trained model;

[0017] a segmentation training unit, configured to input the bone CT training image into the pre-trained model, extract a second feature map from the bone CT training image using an encoder in the pre-trained model, and then generate a training image segmentation result from the second feature map using a decoder in the pre-trained model, thereby constructing an initial segmentation model;

[0018] a second parameter updating unit, configured to calculate the mutual information between the first feature map and the second feature map, construct a second loss function based on the mutual information, and update the parameters of the initial segmentation model using the second loss function to construct a bone CT image segmentation model;

[0019] The designated image segmentation unit is used to perform image segmentation processing on the designated bone CT image using the bone CT image segmentation model.

[0020] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the bone CT image segmentation method as described in the first aspect is implemented.

[0021] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the bone CT image segmentation method as described in the first aspect is implemented.

[0022] Embodiments of the present invention disclose a bone CT image segmentation method, apparatus, computer equipment, and storage medium. The method uses the real mask of the bone CT training image as input for training in a pre-training stage to obtain a first feature map, and obtains a pre-trained model with the ability to extract target shape features. Then, in the segmentation training stage, the bone CT training image is directly used as input for training to obtain a second feature map. Then, the mutual information between the first feature map and the second feature map is calculated, and a loss function is constructed based on the mutual information to update the model parameters to obtain a bone CT image segmentation model. In this way, the features extracted by the bone CT image segmentation model and the features extracted by the pre-training model are separated, so that the bone CT image segmentation model reduces its focus on the target shape features and learns more high-dimensional features. In this way, the generalization of the bone CT image segmentation model can be improved while ensuring segmentation accuracy and reducing the difficulty of model training, thereby improving the accuracy and stability of the model segmentation prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A schematic flow chart of a bone CT image segmentation method provided by an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of the principle of a bone CT image segmentation method provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a sub-process of a bone CT image segmentation method provided by an embodiment of the present invention;

[0027] Figure 4 A schematic block diagram of a bone CT image segmentation device provided by an embodiment of the present invention;

[0028] Figure 5 This is a sub-schematic block diagram of a device for bone CT image segmentation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0030] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0031] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0032] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0033] See below Figure 1 , Figure 1 A flowchart of a bone CT image segmentation method provided by an embodiment of the present invention specifically includes steps S101 to S106.

[0034] S101, obtaining a bone CT training image and its true mask;

[0035] S102: Input the true mask of the bone CT training image into an encoder for feature extraction to obtain a first feature map, decode the first feature map through a decoder to obtain a corresponding generated mask, and then use a discriminator to distinguish the true mask from the generated mask;

[0036] S103: constructing a first loss function based on the discrimination result of the discriminator, and updating the parameters of the encoder, decoder and discriminator by using the first loss function to construct a pre-trained model;

[0037] S104: input the bone CT training image into the pre-trained model, extract the bone CT training image using the encoder in the pre-trained model to obtain a second feature map, and then generate a training image segmentation result using the decoder in the pre-trained model for the second feature map, thereby constructing an initial segmentation model;

[0038] S105. Calculate the mutual information between the first feature map and the second feature map, construct a second loss function based on the mutual information, and update the parameters of the initial segmentation model using the second loss function to construct a bone CT image segmentation model.

[0039] S106 : Perform image segmentation processing on the designated bone CT image using the bone CT image segmentation model.

[0040] Combine Figure 2 As shown, in this embodiment, the training steps of the bone CT image segmentation model are divided into two stages: pre-training and segmentation training. In the pre-training stage, it is first necessary to obtain the bone CT training image for training and the real mask corresponding to the bone CT training image. Secondly, the real mask is input into the encoder for feature extraction, and then the first feature map obtained by the feature extraction is decoded by the decoder to obtain the corresponding generated mask. Then, the real mask and the generated mask are input into the discriminator for discrimination, and a first loss function for updating parameters is constructed according to the discrimination result. Finally, the parameters of the encoder, decoder and discriminator are updated by the first loss function to construct a pre-trained model. After that, the segmentation training stage begins. In the segmentation training stage, the bone CT training image is first input into the pre-training model constructed in the pre-training stage, and the encoder in the pre-training model is used to extract features to obtain the second feature map of the bone CT training image. Secondly, the decoder in the pre-training model performs segmentation prediction on the second feature map output by the encoder, and generates the training image segmentation result to construct an initial segmentation model. Then, the first feature map and the second feature map are combined to calculate the mutual information. Then, the second function is constructed according to the calculated mutual information. Finally, the parameters of the initial segmentation model are updated through the second function, thereby obtaining a bone CT image segmentation model.

[0041] In this embodiment, the pre-training phase uses the real mask of the bone CT training image as input for training, obtaining a first feature map and a pre-trained model capable of extracting target shape features. Then, in the segmentation training phase, the bone CT training image is directly used as input for training, obtaining a second feature map. The mutual information between the first and second feature maps is then calculated, and a loss function is constructed based on the mutual information to update the model parameters, resulting in a bone CT image segmentation model. This approach allows the features extracted by the bone CT image segmentation model to be further separated from those extracted by the pre-training model, reducing the focus on target shape features and learning more high-dimensional features. This improves the generalization of the bone CT image segmentation model while maintaining segmentation accuracy and reducing the difficulty of model training, thereby enhancing the accuracy and stability of the model's segmentation predictions. In particular, this embodiment can be trained using only a single type of bone CT training image, without the need for fine-tuning the input image, and can achieve good segmentation results for bone CT images of different locations. Among them, mutual information represents the correlation between two sets of features. For example, when the mutual information decreases, it means that the correlation between the two sets of features decreases, that is, the two sets of features move away from each other. Using the decrease in correlation as a metric for model training can achieve the effect of increasing the generalization of the model.

[0042] It should also be noted that after the bone CT image segmentation model is trained, when the bone CT image segmentation model is used to perform image segmentation processing on the specified bone CT image, the processing steps of the bone CT image segmentation model are as follows: Figure 2 As shown in the segmentation training stage, the specified bone CT image is first input into the encoder for feature extraction, the corresponding feature map is extracted, and then the feature map is input into the decoder for image segmentation, thereby obtaining the image segmentation result.

[0043] In specific application scenarios, the true mask of the bone CT training image can be obtained by directly acquiring existing data, or it can be drawn by a professional doctor based on information such as the type, number, and distribution of bones in the bone CT training image. In addition, before the model training process is carried out in this embodiment, hyperparameters need to be set, and all training data (bone CT training images and their true masks) are iteratively trained according to the hyperparameters. For example, a stochastic gradient descent optimization algorithm can be used to iteratively train all training data 500 times, and the initial learning rate is set to 0.0001, the learning rate adjustment strategy is 0.01*(1-(epoch / 100)^0.9), and the batch processing capacity is set to 5. The model weights are saved after the 50th generation of training.

[0044] In addition, in other embodiments, this embodiment may also use other types of medical images as training images, and obtain corresponding prediction models through training through the steps described above, thereby using the trained segmentation prediction model to perform segmentation prediction on the medical images of this type.

[0045] In one embodiment, step S101 includes:

[0046] After acquiring the bone CT training image, resampling the bone CT training image according to a preset sampling interval to adjust the image size of the bone CT training image;

[0047] The resampled bone CT training image is subjected to image standardization processing.

[0048] In this embodiment, the bone CT training images are first resampled according to a preset sampling interval, thereby adjusting the image size of the bone CT training images so that their image size meets the input requirements of model training. After the size adjustment, the bone CT training images need to be normalized. The purpose of image normalization is to make the image data conform to the data distribution law, for example, to make the distribution mean of the bone CT training images 0 and the variance 1.

[0049] In practical applications, SimpleITK (a software specialized in medical imaging) can be used to resample and normalize images. After training a bone CT image segmentation model, when segmentation is required for a specific bone CT image, the image can be resampled and normalized. The weights of the bone CT image segmentation model are then read, and the image is divided into multiple image blocks. The segmentation results are then reconstructed using unit sliding window inference, significantly improving the model's efficiency and accuracy.

[0050] Combine Figure 3 As shown, in one embodiment, step S102 includes steps S301 to S304.

[0051] S301, downsampling the true mask by the encoder to extract the first feature map;

[0052] Specifically, step S301 includes:

[0053] The true mask is input into a plurality of visual converters arranged in series, and the output result of the last visual converter is used as the first feature map; wherein each visual converter includes a vector embedding layer, an attention layer, a multi-layer perceptron and a normalization layer in sequence.

[0054] The encoder in this step is continuously equipped with multiple visual transformers (VIT, Vision Transformer). The real mask is sequentially input into the multiple visual transformers set in succession in the encoder, and the real mask is processed in sequence by the vector embedding layer, attention layer, multi-layer perceptron and normalization layer in each visual transformer. When the last visual transformer is processed and the output result is obtained, it is used as the first feature map for subsequent processing. Among them, after the real mask passes through the vector embedding layer in the visual transformer, its size will become 1 / 16 before the input vector embedding layer, and its channel will also become the size set by the hyperparameter. In addition, the size of the output image of each visual transformer is equal to the size of the image input. In actual application scenarios, the number of visual transformers in the encoder depends on the specific encoder design and application requirements. Specifically, the appropriate number of visual transformers can be selected according to the complexity of the task and the limitation of computing resources. For example, 12 visual transformers can be set continuously in the encoder.

[0055] S302, obtaining the number of target bone types contained in the bone CT training image according to the true mask, and initializing the latent space code according to the number of target bone types to obtain an initial latent space code;

[0056] This step initializes the latent space encoding according to the number of target bone types, which enables the model to adapt and learn the characteristics and patterns of bones in bone CT training images more quickly during training, thereby improving the accuracy of subsequent segmentation predictions.

[0057] S303, jump-connecting the first feature map and the initial latent space encoding and inputting them into the decoder for decoding and projection to obtain a generation mask of the bone CT training image;

[0058] Specifically, step S303 includes:

[0059] Performing a first skip connection on the first feature map and the initial latent space code, and inputting the result of the first skip connection into the decoder having multiple convolutional layers;

[0060] Performing a first convolution process on the result of the first skip connection using the first convolution layer of the decoder, and outputting the result of the first convolution process;

[0061] Performing a second skip connection on the result of the first convolution process and the initial latent space code, and inputting the result of the second skip connection into the second convolution layer for a second convolution process, and then outputting the result of the second convolution process by the second convolution layer;

[0062] Inputting the result of the second convolution process to the third convolution layer for a third convolution process, and the third convolution layer outputs the result of the third convolution process;

[0063] The result of the third convolution processing is input to the next convolution layer to continue the convolution processing, and the next convolution layer outputs the corresponding convolution processing result, and so on, until the last convolution layer outputs the last convolution processing result, and the last convolution processing result is used as the generated mask.

[0064] In this step, the first feature map is first skip-connected with the initial latent space code. The result of the first skip connection is then input into a decoder having multiple convolutional layers arranged in sequence. The result of the first skip connection is then skip-connected and processed multiple times within the decoder, ultimately yielding a generated mask. This embodiment, by skip-connecting the feature map with the initial latent space code, strengthens the control of the initial latent space code over the decoder, preventing information from being lost in long-distance computations. This allows low-level feature information to be directly passed to higher levels, helping to retain more spatial information and details, thereby generating a more accurate generated mask.

[0065] Among them, the result of the first jump connection first passes through the first convolution layer for the first convolution processing, and the result of the first convolution processing is output. Then the result of the first convolution processing will be again processed by the second jump connection with the initial latent space code. The result of the second jump connection processing is then passed through the second convolution layer for the second convolution processing. The result of the second convolution processing is directly passed through the third convolution layer for the third convolution processing, and so on until the last convolution layer outputs the last convolution processing result, and the last convolution processing result is output as the generated mask.

[0066] In a specific embodiment, the multi-layer convolution layer of the decoder is a plurality of decoding modules, each of which includes convolution, activation function and normalization operations. Before the feature map passes through the decoding module, it will first undergo a transposed convolution, and its size will be doubled by upsampling. A jump connection is used to splice the shallow feature map of the corresponding size with it, and then it is input into the decoding module for processing. The number of decoding modules here can be adjusted according to actual conditions, such as setting four decoding modules. In addition, in addition to the single jump connection before the feature map is input into the decoder, the number of jump connections performed in the decoder can also be adjusted according to different actual conditions. For example, a larger number of jump connections can be set to generate a more accurate generation mask, or a smaller number of jump connections can be set to avoid destroying the feature map information, etc.

[0067] S304: Input the real mask and the generated mask into the discriminator for discrimination, and output a discrimination result including a real value and a false value through an output function in the discriminator.

[0068] In this step, the discriminator for distinguishing the real mask from the generated mask consists of four visual converters, three downsampling modules and an output module. The downsampling module contains convolution, activation function and normalization operations. The number of channels of the real mask and the generated mask input to the downsampling module will become twice the original number, and the image size will become 1 / 3 of the original number; the output module includes a linear layer and a normalization layer. The output module will scale the input features to pseudo values ​​or real values, which are used to represent the discriminator to judge whether the input feature is a real mask. Specifically, if the discrimination result is a pseudo value, it means that the corresponding input feature is not a real mask, and if the discrimination result is a real value, it means that the corresponding input feature is a real mask.

[0069] In this embodiment, the true mask is first used as the input of the encoder, and the encoder performs downsampling processing on it to extract the first feature map, and then obtains the number of target bone types contained in the corresponding bone CT training image based on the true mask, and initializes the latent space encoding according to the target bone type number to obtain the initial latent space encoding, and then the first feature map and the initial latent space encoding are jump-connected, and the first feature map and the initial latent space encoding after the jump connection are input to the decoder, and the decoder decodes and projects them, that is, the first feature map and the initial latent space encoding are projected back to the image space to generate a generated mask of the bone CT training image, and finally the true mask and the generated mask are input to the discriminator for discrimination, and the output function in the discriminator outputs the corresponding discrimination result, which contains the true value ( Figure 2 "1" in the middle) and dummy values ​​( Figure 2 (with "0" in the middle).

[0070] In one embodiment, step S103 includes:

[0071] Reverse reasoning the generated mask to obtain a reverse latent space encoding;

[0072] Calculating the entropy of the true mask and the true value in the discrimination result to obtain a first cross entropy loss function; calculating the entropy of the generated mask and the pseudo value in the discrimination result to obtain a second cross entropy loss function; calculating the entropy of the reverse latent space encoding and the true mask to obtain a third cross entropy loss function;

[0073] According to the following formula, the first cross entropy loss function, the second cross entropy loss function and the third cross entropy loss function are summed to obtain the first loss function:

[0074]

[0075] in, represents the first loss function, represents the first cross entropy loss function, D represents the decoder, G represents the discriminator, f' represents the first feature map, c represents the initial latent space encoding, 0 represents the pseudo value, represents the second cross entropy loss function, y represents the true mask, 1 represents the true value, represents the third cross entropy loss function, and c' represents the reverse latent space encoding.

[0076] In this embodiment, reverse reasoning is first performed on the generated mask to obtain the corresponding reverse latent space encoding, and then multiple cross-entropy loss functions are calculated by combining the reverse latent space encoding, the true mask, the generated mask, and the discrimination result. The calculated cross-entropy loss functions are summed to obtain a first loss function for updating the model parameters. Among them, the first cross-entropy loss function is obtained by calculating the entropy of the true mask and the true value in the discrimination result, and the second cross-entropy loss function is obtained by calculating the entropy of the generated mask and the pseudo value in the discrimination result. This can improve the discriminator's ability to distinguish between the true mask and the generated mask, thereby prompting the decoder to generate a more accurate mask and compete with the discriminator, so that the model learns more features; the third cross-entropy loss function is obtained by calculating the entropy of the reverse latent space encoding and the true mask, which can make the relevant features learned by the model focus on the appearance features, and ultimately enable the model to generate more accurate segmentation results.

[0077] This embodiment can obtain the reverse latent space encoding through reverse reasoning, so that the skeletal features of the training image are strongly associated with the latent space encoding, while making the discriminator and the decoder compete, urging the decoder to generate an image that is closer to the real mask, thereby improving accuracy and achieving Nash equilibrium.

[0078] In a specific application scenario, the output module of the discriminator has two output heads, one output head is used to calculate and output the first cross entropy loss function and the second cross entropy loss function, and the other output head is used to calculate and output the third cross entropy loss function.

[0079] In some other application scenarios, the model parameters can also be updated by calculating the mutual information in the pre-training stage and using the mutual information in this stage instead of the cross entropy loss function.

[0080] In one embodiment, step S105 includes:

[0081] The mutual information between the first feature map and the second feature map is calculated according to the following formula:

[0082]

[0083] Among them, Isg represents the mutual information, I represents the mutual information operator, H(f) represents the entropy of the second feature map, H(f') represents the entropy of the first feature map, H(f,f') represents the entropy of the joint distribution of the first feature map and the second feature map, P represents the joint probability distribution, and log represents the logarithmic function;

[0084] The second loss function is constructed by combining the mutual information and the Dice loss function:

[0085]

[0086] in, represents the second loss function, and DiceLoss represents the Dice loss function.

[0087] In this embodiment, mutual information is obtained by calculating the entropy of each item. Entropy is an indicator of feature uncertainty, and mutual information measures the correlation between two sets of features by comparing the entropy of the two sets of features. Mutual information represents the correlation between the two sets of features. Since the direction of updating the model parameters is to expect the mutual information to decrease during the calculation process, the mutual information can be used as part of the loss function to update the model parameters. Using the decrease in correlation as a metric for model training can achieve the effect of increasing the generalization of the model. In addition, this embodiment also uses the Dice loss function of the model as part of the second loss function, and adds it to the mutual information to obtain the second loss function. The Dice loss function is a commonly used loss function used to measure the similarity between the segmentation results generated by the model and the actual segmentation labels. In addition, the mutual information in this embodiment can be calculated using a continuous calculation method or a discrete calculation method.

[0088] Figure 4 This is a schematic block diagram of a bone CT image segmentation device 400 provided in an embodiment of the present invention. The device 400 includes:

[0089] An image acquisition unit 401 is used to acquire a bone CT training image and its true mask;

[0090] The mask processing unit 402 is configured to input the true mask of the bone CT training image into an encoder for feature extraction to obtain a first feature map, decode the first feature map through a decoder to obtain a corresponding generated mask, and then use a discriminator to distinguish the true mask from the generated mask.

[0091] A first parameter updating unit 403 is configured to construct a first loss function based on the discrimination result of the discriminator, and to update the parameters of the encoder, decoder, and discriminator using the first loss function to construct a pre-trained model;

[0092] a segmentation training unit 404 configured to input the bone CT training image into the pre-trained model, extract a second feature map from the bone CT training image using an encoder in the pre-trained model, and then generate a training image segmentation result from the second feature map using a decoder in the pre-trained model, thereby constructing an initial segmentation model;

[0093] A second parameter updating unit 405 is configured to calculate the mutual information between the first feature map and the second feature map, construct a second loss function based on the mutual information, and update the parameters of the initial segmentation model using the second loss function to construct a bone CT image segmentation model;

[0094] The designated image segmentation unit 406 is configured to perform image segmentation processing on the designated bone CT image using the bone CT image segmentation model.

[0095] In one embodiment, the image acquisition unit 401 includes:

[0096] a resampling unit, configured to, after acquiring the bone CT training image, resample the bone CT training image according to a preset sampling interval, so as to adjust the image size of the bone CT training image;

[0097] The standardization processing unit is used to perform image standardization processing on the resampled bone CT training image.

[0098] Combine Figure 5 As shown, in one embodiment, the mask processing unit 402 includes:

[0099] A downsampling processing unit 501 is configured to perform downsampling processing on the true mask through the encoder to extract the first feature map;

[0100] A code acquisition unit 502 is configured to acquire the number of target bone types contained in the bone CT training image according to the true mask, and initialize a latent space code according to the number of target bone types to obtain an initial latent space code;

[0101] A decoding and projection unit 503 is configured to perform a jump connection between the first feature map and the initial latent space code and input the resultant data into the decoder for decoding and projection, so as to obtain a generation mask of the bone CT training image;

[0102] The mask discrimination unit 504 is configured to input the real mask and the generated mask into the discriminator for discrimination, and output a discrimination result including a real value and a false value through an output function in the discriminator.

[0103] In one embodiment, the first parameter updating unit 403 includes:

[0104] A reverse reasoning unit, configured to reversely reason the generated mask to obtain a reverse latent space encoding;

[0105] A cross entropy calculation unit is used to calculate the entropy of the true mask and the true value in the discrimination result to obtain a first cross entropy loss function; calculate the entropy of the generated mask and the pseudo value in the discrimination result to obtain a second cross entropy loss function; calculate the entropy of the reverse latent space encoding and the true mask to obtain a third cross entropy loss function;

[0106] The first loss calculation unit is configured to sum the first cross entropy loss function, the second cross entropy loss function, and the third cross entropy loss function according to the following formula to obtain the first loss function:

[0107]

[0108] in, represents the first loss function, represents the first cross entropy loss function, D represents the decoder, G represents the discriminator, f' represents the first feature map, c represents the initial latent space encoding, 0 represents the pseudo value, represents the second cross entropy loss function, y represents the true mask, 1 represents the true value, represents the third cross entropy loss function, and c' represents the reverse latent space encoding.

[0109] In one embodiment, the second parameter updating unit 405 includes:

[0110] A mutual information calculation unit is used to calculate the mutual information between the first feature map and the second feature map according to the following formula:

[0111]

[0112] Among them, I sg represents the mutual information, I represents the mutual information operator, H(f) represents the entropy of the second feature map, H(f') represents the entropy of the first feature map, H(f,f') represents the entropy of the joint distribution of the first feature map and the second feature map, P represents the joint probability distribution, and log represents the logarithmic function;

[0113] The second loss calculation unit is configured to construct the second loss function by combining the mutual information and the Dice loss function:

[0114]

[0115] in, represents the second loss function, and DiceLoss represents the Dice loss function.

[0116] In one embodiment, the decoding and projection unit 503 includes:

[0117] a skip connection unit, configured to perform a first skip connection on the first feature map and the initial latent space code, and input the result of the first skip connection into the decoder having multiple convolutional layers;

[0118] A first convolution unit is configured to perform a first convolution process on the result of the first skip connection using the first convolution layer of the decoder, and output the result of the first convolution process;

[0119] A second convolution unit is configured to perform a second skip connection on the result of the first convolution process and the initial latent space code, and input the result of the second skip connection to the second convolution layer for a second convolution process, and then the second convolution layer outputs the result of the second convolution process;

[0120] a third convolution unit, configured to input the result of the second convolution process into a third convolution layer for a third convolution process, and the third convolution layer outputs the result of the third convolution process;

[0121] The mask generation unit is used to input the result of the third convolution processing to the next convolution layer to continue the convolution processing, and the next convolution layer outputs the corresponding convolution processing result, and so on, until the last convolution layer outputs the last convolution processing result, and the last convolution processing result is used as the generated mask.

[0122] In one embodiment, the downsampling processing unit 501 includes:

[0123] A visual conversion unit is used to input the true mask into a plurality of visual converters arranged in series, and use the output result of the last visual converter as the first feature map; wherein each visual converter includes a vector embedding layer, an attention layer, a multi-layer perceptron and a normalization layer in sequence.

[0124] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, and they will not be repeated here.

[0125] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed, the computer program can implement the steps provided in the above embodiments. The storage medium can include a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, among other media capable of storing program code.

[0126] The present invention also provides a computer device that may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, the steps provided in the above embodiment can be implemented. Of course, the computer device may also include various network interfaces, a power supply, and other components.

[0127] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0128] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

Claims

1. A bone CT image segmentation method, characterized in that: include: Obtain bone CT training images and their true masks; Downsampling the true mask through an encoder to extract a first feature map; Acquire the number of target bone types contained in the bone CT training image according to the true mask, and initialize the latent space encoding according to the number of target bone types to obtain an initial latent space encoding; The first feature map and the initial latent space encoding are jump-connected and input into a decoder for decoding and projection to obtain a generation mask of the bone CT training image; Inputting the true mask and the generated mask into a discriminator for discrimination, and outputting a discrimination result including a true value and a false value by an output function in the discriminator; Constructing a first loss function based on the discrimination result of the discriminator, and updating the parameters of the encoder, decoder and discriminator by using the first loss function to construct a pre-trained model; Inputting the bone CT training image into the pre-trained model, extracting a second feature map from the bone CT training image using an encoder in the pre-trained model, and then generating a training image segmentation result from the second feature map using a decoder in the pre-trained model, thereby constructing an initial segmentation model; Calculating the mutual information between the first feature map and the second feature map, constructing a second loss function based on the mutual information, and updating the parameters of the initial segmentation model using the second loss function to construct a bone CT image segmentation model; Performing image segmentation processing on the specified bone CT image using the bone CT image segmentation model; The constructing of a first loss function based on the discrimination result of the discriminator includes: Reverse reasoning the generated mask to obtain a reverse latent space encoding; Calculating the entropy of the true mask and the true value in the discrimination result to obtain a first cross entropy loss function; calculating the entropy of the generated mask and the pseudo value in the discrimination result to obtain a second cross entropy loss function; calculating the entropy of the reverse latent space encoding and the true mask to obtain a third cross entropy loss function; According to the following formula, the first cross entropy loss function, the second cross entropy loss function and the third cross entropy loss function are summed to obtain the first loss function: in, represents the first loss function, represents the first cross entropy loss function, D represents the decoder, G represents the discriminator, f' represents the first feature map, c represents the initial latent space encoding, 0 represents the pseudo value, represents the second cross entropy loss function, y represents the true mask, 1 represents the true value, represents the third cross entropy loss function, and c' represents the reverse latent space encoding.

2. The bone CT image segmentation method according to claim 1, characterized in that: The step of obtaining a bone CT training image and its true mask includes: After acquiring the bone CT training image, resampling the bone CT training image according to a preset sampling interval to adjust the image size of the bone CT training image; The resampled bone CT training image is subjected to image standardization processing.

3. The bone CT image segmentation method according to claim 1, characterized in that: The calculating the mutual information between the first feature map and the second feature map, and constructing a second loss function based on the mutual information, includes: The mutual information between the first feature map and the second feature map is calculated according to the following formula: Among them, I sg represents the mutual information, I represents the mutual information operator, H(f) represents the entropy of the second feature map, H(f') represents the entropy of the first feature map, H(f,f') represents the entropy of the joint distribution of the first feature map and the second feature map, p represents the joint probability distribution, and log represents the logarithmic function; The second loss function is constructed by combining the mutual information and the Dice loss function: in, represents the second loss function, and DiceLoss represents the Dice loss function.

4. The bone CT image segmentation method according to claim 1, characterized in that: The step of jump-connecting the first feature map and the initial latent space encoding and inputting the resultant data into the decoder for decoding and projection to obtain a generation mask for the bone CT training image includes: Performing a first skip connection on the first feature map and the initial latent space code, and inputting the result of the first skip connection into the decoder having multiple convolutional layers; Performing a first convolution process on the result of the first skip connection using the first convolution layer of the decoder, and outputting the result of the first convolution process; Performing a second skip connection on the result of the first convolution process and the initial latent space code, and inputting the result of the second skip connection into the second convolution layer for a second convolution process, and then outputting the result of the second convolution process by the second convolution layer; Inputting the result of the second convolution process to the third convolution layer for a third convolution process, and the third convolution layer outputs the result of the third convolution process; The result of the third convolution processing is input to the next convolution layer to continue the convolution processing, and the next convolution layer outputs the corresponding convolution processing result, and so on, until the last convolution layer outputs the last convolution processing result, and the last convolution processing result is used as the generated mask.

5. The bone CT image segmentation method according to claim 1, characterized in that: The downsampling the true mask by the encoder to extract the first feature map includes: The true mask is input into a plurality of visual converters arranged in series, and the output result of the last visual converter is used as the first feature map; wherein each visual converter includes a vector embedding layer, an attention layer, a multi-layer perceptron and a normalization layer in sequence.

6. A bone CT image segmentation device, characterized in that: include: An image acquisition unit, used for acquiring bone CT training images and their true masks; a downsampling processing unit, configured to perform downsampling processing on the true mask through an encoder to extract a first feature map; a code acquisition unit, configured to acquire the number of target bone types contained in the bone CT training image according to the true mask, and initialize the latent space code according to the number of target bone types to obtain an initial latent space code; a decoding and projection unit, configured to perform a jump connection between the first feature map and the initial latent space encoding and input the resultant data into a decoder for decoding and projection, so as to obtain a generation mask of the bone CT training image; a mask discrimination unit, configured to input the true mask and the generated mask into a discriminator for discrimination, and output a discrimination result including a true value and a false value by an output function in the discriminator; A first parameter updating unit is configured to construct a first loss function based on a discrimination result of the discriminator, and to update parameters of the encoder, decoder, and discriminator using the first loss function to construct a pre-trained model; a segmentation training unit, configured to input the bone CT training image into the pre-trained model, extract a second feature map from the bone CT training image using an encoder in the pre-trained model, and then generate a training image segmentation result from the second feature map using a decoder in the pre-trained model, thereby constructing an initial segmentation model; a second parameter updating unit, configured to calculate the mutual information between the first feature map and the second feature map, construct a second loss function based on the mutual information, and update the parameters of the initial segmentation model using the second loss function to construct a bone CT image segmentation model; a designated image segmentation unit, configured to perform image segmentation processing on the designated bone CT image using the bone CT image segmentation model; The first parameter updating unit includes: A reverse reasoning unit, configured to reversely reason the generated mask to obtain a reverse latent space encoding; A cross entropy calculation unit is used to calculate the entropy of the true mask and the true value in the discrimination result to obtain a first cross entropy loss function; calculate the entropy of the generated mask and the pseudo value in the discrimination result to obtain a second cross entropy loss function; calculate the entropy of the reverse latent space encoding and the true mask to obtain a third cross entropy loss function; The first loss calculation unit is configured to sum the first cross entropy loss function, the second cross entropy loss function, and the third cross entropy loss function according to the following formula to obtain the first loss function: in, represents the first loss function, represents the first cross entropy loss function, D represents the decoder, G represents the discriminator, f' represents the first feature map, c represents the initial latent space encoding, 0 represents the pseudo value, represents the second cross entropy loss function, y represents the true mask, 1 represents the true value, represents the third cross entropy loss function, and c' represents the reverse latent space encoding.

7. A computer device, characterized in that: The device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the bone CT image segmentation method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the bone CT image segmentation method according to any one of claims 1 to 5 is implemented.

Images