A method and system for multi-organ segmentation of CT images paired with text of voxels

Abstract

The application provides a voxel-textual-pairing CT image multi-organ segmentation method and system, and relates to the field of medical image processing.The method comprises the following steps: constructing a text template to describe an organ to be segmented and obtaining a category text; acquiring a sample CT image; constructing a feature coding network to code the sample CT image into a feature map and code the category text into a text feature vector; constructing a conditional variational autoencoder to generate an uncertainty pseudo-heat map for the feature map; extracting voxel feature vectors corresponding to each voxel point in the feature map to calculate voxel-textual-pairing similarity and construct a pairing result pseudo-heat map, using the pairing result pseudo-heat map as the original input, using the feature map as the conditional input, and performing self-supervised training on the conditional variational autoencoder; acquiring a CT image to be segmented; and calculating the voxel-textual-pairing similarity through the trained feature coding network as the multi-organ segmentation result of the CT image.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and in particular to a method and system for multi-organ segmentation of CT images using voxel-text pairing. Background Technology

[0002] Medical image segmentation is fundamental to medical tasks. In computer vision tasks based on medical images, such as disease prediction and prognosis assessment, lesion segmentation is an essential preparatory step. Designing multi-organ joint segmentation methods with strong interpretability is of great significance for medical image segmentation and even downstream medical tasks. Cross-modal contrastive learning techniques pair signals from different modalities, such as text and images, using a certain metric to map data from different sources to the same feature space. During inference, it can generate the output of another modality based on a prompt from one modality, exhibiting strong generalization performance. Deep learning has made significant progress in the field of medical image segmentation, especially the introduction of convolutional neural networks (CNNs) and generative adversarial networks (GANs), which have greatly improved segmentation accuracy and driven the continuous development of multi-organ joint segmentation techniques.

[0003] However, most current medical segmentation methods are trained and inferred only on a single dataset or for a single segmentation target. Furthermore, image information from a single modality often fails to fully capture complex medical information, resulting in poor model generalization and reduced recognition accuracy. In addition, the "black box" problem of deep learning models is very pronounced, leading to poor interpretability of the output results and reduced clinical trust. Summary of the Invention

[0004] To address the current limitations of most medical segmentation methods, which train and infer only on a single dataset or for a single segmentation target, and the fact that single-modal image information often fails to fully capture complex medical information, resulting in poor model generalization and reduced recognition accuracy, this invention provides a voxel-text pairing method and system for multi-organ segmentation of CT images.

[0005] The technical solutions provided by the embodiments of the present invention are as follows:

[0006] First aspect:

[0007] This invention provides a method for multi-organ segmentation of CT images using voxel-text pairing, comprising:

[0008] S1: Construct a text template, describe the organs to be segmented using the text template, and obtain category text;

[0009] S2: Acquire sample CT images;

[0010] S3: Construct a feature encoding network, and through the feature encoding network, encode the sample CT image into a feature map and the category text into a text feature vector;

[0011] S4: Construct a conditional variational autoencoder, and generate an uncertain pseudo-heatmap from the feature map using the conditional variational autoencoder;

[0012] S5: Sample a first preset number of voxel points with the highest uncertainty from the uncertain pseudo-heatmap, and randomly sample a second preset number of voxel points from other regions of the feature map;

[0013] S6: Extract the voxel feature vector corresponding to each voxel point in the feature map;

[0014] S7: Calculate the voxel text pairing similarity based on the voxel feature vectors corresponding to each voxel point;

[0015] S8: Based on the similarity of voxel text pairings, construct a pseudo heatmap of the pairing results, and train the feature encoding network based on the gradient of the similarity loss function;

[0016] S9: Using the pseudo-heatmap of the pairing result as the original input of the conditional variational autoencoder, and using the feature map output by the feature encoding network as the conditional input of the conditional variational autoencoder, the conditional variational autoencoder is trained under self-supervised supervision.

[0017] S10: Acquire the CT image to be segmented;

[0018] S11: Calculate the voxel text pairing similarity through the trained feature encoding network, and use it as the multi-organ segmentation result of the obtained CT image to be segmented.

[0019] The second aspect:

[0020] This invention provides a voxel-text paired CT image multi-organ segmentation system, comprising:

[0021] processor;

[0022] A memory storing computer-readable instructions, which, when executed by the processor, implement the voxel-text pairing method for multi-organ segmentation of CT images as described in the first aspect.

[0023] Third aspect:

[0024] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the voxel text pairing method for multi-organ segmentation of CT images as described in the first aspect.

[0025] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0026] (1) In this invention, by constructing a conditional variational autoencoder, an uncertainty pseudo-heatmap is generated for the feature map. A first preset number of voxel points with the highest uncertainty are sampled from the uncertainty pseudo-heatmap. A second preset number of voxel points are randomly sampled from other regions of the feature map to extract the voxel feature vectors corresponding to each voxel point in the feature map. Based on the voxel feature vectors corresponding to each voxel point, the voxel text pairing similarity is calculated, which enhances the generalization ability of the model and further improves the accuracy of recognition.

[0027] (2) In this invention, a pseudo-heatmap of the pairing results is constructed based on the similarity of the voxel text pairings, and a feature encoding network is trained according to the gradient of the similarity loss function. The pseudo-heatmap of the pairing results is used as the original input of the conditional variational autoencoder, and the feature map output by the feature encoding network is used as the conditional input of the conditional variational autoencoder. Self-supervised training is then performed on the conditional variational autoencoder. This increases the interpretability of the model and further improves clinical trust. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart illustrating a method for multi-organ segmentation of CT images using voxel-text pairing, provided in an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the structure of a voxel text pairing CT image multi-organ segmentation system provided in an embodiment of the present invention. Detailed Implementation

[0031] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0032] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0033] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0034] In this embodiment of the invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0035] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0036] Reference manual attached Figure 1 The diagram illustrates a flowchart of a voxel-text pairing method for multi-organ segmentation of CT images provided by an embodiment of the present invention.

[0037] This invention provides a method for multi-organ segmentation of CT images using voxel-text pairing. This method can be implemented using a voxel-text pairing CT image multi-organ segmentation device, which can be a terminal or a server. The processing flow of the voxel-text pairing CT image multi-organ segmentation method may include the following steps:

[0038] S1 constructs a text template, which describes the organs to be segmented, and obtains category text.

[0039] The text template is a predefined format or structure that contains relevant information about the target organ, enabling the model to understand and associate the text description with the corresponding region in the CT image.

[0040] In this invention, text templates provide clear target descriptions, helping the model accurately understand the organs that need to be segmented. Furthermore, by constructing and using text templates, the complementary information of text and images can be fully utilized, improving model performance and applicability while reducing reliance on large amounts of labeled data.

[0041] S2: Acquire sample CT images.

[0042] S3: Construct a feature encoding network to encode sample CT images into feature maps and class text into text feature vectors.

[0043] Feature encoding networks play a crucial role in medical image segmentation, as they encode input CT images and category text into feature maps and text feature vectors for subsequent processing and segmentation.

[0044] Optionally, the feature encoding network includes a multi-level image encoder, a text encoder, and a multi-scale feature fusion decoder.

[0045] In one possible implementation, S3 specifically includes:

[0046] S301: Encodes CT images into feature maps using multiple residual 3D convolutional modules in a multi-level image encoder.

[0047] u i,0 =f i,0 (x i ,θ i,0 )

[0048] u i,k =f i,k (u i,k-1 ,θ i,k ),k≥1

[0049] Among them, f i,k Represents the residual 3D convolution module, x i Represents the input image, θ i,0 θ represents the parameters contained in the initial residual 3D convolution module. i,k u represents the parameters contained in the k-th residual 3D convolutional module. i,k u represents the feature output by the k-th residual 3D convolutional module. i,k-1 This represents the input feature of the k-th residual 3D convolutional module, and also represents the output feature of the (k-1)-th residual 3D convolutional module.

[0050] S302: Encodes categorical text into text feature vectors using multiple self-attention modules in the text encoder.

[0051] y t =f t (x t ,θ t )

[0052] Among them, f t Represents a text encoder, y t x represents the text feature vector. t Represents category text, θ t This indicates the parameters contained in the text encoder.

[0053] In this invention, the residual 3D convolution module in a multi-level image encoder can extract detailed and global information from CT images layer by layer, ensuring that the model can accurately capture the shape and location of organs. Simultaneously, encoding category text into feature vectors helps the model better match text descriptions with corresponding regions in the image.

[0054] Furthermore, through multi-level and multi-scale feature extraction, the model can adapt to information at different resolutions and levels, improving its generalization ability on various CT images. Simultaneously, multi-scale feature extraction for organs with complex structures enhances the model's ability to handle complex situations.

[0055] S4: Construct a conditional variational autoencoder to generate an uncertain pseudo-heatmap from the feature map.

[0056] Among them, the Conditional-Variational-Autoencoder (CVAE) is a generative model used to generate data samples that meet specific conditions.

[0057] In this invention, generating uncertain pseudo-heatmaps through a conditional variational autoencoder not only improves the robustness, accuracy, and generative capability of the model, but also provides effective self-supervised signals, improves training and inference efficiency, enhances the interpretability and scalability of the model, and supports the fusion of multimodal data. These advantages collectively improve the overall performance and application value of medical image segmentation tasks.

[0058] S5: Sample a first preset number of voxel points with the highest uncertainty from the uncertain pseudo-heatmap, and randomly sample a second preset number of voxel points from other regions of the feature map.

[0059] S6: Extract the voxel feature vectors corresponding to each voxel point in the feature map.

[0060] In one possible implementation, S6 specifically includes:

[0061] S601: Feature maps are fused using multiple convolutional modules in the multi-scale feature fusion decoder.

[0062] S602: The upsampling module in the multi-scale feature fusion decoder performs upsampling processing on the fused feature map to extract voxel feature vectors corresponding to a first preset number and a second preset number of voxel points:

[0063] y i =g u (u i,0 ,u i,1 ,…,u i,n ,θ u )

[0064] Among them, g u This represents a multi-scale feature fusion decoder, y i u represents the voxel feature vectors corresponding to the first preset number and the second preset number of voxel points. i,k(k = 0, 1, ..., n) represents a multi-scale feature map, θ u This represents the parameters contained in the multi-scale feature fusion encoder.

[0065] In this invention, fusing feature maps at different scales allows for the comprehensive utilization of both low-level detailed information and high-level semantic information, enhancing the model's ability to understand complex structures. Simultaneously, multi-scale feature fusion can adapt to input data of varying resolutions and complexities, improving the model's generalization ability across different datasets and strengthening its interpretability.

[0066] Furthermore, by upsampling the fused feature map using the upsampling module, the voxel feature vectors of the region of interest can be accurately extracted. The upsampling process can retain more feature detail information, ensuring the accuracy and integrity of the feature vectors.

[0067] S7: Calculate the voxel text pairing similarity based on the voxel feature vectors corresponding to each voxel point.

[0068] In one possible implementation, the formula for calculating the similarity of voxel text pairings is:

[0069]

[0070] Where, sim i,j v represents the similarity between voxel text pairs. i Represents the voxel eigenvector, l j τ represents the text feature vector, and τ represents the temperature coefficient.

[0071] In this invention, by directly calculating the similarity between voxel feature vectors and text feature vectors, voxels in an image can be matched more accurately with their corresponding text descriptions, thus improving the accuracy of the segmentation results. Simultaneously, identifying fine-grained features in the image helps to accurately segment complex organs or lesion regions.

[0072] Furthermore, similarity scores can be used to generate visual heatmaps, helping medical experts intuitively view the model's predictions and their underlying rationale. Additionally, the results of similarity calculations can be directly used for gradient optimization, helping the model better learn feature representations.

[0073] S8: Based on the similarity of voxel text pairings, construct a pseudo heatmap of the pairing results, and train the feature encoding network based on the gradient of the similarity loss function.

[0074] In one possible implementation, the pseudo-heatmap of the pairing results in S8 specifically includes:

[0075] The voxel points in the voxel text pairing similarity are integrated into voxel coordinates.

[0076] The similarity of voxel text pairs is sorted in descending order based on similarity, and the coordinates of the third preset number of voxel points in the voxel text pair similarity are obtained.

[0077] Within a preset range, using the coordinates of a third preset number of voxel points as an index and the reciprocal of the third preset number as a step size, each voxel point is assigned a value in descending order and uniformly to obtain a discrete point map.

[0078] A pseudo-heatmap of the pairing results is obtained by smoothing the discrete point map using a Gaussian filter.

[0079] It should be noted that v i ·l j This can be viewed as a vector inner product, therefore the pairing similarity is positively correlated with the cosine similarity. Thus, voxel-class text similarity can be seen as the confidence level of the model in classifying the voxel point into each segmentation category, and a similarity loss function can be further constructed based on this.

[0080] In one possible implementation, the similarity loss function is:

[0081]

[0082] in, Let represent the similarity loss function, exp represent the exponential function with base e, and N represent the total number of segmentation categories.

[0083] In this invention, by constructing a pseudo-heatmap of pairing results and training using a similarity loss function, not only can the segmentation accuracy and robustness of the model be improved, but also the interpretability of the model can be enhanced, the training process optimized, multi-class segmentation tasks supported, and generalization ability improved, thereby comprehensively improving the effectiveness and practicality of medical image segmentation tasks.

[0084] S9: Use the pseudo-heatmap of the pairing result as the original input of the conditional variational autoencoder, and use the feature map output by the feature encoding network as the conditional input of the conditional variational autoencoder to perform self-supervised training on the conditional variational autoencoder.

[0085] In one possible implementation, the conditional variational autoencoder includes an input embedding layer, a conditional embedding layer, a joint encoder, a Gaussian embedding layer, a Gaussian decoder, and a joint decoder.

[0086] In one possible implementation, S9 specifically includes:

[0087] S901: The pseudo-heatmap and feature map are encoded and embedded through the input embedding layer and the conditional embedding layer, respectively, to obtain the input embedding layer features and the conditional embedding layer features.

[0088] e i =fi,v (H g ,θ i,v )

[0089] e c =f c,v (x i ,θ c,v )

[0090] Among them, e i f represents the input embedding layer features. i,v H represents the input embedding layer. g Represents a pseudo-heatmap, θ i,v e represents the parameters contained in the input embedding layer. c f represents the feature of the conditional embedding layer. c,v Represents the conditional embedding layer, x i Represents the feature map, θ c,v This represents the parameters contained in the conditional embedding layer.

[0091] S902: The latent features are obtained by jointly encoding the input embedding layer features and the conditional embedding layer features using a joint encoder.

[0092] u i,v =f v (e i ,e c ,θ v )

[0093] Among them, u i,v f represents a latent feature. v Represents a combined encoder, θ v This indicates the parameters contained in the combined encoder.

[0094] S903: The joint features are mapped to the mean and variance of a Gaussian function through a Gaussian embedding layer.

[0095] μ = f μ,g (u i,v ,θ μ,g )

[0096] σ=f σ,g (u i,v ,θ σ,g )

[0097] Where μ represents the Gaussian mean vector, σ represents the Gaussian variance vector, and f μ,h f represents a mean Gaussian embedding layer. σ,g Denotes a variance Gaussian embedding layer, θ μ,g θ represents the parameters contained in the mean Gaussian embedding layer. σ,g This represents the parameters contained in the variance Gaussian embedding layer.

[0098] S904: Using a Gaussian decoder, the standard Gaussian distributed random sample values, determined using the mean and variance, are reparameterized into Gaussian noise.

[0099]

[0100] Where z represents Gaussian noise, This indicates a Gaussian decoder.

[0101] S905: The pseudo-heatmap is reconstructed using a joint decoder based on the reparameterized standard Gaussian distribution random sampled values ​​and joint features.

[0102]

[0103] in, G represents the reconstructed pseudo-heatmap. v This indicates a combined decoder.

[0104] The reconstruction error is calculated using the error loss function, and the gradient of the conditional variational autoencoder is updated accordingly.

[0105]

[0106] Where MSE represents the mean square error, x (i) This indicates the input original image. This indicates the output reconstructed image. This represents the Gaussian variance vector corresponding to the image. J represents the Gaussian mean vector corresponding to the image, and J represents the vector dimension.

[0107] In this invention, the conditional variational autoencoder combines multi-level feature embedding and decoder, and improves the stability and robustness of the model through reasonable structural design and loss function optimization, making it more suitable for complex medical image analysis tasks.

[0108] Furthermore, by using a loss function reparameterized based on mean squared error and Gaussian distribution for self-supervised training, the model can self-correct and optimize during the generation process, improving its adaptability to multi-sample and multi-class scenarios and enhancing its generalization ability. Simultaneously, the generated pseudo-heatmaps not only contribute to generating new samples but also help explain the model's decision-making process. In medical image segmentation, pseudo-heatmaps can reflect the model's recognition and understanding of different organs or structures, enhancing the interpretability of the model's output and further increasing clinical trust.

[0109] S10: Acquire the CT image to be segmented.

[0110] S11: The trained feature encoding network is used to calculate the voxel text pairing similarity as the result of multi-organ segmentation of the CT image to be segmented.

[0111] Specifically, when using the model for inference, the CT image sequence to be segmented and the text indicating the organ category to be segmented are input into the trained model. The model then passes through a multi-level image encoder, text encoder, and multi-scale feature fusion decoder to obtain the feature vectors of each voxel in the image and the text. Then, pairwise similarity is used... i,j By measuring the similarity between voxels and text of each category, voxels can be classified, and the classification results can be integrated to obtain the segmentation results of each target organ.

[0112] In this invention, multi-level image encoders and text encoders encode CT image and text features, capturing fine-grained features and ensuring more accurate similarity calculations between each voxel and the text category, thus improving segmentation accuracy. Simultaneously, through voxel-text pairing similarity, the model can provide the degree of matching between each voxel and different categories, making the segmentation results more transparent and interpretable, helping medical experts understand the model's segmentation rationale.

[0113] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0114] (1) In this invention, by constructing a conditional variational autoencoder, an uncertainty pseudo-heatmap is generated for the feature map. A first preset number of voxel points with the highest uncertainty are sampled from the uncertainty pseudo-heatmap. A second preset number of voxel points are randomly sampled from other regions of the feature map to extract the voxel feature vectors corresponding to each voxel point in the feature map. Based on the voxel feature vectors corresponding to each voxel point, the voxel text pairing similarity is calculated, which enhances the generalization ability of the model and further improves the accuracy of recognition.

[0115] (2) In this invention, a pseudo-heatmap of the pairing results is constructed based on the similarity of the voxel text pairings, and a feature encoding network is trained according to the gradient of the similarity loss function. The pseudo-heatmap of the pairing results is used as the original input of the conditional variational autoencoder, and the feature map output by the feature encoding network is used as the conditional input of the conditional variational autoencoder. Self-supervised training is then performed on the conditional variational autoencoder. This increases the interpretability of the model and further improves clinical trust.

[0116] Reference manual attached Figure 2 The diagram shows a structural schematic of a voxel text pairing CT image multi-organ segmentation system provided by the present invention.

[0117] The present invention also provides a voxel-text pairing CT image multi-organ segmentation system 20, applied to the above-mentioned voxel-text pairing CT image multi-organ segmentation method, comprising:

[0118] Processor 201;

[0119] The memory 202 stores computer-readable instructions, which, when executed by the processor 201, implement the voxel text pairing method for multi-organ segmentation of CT images as described in the method embodiment.

[0120] The voxel text pairing CT image multi-organ segmentation system 20 provided by the present invention can perform the above-mentioned voxel text pairing CT image multi-organ segmentation method and achieve the same or similar technical effects. To avoid repetition, the present invention will not elaborate further.

[0121] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0122] (1) In this invention, by constructing a conditional variational autoencoder, an uncertainty pseudo-heatmap is generated for the feature map. A first preset number of voxel points with the highest uncertainty are sampled from the uncertainty pseudo-heatmap. A second preset number of voxel points are randomly sampled from other regions of the feature map to extract the voxel feature vectors corresponding to each voxel point in the feature map. Based on the voxel feature vectors corresponding to each voxel point, the voxel text pairing similarity is calculated, which enhances the generalization ability of the model and further improves the accuracy of recognition.

[0123] (2) In this invention, a pseudo-heatmap of the pairing results is constructed based on the similarity of the voxel text pairings, and a feature encoding network is trained according to the gradient of the similarity loss function. The pseudo-heatmap of the pairing results is used as the original input of the conditional variational autoencoder, and the feature map output by the feature encoding network is used as the conditional input of the conditional variational autoencoder. Self-supervised training is then performed on the conditional variational autoencoder. This increases the interpretability of the model and further improves clinical trust.

[0124] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0125] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0126] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0127] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0128] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0129] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0130] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0131] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0132] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0135] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] This invention provides a computer-readable storage medium storing a computer program thereon, characterized in that, when executed by a processor, the program implements the voxel text pairing method for multi-organ segmentation of CT images as described in the method embodiment.

[0137] The present invention provides a computer-readable storage medium that can implement the steps and effects of the voxel text pairing CT image multi-organ segmentation method of the above-described method embodiments. To avoid repetition, the present invention will not repeat them.

[0138] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0139] (1) In this invention, by constructing a conditional variational autoencoder, an uncertainty pseudo-heatmap is generated for the feature map. A first preset number of voxel points with the highest uncertainty are sampled from the uncertainty pseudo-heatmap. A second preset number of voxel points are randomly sampled from other regions of the feature map to extract the voxel feature vectors corresponding to each voxel point in the feature map. Based on the voxel feature vectors corresponding to each voxel point, the voxel text pairing similarity is calculated, which enhances the generalization ability of the model and further improves the accuracy of recognition.

[0140] (2) In this invention, a pseudo-heatmap of the pairing results is constructed based on the similarity of the voxel text pairings, and a feature encoding network is trained according to the gradient of the similarity loss function. The pseudo-heatmap of the pairing results is used as the original input of the conditional variational autoencoder, and the feature map output by the feature encoding network is used as the conditional input of the conditional variational autoencoder. Self-supervised training is then performed on the conditional variational autoencoder. This increases the interpretability of the model and further improves clinical trust.

[0141] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0142] The following points need to be explained:

[0143] (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0144] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the present invention; that is, these drawings are not drawn to actual scale. It is understood that when an element such as a layer, film, region, or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element, or there may be intermediate elements.

[0145] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0146] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for multi-organ segmentation of CT images using voxel-text pairing, characterized in that, include: S1: Construct a text template, describe the organs to be segmented using the text template, and obtain category text; S2: Acquire sample CT images; S3: Construct a feature encoding network, and through the feature encoding network, encode the sample CT image into a feature map and the category text into a text feature vector; S4: Construct a conditional variational autoencoder, and generate an uncertain pseudo-heatmap from the feature map using the conditional variational autoencoder; S5: Sample a first preset number of voxel points with the highest uncertainty from the uncertain pseudo-heatmap, and randomly sample a second preset number of voxel points from other regions of the feature map; S6: Extract the voxel feature vector corresponding to each voxel point in the feature map; S7: Calculate the voxel text pairing similarity based on the voxel feature vectors corresponding to each voxel point; The formula for calculating the similarity of the voxel text pairings is as follows: ； in, v represents the similarity between voxel text pairs. i Represents the voxel eigenvector, l j τ represents the text feature vector, and τ represents the temperature coefficient. S8: Based on the similarity of voxel text pairings, construct a pseudo heatmap of the pairing results, and train the feature encoding network based on the gradient of the similarity loss function; Specifically, the construction of the pairing result pseudo-heatmap in S8 includes: The voxel points in the voxel text pairing similarity are integrated into voxel coordinates; The similarity of the voxel text pairings is sorted in descending order to obtain the coordinates of a third preset number of voxel points in the similarity of the voxel text pairings. Within a preset range, using the coordinates of a third preset number of voxel points as an index and the reciprocal of the third preset number as a step size, each voxel point is assigned a value in descending order and uniformly to obtain a discrete point map. The discrete point map is smoothed by a Gaussian filter to obtain the pseudo heatmap of the pairing results; S9: Using the pseudo-heatmap of the pairing result as the original input of the conditional variational autoencoder, and using the feature map output by the feature encoding network as the conditional input of the conditional variational autoencoder, the conditional variational autoencoder is trained under self-supervised supervision. S10: Acquire the CT image to be segmented; S11: Calculate the voxel text pairing similarity through the trained feature encoding network, and use it as the multi-organ segmentation result of the obtained CT image to be segmented.

2. The method for multi-organ segmentation of CT images using voxel-text pairing according to claim 1, characterized in that, The feature encoding network includes a multi-level image encoder, a text encoder, and a multi-scale feature fusion decoder.

3. The method for multi-organ segmentation of CT images using voxel-text pairing according to claim 2, characterized in that, S3 specifically includes: S301: The CT image is encoded into a feature map using multiple residual 3D convolutional modules in the multi-level image encoder. ； ； in, This represents the residual 3D convolution module. Indicates the input image. This represents the parameters contained in the initial residual 3D convolution module. Indicates the following The parameters contained in a residual 3D convolution module Indicates the first The features output by each residual 3D convolutional module Indicates the first The input features of the residual 3D convolutional module also represent the features of the th residual 3D convolutional module. Features output by a residual 3D convolutional module; S302: The categorical text is encoded into a text feature vector using multiple self-attention modules in the text encoder. ； in, Indicates a text encoder. Represents the text feature vector. Indicates category text, This indicates the parameters contained in the text encoder.

4. The method for multi-organ segmentation of CT images using voxel-text pairing according to claim 1, characterized in that, S6 specifically includes: S601: The feature maps are fused using multiple convolutional modules in the multi-scale feature fusion decoder; S602: The upsampling module in the multi-scale feature fusion decoder performs upsampling processing on the fused feature map to extract the voxel feature vectors corresponding to the first preset number and the second preset number of voxel points: ； in, This represents a multi-scale feature fusion decoder. This represents the voxel feature vectors corresponding to the first preset number and the second preset number of voxel points. Indicates the first The features output by each residual 3D convolutional module This represents the parameters contained in the multi-scale feature fusion encoder.

5. The method for multi-organ segmentation of CT images using voxel-text pairing according to claim 1, characterized in that, The similarity loss function is: ； in, Let represent the similarity loss function, and exp represent the exponential function with base e. The similarity of voxel text pairings is represented by N, which represents the total number of segmentation categories.

6. The method for multi-organ segmentation of CT images using voxel-text pairing according to claim 1, characterized in that, The conditional variational autoencoder includes an input embedding layer, a conditional embedding layer, a joint encoder, a Gaussian embedding layer, a Gaussian decoder, and a joint decoder.

7. The method for multi-organ segmentation of CT images using voxel-text pairing according to claim 6, characterized in that, S9 specifically includes: S901: The pseudo-heatmap and feature map are encoded and embedded through the input embedding layer and the conditional embedding layer respectively to obtain the input embedding layer features and the conditional embedding layer features: ； ； in, This represents the input embedding layer features. Indicates the input embedding layer. This indicates a pseudo-heatmap. This represents the parameters contained in the input embedding layer. Represents the features of the conditional embedding layer. This indicates a conditional embedding layer. Representing feature maps, This represents the parameters contained in the conditional embedding layer; S902: The latent features are obtained by jointly encoding the input embedding layer features and the conditional embedding layer features using the joint encoder. ； in, Indicates implicit features. Indicates a combined encoder. This indicates the parameters contained in the combined encoder; S903: The joint features are mapped to the mean and variance of a Gaussian function through the Gaussian embedding layer. ； ； in, Represents the Gaussian mean vector. Represents the Gaussian variance vector. This indicates a mean Gaussian embedding layer. This indicates a variance Gaussian embedding layer. This represents the parameters contained in the mean Gaussian embedding layer. Indicates the parameters contained in the variance Gaussian embedding layer; S904: The standard Gaussian distributed random sample values ​​determined using the mean and variance are reparameterized into Gaussian noise using the Gaussian decoder. ； in, Indicates Gaussian noise. Indicates a Gaussian decoder; S905: The pseudo-heatmap is reconstructed using the joint decoder based on the reparameterized standard Gaussian distribution random sampled values ​​and the joint features. ； in, This represents the reconstructed pseudo-heatmap. Indicates a joint decoder; The reconstruction error is calculated using the error loss function, and the gradient of the conditional variational autoencoder is updated accordingly. ； Where MSE represents the mean squared error. This indicates the input original image. This indicates the output reconstructed image. This represents the Gaussian variance vector corresponding to the image. J represents the Gaussian mean vector corresponding to the image, and J represents the vector dimension.

8. A voxel-text pairing system for multi-organ segmentation of CT images, characterized in that, include: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the voxel text pairing method for multi-organ segmentation of CT images as described in any one of claims 1 to 7.

Images