Medical image segmentation model training method and device, medium and equipment

Abstract

The invention discloses a medical image segmentation model training method and device, a medium and equipment. Comprises: acquiring a medical image sample; constructing a semi-supervised segmentation framework of the initial segmentation model; performing feature interaction fusion processing on the medical image sample through the initial segmentation model to generate a feature map of the medical image sample; calculating a first loss of the slice matrix of the feature maps of the plurality of visual coding branches in the orthogonal direction based on the kernel function; calculating a second loss of the feature map corresponding to the first image sample under the plurality of visual coding branches based on a supervision loss function; calculating a third loss of the feature map corresponding to the second image sample under the plurality of visual coding branches based on a contrast pseudo label supervision loss function; and training the initial segmentation model by taking the weighted sum minimization of the first loss, the second loss and the third loss as a target, and outputting a target segmentation model after an iteration termination condition is met. Therefore, the multi-organ segmentation accuracy of the medical image is improved, and the overall segmentation precision and reliability are improved.

Description

Technical Field

[0001] The present application relates to the technical field of medical image analysis, and in particular to a medical image segmentation model training method, device, medium and equipment. Background Art

[0002] When dealing with multi-organ segmentation in CT images, the visual feature extraction and model training at the image pixel level are mainly used, which makes it difficult for the model to accurately distinguish different organs when facing complex situations or limited data, affecting the segmentation accuracy. In addition, due to the complex layout and morphological relationship of human organs in space, relying solely on general image feature learning will ignore the constraints of the orthogonal geometric relationship between the cross-sectional and longitudinal planes in 3D medical images on the model, resulting in segmentation errors or blurred boundaries caused by the inability to accurately define boundaries. In particular, the segmentation effect of small-sized and complex-shaped organs is poor, which cannot meet the clinical needs for high-precision medical image segmentation. Summary of the invention

[0003] In view of this, the present application provides a medical image segmentation model training method, device, medium and equipment to solve the problem of insufficient accuracy of medical image segmentation.

[0004] According to one aspect of the present application, a medical image segmentation model training method is provided, comprising: Acquire a medical image sample, wherein the medical image sample includes a first image sample labeled with an organ category and a second image sample not labeled with an organ category; Constructing a semi-supervised segmentation framework of the initial segmentation model, wherein the semi-supervised segmentation framework includes a plurality of parallel visual encoding branches and a text encoding branch; Performing feature interactive fusion processing on the medical image sample through the initial segmentation model to generate a feature map of the medical image sample; Based on the kernel function, calculating the first loss of the slicing matrix of the feature maps of the multiple visual encoding branches in the orthogonal direction; Calculate the second loss of the feature map corresponding to the first image sample under multiple visual encoding branches based on the supervised loss function; Calculate a third loss of the feature map corresponding to the second image sample under multiple visual encoding branches based on the comparison pseudo-label supervision loss function; The initial segmentation model is trained with the goal of minimizing the weighted sum of the first loss, the second loss, and the third loss, and a target segmentation model is output after an iteration termination condition is met.

[0005] Optionally, performing feature interactive fusion processing on the medical image sample by using the initial segmentation model to generate a feature map of the medical image sample includes: The text features and the visual features in the medical image sample are respectively encoded by the visual encoding branch and the text encoding branch to obtain text feature encoding and visual feature encoding with the same number of channels, wherein the text features are used to describe the organ category; Performing multimodal semantic fusion processing on the text feature code and the visual feature code to generate a text-visual fusion code; The text-visual fusion code and the visual feature code are connected via a jump connection to form the feature map.

[0006] Optionally, performing multimodal semantic fusion processing on the text feature code and the visual feature code to generate a text-visual fusion code includes: Respectively projecting the text feature code and the visual feature code into a query space to generate a text query code and a visual query code; Respectively projecting the text feature code and the visual feature code into a key-value pair space to generate a text key-value pair code and a visual key-value pair code; Performing an alignment operation on the text query code based on the visual key-value pair code to generate a target text feature code; Performing an alignment operation on the visual query encoding based on the text key-value pair encoding to generate a target visual feature encoding; A fusion operation is performed on the target text feature code and the target visual feature code to determine a text-visual fusion code.

[0007] Optionally, encoding the text features and visual features in the medical image sample respectively through the visual encoding branch and the text encoding branch to obtain text feature encoding and visual feature encoding with the same number of channels includes: identifying text features in the medical image sample; Inputting the text feature into the text encoder of the text encoding branch to obtain the text feature encoding; Inputting the medical image sample into the image segmentation network of the visual encoding branch to obtain the image feature encoding of the medical image sample; The image feature code is reshaped based on the channel dimension to which the text feature code belongs to determine the visual feature code.

[0008] Optionally, the image segmentation network of the visual encoding branch includes a plurality of encoder layers; and performing multimodal semantic fusion processing on the text feature encoding and the visual feature encoding to determine the text-visual fusion encoding includes: The text feature code and the visual feature code corresponding to the last encoder layer are subjected to a multimodal semantic fusion operation to determine the text-visual fusion code of a target encoder layer, wherein the target encoder layer is the penultimate encoder layer of the image segmentation network.

[0009] Optionally, the connecting operation of the text-visual fusion code and the visual feature code through a skip connection to form the feature map includes: The text-visual fusion encoding and the visual feature encoding belonging to the same encoder layer are connected through a jump connection.

[0010] Optionally, the orthogonal direction includes a first direction and a second direction perpendicular to each other, and the kernel function-based calculation of a first loss of a slice matrix of the feature graphs of the plurality of visual encoding branches in the orthogonal direction includes: Slicing the feature map along the first direction and the second direction respectively to obtain a first direction slice matrix and a second direction slice matrix; Based on the kernel function, respectively calculating the kernel loss of the first direction slice matrix and the second direction slice matrix under different visual encoding branches; The kernel losses corresponding to the multiple visual encoding branches and the total kernel losses corresponding to the multiple visual encoding branches are summed to obtain the first loss.

[0011] Optionally, the kernel function-based, respectively calculating the kernel losses of the first direction slice matrix and the second direction slice matrix under different visual encoding branches, includes: Determine a Euclidean distance matrix corresponding to the first direction slice matrix and the second direction slice matrix respectively under a target visual coding branch, wherein the target visual coding branch is any visual coding branch among the multiple visual coding branches; Extracting a triangular matrix from the Euclidean distance matrix; Calculating the gamma value of the first-direction slice matrix based on the median of the elements in the triangular matrix corresponding to the first-direction slice matrix, and calculating the gamma value of the second-direction slice matrix based on the median of the elements in the triangular matrix corresponding to the second-direction slice matrix; Calculate a kernel matrix based on a kernel function, a gamma value of the first-direction slice matrix, and a gamma value of the second-direction slice matrix; A kernel loss under the target visual encoding branch is determined for the kernel matrix.

[0012] According to another aspect of the present application, a medical image segmentation model training device is provided, comprising: A data acquisition module, used to acquire medical image samples, wherein the medical image samples include first image samples labeled with organ categories and second image samples not labeled with organ categories; A construction module, used to construct a semi-supervised segmentation framework of an initial segmentation model, wherein the semi-supervised segmentation framework includes a plurality of parallel visual encoding branches and a text encoding branch; A feature extraction module, used for performing feature interactive fusion processing on the medical image sample through the initial segmentation model to generate a feature map of the medical image sample; A training module is used to calculate the first loss of the slicing matrix of the feature maps of multiple visual encoding branches in an orthogonal direction based on a kernel function; and, based on a supervised loss function, calculate the second loss of the feature maps corresponding to the first image samples under multiple visual encoding branches; and, based on a comparative pseudo-label supervised loss function, calculate the third loss of the feature maps corresponding to the second image samples under multiple visual encoding branches; and, with the goal of minimizing the weighted sum of the first loss, the second loss, and the third loss, train the initial segmentation model, and output the target segmentation model after the iteration termination condition is met.

[0013] Optionally, the feature extraction module includes: an encoding module, used for encoding text features and visual features in the medical image sample respectively through the visual encoding branch and the text encoding branch to obtain text feature codes and visual feature codes with the same number of channels, wherein the text features are used to describe the organ category; A multimodal fusion module, used for performing multimodal semantic fusion processing on the text feature code and the visual feature code to generate a text-visual fusion code; The encoding module is further used to connect the text-visual fusion code and the visual feature code through a jump connection to form the feature map.

[0014] Optionally, the multimodal fusion module is specifically used to project the text feature code and the visual feature code to the query space, respectively, to generate a text query code and a visual query code; project the text feature code and the visual feature code to the key-value pair space, respectively, to generate a text key-value pair code and a visual key-value pair code; align the text query code based on the visual key-value pair code to generate a target text feature code; align the visual query code based on the text key-value pair code to generate a target visual feature code; and fuse the target text feature code and the target visual feature code to determine a text-visual fusion code.

[0015] Optionally, the encoding module is specifically used to identify text features in the medical image samples; input the text features into the text encoder of the text encoding branch to obtain the text feature coding; input the medical image samples into the image segmentation network of the visual encoding branch to obtain the image feature coding of the medical image samples; perform a reshape operation on the image feature coding based on the channel dimension to which the text feature coding belongs to determine the visual feature coding.

[0016] Optionally, the image segmentation network of the visual encoding branch includes multiple encoder layers; the multimodal fusion module is specifically used to perform a multimodal semantic fusion operation on the text feature encoding and the visual feature encoding corresponding to the last encoder layer, and determine the text-visual fusion encoding of the target encoder layer, wherein the target encoder layer is the second to last encoder layer of the image segmentation network.

[0017] Optionally, the encoding module is specifically configured to connect the text-visual fusion encoding and the visual feature encoding belonging to the same encoder layer through a skip connection.

[0018] Optionally, the orthogonal direction includes a first direction and a second direction that are perpendicular to each other, and the medical image segmentation model training device further includes: A segmentation module, used to perform a slicing operation on the feature map along the first direction and the second direction respectively, to obtain a first direction slicing matrix and a second direction slicing matrix; The training module is specifically used to calculate the kernel losses of the first direction slice matrix and the second direction slice matrix under different visual coding branches based on the kernel function; and to sum the kernel losses corresponding to multiple visual coding branches and the total kernel losses corresponding to multiple visual coding branches to obtain the first loss.

[0019] Optionally, the training module is specifically used to determine the Euclidean distance matrices corresponding to the first direction slice matrix and the second direction slice matrix respectively under the target visual coding branch, wherein the target visual coding branch is any visual coding branch among multiple visual coding branches; extract the triangular matrix in the Euclidean distance matrix; calculate the gamma value of the first direction slice matrix based on the median of the elements in the triangular matrix corresponding to the first direction slice matrix, and calculate the gamma value of the second direction slice matrix based on the median of the elements in the triangular matrix corresponding to the second direction slice matrix; calculate the kernel matrix based on the kernel function, the gamma value of the first direction slice matrix and the gamma value of the second direction slice matrix; determine the kernel loss under the target visual coding branch for the kernel matrix.

[0020] According to another aspect of the present application, a readable storage medium is provided, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the above-mentioned medical image segmentation model training method are implemented.

[0021] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, and the processor implements the steps of the above-mentioned medical image segmentation model training method when executing the program.

[0022] Through the above technical solution, on the one hand, the category text is encoded and then interactively fused with the image visual feature encoding, so that the features recognized by the model can combine the visual features and the text description of the organ category knowledge. In this way, the text information is fully utilized to assist the model in identifying organs, enhance the recognition ability of the segmentation model for different organs, improve the accuracy of multi-organ segmentation of medical images, and especially improve the segmentation effect of small-sized and complex-shaped organs. On the second hand, the orthogonal geometric characteristics and independent relationships of 3D medical images are fully considered, and the dependencies between feature sets are quantified using kernel functions. In this way, pixel geometry and feature learning are closely combined to solve the problem that the existing technology ignores the geometric structure information of the image, further enhance the segmentation model's understanding and processing ability of the spatial layout and morphological relationship of organs, reduce segmentation errors and boundary fuzziness, and thus improve the overall segmentation accuracy and reliability. At the same time, through semi-supervised learning, the limited labeled data and a large amount of unlabeled data are fully utilized to improve the performance of the image segmentation model. Thirdly, through cross-pseudo-supervision of multiple visual branches, the training results of multiple visual branches can learn from each other, which improves the generalization ability of the segmentation model. At the same time, multiple loss functions (kernel function, supervision loss function, comparative pseudo-label supervision loss function) are used to perform joint optimization of two visual branches in parallel. While ensuring consistency constraints, the learning process of the model can be constrained from different angles, thereby improving the performance and robustness of the model.

[0023] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A schematic diagram of the process of the medical image segmentation model training method provided in an embodiment of the present application is shown; Figure 2 A logical schematic diagram of a medical image segmentation model training method provided in an embodiment of the present application is shown; Figure 3 A schematic diagram of the coding mapping relationship provided in an embodiment of the present application is shown; Figure 4 A schematic diagram of the calculation logic of the first loss provided in an embodiment of the present application is shown; Figure 5 A structural block diagram of a medical image segmentation model training device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0026] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0027] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, or there may be intermediate elements. In addition, the "connection" or "connection" used herein may include wireless connection or wireless fusion. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0028] Now, exemplary embodiments according to the present application will be described in more detail with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments set forth herein. It should be understood that these embodiments are provided to make the disclosure of the present application thorough and complete, and to fully convey the concepts of these exemplary embodiments to those of ordinary skill in the art.

[0029] In this embodiment, a medical image segmentation model training method is provided. Figure 1 As shown, the method includes: Step 110: Acquire a medical image sample.

[0030] The medical image samples include first image samples labeled with organ categories and second image samples not labeled with organ categories.

[0031] Specifically, the medical image samples may be CT (computed tomography) images, MRI (magnetic resonance imaging) images, ultrasound images, digital X-ray images, PET (positron emission tomography) images, etc., which are used to capture images of internal organs or bones of organisms.

[0032] In this embodiment, the model training can be performed by combining labeled and unlabeled data, thereby reducing the reliance on labeled data and helping to reduce the overall data collection cost. At the same time, while ensuring that the model learns accurate features and patterns, it helps the model to better generalize and cope with unseen data.

[0033] The medical image segmentation model training method provided in the embodiments of the present application can be applied to a terminal or a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; Step 120, construct a semi-supervised segmentation framework of the initial segmentation model.

[0034] Among them, the semi-supervised segmentation framework includes multiple parallel visual encoding branches and text encoding branches. Multiple parallel visual branches use the same image segmentation network (such as U-Net, DeepLabResNet, VGG, etc.), but are initialized to different values, and the parameters are updated independently. In this way, multiple visual encoding branches can process the same input sample through different feature extraction methods, thereby reducing the risk of overfitting in a single branch and improving the generalization ability of the model on new samples. To increase the diversity and generalization ability of the model. The text encoding branch usually uses a pre-trained language model (such as BERT, CLIP, GPT, etc.) to extract text features.

[0035] Step 130 , performing feature interactive fusion processing on the medical image sample through the visual encoding branch and the text encoding branch of the initial segmentation model to generate a feature map of the medical image sample.

[0036] Among them, the feature map is the direct output form of feature encoding, that is, the multidimensional tensor output by the encoder. The feature map usually contains spatial information and channel information. Through operations such as convolution, downsampling, and nonlinear activation, the feature map gradually transitions from low-level features to high-level features.

[0037] In this embodiment, the feature description of the organ category of the medical image sample can be identified through the text encoding branch and combined with the visual features identified by the visual encoding branch. This allows the trained segmentation model to make full use of text information to assist the model in identifying organs, enhance the segmentation model's ability to recognize different organs, and improve the accuracy of multi-organ segmentation in medical images, especially improving the segmentation effect of small-sized and complex-shaped organs. In addition, the prediction results (pseudo-labels) of one visual branch can be used as supervisory signals for other visual branches, so that multiple visual branches can learn from each other, thereby improving the generalization ability of the target segmentation model.

[0038] In one embodiment, step 130, i.e., performing feature interactive fusion processing on the medical image sample through the visual encoding branch and the text encoding branch of the initial segmentation model to generate a feature map of the medical image sample, specifically includes the following steps: Step 131 , encoding the text features and visual features in the medical image sample through the visual encoding branch and the text encoding branch respectively, to obtain text feature encoding and visual feature encoding with the same number of channels.

[0039] Among them, text features are used to describe organ categories.

[0040] In this embodiment, the text feature codes and visual feature codes in the medical image samples are extracted through the visual coding branch and the text coding branch, and the visual and text codes are mapped to the same space so that the text feature codes and the visual feature codes are in the same dimension. This allows information from different modalities to be more easily integrated, so that the model will focus more on the regional segmentation of organ category-related characteristics, enhance the model's understanding of medical images, and thus improve the performance of the model's classification and segmentation tasks.

[0041] In a specific application scenario, step 131 specifically includes: identifying text features in medical image samples; inputting text features into the text encoder of the text encoding branch to obtain text feature encoding; inputting medical image samples into the image segmentation network of the visual encoding branch to obtain image feature encoding of the medical image samples; reshaping the image feature encoding based on the channel dimension to which the text feature encoding belongs to determine the visual feature encoding.

[0042] In this embodiment, the encoders of the visual encoding branch and the text encoding branch extract features present in the medical image sample, and then align the channel dimension of the text encoding with the visual encoding through the reshaping operation to facilitate cross-modal feature fusion, so that the visual features are more focused on the target area described by the text. In this way, the trained segmentation model can more accurately distinguish the target organ from the background in complex multi-organ scenes.

[0043] For example, Figure 2 As shown in the figure, the text prompts related to the organ category are input into the CLIP text encoder, and the text feature encoding is obtained after the encoding operation. , where K represents the number of organ categories and C represents the number of channels for text feature encoding. At the same time, the image sample first passes through a 3D convolution layer Conv3d to obtain the image feature encoding. , where C1 represents the number of channels of image feature encoding, D, H and W represent image depth, height and width respectively; then use the reshape function to reshape and obtain the visual feature encoding with the same channel dimension as the text feature encoding ,in .

[0044] Step 132, performing multimodal semantic fusion processing on the text feature coding and the visual feature coding to generate a text-visual fusion coding.

[0045] In this embodiment, the category text (such as "liver", "spleen", etc.) is encoded and interactively fused with the visual feature encoding, so that the features recognized by the model can combine the visual features and the text description of the organ category knowledge. This allows the model to obtain a more comprehensive feature representation, thereby improving the performance of classification, segmentation or other tasks, enhancing the segmentation model's ability to recognize different organs, and improving the accuracy of multi-organ segmentation in medical images.

[0046] It should be noted that since the semi-supervised segmentation framework includes multiple parallel visual encoding branches, the visual feature encoding extracted by each visual encoding branch is fused with the text feature encoding to ensure the consistency of subsequent supervised training of multiple parallel visual encoding branches.

[0047] In a specific application scenario, step 132 specifically includes: projecting the text feature code and the visual feature code to the query space respectively to generate a text query code and a visual query code; projecting the text feature code and the visual feature code to the key-value pair space respectively to generate a text key-value pair code and a visual key-value pair code; aligning the text query code based on the visual key-value pair code to generate a target text feature code; aligning the visual query code based on the text key-value pair code to generate a target visual feature code; fusing the target text feature code and the target visual feature code to determine the text-visual fusion code.

[0048] In this embodiment, the text and visual features are first projected to a specific space respectively. Cross-Attention is used to encode the text feature as Query and the visual feature as Key / Value to generate a visually enhanced target text feature encoding. Similarly, cross-attention is used to encode the visual feature as Query and the text feature as Key / Value to generate a semantically enhanced target visual feature encoding. Finally, the text-visual fusion encoding is obtained by dot product and summation. Thus, through the cross-attention operations from text to vision and from vision to text, the alignment of the two in the feature space is promoted and the domain gap is narrowed. Then, the text information is fully utilized to assist the model in identifying organs, thereby realizing the effective fusion of text and visual information, providing richer semantic information for subsequent segmentation tasks, and solving the problem of insufficient utilization of semantic information in the prior art.

[0049] For example, Figure 3 As shown, in the text-visual cross-attention mechanism, the text features are encoded Projected into space Q as query , while encoding the visual features Projected into space K and V respectively as keys Sum In the visual-text cross-attention mechanism, the visual features are encoded Projected into space Q as query , encode the text features Projection into space K and V as keys Sum Through the residual connection, the transpose of space Q and space K is multiplied (QK T ), after softmax normalization (marked as S in the figure), the aligned text feature encoding is obtained and visual feature encoding The specific calculation formula is: ; ; In the formula, C represents the number of channels for text feature encoding. α and β are learnable parameters used to balance the impact of the cross-fusion part.

[0050] On the C dimension of text feature encoding and Perform dot product and sum operations, and use enisum string form to represent text-visual fusion encoding , the calculation formula of text-visual fusion coding is: ; In the formula, Represents the dot product operation.

[0051] Step 133, connecting the text-visual fusion code and the visual feature code through a jump connection to form a feature map.

[0052] In this embodiment, the low-level features of the encoder (downsampling path) are combined with the high-level features of the decoder (upsampling path) through skip connections to restore spatial details, allowing the model to better understand and represent the details and contextual information in the image, avoiding information loss in deep networks, and improving the model's segmentation and generalization capabilities for complex medical images.

[0053] It should be noted that, for the case where the image segmentation network of the visual encoding branch includes multiple encoder layers, the feature map, that is, a direct representation of the feature encoding output by the last layer of the encoder, is the final result after the encoder performs multi-layer feature extraction on the input image.

[0054] In one embodiment, when the image segmentation network of the visual encoding branch includes multiple encoder layers, step 133 specifically includes: connecting the text-visual fusion encoding and the visual feature encoding belonging to the same encoder layer through a skip connection.

[0055] Furthermore, for the case where the image segmentation network of the visual coding branch includes multiple encoder layers, step 132 specifically includes: performing a multimodal semantic fusion operation on the text feature coding and the visual feature coding corresponding to the last encoder layer to determine the text-visual fusion coding of the target encoder layer.

[0056] Among them, the target encoder layer is the penultimate encoder layer of the image segmentation network.

[0057] In this embodiment, considering that the output of the last layer of the encoder is already the final high-level feature map, there is no need to further fuse the low-level features. The visual features corresponding to the penultimate encoder layer are replaced with the visual features corresponding to the last encoder layer, and interactively fused with the text feature encoding to generate the text-visual fusion encoding of the penultimate encoder layer. In this way, in the subsequent encoder-decoder jump connection process, according to the characteristics of semantic encoding at different network scales, more detailed semantic priors are obtained at specific layers of the visual branch, and these semantic priors are injected into the corresponding decoder layer according to specific rules. This allows the model to simultaneously utilize multi-scale information at different network scales, improve the segmentation model's ability to learn complex patterns, and enhance the model's segmentation effect on different organs.

[0058] For example, the visual branch uses a 5-layer U-Net encoder, where the encoder layers are denoted as L1-L5, and the corresponding layer outputs (before downsampling or upsampling operations) are denoted as E v1 -E v5 For the first three layers (L1-L3), output them respectively Text feature encoding Perform interactive fusion operations to obtain text-visual fusion encoding of each layer ; For the fourth layer L4, the output of the fifth layer Through interactive fusion operation and text feature encoding The fused text-visual fusion encoding is used as The visual feature encoding of the encoder layer is connected to the fused text-visual fusion encoding through a jump connection. The specific connection method is:

[0059] In the formula, Represents a join operation.

[0060] Step 140, based on the kernel function, calculate the first loss of the slicing matrix of the feature maps of the multiple visual encoding branches in the orthogonal direction.

[0061] The orthogonal directions include a first direction and a second direction that are perpendicular to each other.

[0062] Specifically, the kernel function is used to map data to a high-dimensional space. Kernel functions include but are not limited to Gaussian kernel function, Sigmoid kernel function, Laplace kernel function, exponential kernel function, and chi-square kernel function.

[0063] In this embodiment, on the one hand, the orthogonal geometric characteristics and independent relationships of multiple planes of 3D medical images are fully considered, and pixel geometry is closely combined with feature learning, which can ensure that the model can not only focus on the global information of the features, but also capture the similarity of local features. This improves the model's adaptability to spatial transformations, solves the problem of prior art ignoring image geometric structure information, further improves the segmentation model's understanding and processing capabilities of the spatial layout and morphological relationships of organs, reduces segmentation errors and boundary fuzziness, and thus improves overall segmentation accuracy and reliability. On the other hand, by calculating the first loss of different branches, multi-view features can be effectively fused, which helps to enhance the model's understanding and expression capabilities of information from different perspectives, enabling the model to better capture complex patterns and structures in images.

[0064] In one embodiment, step 140, i.e., calculating the first loss of the slicing matrix of the feature graphs of the plurality of visual encoding branches in the orthogonal direction based on the kernel function, specifically comprises the following steps: Step 141, performing a slicing operation on the feature map along a first direction and a second direction respectively, to obtain a first direction slicing matrix and a second direction slicing matrix.

[0065] It is understandable that the interval distance between slices can be reasonably set according to the orthogonal geometry accuracy, and the embodiments of the present application do not make specific limitations.

[0066] Step 142, based on the kernel function, respectively calculate the kernel loss of the first direction slice matrix and the second direction slice matrix under multiple visual encoding branches.

[0067] In a specific application scenario, step 142 specifically includes: determining the Euclidean distance matrices corresponding to the first direction slice matrix and the second direction slice matrix under the target visual coding branch, wherein the target visual coding branch is any visual coding branch among multiple visual coding branches; extracting the triangular matrix in the Euclidean distance matrix; calculating the gamma value of the first direction slice matrix based on the median of the elements in the triangular matrix corresponding to the first direction slice matrix, and calculating the gamma value of the second direction slice matrix based on the median of the elements in the triangular matrix corresponding to the second direction slice matrix; calculating the kernel matrix based on the kernel function, the gamma value of the first direction slice matrix and the gamma value of the second direction slice matrix; determining the kernel loss under the target visual coding branch for the kernel matrix.

[0068] Step 143, summing the kernel losses corresponding to the multiple visual encoding branches and the total kernel losses corresponding to the multiple visual encoding branches to obtain a first loss.

[0069] In this embodiment, slicing in orthogonal directions provides feature representations of feature maps in different directions. For any visual encoding branch, a kernel function is used to calculate the kernel loss of the first direction slice matrix and the second direction slice matrix respectively, so that the relationship between the features in the first direction and the second direction can be captured more accurately, especially when processing high-dimensional data. Finally, by summing the kernel losses of slices under multiple visual encoding branches, the kernel loss under a single visual encoding branch is integrated into the total loss function of the model for optimization, so that pixel geometry and feature learning are closely combined, solving the problem of ignoring image geometric structure information in the prior art and improving the segmentation accuracy of complex abdominal organs.

[0070] For example, the feature map For example, we first extract the horizontal center slice along the width dimension W from the feature map and reshape it into a horizontal slice matrix At the same time, the vertical center slice along the height dimension H is extracted and reshaped into . Calculate the pairwise Euclidean distance of samples in matrices X and Y. The Euclidean distance calculation formula is: ; ; Where n represents the number of elements in the reshaped matrix. represents two n-dimensional vectors in the matrix X, represents two n-dimensional vectors in the matrix Y, and d represents the Euclidean distance.

[0071] The Euclidean distance matrix corresponding to the horizontal slice matrix is ​​obtained by Euclidean distance calculation And the Euclidean distance matrix corresponding to the vertical slice matrix The size of the Euclidean distance matrix is , where m=B×C.

[0072] Then, Take as an example, first extract its upper triangular matrix (excluding the diagonal), and then calculate the median of the matrix elements, denoted as Gamma value The calculation formula is:

[0073] And calculate in the same way . Get the gamma value and Then calculate the kernel matrix K and L, the calculation formulas are: ; ; Finally, the kernel loss of the feature map in the horizontal and vertical directions is calculated based on the kernel matrix and integrated.

[0074] The formula for calculating the core loss in the orthogonal direction can be simplified to: ; In the formula, , is the identity matrix, is a matrix of all 1s.

[0075] like Figure 1 and Figure 4 As shown in the figure, a semi-supervised segmentation framework containing two parallel visual branches (A and G) is taken as an example. For multiple visual branches A and G, through orthogonal and independent constraints, the respective HSIC (Hilbert-Schmidt Independence Criterion) losses are calculated respectively: ; ; In the formula, and represents the orthogonal central visual feature slice of visual branch A, Represents the kernel loss of the output feature map of the visual branch A in the orthogonal direction, and represents the orthogonal central visual feature slice of the visual branch G, Represents the kernel loss of the output feature map of the visual branch G in the orthogonal direction. At the same time, the HSIC loss between branches A and G is also calculated, and the calculation formula is: ; Integrate these HSIC losses into the total loss function of the model, namely: .

[0076] Step 150, calculating a second loss of a feature map corresponding to the first image sample under multiple visual encoding branches based on a supervised loss function.

[0077] In this embodiment, by calculating the supervised loss, i.e., the second loss, the model can be guided to suppress irrelevant or redundant features, so that the segmentation prediction results of the model on the labeled data are as close as possible to the real mask, thereby improving the segmentation effect of the model. And by sharing feature representations between different branches of the same input sample, the existing labeled information can be effectively utilized to improve the learning efficiency of the model.

[0078] Exemplarily, for samples with labeled organ types, the following supervised loss function is used to calculate the second loss:

[0079] ; In the formula, represents the second loss, represents the batch size of the labeled data, and Represent image segmentation loss and cross entropy loss respectively, A probability map representing the output of a neural network.

[0080] Step 160, calculating a third loss of the feature map corresponding to the second image sample under multiple visual encoding branches based on the comparison pseudo-label supervision loss function.

[0081] In this embodiment, a pseudo-label is generated on unlabeled data through a visual encoding branch and used as a supervision signal for other visual branches. Then, by comparing the pseudo-label supervision loss function, the cross pseudo-supervision (CPS) loss is calculated to constrain the consistency between the prediction results of multiple branches and the pseudo-label. In this way, the difference between the model prediction results and the pseudo-label can be quantified. Even if the model has fewer real labels, it can still learn useful features, thereby reducing the dependence on labeled data and reducing the cost and time of data labeling.

[0082] Exemplarily, for samples with labeled organ types, the following supervised loss function is used to calculate the second loss:

[0083] ; In the formula, represents the batch size of unlabeled data, Represents the predicted probability map of the i-th unlabeled sample. By penalizing the prediction difference, the model's prediction on unlabeled data is made more accurate and stable, thereby improving the generalization ability of the model.

[0084] Step 170, training the initial segmentation model with the goal of minimizing the weighted sum of the first loss, the second loss, and the third loss, and outputting the target segmentation model after the iteration termination condition is met.

[0085] In this embodiment, multiple loss functions (kernel function, supervision loss function, comparative pseudo-label supervision loss function) are used to perform joint optimization of two visual branches in parallel. While ensuring consistency constraints, the learning process of the model can be constrained from different angles to improve the performance and robustness of the target segmentation model.

[0086] For example, Figure 2 As shown in the figure, taking the semi-supervised segmentation framework containing two parallel visual branches (A and G) as an example, medical image samples are first collected to form a training data set. The training data set includes N labeled data and M unlabeled data, and satisfies The labeled data is recorded as ,in represents the input volume data, Represents the corresponding true mask; unlabeled data is recorded as . During the training process, each training step will sample from the labeled data and the unlabeled data respectively, and send them to the networks A and G for processing. During training, the text feature codes in the medical image samples are extracted through the text encoder, and the visual feature codes in the medical image samples are extracted through the image encoder. The text feature codes are interactively fused with the visual feature codes obtained from the two visual branches, and the fused feature codes and the visual feature codes are input into the image decoder together. The visual feature codes finally output by the image encoder are used as the basis for orthogonal central visual feature slicing, and the HSIC loss of the branch is calculated at the same time. For the labeled data, the visual feature codes finally output by the image encoder ( Figure 2 The supervised loss function is used to calculate the SUP loss based on the feature map segmented by different colors in the image. For unlabeled data, the visual feature encoding ( Figure 2 The CPS loss is calculated based on the feature maps segmented by different colors.

[0087] HSIC loss is applied to all sample data, CPS loss is applied to unlabeled data, and SUP loss is applied to labeled data. The total loss function is: .

[0088] In practical applications, empirically Set to 0.1, Set it to 0.01 and use the CPS weight growth function to gradually increase and The proportion of the total loss is used to guide model training with the help of the loss function, so as to balance the impact of different loss items on model training, so that the model can make full use of labeled data and unlabeled data for learning and optimization.

[0089] In one embodiment, after obtaining a trained target segmentation model, the medical image to be processed can be input into the target segmentation model, and the multi-organ segmentation result in the medical image can be obtained through inference of the target segmentation model.

[0090] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0091] Furthermore, if Figure 5 As shown, as a specific implementation of the above-mentioned medical image segmentation model training method, an embodiment of the present application provides a medical image segmentation model training device 500, and the medical image segmentation model training device 500 includes: a data acquisition module 501, a construction module 502, a feature extraction module 503 and a training module 504.

[0092] The data acquisition module 501 is used to acquire medical image samples, wherein the medical image samples include first image samples labeled with organ categories and second image samples not labeled with organ categories; A construction module 502 is used to construct a semi-supervised segmentation framework of the initial segmentation model, wherein the semi-supervised segmentation framework includes a plurality of parallel visual encoding branches and a text encoding branch; A feature extraction module 503 is used to perform feature interactive fusion processing on the medical image sample through the visual encoding branch and the text encoding branch of the initial segmentation model to generate a feature map of the medical image sample; The training module 504 is used to calculate the first loss of the slicing matrix of the feature maps of multiple visual encoding branches in the orthogonal direction based on the kernel function; and, based on the supervision loss function, calculate the second loss of the feature map corresponding to the first image sample under the multiple visual encoding branches; and, based on the comparison pseudo-label supervision loss function, calculate the third loss of the feature map corresponding to the second image sample under the multiple visual encoding branches; and, train the initial segmentation model with the goal of minimizing the weighted sum of the first loss, the second loss, and the third loss, and output the target segmentation model after the iteration termination condition is met.

[0093] Optionally, the feature extraction module 503 includes: An encoding module (not shown in the figure) is used to encode text features and visual features in the medical image sample through a visual encoding branch and a text encoding branch, respectively, to obtain text feature encoding and visual feature encoding with the same number of channels, wherein the text feature is used to describe the organ category; A multimodal fusion module (not shown in the figure) is used to perform multimodal semantic fusion processing on the text feature coding and the visual feature coding to generate a text-visual fusion coding; The encoding module is also used to connect the text-visual fusion encoding and the visual feature encoding through a jump connection to form a feature map.

[0094] Optionally, a multimodal fusion module is specifically used to project the text feature code and the visual feature code to the query space, respectively, to generate a text query code and a visual query code; project the text feature code and the visual feature code to the key-value pair space, respectively, to generate a text key-value pair code and a visual key-value pair code; align the text query code based on the visual key-value pair code to generate a target text feature code; align the visual query code based on the text key-value pair code to generate a target visual feature code; fuse the target text feature code and the target visual feature code to determine the text-visual fusion code.

[0095] Optionally, the encoding module is specifically used to identify text features in medical image samples; input the text features into the text encoder of the text encoding branch to obtain text feature encoding; input the medical image samples into the image segmentation network of the visual encoding branch to obtain image feature encoding of the medical image samples; reshape the image feature encoding based on the channel dimension to which the text feature encoding belongs to determine the visual feature encoding.

[0096] Optionally, the image segmentation network of the visual coding branch includes multiple encoder layers; a multimodal fusion module, specifically used to perform a multimodal semantic fusion operation on the text feature coding and the visual feature coding corresponding to the last encoder layer, and determine the text-visual fusion coding of the target encoder layer, wherein the target encoder layer is the second to last encoder layer of the image segmentation network.

[0097] Optionally, the encoding module is specifically used to connect the text-visual fusion encoding and the visual feature encoding belonging to the same encoder layer through a skip connection.

[0098] Optionally, the orthogonal direction includes a first direction and a second direction that are perpendicular to each other, and the medical image segmentation model training device 500 further includes: A segmentation module (not shown in the figure) is used to perform a slicing operation on the feature map along a first direction and a second direction respectively to obtain a first direction slicing matrix and a second direction slicing matrix; The training module 504 is specifically used to calculate the kernel loss of the first direction slice matrix and the second direction slice matrix under different visual coding branches based on the kernel function; and to sum the kernel losses corresponding to multiple visual coding branches and the total kernel losses corresponding to multiple visual coding branches to obtain the first loss.

[0099] Optionally, the training module 504 is specifically used to determine the Euclidean distance matrices corresponding to the first direction slice matrix and the second direction slice matrix under the target visual coding branch, wherein the target visual coding branch is any visual coding branch among multiple visual coding branches; extract the triangular matrix in the Euclidean distance matrix; calculate the gamma value of the first direction slice matrix based on the median of the elements in the triangular matrix corresponding to the first direction slice matrix, and calculate the gamma value of the second direction slice matrix based on the median of the elements in the triangular matrix corresponding to the second direction slice matrix; calculate the kernel matrix based on the kernel function, the gamma value of the first direction slice matrix and the gamma value of the second direction slice matrix; determine the kernel loss under the target visual coding branch for the kernel matrix.

[0100] For the specific limitations of the medical image segmentation model training device, please refer to the limitations of the medical image segmentation model training method above, which will not be repeated here. Each module in the above-mentioned medical image segmentation model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0101] Based on the above Figure 1 The method shown in the embodiment of the present application accordingly provides a readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned Figure 1 The medical image segmentation model training method shown.

[0102] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0103] Based on the above Figure 1 The method shown, and Figure 5 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a computer device, which can be a personal computer, a server, a network device, etc. The computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 The medical image segmentation model training method shown.

[0104] Optionally, the computer device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Bluetooth interface, a WI-FI interface), etc.

[0105] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not limit the computer device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0106] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and saves the hardware and software resources of the computer device, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the physical device.

[0107] Through the description of the above implementation methods, the technicians in this field can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware to obtain medical image samples, wherein the medical image samples include a first image sample labeled with an organ category, and a second image sample without an organ category; construct a semi-supervised segmentation framework for the initial segmentation model, wherein the semi-supervised segmentation framework includes multiple parallel visual encoding branches and text encoding branches; perform feature interactive fusion processing on the medical image samples through the initial segmentation model to generate a feature map of the medical image samples; based on the kernel function, calculate the first loss of the feature map of the multiple visual encoding branches in the orthogonal direction of the slice matrix; based on the supervised loss function, calculate the second loss of the feature map corresponding to the first image sample under the multiple visual encoding branches; based on the comparison of the pseudo-label supervised loss function, calculate the third loss of the feature map corresponding to the second image sample under the multiple visual encoding branches; train the initial segmentation model with the goal of minimizing the weighted sum of the first loss, the second loss, and the third loss, and output the target segmentation model after satisfying the iteration termination condition. In the first aspect of the embodiment of the present application, the category text is encoded and interactively fused with the image visual feature encoding, so that the features recognized by the model can combine the features at the visual level and the text description of the organ category knowledge. In this way, we can make full use of text information to assist the model in identifying organs, enhance the recognition ability of the segmentation model for different organs, improve the accuracy of multi-organ segmentation in medical images, and especially improve the segmentation effect of small-sized and complex-shaped organs. Secondly, we fully consider the orthogonal geometric characteristics and independent relationships of multiple planes in 3D medical images, and use kernel functions to quantify the dependencies between feature sets. In this way, we can closely combine pixel geometry with feature learning, solve the problem that the existing technology ignores the geometric structure information of images, further enhance the segmentation model's understanding and processing ability of the spatial layout and morphological relationship of organs, reduce segmentation errors and boundary fuzziness, and thus improve the overall segmentation accuracy and reliability. At the same time, through semi-supervised learning, we can make full use of limited labeled data and a large amount of unlabeled data to improve the performance of the image segmentation model. Thirdly, through the cross-pseudo-supervision of multiple visual branches, the training results of multiple visual branches can learn from each other, which improves the generalization ability of the segmentation model. At the same time, we use multiple loss functions (kernel function, supervision loss function, and comparison of pseudo-label supervision loss function) to perform the joint optimization of two visual branches in parallel. Under the condition of ensuring consistency constraints, we can constrain the learning process of the model from different angles and improve the performance and robustness of the model.

[0108] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0109] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A medical image segmentation model training method, characterized in that: The method comprises: Acquire a medical image sample, wherein the medical image sample includes a first image sample labeled with an organ category and a second image sample not labeled with an organ category; Constructing a semi-supervised segmentation framework of the initial segmentation model, wherein the semi-supervised segmentation framework includes a plurality of parallel visual encoding branches and a text encoding branch; Performing feature interactive fusion processing on the medical image sample through the initial segmentation model to generate a feature map of the medical image sample; Based on the kernel function, calculating the first loss of the slicing matrix of the feature maps of the multiple visual encoding branches in the orthogonal direction; Calculate the second loss of the feature map corresponding to the first image sample under multiple visual encoding branches based on the supervised loss function; Calculate a third loss of the feature map corresponding to the second image sample under multiple visual encoding branches based on the comparison pseudo-label supervision loss function; The initial segmentation model is trained with the goal of minimizing the weighted sum of the first loss, the second loss, and the third loss, and a target segmentation model is output after an iteration termination condition is met.

2. The medical image segmentation model training method according to claim 1, characterized in that: The performing feature interactive fusion processing on the medical image sample by using the initial segmentation model to generate a feature map of the medical image sample includes: The text features and the visual features in the medical image sample are respectively encoded by the visual encoding branch and the text encoding branch to obtain text feature encoding and visual feature encoding with the same number of channels, wherein the text features are used to describe the organ category; Performing multimodal semantic fusion processing on the text feature code and the visual feature code to generate a text-visual fusion code; The text-visual fusion code and the visual feature code are connected via a jump connection to form the feature map.

3. The medical image segmentation model training method according to claim 2, characterized in that: The performing multimodal semantic fusion processing on the text feature code and the visual feature code to generate a text-visual fusion code includes: Respectively projecting the text feature code and the visual feature code into a query space to generate a text query code and a visual query code; Respectively projecting the text feature code and the visual feature code into a key-value pair space to generate a text key-value pair code and a visual key-value pair code; Performing an alignment operation on the text query code based on the visual key-value pair code to generate a target text feature code; Performing an alignment operation on the visual query encoding based on the text key-value pair encoding to generate a target visual feature encoding; A fusion operation is performed on the target text feature code and the target visual feature code to determine a text-visual fusion code.

4. The medical image segmentation model training method according to claim 2, characterized in that: The encoding process of the text features and the visual features in the medical image sample by the visual encoding branch and the text encoding branch to obtain the text feature encoding and the visual feature encoding with the same number of channels includes: identifying text features in the medical image sample; Inputting the text feature into the text encoder of the text encoding branch to obtain the text feature encoding; Inputting the medical image sample into the image segmentation network of the visual encoding branch to obtain the image feature encoding of the medical image sample; The image feature code is reshaped based on the channel dimension to which the text feature code belongs to determine the visual feature code.

5. The medical image segmentation model training method according to claim 3, characterized in that: The image segmentation network of the visual coding branch includes a plurality of encoder layers; the multimodal semantic fusion processing of the text feature coding and the visual feature coding to determine the text-visual fusion coding includes: Performing a multimodal semantic fusion operation on the text feature code and the visual feature code corresponding to the last encoder layer to determine a text-visual fusion code of a target encoder layer, wherein the target encoder layer is the penultimate encoder layer of the image segmentation network; The step of connecting the text-visual fusion code and the visual feature code through a skip connection to form the feature map includes: The text-visual fusion encoding and the visual feature encoding belonging to the same encoder layer are connected through a jump connection.

6. The medical image segmentation model training method according to claim 1, characterized in that: The orthogonal direction includes a first direction and a second direction perpendicular to each other, and the kernel function-based calculation of the first loss of the slicing matrix of the feature graphs of the plurality of visual encoding branches in the orthogonal direction includes: Slicing the feature map along the first direction and the second direction respectively to obtain a first direction slice matrix and a second direction slice matrix; Based on the kernel function, respectively calculating the kernel loss of the first direction slice matrix and the second direction slice matrix under multiple visual encoding branches; The kernel losses corresponding to the multiple visual encoding branches and the total kernel losses corresponding to the multiple visual encoding branches are summed to obtain the first loss.

7. The medical image segmentation model training method according to claim 6, characterized in that: The kernel function-based, respectively calculating the kernel loss of the first direction slice matrix and the second direction slice matrix under different visual encoding branches, comprises: Determine a Euclidean distance matrix corresponding to the first direction slice matrix and the second direction slice matrix respectively under a target visual coding branch, wherein the target visual coding branch is any visual coding branch among the multiple visual coding branches; Extracting a triangular matrix from the Euclidean distance matrix; Calculating the gamma value of the first-direction slice matrix based on the median of the elements in the triangular matrix corresponding to the first-direction slice matrix, and calculating the gamma value of the second-direction slice matrix based on the median of the elements in the triangular matrix corresponding to the second-direction slice matrix; Calculate a kernel matrix based on a kernel function, a gamma value of the first-direction slice matrix, and a gamma value of the second-direction slice matrix; A kernel loss under the target visual encoding branch is determined based on the kernel matrix.

8. A medical image segmentation model training device, characterized in that: The device comprises: A data acquisition module, used to acquire medical image samples, wherein the medical image samples include first image samples labeled with organ categories and second image samples not labeled with organ categories; A construction module, used to construct a semi-supervised segmentation framework of an initial segmentation model, wherein the semi-supervised segmentation framework includes a plurality of parallel visual encoding branches and a text encoding branch; A feature extraction module, used for performing feature interactive fusion processing on the medical image sample through the initial segmentation model to generate a feature map of the medical image sample; A training module, used for calculating the first loss of the slicing matrix of the feature maps of the plurality of visual encoding branches in an orthogonal direction based on a kernel function; and Calculating a second loss of the feature map corresponding to the first image sample under the plurality of visual encoding branches based on a supervised loss function; and Calculating a third loss of the feature map corresponding to the second image sample under the plurality of visual encoding branches based on the comparison pseudo-label supervision loss function; and The initial segmentation model is trained with the goal of minimizing the weighted sum of the first loss, the second loss, and the third loss, and a target segmentation model is output after an iteration termination condition is met.

9. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the medical image segmentation model training method as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the program, the medical image segmentation model training method according to any one of claims 1 to 7 is implemented.

Images