Training method and device of deep learning-based multi-anatomical region segmentation model

Abstract

The application provides a kind of training method and device of multi-anatomical region segmentation model based on deep learning, it is related to computer technology field, comprising: obtaining medical image sample, extracting multi-level local feature representation and global classification label feature by feature extraction backbone network, based on the pixel-level segmentation prediction result and global class prediction result output by multi-anatomical region segmentation model, total loss function is calculated, and the network parameter of the multi-anatomical region segmentation model is updated.The method and device provided by the application output pixel-level segmentation prediction result and global class prediction result through multi-anatomical region segmentation model, calculate total loss function, and update the network parameter of multi-anatomical region segmentation model based on total loss function, guide network model to optimize its local pixel division ability and global overall discriminant ability in the process of reverse propagation simultaneously, significantly improve the accuracy and stability of digestive tract anatomical segmentation region.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a training method and apparatus for a multi-anatomical region segmentation model based on deep learning. Background Technology

[0002] Medical images of the digestive tract (such as the esophagus, stomach, and small intestine) typically contain multiple sites or subregions with clear anatomical significance. Taking the stomach as an example, its interior can usually be further divided into multiple anatomical structures such as the cardia, fundus, body, angle, antrum, and pylorus. Precise automated image segmentation of multiple parts of the digestive tract is a crucial foundation for assisting in the diagnosis of digestive system diseases, accurately locating lesions, assessing the extent of lesions, and subsequent three-dimensional reconstruction.

[0003] Current methods for identifying body parts in capsule endoscopy images primarily rely on subjective human judgment. For automation, existing technologies typically employ mainstream deep learning-based object detection and instance segmentation schemes. For example, a common approach is to use an architecture based on ResNet with Split-Attention (ResNeSt) and Cascade Mask Region-based Convolutional Neural Network (CascadeMask R-CNN) to automatically identify stomach images acquired by capsule endoscopy. These existing methods mainly depend on candidate box mechanisms for local region identification and use convolutional networks to extract features from local receptive fields to achieve pixel-level mask generation and classification. However, in practical applications, the output results of these existing technologies are not accurate enough.

[0004] Therefore, improving the accuracy of segmentation of multiple anatomical regions of the digestive tract has become a technical problem that the industry urgently needs to solve. Summary of the Invention

[0005] This invention provides a training method and apparatus for a multi-anatomical region segmentation model based on deep learning, which solves the problem of low accuracy of the output results of multi-anatomical region segmentation of the digestive tract in the prior art and improves the accuracy of multi-anatomical region segmentation of the digestive tract.

[0006] This invention provides a training method for a deep learning-based multi-anatomical region segmentation model, comprising: Acquire medical image samples; The medical image samples are input into the feature extraction backbone network of the multi-anatomical region segmentation model to extract multi-level local feature representations and global classification label features of the medical image samples; The multi-anatomical region segmentation model performs feature decoding on the multi-level local feature representation through its segmentation branches, and outputs pixel-level segmentation prediction results. The classification branch of the multi-anatomical region segmentation model performs category discrimination based on the global classification label features and outputs the global category prediction result. Based on the pixel-level segmentation prediction results and the global category prediction results, the total loss function is calculated; The network parameters of the multi-anatomical region segmentation model are updated based on the total loss function.

[0007] In some embodiments, calculating the total loss function based on the pixel-level segmentation prediction result and the global category prediction result includes: Pixel-level segmentation loss, region consistency loss, dissecting adjacency loss, and boundary constraint loss are calculated based on the pixel-level segmentation prediction results. Calculate the classification head loss based on the global category prediction results; The total loss function is obtained by weighted summation of the pixel-level segmentation loss, the region consistency loss, the anatomical adjacency loss, the boundary constraint loss, and the classification head loss.

[0008] In some embodiments, updating the network parameters of the multi-anatomical region segmentation model based on the total loss function includes: In the first training phase, the weight coefficients of the anatomical adjacency loss, the boundary constraint loss, and the classification head loss are set to zero, and the network parameters are updated based on the total loss function after weighted summation. In the second training phase, the weight coefficients of the dissection adjacency loss, the boundary constraint loss, and the classification head loss are adjusted to non-zero preset values, and the network parameters are updated based on the total loss function after weighted summation.

[0009] In some embodiments, the dissecting adjacency loss is calculated based on the following steps: Obtain the adjacency matrix constructed based on the actual spatial topological relationships between various anatomical regions of the digestive tract; Based on the pixel-level segmentation prediction results, the predicted probabilities of adjacent pixel pairs belonging to different categories are obtained. Based on the predicted probabilities, a penalty term for class co-occurrence relationships is constructed; The penalty terms for all the adjacent pixel pairs are aggregated and calculated to obtain the dissected adjacency loss.

[0010] In some embodiments, the step of inputting the medical image sample into the feature extraction backbone network of a multi-anatomical region segmentation model to extract multi-level local feature representations and global classification label features of the medical image sample includes: Extract the classification labels output by the feature extraction backbone network at different coding levels; The classification labels output from different coding levels are fused to generate fused features; Based on the fusion features, the global classification label features are generated; The features are extracted from the local structured features output by the backbone network at different coding levels to generate the multi-level local feature representation.

[0011] In some embodiments, before inputting the medical image sample into the feature extraction backbone network of a multi-anatomical region segmentation model to extract multi-level local feature representations and global classification label features of the medical image sample, the method further includes: Obtain an unlabeled set of images of the digestive tract; The initial feature extraction network is trained under self-supervised conditions based on the digestive tract image set to obtain the pre-trained feature extraction backbone network.

[0012] This invention provides a deep learning-based multi-anatomical region segmentation method, comprising: Obtain the image of the digestive tract to be segmented; The digestive tract image to be segmented is input into a pre-trained multi-anatomical region segmentation model; the multi-anatomical region segmentation model is trained using the training method for a deep learning-based multi-anatomical region segmentation model as described in any one of claims 1 to 6. The target global category information is output through the classification branch of the multi-anatomical region segmentation model, and the initial pixel-level segmentation probability map is output through the segmentation branch of the multi-anatomical region segmentation model. Threshold filtering and connected component analysis are performed on the initial pixel-level segmentation probability map to obtain a denoised pixel-level segmentation probability map; Based on the target global category information, semantic consistency correction is performed on the denoised pixel-level segmentation probability map to obtain a corrected segmentation probability map; Based on a preset anatomical adjacency matrix, the corrected segmentation probability map is modified by prior anatomical structure, and the segmentation result of the target anatomical region is finally output.

[0013] This invention provides a training device for a deep learning-based multi-anatomical region segmentation model, comprising: The first acquisition module is used to acquire medical image samples; The feature extraction module is used to input the medical image sample into the feature extraction backbone network of the multi-anatomical region segmentation model, and extract the multi-level local feature representation and global classification label feature of the medical image sample; The segmentation module is used to perform feature decoding on the multi-level local feature representation through the segmentation branch of the multi-anatomical region segmentation model and output pixel-level segmentation prediction results. The classification module is used to perform category discrimination based on the global classification label features through the classification branch of the multi-anatomical region segmentation model and output the global category prediction result; The loss calculation module is used to calculate the total loss function based on the pixel-level segmentation prediction result and the global category prediction result; The backpropagation module is used to update the network parameters of the multi-anatomical region segmentation model based on the total loss function.

[0014] This invention provides a deep learning-based multi-anatomical region segmentation device, comprising: The second acquisition module is used to acquire images of the digestive tract to be segmented; An input module is used to input the digestive tract image to be segmented into a pre-trained multi-anatomical region segmentation model; the multi-anatomical region segmentation model is trained using the training method for a deep learning-based multi-anatomical region segmentation model as described in any one of claims 1 to 6. The model output module is used to output global category information of the target through the classification branch in the multi-anatomical region segmentation model, and to output an initial pixel-level segmentation probability map through the segmentation branch in the multi-anatomical region segmentation model. The denoising module is used to perform threshold filtering and connected component analysis on the initial pixel-level segmentation probability map to obtain a denoised pixel-level segmentation probability map. The correction module is used to perform semantic consistency correction on the denoised pixel-level segmentation probability map based on the target global category information to obtain a corrected segmentation probability map; The output module is used to perform anatomical structure prior correction on the corrected segmentation probability map based on a preset anatomical adjacency matrix, and finally output the segmentation result of the target anatomical region.

[0015] The present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the training method of the deep learning-based multi-anatomical region segmentation model, or implements the deep learning-based multi-anatomical region segmentation method.

[0016] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the training method of the deep learning-based multi-anatomical region segmentation model, or implements the deep learning-based multi-anatomical region segmentation method.

[0017] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the training method of the deep learning-based multi-anatomical region segmentation model, or implements the deep learning-based multi-anatomical region segmentation method.

[0018] The present invention provides a training method and apparatus for a deep learning-based multi-anatomical region segmentation model. By acquiring medical image samples and inputting them into the feature extraction backbone network of the multi-anatomical region segmentation model, the model extracts multi-level local feature representations and global classification label features from the medical image samples. During the feature encoding stage, it simultaneously captures local structured information for detail segmentation and global semantic information for macro-level category perception, laying a comprehensive feature foundation for achieving multi-level consistency constraints. The multi-level local feature representations are then decoded by the segmentation branch of the multi-anatomical region segmentation model, outputting pixel-level segmentation prediction results. This achieves precise localization and contour division of each anatomical region in the medical image at a micro-scale, ensuring the model's basic local pixel-level segmentation accuracy. The classification branch of the multi-anatomical region segmentation model further enhances the model's performance. The model performs category discrimination based on global classification features and outputs global category prediction results, enabling it to identify the overall anatomical environment to which the current image belongs at a macro level. By calculating the total loss function, local pixel discrimination and global semantic cognition are deeply integrated at the loss calculation level. This allows the prediction of local micro-pixels to be guided and constrained by macro-global prior semantics, thereby effectively reducing the problem of misjudgment of anatomical regions caused by similar local features. The network parameters of the multi-anatomical region segmentation model are updated based on the total loss function, guiding the network model to simultaneously optimize its local pixel segmentation ability and global overall discrimination ability during backpropagation. Finally, a deep learning model is trained that maintains segmentation accuracy while possessing extremely high structural consistency and medical anatomical rationality, significantly improving the accuracy and stability of digestive tract anatomical segmentation regions. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the training method for a deep learning-based multi-anatomical region segmentation model provided by the present invention.

[0022] Figure 2 This is a diagram showing the overall network structure of the training method for a deep learning-based multi-anatomical region segmentation model provided by this invention.

[0023] Figure 3 This is a flowchart illustrating the deep learning-based multi-anatomical region segmentation method provided by the present invention.

[0024] Figure 4 This is a schematic diagram of the training device for the deep learning-based multi-anatomical region segmentation model provided by the present invention.

[0025] Figure 5 This is a schematic diagram of the structure of the deep learning-based multi-anatomical region segmentation device provided by the present invention.

[0026] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps, units, or modules is not necessarily limited to those explicitly listed, but may include other steps, units, or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0029] Figure 1 This is a flowchart illustrating the training method for a deep learning-based multi-anatomical region segmentation model provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 110, 120, 130, 140, 150 and 160.

[0030] Step 110: Obtain medical image samples.

[0031] Specifically, the execution entity of the training method for a deep learning-based multi-anatomical region segmentation model provided in this embodiment of the invention is a training device for a deep learning-based multi-anatomical region segmentation model. This device can be implemented in software, such as a training program for a deep learning-based multi-anatomical region segmentation model running on a computer; or it can be implemented in hardware, such as a computer or server that executes the training method for the deep learning-based multi-anatomical region segmentation model.

[0032] Figure 2 This is a diagram of the overall network structure of the training method for the deep learning-based multi-anatomical region segmentation model provided by the present invention, as shown in the figure. Figure 2 As shown, unlike object detection and instance segmentation schemes based on the ResNeSt backbone and Cascade Mask R-CNN, the training method of the deep learning-based multi-anatomical region segmentation model provided in this embodiment of the invention does not rely on candidate box mechanisms for region recognition. Instead, it adopts an end-to-end semantic segmentation framework, combining multi-class classification heads, and using pixel-level segmentation loss, region consistency loss, anatomical adjacency loss, boundary constraint loss, and classification head loss to collaboratively constrain the same segmentation result from multiple levels, such as pixel discrimination, region integrity, inter-region topological relationships, anatomical contour accuracy, and global semantics. This ensures segmentation accuracy while improving the structural consistency and rationality of the segmentation result.

[0033] Medical image samples refer to digitized visual data containing specific organs or tissue structures within the human body. In the specific application scenario described in this article, medical image samples primarily refer to endoscopic images of the digestive tract (including but not limited to the esophagus, various subregions of the stomach, and the small intestine), such as color digital photographs continuously captured inside the patient using a capsule endoscopy system. Furthermore, a complete medical image sample typically contains multiple sites or subregions with clear anatomical significance (e.g., cardia, fundus, body, angle, antrum, pylorus, and duodenum).

[0034] In this embodiment of the invention, the capsule endoscopy system first acquires relevant images of the digestive tract. The acquired images are then quality-assessed, and capsule endoscopy image data is collected. Low-quality images, such as those that are too dark, overexposed, blurry, contain floating objects, mucus bubbles, or food residue, are manually annotated to remove them, resulting in higher-quality images. The esophagus, cardia, fundus, body, angle, antrum, pylorus, duodenum, and small intestine are then manually outlined using Labelme software to obtain a dataset.

[0035] Step 120: Input the medical image sample into the feature extraction backbone network of the multi-anatomical region segmentation model to extract the multi-level local feature representation and global classification label features of the medical image sample.

[0036] Specifically, the feature extraction backbone network refers to the network skeleton in a multi-anatomical region segmentation model used to encode the basic features of the original input image. In this embodiment of the invention, the backbone network preferably adopts a large visual model structure based on the Transformer architecture. Its main function is to map the two-dimensional medical image into a high-dimensional feature space, providing a data foundation for subsequent segmentation and classification tasks.

[0037] Multi-level local feature representations refer to the set of feature maps with spatial dimensionality output by the feature extraction backbone network in coding layers of different depths. Generally speaking, features extracted by shallow networks contain more high-resolution local structural information such as image edges and textures, while features extracted by deep networks contain more abstract semantic information. Preserving multi-level feature representations is an important foundation for achieving high-precision pixel-level dense prediction (i.e., image segmentation).

[0038] Global classification tokens (Cls Tokens) are a type of learnable feature vector unique to the Transformer network architecture. Before an image is input into the network, this token vector is added to the sequence of image patches. As the network depth increases, it continuously absorbs information from various regions of the entire image through a self-attention mechanism. The final output global classification token can highly condense the macroscopic anatomical semantic information of the entire image; for example, it can indicate whether the current image as a whole belongs to the "stomach" or the "intestines".

[0039] In this embodiment of the invention, the preprocessed medical image samples are divided into multiple image patches, and these image patches are converted into sequence vectors, which, along with the initial classification label vectors, are input into the feature extraction backbone network. During the forward propagation encoding process of the network, information interaction between different image patches is achieved through a multi-head attention mechanism. In this process, the backbone network simultaneously performs two feature outputs: on the one hand, it outputs high-dimensional feature encodings of each local image patch at different levels, forming the multi-level local feature representation; on the other hand, it extracts the classification label vectors after multi-level attention interaction as global classification label features.

[0040] In this embodiment of the invention, the feature extraction backbone network can be a self-disciplined pre-trained version 2 of the unlabeled self-distillation model (DINOv2).

[0041] Step 130: Decode the multi-level local feature representation through the segmentation branch of the multi-anatomical region segmentation model, and output pixel-level segmentation prediction results.

[0042] Specifically, the segmentation branch refers to a decoder network in a multi-task deep learning model architecture that is specifically designed to handle spatially dense prediction tasks. Its main function is to gradually restore the low-resolution, high-dimensional abstract feature maps extracted by the backbone network to the same spatial resolution as the original input image, thereby achieving pixel-level classification.

[0043] Pixel-level segmentation prediction results refer to the spatial probability matrix or mask output by the model for the input image. In this matrix, each pixel is assigned a set of probability values ​​to characterize the likelihood that the pixel belongs to various preset anatomical regions of the digestive tract (such as the esophagus, cardia, stomach body, etc.) and the background category.

[0044] In this embodiment of the invention, multi-level feature representations are input to the Dense Prediction Transformers (DPT) segmentation and decoding module. Through multi-scale feature decoding and fusion, and by utilizing an attention mechanism to achieve information interaction between features of different scales, pixel-level segmentation results are output in combination with the segmentation head.

[0045] Because the multi-level local feature representation contains feature maps of different scales, the segmentation branch effectively fuses and upsamples these cross-scale features during feature decoding. Through this decoding process, the network can simultaneously utilize the powerful semantic discrimination capabilities of deep features to roughly locate the extent of each anatomical region, and utilize the fine texture information of shallow features to accurately depict the boundary contours between different anatomical regions, ultimately outputting pixel-level segmentation prediction results corresponding to the original input image size.

[0046] Step 140: Based on the global classification label features, the classification branch of the multi-anatomical region segmentation model is used to determine the category and output the global category prediction result.

[0047] Specifically, the classification branch (i.e., the classification head module) is an independent decoding branch in a multi-anatomical region segmentation model, distinct from the pixel-level segmentation branch. In terms of network structure, it is typically constructed from fully connected layers or multi-layer perceptrons, specifically designed to process global-level feature vectors.

[0048] Global category prediction results refer to the probability distribution of the current input medical image belonging to one or more major anatomical regions in the overall macro scene. For example, in a digestive tract scene, this result is used to macroscopically determine whether the current capsule endoscopy image belongs to "stomach image", "esophagus image", or "small intestine image".

[0049] In this embodiment of the invention, Cls Tokens output from different layers of the backbone network are fused to construct a multi-level global semantic representation. Based on this fused feature, classification modeling is performed to globally classify the input digestive tract image, achieving full-image classification. The Cls Tokens output from the Transformer structure in the segmentation network backbone are used as image-level global semantic representations. Multi-class classification branches are constructed based on this representation. Overall anatomical semantic information is obtained by classifying the Cls Tokens, thereby introducing global semantic constraints during pixel-level segmentation to achieve overall consistency control of local prediction results.

[0050] During the forward propagation of the model, after extracting the global classification label features representing the semantics of the entire image, the multi-anatomical region segmentation model inputs them into the classification branch. The classification branch performs linear or non-linear mapping calculations on this high-dimensional feature vector through its internal network weight coefficients, reducing its dimensionality and projecting it into a preset medical category classification space. Subsequently, the probability confidence of the current image belonging to each macroscopic digestive tract scene is calculated, and the category with the highest probability confidence or the entire probability distribution is output as the final global category prediction result.

[0051] A classification branch based on Cls Token is introduced to model the global semantic information at the image level. The overall anatomical structure is constrained by the classification loss and jointly optimized with the pixel-level segmentation loss. The global semantic information is used as an auxiliary supervision signal to act on the segmentation result, so that the model can take into account the global semantic consistency while performing pixel-level prediction.

[0052] Step 150: Calculate the total loss function based on the pixel-level segmentation prediction results and the global category prediction results.

[0053] Specifically, in deep learning model training, the loss function is used to measure the degree of difference between the model's predicted output and the true annotation. In this embodiment of the invention, the total loss function is not a single-dimensional error calculation, but a comprehensive objective function that integrates multiple levels of constraints, used to guide the model parameters to be updated in the optimal direction during backpropagation.

[0054] Traditional medical image segmentation schemes typically rely solely on pixel-level errors to calculate loss, which can easily lead to the model getting bogged down in local features, resulting in misclassifications of structurally similar but semantically incompatible areas (e.g., segmenting an isolated small intestine region in a stomach image). To address this issue, this step innovatively integrates and jointly evaluates the "global category prediction result" and the "pixel-level segmentation prediction result" when calculating the total loss function. In this way, the total loss function not only penalizes errors in local pixel boundary delineation but also penalizes biases in overall anatomical semantic cognition. This leverages global prior knowledge to provide macroscopic guidance for local pixel segmentation, ensuring that the features ultimately learned by the model possess both pixel-level accuracy and conform to overall anatomical common sense.

[0055] In this embodiment of the invention, the total loss function is composed of multiple loss sub-items with specific dimensions. Based on the pixel-level segmentation prediction results, multiple local loss sub-items can be further calculated to constrain pixel discrimination accuracy, region morphological integrity, local boundary sharpness, and the spatial topological rationality between organs; simultaneously, based on the global category prediction results, a global loss sub-item can be calculated to constrain the overall image semantics. Finally, by fusing these loss sub-items covering different constraint levels, the total loss function used for updating the overall network parameters can be obtained.

[0056] By introducing a classification head, a collaborative enhancement mechanism for global semantics and local pixel prediction is achieved. The classification branch provides image-level prior semantic constraints, guiding the segmentation branch to maintain overall semantic consistency during prediction, thereby effectively reducing misjudgment problems between structurally similar regions (such as confusion between the pylorus and the duodenum).

[0057] Step 160: Update the network parameters of the multi-anatomical region segmentation model based on the total loss function.

[0058] Specifically, network parameters refer to the learning variables that can be dynamically adjusted during the training process within a multi-anatomical region segmentation model (including the pre-trained feature extraction backbone network, segmentation branch, and classification branch). These mainly include the weight matrix and bias terms in each layer of the neural network (such as the Transformer module, convolutional layer, fully connected layer, etc.).

[0059] After calculating the total loss function under multiple constraints, including pixel-level and global category, the system automatically calculates the partial derivative (i.e., gradient) of this total loss function with respect to each network parameter in the model. Subsequently, the optimization algorithm fine-tunes the network parameters of the model based on the learning rate and the calculated gradient. By repeatedly performing the iterative process of "forward propagation output prediction - calculating total loss - backpropagation gradient calculation - updating network parameters" on a large number of medical image samples, the value of the total loss function gradually converges. This process enables the multi-anatomical region segmentation model to autonomously learn the deep features of digestive tract images from the data, ultimately achieving a generalization ability that possesses both high-precision local pixel segmentation capabilities and strictly adheres to the rules of digestive tract anatomical topology.

[0060] It should be noted that, in this embodiment of the invention, the total loss function is composed of multiple sub-loss terms with different physical and anatomical meanings (e.g., pixel-level segmentation loss, region consistency loss, anatomical adjacency loss, boundary constraint loss, and classifier head loss). These terms collaboratively constrain the same segmentation result at multiple levels, including pixel discrimination, region integrity, inter-region topological relationships, and anatomical contour accuracy, thereby achieving overall consistency of the digestive tract anatomical structure during training. By employing a collaborative optimization strategy of global classification loss, pixel-level segmentation loss, and structural constraint loss, multi-level consistency constraints from global semantics to local pixels are achieved.

[0061] The training method for a deep learning-based multi-anatomical region segmentation model provided in this invention involves acquiring medical image samples and inputting them into the feature extraction backbone network of the multi-anatomical region segmentation model. This extracts multi-level local feature representations and global classification label features from the medical image samples. During the feature encoding stage, local structured information for detail segmentation and global semantic information for macro-level category perception are simultaneously captured, laying a comprehensive feature foundation for achieving multi-level consistency constraints. The segmentation branch of the multi-anatomical region segmentation model decodes these multi-level local feature representations, outputting pixel-level segmentation prediction results. This achieves precise localization and contour segmentation of each anatomical region in the medical image at a micro-scale, ensuring the model's basic local pixel-level segmentation accuracy. The classification branch of the multi-anatomical region segmentation model further enhances the model's performance. The model performs category discrimination based on global classification features and outputs global category prediction results, enabling it to identify the overall anatomical environment to which the current image belongs at a macro level. By calculating the total loss function, local pixel discrimination and global semantic cognition are deeply integrated at the loss calculation level. This allows the prediction of local micro-pixels to be guided and constrained by macro-global prior semantics, thereby effectively reducing the problem of misjudgment of anatomical regions caused by similar local features. The network parameters of the multi-anatomical region segmentation model are updated based on the total loss function, guiding the network model to simultaneously optimize its local pixel segmentation ability and global overall discrimination ability during backpropagation. Finally, a deep learning model is trained that maintains segmentation accuracy while possessing extremely high structural consistency and medical anatomical rationality, significantly improving the accuracy and stability of digestive tract anatomical segmentation regions.

[0062] In some embodiments, calculating the total loss function based on the pixel-level segmentation prediction result and the global category prediction result includes: Pixel-level segmentation loss, region consistency loss, dissecting adjacency loss, and boundary constraint loss are calculated based on the pixel-level segmentation prediction results. Calculate the classification head loss based on the global category prediction results; The total loss function is obtained by weighted summation of the pixel-level segmentation loss, the region consistency loss, the anatomical adjacency loss, the boundary constraint loss, and the classification head loss.

[0063] Specifically, pixel-level segmentation loss is used to evaluate the difference between the class prediction probability of each individual pixel and the true label, focusing on the pixel discrimination accuracy of local features.

[0064] Region consistency loss is used to measure the degree of overlap between the predicted anatomical regions and the actual labeled regions from the perspective of overall morphology and region level, guide the model to learn the overall morphological features of each anatomical region, and solve the problem of positive and negative sample imbalance.

[0065] Anatomical adjacency loss is a topological constraint term based on medical prior knowledge introduced in this embodiment of the invention. It aims to evaluate whether the contact and connectivity relationships between anatomical regions in the predicted results conform to the actual spatial topology of the digestive tract, and to penalize pathological segmentation results that violate medical common sense.

[0066] Boundary constraint loss is used to measure the distance or difference between the predicted segmentation contour and the actual anatomical boundary. It aims to refine and sharpen the boundary line between anatomical regions with similar structures (such as the pylorus and duodenum) and improve the accuracy of boundary localization.

[0067] The classification head loss is used to evaluate the consistency between the global semantic category of the entire image output by the classification branch and the true image category. Through image-level prior judgment, it provides a top-down macroscopic semantic constraint on the underlying pixel segmentation.

[0068] After the model outputs its predictions via forward propagation, the system evaluates the results from both local and global dimensions. First, for the pixel-level segmentation predictions output by the segmentation branch, the system calculates the pixel-level segmentation loss, region consistency loss, anatomical adjacency loss, and boundary constraint loss from four dimensions: absolute pixel classification, connected region integrity, medical topological common sense, and contour edge sharpness. Simultaneously, for the global category predictions output by the classification branch, the system calculates the classification head loss in parallel.

[0069] The system then performs a weighted summation of the loss terms across the five dimensions. Since the magnitude, convergence speed, and dominant role of each type of loss differ at different training stages, the system assigns a corresponding weight coefficient to each loss term, generating a unique total loss function through a linear combination.

[0070] In this embodiment of the invention, the formula for calculating the pixel-level segmentation loss function is as follows: Where N is the total number of pixels; C is the number of categories; The true label of the i-th category; Let be the predicted probability of the i-th category.

[0071] In this embodiment of the invention, the formula for calculating the regional consistency loss function is as follows: Where N is the total number of pixels; C is the number of categories; To predict mask probability; This is a real label.

[0072] In this embodiment of the invention, the formula for calculating the boundary constraint loss is: in, The set of true boundary pixels; These are the true boundary values; To predict boundary values.

[0073] In this embodiment of the invention, the calculation formula for the cross-entropy loss function of the classification head is as follows: Where C is the number of categories; The true label of the i-th category; Let be the predicted probability of the i-th category.

[0074] In this embodiment of the invention, the formula for calculating the total loss function of network training is as follows: in, Pixel-level segmentation loss; This represents a loss of regional consistency. For boundary constraint loss; The cross-entropy loss is for the classification head; To dissect adjacent loss; , , , , Assign weights to each loss.

[0075] The training method for a deep learning-based multi-anatomical region segmentation model provided in this invention constructs a total loss function to achieve collaborative constraints on the segmentation results of the same medical image, from pixel discrimination, region integrity, inter-region topological relationships, anatomical contour accuracy to global semantics. This multi-level constraint enables the model to greatly improve the medical consistency and rationality of macroscopic anatomical structures while ensuring microscopic segmentation accuracy.

[0076] In some embodiments, updating the network parameters of the multi-anatomical region segmentation model based on the total loss function includes: In the first training phase, the weight coefficients of the anatomical adjacency loss, the boundary constraint loss, and the classification head loss are set to zero, and the network parameters are updated based on the total loss function after weighted summation. In the second training phase, the weight coefficients of the dissection adjacency loss, the boundary constraint loss, and the classification head loss are adjusted to non-zero preset values, and the network parameters are updated based on the total loss function after weighted summation.

[0077] Specifically, the weighting coefficient refers to the numerical multiplication factor assigned to each sub-loss term when calculating the total loss function for multiple tasks. It is used to adjust the penalty proportion of errors in different dimensions in the overall network gradient backpropagation.

[0078] In this embodiment of the invention, during network training, a phased, multi-loss joint optimization approach is used to train the model parameters in order to gradually improve segmentation accuracy and the rationality of anatomical structure.

[0079] In the early stages of training, the network is mainly trained under supervision using labeled samples. The class prediction results of each pixel are constrained by pixel-level segmentation loss, and the network is guided to learn the overall morphological features of each anatomical region by combining region consistency loss, so that the network can converge quickly and obtain stable initial segmentation ability.

[0080] In the later stages of training, while maintaining pixel-level segmentation loss and region consistency loss, an anatomical adjacency loss is introduced to constrain the spatial topological relationships between different anatomical regions, suppressing region adjacency that does not conform to the actual anatomical structure, thereby improving the structural rationality of the segmentation results. Simultaneously, a boundary constraint loss is introduced to further refine the segmentation boundaries between anatomical regions, improving the accuracy of boundary localization. A global classification branch is also introduced during training, using classification loss to impose region-level semantic constraints on the network, enabling the network to consider the overall category consistency of the entire image while performing pixel-level segmentation prediction.

[0081] It should be noted that during network training, the various loss terms are jointly optimized through a weighted approach. In the early stages of training, pixel-level segmentation loss and region consistency loss are the main optimization objectives to ensure that the network converges quickly and obtains stable segmentation results. In the later stages of training, dissecting adjacency loss and boundary constraint loss are gradually introduced to improve the rationality of the segmentation structure and the accuracy of the boundaries. At the same time, the classification head loss is used to provide global semantic constraints, so that the network can take into account the class consistency of the entire image on the basis of pixel-level segmentation.

[0082] The training method for a deep learning-based multi-anatomical region segmentation model provided in this invention employs a phased, multi-loss joint optimization approach to train model parameters during network training. This aligns with the cognitive pattern of deep learning models, which "learn to walk first (basic pixel classification), then learn to run (following medical anatomical logic)". This method ensures rapid and stable convergence of the network in the initial stage, while also refining it in the later stages, significantly improving the rationality and boundary accuracy of the final segmentation results at the medical structural level.

[0083] In some embodiments, the dissecting adjacency loss is calculated based on the following steps: Obtain the adjacency matrix constructed based on the actual spatial topological relationships between various anatomical regions of the digestive tract; Based on the pixel-level segmentation prediction results, the predicted probabilities of adjacent pixel pairs belonging to different categories are obtained. Based on the predicted probabilities, a penalty term for class co-occurrence relationships is constructed; The penalty terms for all the adjacent pixel pairs are aggregated and calculated to obtain the dissected adjacency loss.

[0084] Specifically, actual spatial topological relationships refer to the inherent physical connections between various organs or subregions of the digestive tract in real medical anatomy. For example, the esophagus is necessarily connected to the cardia, and the pylorus is necessarily connected to the duodenum; however, under normal physiological structure, it is absolutely impossible for the esophagus and duodenum, or the cardia and small intestine, to be in direct contact.

[0085] An adjacency matrix is ​​a mathematical representation of the actual spatial topological relationships described above. It is typically defined as a two-dimensional matrix (e.g., matrix M), where the rows and columns represent different anatomical region categories. When category i and category j are anatomically allowed to be adjacent, the corresponding element in the matrix is ​​set to 1 (i.e., legal contact); when category i and category j are anatomically impossible to be adjacent, the corresponding element in the matrix is ​​set to 0 (i.e., illegal contact).

[0086] In this embodiment of the invention, an adjacency rule matrix M is defined for the spatial adjacency relationships of the anatomical regions of the digestive tract: This matrix is ​​used to constrain the legal contact relationships between different categories.

[0087] In the field of image segmentation, category co-occurrence relationship refers to the joint probability that two spatially adjacent pixels in an image are predicted by the model to belong to two specific categories.

[0088] In this embodiment of the invention, boundary strength is obtained by calculating the spatial gradient of the probability maps of each category, and the probability co-occurrence between different categories is statistically analyzed based on the boundary regions to characterize the adjacency relationship between categories. For category pairs that do not satisfy the adjacency constraint, a penalty term is introduced to constrain them, thereby effectively suppressing the region contact phenomenon that does not conform to the anatomical structure and improving the spatial structural rationality and medical consistency of the segmentation results. The adjacency loss is calculated for the adjacent pixel pairs (p,q) of the predicted mask: in, Let p be the predicted probability that pixel p belongs to category i; Let q be the predicted probability that pixel q belongs to category j. Adjacency rules that represent the physical contact between organs in the human digestive tract.

[0089] The training method for a deep learning-based multi-anatomical region segmentation model provided in this invention constructs an adjacency matrix to model the spatial topological relationship between different anatomical regions. During the training process, it penalizes category combinations that do not conform to adjacency constraints, thereby effectively suppressing unreasonable region contact and improving the structural rationality of the segmentation results.

[0090] In some embodiments, the step of inputting the medical image sample into the feature extraction backbone network of a multi-anatomical region segmentation model to extract multi-level local feature representations and global classification label features of the medical image sample includes: Extract the classification labels output by the feature extraction backbone network at different coding levels; The classification labels output from different coding levels are fused to generate fused features; Based on the fusion features, the global classification label features are generated; The features are extracted from the local structured features output by the backbone network at different coding levels to generate the multi-level local feature representation.

[0091] Specifically, different coding levels refer to the neural network computation layers at different depths within the feature extraction backbone network, arranged from bottom to top. Typically, shallow coding levels mainly capture low-level visual details such as edges and colors in medical images, while deep coding levels can aggregate high-level abstract semantic features such as organ locations and anatomical categories.

[0092] In this embodiment of the invention, the pre-trained DINOv2 model is used as the feature extraction backbone network to encode the input digestive tract image, obtaining a high-dimensional feature representation containing global semantic information and local structural information. During the encoding process, the feature extraction backbone network simultaneously outputs a classification token (Cls token) and multi-level feature representations.

[0093] In the feature extraction stage, this embodiment of the invention not only relies on the output of the final layer of the network, but also performs in-depth mining from the inside of the backbone network. The specific processing logic is divided into two parallel operations: "global semantic flow" and "local structural flow".

[0094] In terms of global semantic information extraction, the steps are as follows: extracting classification labels output by the backbone network at different coding levels; fusing classification labels output by different coding levels to generate fused features; and generating global classification label features based on the fused features.

[0095] Traditional methods often extract only a single classification label from the last layer of the backbone network for subsequent discrimination. However, this invention, considering the complexity and variability of gastrointestinal medical images, configures the model to simultaneously extract classification labels from multiple stages at different depths within the backbone network (such as shallow, mid, and deep encoders at specific intervals). Since the classification labels at different levels focus on cross-scale information ranging from macroscopic texture to overall anatomical structure, this invention fuses these labels through methods such as concatenation, weighted summation, or attention aggregation. The resulting global classification label feature possesses extremely strong multi-dimensional semantic expressive power. This feature will then be specifically fed into an independent classification branch, serving as the cornerstone of macroscopic constraints and providing the network with stable and highly accurate prior knowledge of the whole-image category.

[0096] In terms of local structural information extraction, the process involves extracting local structured features output by the feature extraction backbone network at different coding levels to generate multi-level local feature representations. Simultaneously with acquiring classification labels, the feature extraction backbone network also outputs local feature sequences corresponding to the spatial locations of each input image patch. This embodiment of the invention simultaneously extracts high-resolution shallow features and low-resolution deep features distributed across different coding levels. These local structured features, preserving the original image's spatial topology, are combined to form multi-level local feature representations.

[0097] The training method for a deep learning-based multi-anatomical region segmentation model provided in this invention extracts multi-level local feature representations and global classification label features from medical image samples. This not only ensures that the model can "see the big picture"—that is, use global classification label features to clearly define whether the current image is the stomach fundus or the small intestine macroscopically—but also ensures that the model can "detail"—that is, use multi-level local feature representations to accurately delineate the complex tissue boundaries of each anatomical location. The effective splitting and extraction of these two information streams lays a solid feature data foundation for the subsequent collaborative optimization of segmentation and classification branches.

[0098] In some embodiments, before inputting the medical image sample into the feature extraction backbone network of a multi-anatomical region segmentation model to extract multi-level local feature representations and global classification label features of the medical image sample, the method further includes: Obtain an unlabeled set of images of the digestive tract; The initial feature extraction network is trained under self-supervised conditions based on the digestive tract image set to obtain the pre-trained feature extraction backbone network.

[0099] Specifically, unlabeled digestive tract image sets refer to raw medical image collections acquired using medical devices such as capsule endoscopes. These images may have undergone basic quality assessments (such as removing severely overexposed, underexposed, or obscured images with significant amounts of floating debris), but they have not been professionally analyzed by physicians to perform pixel-level anatomical region contour outlining or category labeling. Compared to labeled data, this type of data is extremely inexpensive to acquire and can easily be amassed in massive quantities.

[0100] Self-supervised training is a deep learning paradigm that does not require manual intervention in labeling. Its core lies in utilizing the inherent structure or contextual relationships of the data itself to construct proxy tasks, such as image mask reconstruction or multi-view contrast learning. This forces the network to continuously update weights by uncovering implicit patterns within the data, thereby autonomously learning high-dimensional feature representations of the image.

[0101] In this embodiment of the invention, the DINOv2 network is trained in a self-supervised manner using unannotated digestive tract capsule image data, enabling the network to learn global semantic information and local structural features in the images without manual annotation.

[0102] The system first acquires a massive set of unlabeled digestive tract images. These raw images are then input into an initial feature extraction network with initial weights (e.g., randomly initialized or based on general natural image weights). During self-supervised training, the initial feature extraction network does not need prior knowledge of the specific locations and categories of each anatomical region (e.g., cardia, antrum, pylorus) in the images. By performing self-supervised tasks on the massive unlabeled data, the network continuously learns and refines the unique texture features of digestive tract tissues, mucosal morphological distribution, illumination variation patterns, and macroscopic visual relationships between different local regions. After sufficient iterative optimization, this initial feature extraction network "internalizes" rich prior knowledge of the digestive tract domain. At this point, its network parameters are extracted and saved, resulting in the pre-trained feature extraction backbone network.

[0103] The training method for a deep learning-based multi-anatomical region segmentation model provided in this invention significantly improves the expression quality of basic features by using unannotated digestive tract capsule image data for self-supervised training of the DINOv2 network. It directly extracts highly representative multi-level local structural features and global classification labels rich in macroscopic contextual semantics from the input medical image samples, thereby laying a solid and high-quality underlying feature foundation for subsequent multi-branch feature decoding and anatomical topological constraints.

[0104] Figure 3 This is a flowchart illustrating the deep learning-based multi-anatomical region segmentation method provided by the present invention, as shown below. Figure 3 As shown, the method includes steps 310, 320, 330, 340, 350 and 360.

[0105] Step 310: Obtain the image of the digestive tract to be segmented.

[0106] Specifically, the execution entity of the deep learning-based multi-anatomical region segmentation method provided in this embodiment of the invention is a deep learning-based multi-anatomical region segmentation device. This device can be implemented in software, such as a deep learning-based multi-anatomical region segmentation program running on a computer; or it can be implemented in hardware, such as a computer or server executing the deep learning-based multi-anatomical region segmentation method.

[0107] The digestive tract image to be segmented refers to medical image data that needs to be automatically analyzed by electronic devices during the actual clinical auxiliary diagnosis or model inference stage. At the anatomical level, the image usually contains one or more digestive tract parts or sub-regions with specific medical significance, such as including but not limited to: esophagus, cardia, fundus, body, angle, antrum, pylorus, duodenum, and small intestine.

[0108] In this embodiment of the invention, an image of the digestive tract to be segmented is first obtained.

[0109] Step 320: Input the digestive tract image to be segmented into a pre-trained multi-anatomical region segmentation model; the multi-anatomical region segmentation model is trained using the training method of the deep learning-based multi-anatomical region segmentation model as described in any one of claims 1 to 6.

[0110] Specifically, the pre-trained multi-anatomical region segmentation model refers to a deep neural network entity that has reached convergence after iterative learning on massive amounts of data using the training method described in the previous embodiments (i.e., using a multi-loss joint optimization strategy with constraints including global classification labels and anatomical adjacency matrices), and whose internal network parameters (weights and biases) have been solidified and saved. The hidden space inside this model has solidified the "medical prior memory" of the macroscopic discrimination ability of the digestive tract and the microscopic boundary topological rules.

[0111] Before inputting images into the model, the raw images may contain low-quality images due to the complex physical environment inside the digestive tract. Therefore, embodiments of this invention typically perform preprocessing operations on the digestive tract images to be segmented, operations that match those used in the training phase. Since deep learning models have strict requirements on the dimensionality and numerical distribution of input tensors, the system performs steps on the raw images to be segmented, including but not limited to image cropping and scaling, pixel color space conversion, and numerical normalization. This process aims to eliminate data distribution biases caused by different endoscopic hardware and lighting environments, ensuring that the data distribution characteristics of the current input data are highly consistent with the feature space learned by the pre-trained model, thereby maximizing the model's inference accuracy.

[0112] After completing the aforementioned adaptation preprocessing, the system formally inputs the standardized digestive tract image to be segmented into the input layer of the pre-trained multi-anatomical region segmentation model. At this point, the model system is in inference mode, i.e., all network parameters are frozen, the loss function gradient is not calculated, and backpropagation is not performed. The image data undergoes strictly unidirectional forward propagation computation within the model.

[0113] During the forward propagation process, the input image is first mapped layer by layer by the feature extraction backbone network and parsed into various high-dimensional feature maps containing local edge texture features and macro semantic information. Subsequently, these pre-trained channel feature streams are input in parallel to the classification and segmentation branches (such as the DPT decoding module) inside the model for decoding and reconstruction according to the predetermined architecture.

[0114] Step 330: Output the global category information of the target through the classification branch in the multi-anatomical region segmentation model, and output the initial pixel-level segmentation probability map through the segmentation branch in the multi-anatomical region segmentation model.

[0115] Specifically, during the model deployment phase (i.e., the forward inference phase), the fully trained multi-anatomical region segmentation model exhibits a decoupled, two-branch parallel prediction architecture. This architecture is able to provide an understanding of the current medical image from both macroscopic and microscopic dimensions, based on the features extracted in the preceding steps.

[0116] First, for macro-level predictions, processing is performed using a classification branch within the multi-anatomical region segmentation model. This classification branch receives global classification label features from the feature extraction backbone network. The classification branch typically contains multilayer perceptrons or fully connected layers, which output global category information by performing dimensionality reduction and activation normalization on the highly condensed global semantic features.

[0117] Global category information for a target is typically presented as a category probability distribution vector or by directly outputting category labels. In medical applications, its physical meaning represents the model's authoritative judgment on the overall macroscopic classification of the current digestive tract image to be segmented (e.g., determining whether the overall field of view belongs to the "stomach," "esophagus," or "small intestine"). This global category information is not merely an isolated classification result, but rather a powerful prior knowledge of macroscopic anatomy, providing a guideline for subsequent semantic error prevention and filtering.

[0118] For predictions at the micro-dimensional level, parallel processing is performed through a segmentation branch in a multi-anatomical region segmentation model. This segmentation branch receives multi-level local feature representations extracted by the backbone network. In a specific implementation of this invention, the segmentation branch can employ a modular architecture with powerful decoding capabilities, such as DPT. During the decoding stage, the segmentation branch achieves depth information interaction and progressive upsampling between feature maps of different resolutions through skip connections, multi-scale feature fusion, and attention mechanisms, ultimately outputting an initial pixel-level segmentation probability map through the top-level segmentation head.

[0119] The initial pixel-level segmentation probability map is typically a multi-channel tensor matrix with the same spatial dimensions as the original image to be segmented. The number of channels in the matrix corresponds to the number of specific anatomical sub-regions (such as cardia, fundus, antrum, pylorus, etc.) to be segmented. The value at each coordinate position in the matrix represents the prediction confidence or probability distribution of the pixel at the corresponding position in the original medical image belonging to each type of anatomical sub-region.

[0120] Step 340: Perform threshold filtering and connected component analysis on the initial pixel-level segmentation probability map to obtain a denoised pixel-level segmentation probability map.

[0121] Specifically, threshold filtering is a basic pixel-level classification filtering mechanism. Since the model outputs probability values ​​in the range [0, 1], the system sets a predefined activation threshold (e.g., 0.5 or higher) and classifies pixels with probabilities below this threshold as non-class regions or background, thereby initially filtering out ambiguous points with low model prediction accuracy.

[0122] Connected component analysis is a classic method in digital image topology analysis. In medical images, pixels of the same organ or anatomical location are necessarily continuous and contiguous in physical space. Connected component analysis aims to scan an image, find and label sets of spatially adjacent pixels that are predicted to be of the same category (i.e., connected components), and statistically analyze the geometric properties of each connected component (such as pixel area or physical size).

[0123] In this embodiment of the invention, after training is completed, the same preprocessing is performed on the image to be predicted during the inference stage. The image is then input into the trained model to obtain the prediction result. First, threshold filtering and connected component analysis are performed on the probability map output by the model to remove small noise regions.

[0124] Although deep learning networks can extract high-dimensional features, the actual medical imaging environment of the digestive tract, such as that of capsule endoscopy, is extremely complex and is often affected by mucus bubbles, food residue, local reflections, or floating objects. These disturbances can easily cause the network to misjudge within local receptive fields, resulting in the output initial pixel-level segmentation probability map containing small, abruptly located, and isolated false positive prediction patches.

[0125] To eliminate the local prediction anomalies caused by poor image quality, this embodiment of the invention performs morphological post-processing during the model inference stage. First, threshold filtering is used to perform hard voting or binarization on the initial pixel-level segmentation probability maps of multiple categories, stripping away low-confidence edge pixels. Then, based on pixel adjacency relationships (such as 4-adjacency or 8-adjacency algorithms), connected component analysis is performed on the filtered prediction results. The system traverses the entire segmentation result, identifies each independent connected patch of the same category, and accurately calculates the connected area of ​​each patch.

[0126] Based on common knowledge of medical anatomy, any substantial anatomical region of the digestive tract occupies a certain continuous area in a single frame image. Therefore, this embodiment of the invention presets a minimum area tolerance threshold corresponding to each anatomical site. The system compares the calculated connected component areas and determines tiny connected components with areas smaller than the corresponding area tolerance threshold as scattered misclassifications caused by noise. The predicted category of the pixels occupied by these components is forcibly reset to zero (classified as background) or integrated into the surrounding category with the largest area and highest confidence through morphological methods.

[0127] After the above dual filtering based on pixel confidence and spatial coherence, a denoised pixel-level segmentation probability map is finally generated.

[0128] Step 350: Based on the target global category information, perform semantic consistency correction on the denoised pixel-level segmentation probability map to obtain a corrected segmentation probability map.

[0129] Specifically, semantic consistency correction refers to breaking away from the limitations of traditional deep learning models that rely solely on local pixel features for inference. It utilizes global semantic labels in the macro dimension as the highest-level logical criterion to perform cross-scale conflict verification and logical filtering on the prediction results of local pixels in the micro dimension.

[0130] In this embodiment of the invention, semantic consistency correction is performed on the segmentation results by combining the global category information output by the classification head. For example, when the classification result is determined to be a stomach image, unreasonable predictions of small intestine regions are suppressed. The system extracts the global category information of the target output from the classification branch and logically compares it with the predicted category of each target region in the denoised pixel-level segmentation probability map. Because the various tissues and organs within the digestive tract have a strict distribution pattern in their physiological structure, certain anatomical regions of specific categories are absolutely impossible to appear from a specific global perspective. Therefore, this embodiment of the invention constructs cross-validation logic to forcibly eliminate the "medical logical paradox" in the prediction.

[0131] Step 360: Based on the preset anatomical adjacency matrix, perform anatomical structure prior correction on the corrected segmentation probability map, and finally output the segmentation result of the target anatomical region.

[0132] Specifically, anatomical structure prior correction refers to moving beyond the realm of probabilistic prediction driven purely by deep learning network data, and using rigid logical rules constructed based on objective human medical common sense (i.e., prior knowledge) to perform mandatory secondary verification and logical intervention to correct local boundary defects in the network output.

[0133] In this embodiment of the invention, anatomical priors are introduced based on the adjacency matrix to constrain and correct abnormal adjacency regions, thereby reducing prediction results that do not conform to anatomical rules.

[0134] The deep learning-based multi-anatomical region segmentation method provided in this invention inputs the digestive tract image to be segmented into a multi-anatomical region segmentation model pre-trained based on anatomical topological constraints. It simultaneously acquires global target category information and an initial pixel-level segmentation probability map through classification and segmentation branches, achieving dual information extraction from macroscopic image-level semantics to microscopic pixel-level features, providing a comprehensive data foundation for subsequent multi-dimensional logical correction. By performing threshold filtering and connected component analysis on the initial pixel-level segmentation probability map, it effectively removes small-area isolated noise and artifact patches caused by the complex internal environment of the digestive tract (such as floating objects, reflections, and bubbles), initially purifying the segmentation mask and reducing subsequent computational interference. Furthermore, by using a method based on the target global category... Information is used to perform semantic consistency correction on the denoised pixel-level segmentation probability map, constructing a top-down constraint mechanism of "macro-qualitative guidance for micro-quantitative analysis". This effectively suppresses absurd region predictions that do not match the main organs in the whole image, and significantly improves the overall semantic logic consistency of the image. By performing anatomical structure prior correction on the corrected segmentation probability map based on a preset anatomical adjacency matrix, objective clinical medical anatomical rules are forcibly embedded into the post-processing closed loop of deep learning. This forcibly detects and corrects abnormal contact phenomena at local junction edges that violate the connection rules of real digestive tract organs, ultimately ensuring the absolute rationality and interpretability of the output target anatomical region segmentation results in the medical spatial topology, and improving the accuracy and stability of digestive tract anatomical segmentation regions.

[0135] The apparatus provided in the embodiments of the present invention will be described below. The apparatus described below can be referred to in correspondence with the method described above.

[0136] Figure 4 This is a schematic diagram of the training device for the deep learning-based multi-anatomical region segmentation model provided by the present invention, as shown in the figure. Figure 4 As shown, the device includes a first acquisition module 410, a feature extraction module 420, a segmentation module 430, a classification module 440, a loss calculation module 450, and a backpropagation module 460 connected in sequence.

[0137] The first acquisition module 410 is used to acquire medical image samples; The feature extraction module 420 is used to input the medical image sample into the feature extraction backbone network of the multi-anatomical region segmentation model, and extract the multi-level local feature representation and global classification label feature of the medical image sample; The segmentation module 430 is used to perform feature decoding on the multi-level local feature representation through the segmentation branch of the multi-anatomical region segmentation model and output pixel-level segmentation prediction results. The classification module 440 is used to perform category discrimination based on the global classification label features through the classification branch of the multi-anatomical region segmentation model and output the global category prediction result; The loss calculation module 450 is used to calculate the total loss function based on the pixel-level segmentation prediction result and the global category prediction result; The backpropagation module 460 is used to update the network parameters of the multi-anatomical region segmentation model based on the total loss function.

[0138] The training device for a deep learning-based multi-anatomical region segmentation model provided in this invention acquires medical image samples and inputs them into the feature extraction backbone network of the multi-anatomical region segmentation model. This extracts multi-level local feature representations and global classification label features from the medical image samples. During the feature encoding stage, it simultaneously captures local structured information for detail segmentation and global semantic information for macro-level category perception, laying a comprehensive feature foundation for achieving multi-level consistency constraints. The segmentation branch of the multi-anatomical region segmentation model decodes the multi-level local feature representations, outputting pixel-level segmentation prediction results. This achieves precise localization and contour division of each anatomical region in the medical image at the micro-scale, ensuring the model's basic local pixel-level segmentation accuracy. The classification branch of the multi-anatomical region segmentation model further enhances the model's performance. The model performs category discrimination based on global classification features and outputs global category prediction results, enabling it to identify the overall anatomical environment to which the current image belongs at a macro level. By calculating the total loss function, local pixel discrimination and global semantic cognition are deeply integrated at the loss calculation level. This allows the prediction of local micro-pixels to be guided and constrained by macro-global prior semantics, thereby effectively reducing the problem of misjudgment of anatomical regions caused by similar local features. The network parameters of the multi-anatomical region segmentation model are updated based on the total loss function, guiding the network model to simultaneously optimize its local pixel segmentation ability and global overall discrimination ability during backpropagation. Finally, a deep learning model is trained that maintains segmentation accuracy while possessing extremely high structural consistency and medical anatomical rationality, significantly improving the accuracy and stability of digestive tract anatomical segmentation regions.

[0139] Figure 5 This is a schematic diagram of the structure of the deep learning-based multi-anatomical region segmentation device provided by the present invention, as shown below. Figure 5 As shown, the device includes a second acquisition module 510, an input module 520, a model output module 530, a denoising module 540, a correction module 550, and an output module 560 connected in sequence.

[0140] The second acquisition module 510 is used to acquire the digestive tract image to be segmented; The input module 520 is used to input the digestive tract image to be segmented into a pre-trained multi-anatomical region segmentation model; the multi-anatomical region segmentation model is trained using the training method for a deep learning-based multi-anatomical region segmentation model as described in any one of claims 1 to 6. The model output module 530 is used to output global category information of the target through the classification branch in the multi-anatomical region segmentation model, and to output an initial pixel-level segmentation probability map through the segmentation branch in the multi-anatomical region segmentation model. The denoising module 540 is used to perform threshold filtering and connected component analysis on the initial pixel-level segmentation probability map to obtain a denoised pixel-level segmentation probability map. The correction module 550 is used to perform semantic consistency correction on the denoised pixel-level segmentation probability map based on the target global category information to obtain a corrected segmentation probability map. The output module 560 is used to perform anatomical structure prior correction on the corrected segmentation probability map based on a preset anatomical adjacency matrix, and finally output the segmentation result of the target anatomical region.

[0141] The deep learning-based multi-anatomical region segmentation device provided in this invention inputs the digestive tract image to be segmented into a multi-anatomical region segmentation model pre-trained based on anatomical topological constraints. It simultaneously acquires global target category information and an initial pixel-level segmentation probability map through classification and segmentation branches, achieving dual information extraction from macroscopic image-level semantics to microscopic pixel-level features, providing a comprehensive data foundation for subsequent multi-dimensional logical correction. By performing threshold filtering and connected component analysis on the initial pixel-level segmentation probability map, it effectively removes small-area isolated noise and artifact patches caused by the complex environment within the digestive tract (such as floating objects, reflections, and bubbles), initially purifying the segmentation mask and reducing subsequent computational interference. Furthermore, by using a target global category... Information is used to perform semantic consistency correction on the denoised pixel-level segmentation probability map, constructing a top-down constraint mechanism of "macro-qualitative guidance for micro-quantitative analysis". This effectively suppresses absurd region predictions that do not match the main organs in the whole image, and significantly improves the overall semantic logic consistency of the image. By performing anatomical structure prior correction on the corrected segmentation probability map based on a preset anatomical adjacency matrix, objective clinical medical anatomical rules are forcibly embedded into the post-processing closed loop of deep learning. This forcibly detects and corrects abnormal contact phenomena at local junction edges that violate the connection rules of real digestive tract organs, ultimately ensuring the absolute rationality and interpretability of the output target anatomical region segmentation results in the medical spatial topology, and improving the accuracy and stability of digestive tract anatomical segmentation regions.

[0142] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communications bus 640. The processor 610 can call logical commands stored in the memory 630 to execute the methods described in the above embodiments, for example: Acquire medical image samples; input the medical image samples into the feature extraction backbone network of a multi-anatomical region segmentation model to extract multi-level local feature representations and global classification label features of the medical image samples; perform feature decoding on the multi-level local feature representations through the segmentation branch of the multi-anatomical region segmentation model to output pixel-level segmentation prediction results; perform category discrimination based on the global classification label features through the classification branch of the multi-anatomical region segmentation model to output global category prediction results; calculate the total loss function based on the pixel-level segmentation prediction results and the global category prediction results; update the network parameters of the multi-anatomical region segmentation model based on the total loss function.

[0143] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] The processor in the electronic device provided in this embodiment of the invention can call logical instructions in the memory to implement the above method. Its specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, which will not be repeated here.

[0145] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0146] The specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, so it will not be repeated here.

[0147] This invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0148] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0149] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a multi-anatomical region segmentation model based on deep learning, characterized in that, include: Acquire medical image samples; The medical image samples are input into the feature extraction backbone network of the multi-anatomical region segmentation model to extract multi-level local feature representations and global classification label features of the medical image samples; The multi-anatomical region segmentation model performs feature decoding on the multi-level local feature representation through its segmentation branches, and outputs pixel-level segmentation prediction results. The classification branch of the multi-anatomical region segmentation model performs category discrimination based on the global classification label features and outputs the global category prediction result. Based on the pixel-level segmentation prediction results and the global category prediction results, the total loss function is calculated; The network parameters of the multi-anatomical region segmentation model are updated based on the total loss function.

2. The training method for a deep learning-based multi-anatomical region segmentation model according to claim 1, characterized in that, The calculation of the total loss function based on the pixel-level segmentation prediction result and the global category prediction result includes: Pixel-level segmentation loss, region consistency loss, dissecting adjacency loss, and boundary constraint loss are calculated based on the pixel-level segmentation prediction results. Calculate the classification head loss based on the global category prediction results; The total loss function is obtained by weighted summation of the pixel-level segmentation loss, the region consistency loss, the anatomical adjacency loss, the boundary constraint loss, and the classification head loss.

3. The training method for the deep learning-based multi-anatomical region segmentation model according to claim 2, characterized in that, The process of updating the network parameters of the multi-anatomical region segmentation model based on the total loss function includes: In the first training phase, the weight coefficients of the anatomical adjacency loss, the boundary constraint loss, and the classification head loss are set to zero, and the network parameters are updated based on the total loss function after weighted summation. In the second training phase, the weight coefficients of the dissection adjacency loss, the boundary constraint loss, and the classification head loss are adjusted to non-zero preset values, and the network parameters are updated based on the total loss function after weighted summation.

4. The training method for the deep learning-based multi-anatomical region segmentation model according to claim 2, characterized in that, The dissection adjacency loss is calculated based on the following steps: Obtain the adjacency matrix constructed based on the actual spatial topological relationships between various anatomical regions of the digestive tract; Based on the pixel-level segmentation prediction results, the predicted probabilities of adjacent pixel pairs belonging to different categories are obtained. Based on the predicted probabilities, a penalty term for class co-occurrence relationships is constructed; The penalty terms for all the adjacent pixel pairs are aggregated and calculated to obtain the dissected adjacency loss.

5. The training method for a deep learning-based multi-anatomical region segmentation model according to claim 1, characterized in that, The step of inputting the medical image sample into the feature extraction backbone network of the multi-anatomical region segmentation model to extract multi-level local feature representations and global classification label features of the medical image sample includes: Extract the classification labels output by the feature extraction backbone network at different coding levels; The classification labels output from different coding levels are fused to generate fused features; Based on the fusion features, the global classification label features are generated; The features are extracted from the local structured features output by the backbone network at different coding levels to generate the multi-level local feature representation.

6. The training method for a deep learning-based multi-anatomical region segmentation model according to claim 1, characterized in that, Before inputting the medical image sample into the feature extraction backbone network of the multi-anatomical region segmentation model to extract the multi-level local feature representation and global classification label feature of the medical image sample, the method further includes: Obtain an unlabeled set of images of the digestive tract; The initial feature extraction network is trained under self-supervised conditions based on the digestive tract image set to obtain the pre-trained feature extraction backbone network.

7. A multi-anatomical region segmentation method based on deep learning, characterized in that, include: Obtain the image of the digestive tract to be segmented; The digestive tract image to be segmented is input into a pre-trained multi-anatomical region segmentation model; the multi-anatomical region segmentation model is trained using the training method for a deep learning-based multi-anatomical region segmentation model as described in any one of claims 1 to 6. The target global category information is output through the classification branch of the multi-anatomical region segmentation model, and the initial pixel-level segmentation probability map is output through the segmentation branch of the multi-anatomical region segmentation model. Threshold filtering and connected component analysis are performed on the initial pixel-level segmentation probability map to obtain a denoised pixel-level segmentation probability map; Based on the target global category information, semantic consistency correction is performed on the denoised pixel-level segmentation probability map to obtain a corrected segmentation probability map; Based on a preset anatomical adjacency matrix, the corrected segmentation probability map is modified by prior anatomical structure, and the segmentation result of the target anatomical region is finally output.

8. A training device for a deep learning-based multi-anatomical region segmentation model, characterized in that, include: The first acquisition module is used to acquire medical image samples; The feature extraction module is used to input the medical image sample into the feature extraction backbone network of the multi-anatomical region segmentation model, and extract the multi-level local feature representation and global classification label feature of the medical image sample; The segmentation module is used to perform feature decoding on the multi-level local feature representation through the segmentation branch of the multi-anatomical region segmentation model and output pixel-level segmentation prediction results. The classification module is used to perform category discrimination based on the global classification label features through the classification branch of the multi-anatomical region segmentation model and output the global category prediction result; The loss calculation module is used to calculate the total loss function based on the pixel-level segmentation prediction result and the global category prediction result; The backpropagation module is used to update the network parameters of the multi-anatomical region segmentation model based on the total loss function.

9. A multi-anatomical region segmentation device based on deep learning, characterized in that, include: The second acquisition module is used to acquire images of the digestive tract to be segmented; An input module is used to input the digestive tract image to be segmented into a pre-trained multi-anatomical region segmentation model; the multi-anatomical region segmentation model is trained using the training method for a deep learning-based multi-anatomical region segmentation model as described in any one of claims 1 to 6. The model output module is used to output global category information of the target through the classification branch in the multi-anatomical region segmentation model, and to output an initial pixel-level segmentation probability map through the segmentation branch in the multi-anatomical region segmentation model. The denoising module is used to perform threshold filtering and connected component analysis on the initial pixel-level segmentation probability map to obtain a denoised pixel-level segmentation probability map. The correction module is used to perform semantic consistency correction on the denoised pixel-level segmentation probability map based on the target global category information to obtain a corrected segmentation probability map; The output module is used to perform anatomical structure prior correction on the corrected segmentation probability map based on a preset anatomical adjacency matrix, and finally output the segmentation result of the target anatomical region.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the training method of the deep learning-based multi-anatomical region segmentation model as described in any one of claims 1 to 6, or the deep learning-based multi-anatomical region segmentation method as described in claim 7.

Images