Lung anatomical structure identification method, device, equipment and medium

Abstract

This application discloses a method, apparatus, device, and medium for identifying lung anatomical structures, relating to the fields of image processing and artificial intelligence. The method includes: acquiring a lung image to be identified and extracting multi-scale features from the lung image to obtain multi-scale features; determining the lung segment resection status category in the lung image to be identified; determining sampling point features of spatial sampling points in the lung based on the lung segment resection status category and the multi-scale features; determining a target lung segment distribution prior template corresponding to the lung segment resection status category from multiple candidate lung segment distribution prior templates based on the lung segment resection status category; identifying lung segment change information in the lung image to be identified based on the sampling point features, and performing feature fusion based on the lung segment change information and the target lung segment distribution prior template to identify and segment the distribution area of ​​lung segments in the lung image to obtain lung segmentation results.

Description

Technical Field

[0001] This application relates to the fields of image processing, artificial intelligence and other technical fields, and in particular to a method, device, equipment and medium for recognizing lung anatomical structures. Background Technology

[0002] Medical image segmentation is an image processing technique that separates regions of interest in medical images from the background or other regions. Medical images include computed tomography (CT) images, magnetic resonance imaging (MR) images, etc. This technique plays an important role in the application environment of medical imaging, providing doctors with three-dimensional information about anatomical locations or lesion voxels.

[0003] The intersegmental planes of the lung do not have clear boundaries on CT scans and must be determined based on the distribution and spatial location of the bronchi and blood vessels. Segmentectomy is an anatomical lung resection procedure. Compared to lobectomy, its core lies in the precise removal of bronchial segments supplied by the same bronchus and blood vessels, preserving healthy lung tissue to the maximum extent possible. Clinical practice has demonstrated that segmentectomy is not inferior to, and may even be superior to, lobectomy in tumor control, while providing better postoperative quality of life and preservation of lung function. Preoperative CT assessment is a tool for identifying lung segmental structures and planning surgery. Postoperative CT is an important tool for evaluating surgical outcomes, detecting complications, and follow-up. However, after segmentectomy, CT images become very complex, and identifying tissue structures presents many challenges. After the removal of a portion of the lung tissue, the remaining lung tissue expands and shifts to fill the cavity, causing changes in the position and abnormal course of normal anatomical structures (such as interlobar fissures and blood vessels), making it difficult to identify specific structures.

[0004] In summary, accurately and efficiently identifying lung segment regions after pulmonary segmentectomy, when anatomical structures have changed, is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] The embodiments of this application aim to at least partially solve one of the technical problems in the related art. Therefore, the first objective of the embodiments of this application is to provide a method, apparatus, device, and medium for identifying lung anatomical structures.

[0006] This application provides a method for identifying lung anatomical structures. The method includes: acquiring a lung image to be identified and extracting multi-scale features from the lung image to obtain multi-scale features; determining the lung segment resection status category in the lung image to be identified; determining sampling point features of lung spatial sampling points based on the lung segment resection status category and the multi-scale features; determining a target lung segment distribution prior template corresponding to the lung segment resection status category from multiple candidate lung segment distribution prior templates based on the lung segment resection status category; identifying lung segment change information in the lung image to be identified based on the sampling point features, and performing feature fusion based on the lung segment change information and the target lung segment distribution prior template to identify and segment the distribution area of ​​lung segments in the lung image to be identified to obtain lung segmentation results.

[0007] In other embodiments, the lung segment change information includes a lung segment deformation field and a local tissue change correction map; the lung segment change information in the lung image to be identified is identified based on sampling point features, and feature fusion is performed based on the lung segment change information and the target lung segment distribution prior template to identify and segment the distribution area of ​​the lung segment in the lung image to be identified, thereby obtaining the lung segment segmentation result. This includes: based on sampling point features, performing deformation identification on the lung segment in the lung image to be identified to obtain a lung segment deformation field; and based on sampling point features, performing local tissue change identification on the lung segment in the lung image to be identified to obtain a local tissue change correction map; performing deformation processing on the target lung segment distribution prior template based on the lung segment deformation field to obtain a deformed lung segment distribution prior template; fusing the local tissue change correction map and the deformed lung segment distribution prior template to obtain fine distribution features of the lung segment; and obtaining the lung segment segmentation result based on the fine distribution features.

[0008] In other implementations, the multiple candidate lung segment distribution prior templates include Cls lung segment resection prior templates and one lung segment unresection prior template, where Cls is an integer greater than or equal to 1. The multiple candidate lung segment distribution prior templates are obtained as follows: Cls+1 training datasets are acquired, where each Cls-group lung segment resection dataset includes a Cls-group lung segment resection dataset and one lung segment unresection dataset, with each lung segment resection dataset in the Cls-group lung segment resection dataset corresponding to a lung segment resection category; multiple batches of data are determined from the Cls+1-group training dataset and sequentially input into the template generation model for processing. Multiple rounds of training are conducted to identify the resected lung segments, yielding a loss function value for each round. This loss function value includes a similarity loss value and a cross-entropy loss value. The similarity loss value evaluates the similarity between the trained identification results and the true labels, while the cross-entropy loss value evaluates the difference between the distribution of the trained identification results and the true distribution. After multiple rounds of training until the loss function value meets the convergence condition, the model parameters of the template generation model are determined based on the evaluation metrics. Based on the model parameters of the template generation model, Cls+1 training identification results corresponding to the Cls+1 training datasets are determined as prior templates for the Cls+1 candidate lung segment distributions.

[0009] In other implementations, based on the lung segment resection status category and multi-scale features, the sampling point features of the lung spatial sampling points are determined, including: interpolating the multi-scale features to obtain high-level features of the lung spatial sampling points; determining the spatial location of the lung spatial sampling points; and concatenating the high-level features, spatial location, and lung segment resection status category to obtain the sampling point features.

[0010] In other embodiments, interpolation processing is performed on multi-scale features to obtain high-level features of lung spatial sampling points, including: receiving a specified instruction for a local lung region; determining lung spatial sampling points from a voxel grid constituting the lung space based on the specified instruction, wherein the lung spatial sampling points include discrete voxel points of the voxel grid and / or non-voxel points in continuous space; and interpolating the multi-scale features based on the lung spatial sampling points to obtain high-level features corresponding to the lung spatial sampling points, wherein the high-level features corresponding to the non-voxel points are obtained through interpolation.

[0011] In other embodiments, determining the segmentectomy status category in the lung image to be identified includes: inputting multi-scale features into a classification model for identification and outputting the segmentectomy status category; or determining the segmentectomy status category in the lung image to be identified based on medical record data.

[0012] In other embodiments, lung segment change information in the lung image to be identified is identified based on sampling point features, and feature fusion is performed based on the lung segment change information and the target lung segment distribution prior template to identify and segment the distribution area of ​​the lung segments in the lung image to be identified, thereby obtaining lung segment segmentation results. This includes: inputting the sampling point features and the target lung segment distribution prior template into the segmentation model, identifying lung segment change information in the lung image to be identified based on the sampling point features through the segmentation model, and performing feature fusion based on the lung segment change information and the target lung segment distribution prior template to identify and segment the distribution area of ​​the lung segments in the lung image to be identified, thereby obtaining lung segment segmentation results. In this embodiment, the segmentation model and the classification model are jointly trained and optimized.

[0013] In other embodiments, the lung image samples to be identified for jointly training the segmentation model and the classification model are obtained by: acquiring real postoperative image samples and / or real preoperative image samples, wherein a first lung segment in the real postoperative image sample is removed and a second lung segment in the real preoperative image sample is to be removed; performing random offset simulation processing on the real postoperative image samples and / or real preoperative image samples to generate simulated postoperative image samples and / or simulated preoperative image samples, wherein a third lung segment in the simulated postoperative image samples is simulated to be removed and a fourth lung segment in the simulated preoperative image samples is to be removed; and obtaining the lung image samples to be identified based on the real postoperative image samples and simulated postoperative image samples, and / or based on the real preoperative image samples and simulated preoperative image samples.

[0014] This application provides a lung anatomy structure recognition device, comprising: an acquisition module for acquiring a lung image to be recognized and extracting multi-scale features from the lung image to obtain multi-scale features; a first determination module for determining the lung segment resection status category in the lung image to be recognized; a second determination module for determining the sampling point features of lung spatial sampling points based on the lung segment resection status category and the multi-scale features; a third determination module for determining a target lung segment distribution prior template corresponding to the lung segment resection status category from multiple candidate lung segment distribution prior templates based on the lung segment resection status category; and an identification module for identifying lung segment change information in the lung image to be recognized based on the sampling point features, and performing feature fusion based on the lung segment change information and the target lung segment distribution prior template to identify and segment the distribution area of ​​lung segments in the lung image to be recognized to obtain lung segmentation results.

[0015] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the above embodiments.

[0016] This application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0017] This application provides a computer program product that includes instructions that, when executed by a processor of a computer device, enable the computer device to perform the steps of the method described in any of the above embodiments. Attached Figure Description

[0018] Figure 1 A flowchart of a lung anatomy structure recognition method provided for an embodiment of this application.

[0019] Figure 2 and Figure 3 This is a schematic diagram illustrating lung segment identification based on a trained model, provided as an embodiment of this application.

[0020] Figure 4 A schematic diagram of a lung image to be identified provided for an embodiment of this application.

[0021] Figure 5 A schematic diagram of a priori template for lung segment distribution provided for another embodiment of this application.

[0022] Figure 6 A schematic diagram of lung segmentation provided for an embodiment of this application.

[0023] Figure 7 and Figure 8 A schematic diagram illustrating model training provided for an embodiment of this application.

[0024] Figure 9 This is a schematic diagram of the lung segmentation results provided in the embodiments of this application.

[0025] Figure 10 A schematic diagram of a lung anatomy recognition device provided in an embodiment of this application.

[0026] Figure 11 A block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0028] When identifying lung segments, doctors can determine the voxel range of the target lesion and its anatomical location by reviewing the images. This process can also be assisted by artificial intelligence. In AI-powered medical image segmentation tasks, convolutional neural networks (CNNs) downsample the 3D image through an encoder and upsample it through a decoder to obtain the target region, outputting a target region with the same or smaller size as the input image. Improving output resolution depends on either increasing the input size or upsampling the output. The former significantly increases the computational load and resource consumption of the CNN, while the latter results in a loss of detail, especially since medical images often contain small anatomical targets such as trachea and blood vessels.

[0029] Three-dimensional representations involve voxels arranged sequentially in three-dimensional space, with each voxel's value representing a specific attribute. In radiology, several consecutive image slices in the same sequence can represent a three-dimensional anatomical area of ​​the human body. In three-dimensional medical images, the value at each voxel position represents the density of human tissues or organs within a corresponding spatial range, measured in Henle units (HU). However, in the real physical world, objects are continuous, not discrete. This means that the spatial range represented by each pixel can be infinitely subdivided, but due to limitations in expressive power, continuous three-dimensional objects are usually represented as discrete three-dimensional matrices, sacrificing expressive precision. Deep Implicit Fields (DIF or IF) originate from the concept of implicit fields. It can be used to handle modeling tasks of three-dimensional shapes with varying precision. Leveraging the rich expressive power of deep neural networks, it approximates implicit functions of arbitrary shapes and precision, thereby enabling the modeling of complex three-dimensional models. DIF can learn the mapping relationship from input data to implicit field representations, mapping input data to a continuous implicit field representation. Compared to traditional image processing methods, deep implicit fields do not perform manual feature extraction or shape modeling of objects. Instead, they learn high-dimensional feature representations of objects through deep neural networks. Compared to traditional convolutional neural networks, deep implicit fields are continuous functions that can achieve arbitrarily precise three-dimensional representations.

[0030] In medical image segmentation using deep learning technology, the anatomical structure of lung segments is related to their spatial location. Obtaining postoperative anatomical structures is often beyond the processing capabilities of common image post-processing software. This lack of postoperative structure can cause image segmentation models to produce a "illusion," segmenting all lung segments based on spatial location. Deep learning-based medical image segmentation techniques, such as lung segmentation, require training on a large number of labeled lung CT images to accurately divide the lung into five lobes and approximately 18-20 bronchial segments.

[0031] Some models can learn an inherent "anatomical atlas"—the relative positions, morphological features, and interconnections of anatomical structures in three-dimensional space. However, segmentectomy fundamentally disrupts the "rules" upon which this model relies. The model's path dependence on the "complete anatomical atlas" subconsciously searches for every structure it has learned in the image: it expects to see the apical, anterior, and posterior bronchioles and vessels at specific locations in the upper right lobe; it expects to see clear tissue boundaries at the interlobar fissure. This strong spatial prior knowledge normally guarantees the model's high-precision performance. But postoperatively, the target lung segment is removed, and its corresponding bronchi, arteries, and veins are severed. This means that many key "landmarks" the model tries to find no longer exist, and the expansion and displacement of the remaining lung tissue causes blood vessels to be pulled to an atypical position, completely disrupting the original spatial coordinate system. Faced with this chaotic scenario where rules are broken, the model's embedded "anatomical atlas" is severely inaccurate. However, as a tool designed to accomplish the task of "segmenting all lung segments," it must still provide an output. Thus, an "illusion" arises: the model tends to ignore actual, anomalous image evidence, instead relying excessively on its strong spatial priors acquired during training. Where it "believes" a lung segment should exist—for example, the model might incorrectly identify postoperative fibrous strands or fluid effusion planes as normal intersegmental planes—it "segments" a lung segment that no longer exists, ignoring missing structures and inferring and outlining a complete, but actually resected, lung segment morphology. The final segmentation result may still appear as a structurally complete, clearly defined five-lobed, eighteen-segment lung, but this is far removed from the patient's actual postoperative condition and completely loses its clinical reference value. Furthermore, operations are performed on all voxels in the image, making operations on background voxels redundant, which limits the output resolution to the input resolution.

[0032] In some medical scenarios and clinical applications, subtask 1 involves identifying lesion regions on anatomical structures (such as pneumonia or tumors), while subtask 2 involves recognizing the location of target anatomical structures (such as the lungs or brain) in medical images. The expected foreground portion of both outputs is inclusive; that is, the result of subtask 1 is a part of subtask 2. The outputs of subtasks 1 and 2 are obtained separately and then superimposed at the same spatial location to determine if the lesion in subtask 1 is located within a specific anatomical region of subtask 2. However, this method performs calculations for both tasks on all pixels, which is clearly redundant when processing background voxels.

[0033] In view of this, the anatomical structure recognition of CT images after lung segmentectomy in this application uses simulated data and finely annotated CT images after lung segmentectomy for training, allowing the model to learn various postoperative change patterns. A deformable knowledge graph is embedded in the model as prior knowledge to guide the segmentation of postoperative anatomical structures. The surgical procedure type (i.e., the name or category of the resected lung segment) is treated as a parallel task for multi-task prediction, guiding targeted recognition and improving the accuracy of lung segment recognition.

[0034] Figure 1 A flowchart of a lung anatomy structure recognition method provided for an embodiment of this application.

[0035] like Figure 1 As shown, the lung anatomical structure identification method provided in this application includes steps S110-S150.

[0036] Step S110: Obtain the lung image to be identified, and perform multi-scale feature extraction on the lung image to be identified to obtain multi-scale features.

[0037] Step S120: Determine the lung segment resection status category in the lung image to be identified.

[0038] Step S130: Based on the lung segment resection status category and multi-scale features, determine the sampling point features of the lung spatial sampling points.

[0039] Step S140: Based on the segmentectomy status category, determine the target segment distribution prior template corresponding to the segmentectomy status category from multiple candidate segment distribution prior templates.

[0040] Step S150: Based on the features of the sampling points, identify the lung segment change information in the lung image to be identified, and perform feature fusion based on the lung segment change information and the target lung segment distribution prior template, so as to identify and segment the distribution area of ​​the lung segment in the lung image to be identified and obtain the lung segment segmentation result.

[0041] For example, a patient's lung CT image can be obtained as the lung image to be identified. A feature extraction module (such as a 3D convolutional neural network like ResNet or UNet) is used to extract multi-scale features from the lung image to be identified, resulting in multi-scale features (such as multi-scale feature maps) containing different semantic levels (such as edges, textures, organ morphology, etc.).

[0042] The segmentectomy status category can be identified by classifying multi-scale features through a classification module (classification model) (e.g., right upper lobe apical segmentectomy), or the segmentectomy status category of the lung image to be identified can be determined directly from medical record data.

[0043] Based on the lung segment resection status category and multi-scale features, the sampling point features of the lung spatial sampling points are determined. Then, according to the predicted lung segment resection status category, the target lung segment distribution prior template corresponding to the lung segment resection status category is selected from multiple pre-constructed candidate lung segment distribution prior templates (such as preoperative complete template, apical resection template, etc.). For example, if the lung is divided into Cls lung segments, each lung segment resection corresponds to one candidate lung segment distribution prior template, and all lung segments are not resected, there is one candidate lung segment distribution prior template (this candidate lung segment distribution prior template indicates that all lung segments are not resected). Therefore, there are Cls+1 candidate lung segment distribution prior templates. If the lung segment resection status category indicates that a certain lung segment has been resected, then the candidate lung segment distribution prior template indicating that the lung segment has been resected is selected from the Cls+1 candidate lung segment distribution prior templates as the target lung segment distribution prior template. Finally, based on the sampling point features and the prior template of the target lung segment distribution, the range or region of the lung segment in the lung image to be identified is determined through a non-rigid deformation and correction mechanism, thus obtaining the lung segment segmentation result. For example, based on the sampling point features, lung segment change information in the lung image to be identified is identified. This lung segment change information characterizes lung segment deformation and local tissue changes. The lung segment segmentation result is obtained by feature fusion based on the lung segment change information and the prior template of the target lung segment distribution. The lung segment segmentation result includes the segmentation results of all lung segments, i.e., the segmentation results of all lung segments after lung segmentectomy. After removing a portion of the lung segment, the remaining lung tissue expands and shifts to fill the cavity, causing changes in the position of normal anatomical structures and abnormal deformation, making it difficult to determine specific structures. The method of this application can accurately identify abnormal anatomical structures after lung segmentectomy, thereby accurately identifying the distribution of lung segments after lung segmentectomy.

[0044] The embodiments of this application introduce a priori templates of lung segment distribution as prior knowledge, and select corresponding target lung segment distribution prior templates as references for lung segment recognition and segmentation based on this. This effectively solves the "illusion" problem caused by the over-reliance on "complete anatomical atlas" in traditional segmentation models under postoperative anatomical structure variation scenarios, and improves the lung segment recognition effect.

[0045] Figure 2 and Figure 3 This is a schematic diagram illustrating lung segment identification based on a trained model, provided as an embodiment of this application.

[0046] like Figure 2 or Figure 3 As shown, a trained model may include, for example, a feature extraction module, a classification module (classification model), and a segmentation module (segmentation model).

[0047] The lung image to be identified can be a voxel image, specifically including the original lung CT scan image, or segmentation results of multiple important anatomical structures obtained after preprocessing the original lung CT scan image, such as artery segmentation results, vein segmentation results, and bronchial segmentation results. Any one or more of these images can be selected as combined input, providing a rich anatomical information basis for subsequent deformation registration and surgical procedure type (i.e., lung segmentation status category) identification. For example... Figure 4 As shown, the lung image to be identified may include a segmented image of the pulmonary artery (left), a segmented image of the pulmonary veins (middle), and a segmented image of the trachea (right).

[0048] All input images maintain consistency in spatial location and size, specifically in a three-dimensional space of D×H×W. This spatial alignment ensures that different modal features can be fused within a unified coordinate system. For example, the locations of blood vessels marked in artery segmentation and the airway structures in bronchus segmentation have an accurate spatial anatomical correspondence, providing a necessary condition for multi-feature collaborative analysis.

[0049] This application uses the lung image to be identified as a combined input of artery segmentation and bronchial segmentation results for detailed explanation. The artery segmentation results clearly delineate the distribution pattern of the pulmonary vascular tree, reflecting the division of blood supply areas; while the bronchial segmentation results accurately label the topological structure of the airway system, representing the ventilation functional units of lung segments. The combination of these two provides complementary anatomical evidence; when a lung segment is removed, its corresponding segmental artery and segmental bronchus will exhibit characteristic transection. By simultaneously analyzing the segmentation results of these two structures, a more reliable basis for surgical procedure judgment can be obtained than with a single modality, thereby accurately identifying the specific lung segment that was removed, laying a solid foundation for subsequent template registration and 3D reconstruction.

[0050] In addition to inputting lung images to be identified into the model, spatial sampling points of the lungs can also be input. Within the spatial domain, sampling points can be discrete points in a regular voxel grid or arbitrary non-voxel locations in continuous space. The ability to sample in the continuous domain breaks through the discretization limitations of traditional voxel grids, demonstrating the flexibility and practicality of this application in spatial modeling. During the model training phase, diverse sampling strategies are employed: random selection of spatial points can enhance the model's generalization ability and robustness, while proportional sampling can be performed according to specific rules, such as increasing the sampling density in anatomical boundary regions and reducing the number of sampling points in homogeneous tissue regions, thereby achieving optimal allocation of computational resources. During the model inference application phase, the sampling point strategy fully considers actual clinical needs. Users can perform targeted sampling for specific regions of interest to obtain sampling points for those regions, a capability that has significant value in clinical practice. For example, when a doctor finds a suspicious pulmonary nodule in a CT scan, traditional methods require full calculation of the entire lung. This application, however, allows doctors to precisely locate the spatial coordinates of the nodule and sample only that point or a small area around it for analysis, quickly determining which specific pulmonary segment the nodule belongs to. This targeted sampling mechanism significantly improves computational efficiency, enabling real-time interactive analysis. More importantly, it aligns with the actual workflow of clinicians, eliminating the need to wait for batch processing of the entire image. Instead, it allows for real-time analysis of specific anatomical locations, much like using a navigation tool, thus improving efficiency.

[0051] Continue to refer to Figure 2 or Figure 3 The lung image to be identified is input into the feature extraction module, where multi-scale feature extraction is performed to obtain multi-scale features. The feature extraction module includes a convolutional neural network (CNN). In the deep learning feature extraction stage, the CNN performs multi-level feature learning on one or more input medical image data. For example, several feature extraction sub-modules can be organized in a cascaded or parallel manner. Each sub-module focuses on capturing feature information at a specific granular level. For instance, shallow modules mainly extract low-level visual features such as local texture and edge contours, while deep modules gradually abstract high-level semantic features such as organ morphology and anatomical structural relationships. ResNet, UNet, or other backbone networks can be selected as feature extractors. Through these network architectures, L feature maps at different scales are extracted, which can be denoted as... , ..., This forms a feature pyramid, which can have different sizes, denoted as . ,……, . ,……, This refers to the feature channel dimension, representing the number of dimensions of the extracted high-dimensional features. This multi-scale feature representation provides comprehensive feature support for subsequent deformation field prediction and correction map generation, ensuring that the system can grasp both the overall anatomical structure and pay attention to local detail changes.

[0052] Continue to refer to Figure 2 After obtaining the multi-scale features, these features are input into a classification model for identification, outputting the lung segment resection status category. Or, as... Figure 3 It can determine the segmentectomy status category in lung images to be identified based on medical record data.

[0053] For example, there are at least two ways to determine the segmentectomy status category. Method 1 (Data-driven): The extracted multi-scale features are input into a classification network (classification module) consisting of fully connected layers. This network outputs a Cls-dimensional vector, where each element represents the probability that the corresponding lung segment was resected. Thresholding (e.g., a threshold of 0.5, where values ​​greater than 0.5 are 1, and values ​​less than or equal to 0.5 are 0) yields the final 0 / 1 category label, where 1 indicates the corresponding lung segment was not resected, and 0 indicates the corresponding lung segment was resected. Method 2 (Knowledge-driven): Known surgical procedure information (e.g., "thoracoscopic right upper lobe apical segmentectomy") is directly read from the patient's electronic medical record or surgical record and parsed into a standardized segmentectomy status code, which serves as the segmentectomy status category (indicating the resected lung segment is the right upper lobe apical segment).

[0054] For the classification module, based on the diagnostic logic of doctors using postoperative CT images of patients in clinical practice, an intelligent surgical procedure classification module was designed to simulate and extend the expert's diagnostic reasoning process. Specifically, experts judge the surgical situation by observing the transection of the patient's bronchi, arteries, veins, and surrounding tissues in the CT image. When faced with CT images of patients with unknown surgical conditions, this application automatically extracts deep features of the image through a deep learning network, including key anatomical information such as lung lobe morphology, bronchial tree distribution patterns, and vascular course characteristics. It analyzes the transection of bronchi, arteries, and veins, the morphology of residual ends, and changes in vascular perfusion patterns. These features are important bases for determining the extent of lung segment resection. The purpose of the classification module is to identify the lung segment resection status corresponding to the input image and output a classification vector with dimension Cls. Each element in the classification vector corresponds to the predicted resection status of a specific lung segment. In this coding system, a value of 1 represents that the lung segment remains intact and has not been resected, while a value of 0 indicates that the lung segment has been surgically removed. This design has significant clinical implications: on the one hand, it inherits the diagnostic thinking of radiologists who infer surgical history based on changes in anatomical structures, making the algorithm's decision-making process more in line with medical logic; on the other hand, by quantifying the resection probability of each lung segment, it provides accurate prior information for subsequent template selection and deformation registration, ensuring that the registration process can adapt to significant changes in postoperative anatomical structures, thereby achieving accurate image analysis and visualization reconstruction in the context of incomplete anatomy.

[0055] Through the embodiments of this application, when reliable medical record data is available, it can be directly used to determine the segmentectomy status category, simplifying the model task and improving segmentation reliability; when medical records are missing or questionable, the model can autonomously infer the segmentectomy procedure from images. This dual-mode design enhances the clinical adaptability and robustness of the system, and is particularly suitable for patients with unclear medical history or transferred to other hospitals during postoperative follow-up.

[0056] Continue to refer to Figure 2 or Figure 3 Based on the lung segment resection status category and multi-scale features, the sampling point features of the lung spatial sampling points are determined, including: interpolating the multi-scale features to obtain the high-level features of the lung spatial sampling points; determining the spatial location of the lung spatial sampling points; and concatenating the high-level features, spatial location, and lung segment resection status category to obtain the sampling point features.

[0057] (1) This example can be implemented by a sampling point feature stitching module. The sampling points include voxel points in discrete coordinates and / or non-voxel points in continuous space (sub-voxel level). High-level features of the sampling points are obtained from multi-scale features through linear interpolation, denoted as... , ..., . , ..., Each of the dimensions is ,like The dimension is .

[0058] (2) Sampling point coordinate normalization: Remove the differences in spatial location and scale between different data, reflect the relative position of the sampling point in the original space, and obtain the coordinates (spatial location) of three orthogonal coordinate axes in the range of [0,1] or [-1,1], with a dimension of 1×3.

[0059] (3) Classification result serialization: A set of binary numbers (0 and 1) is used to represent the two states of the same lung segment, corresponding to the two results output by the classification network. Based on the lung segment resection state category, the resection results of Cls lung segments are concatenated bit by bit to avoid the model mistakenly believing that there is a numerical relationship between the categories. The dimension is 1×Cls.

[0060] (4) The above three features (high-level features, spatial location and lung segment resection status category) are spliced ​​together to obtain the sampling point features. The dimension of the sampling point features is 1×(L+3+Cls).

[0061] This design offers two significant advantages: First, during model inference, the model does not need to perform intensive computation on all voxels of the entire 3D volume data. Instead, it can flexibly select regions of interest or key anatomical locations for targeted sampling computation based on actual application needs. This sampling-based computation method greatly reduces computational complexity and memory overhead, enabling the model to efficiently process high-resolution medical image data, making it particularly suitable for rapid analysis of large 3D medical images in clinical environments. Through a carefully designed sampling strategy, the system can significantly improve inference efficiency while maintaining accuracy, providing technical feasibility for interactive medical applications. Second, since sampling points can be distributed at any location in continuous space, it breaks the discretization limitations of traditional voxel grids, giving the model the theoretical potential for infinitely subdivided precision. This continuous representation characteristic allows the system to capture and reconstruct anatomical structures and morphological details far finer than the input resolution, achieving true super-resolution output. When focusing on specific anatomical regions, sub-voxel-level precision can be obtained by increasing the local sampling density, thus clearly presenting subtle local features. This continuous spatial modeling method not only improves the geometric accuracy of the results but, more importantly, opens up a transition from discrete quantization to continuous modeling for medical image analysis.

[0062] The embodiments of this application concatenate high-level features, spatial location, and lung segment resection status categories. The introduction of spatial location endows the model with spatial perception of anatomical structures, overcoming the problem of pure convolutional networks being insensitive to location. The introduction of lung segment resection status category features provides the model with explicit semantic priors, guiding it to segment within the corresponding anatomical space (the lung segment resection status category indicates which lung segment was resected, and the subsequent segmentation model segments the region where all lung segments are located based on this). This multi-source information fusion significantly improves the model's discriminative ability in complex postoperative scenarios.

[0063] In another example, interpolation processing is performed on multi-scale features to obtain high-level features of lung spatial sampling points, including: receiving a specified instruction for a local lung region; determining lung spatial sampling points from a voxel grid constituting the lung space based on the specified instruction, wherein the lung spatial sampling points include discrete voxel points of the voxel grid and / or non-voxel points in continuous space; and interpolating the multi-scale features based on the lung spatial sampling points to obtain high-level features corresponding to the lung spatial sampling points, wherein the high-level features corresponding to the non-voxel points are obtained through interpolation.

[0064] For example, during the model inference phase, the system receives a region of interest (e.g., a suspected nodule region) specified by the doctor on an image. Based on this specification, sampling points are determined within this region. These sampling points can be discrete voxel points located at the center of the original voxel grid, or non-voxel points (sub-voxel level) located in continuous space between grids, such as the physical center of a nodule. Then, only for these determined sampling points, their corresponding high-level features are calculated from the multi-scale feature map via interpolation; the high-level features corresponding to non-voxel points are obtained through interpolation. Background regions that do not require computation are not sampled, thus achieving efficient local fine-grained analysis.

[0065] The embodiments of this application implement an intelligent computing mode of "on-demand sampling." Clinicians do not need to wait for the complete segmentation results of the whole lung. Instead, they can click or select any suspicious location in real time, just like using a navigation tool. The system immediately returns the anatomical information of the lung segment to which that location belongs. This interactive workflow significantly reduces redundant calculations, lowers latency, and allows for super-resolution (subvoxel-level) sampling analysis in critical areas, providing strong support for intraoperative navigation and real-time decision-making.

[0066] In another example, the prior templates for multiple candidate lung segment distributions include Cls prior templates for lung segment resection and 1 prior template for lung segment non-resection, where Cls is an integer greater than or equal to 1; the multiple prior templates for candidate lung segment distributions are obtained in the following way: (1) Obtain the Cls+1 training dataset, which includes the Cls group lung segment resection dataset and 1 group lung segment unresected dataset. Each lung segment resection dataset in the Cls group lung segment resection dataset corresponds to a lung segment resection category.

[0067] (2) Multiple batches of data are selected from the Cls+1 training dataset and sequentially input into the template generation model for multiple rounds of training to identify the resected lung segments and obtain the loss function value for each round. The loss function value includes similarity loss value and cross-entropy loss value. The similarity loss value is used to evaluate the similarity between the training recognition result and the real label, and the cross-entropy loss value is used to evaluate the difference between the distribution of the training recognition result and the real distribution.

[0068] (3) After multiple rounds of training until the loss function value meets the convergence condition, the model parameters of the template generation model are determined based on the evaluation index.

[0069] (4) Based on the model parameters of the template generation model, determine the Cls+1 training recognition results corresponding to the Cls+1 training dataset as the Cls+1 candidate lung segment distribution prior templates.

[0070] Specifically, lung segment regions were pre-labeled manually before and after segmentectomy. For each case in the dataset, the data was divided into two groups based on the location of the resected lung segment: a postoperative dataset (Group C1) and a preoperative dataset (Group 1). Each group in the postoperative dataset represents data from lungs that underwent resection of a specific lung segment. The training dataset was randomly divided into a training set and a validation set. The training set was used to train the template generation model and update its parameters, while the validation set was used to select the parameters for the final template generation model.

[0071] Template generation models include networks for compressing features, aiming to summarize the prevalent spatial distribution patterns of anatomical structures to obtain prior templates, rather than predicting a single input. Template generation models consist of three parts: (1) Feature-compressing network: Perform feature compression on each data point in the dataset to obtain a network with dimension 1. Trainable vectors, This indicates compressed feature dimensions.

[0072] (2) Decoder: It consists of several up-decoding modules. Each decoding module consists of an upsampling layer and a convolutional layer, which can reduce the channel dimension and increase the resolution.

[0073] (3) Prior output layer: It consists of a convolutional layer that converts the channel number dimension into Cls+1 dimension, representing the background and Cls lung segments, while maintaining the size of the output of the last decoding module.

[0074] For example, the compression feature is 1×1024×1×1×1 ( =1024), after passing through 7 layers of decoders, a feature map of 1×32×128×128×128 is generated. After passing through the prior output layer, it is converted into an output of 1×(Cls+1)×128×128×128.

[0075] The prior template generation module (template generation model) can be trained separately, as follows.

[0076] Perform the following operations on each dataset. After training the postoperative dataset of the Cls group and the preoperative dataset of 1 group, you can obtain the Cls+1 candidate lung segment distribution prior templates.

[0077] (1) Randomly initialize the parameters of the neural network, including weights and biases.

[0078] (2) Calculate the loss function by manually annotating the dataset with the network output results. The loss function includes the cross-entropy loss function and the Dice loss function (the Dice loss function is a similarity loss function).

[0079] (3) Starting from the output layer, calculate the gradient of the loss function for each parameter in reverse and update the parameters according to the gradient direction.

[0080] (4) When the cycle described above has been completed for the entire training set, one round of training is completed.

[0081] (5) Repeat this process for multiple rounds until the loss function no longer changes significantly, and the loss function and Dice evaluation index are stable on the validation set. Then, select the network parameters corresponding to the round with the highest Dice evaluation index and save them as the final neural network model parameters for this set of data. At this point, the model's output is used as the prior template for this set (lung segment distribution prior template). The prior template is as follows: Figure 5 As shown (left: preoperative template; middle: preoperative template after right upper lobe apical segment resection; right: preoperative template after right upper lobe posterior segment resection).

[0082] After completion, obtain Cls+1 prior templates. These templates will not be used to generate model parameters that will not be involved in the training process of neural network parameters in other parts (such as the sampling point feature splicing module, classification module, and segmentation module).

[0083] This application employs a method that independently and specifically trains dedicated prior templates (lung segment distribution prior templates) for each surgical procedure. This method explicitly encodes clinically common lung segmentectomy patterns (such as the typical morphology of remaining lung tissue after the loss of a specific lung segment) into computable structured knowledge. These templates provide highly relevant initial anatomical structures for subsequent deformation registration, significantly reducing the deformation required to recover anatomical structures from severely mutated images, lowering the registration difficulty, and improving the stability and accuracy of segmentation.

[0084] Figure 6 A schematic diagram of lung segmentation provided for an embodiment of this application.

[0085] like Figure 6 As shown, the lung segment change information includes the lung segment deformation field and the local tissue change correction map; the lung segment change information in the lung image to be identified is identified based on the sampling point features, and feature fusion is performed based on the lung segment change information and the target lung segment distribution prior template to identify and segment the distribution area of ​​the lung segment in the lung image to be identified, and the lung segment segmentation result is obtained, including the following contents.

[0086] First, based on the features of the sampling points, deformation recognition is performed on the lung segments in the lung image to be identified to obtain the lung segment deformation field. Then, based on the features of the sampling points, local tissue changes are recognized in the lung segments in the lung image to be identified to obtain the local tissue change correction map.

[0087] Secondly, based on the lung segment deformation field, the target lung segment distribution prior template (prior information) is deformed (e.g., non-rigid deformation processing) to obtain the deformed lung segment distribution prior template.

[0088] Then, the local tissue change correction map and the prior template of deformed lung segment distribution are fused (fine correction) to obtain the fine distribution features of lung segments, and the lung segment segmentation results are obtained based on the fine distribution features.

[0089] Specifically, this can be implemented by a segmentation module (segmentation network), which includes a bi-branch prediction network comprising a deformation field branch network and a correction map branch network. Since the sampling point features already contain location information, a continuous mapping relationship (i.e., the segmentation network) is trained using the features of several sampling points corresponding to different locations each time. Based on the sampling point features, the bi-branch prediction network processes them separately. The deformation field branch network predicts a dense lung segment deformation field based on the sampling point features, used to characterize the geometric deformations such as expansion and displacement of the remaining lung tissue. The correction map branch network predicts a local tissue change correction map, used to compensate for changes in local intensity distribution caused by surgical trauma, tissue loss, etc. Then, according to the lung segment resection status category, a corresponding target lung segment distribution prior template is selected from multiple candidate lung segment distribution prior templates. The predicted deformation field is used to spatially transform the target lung segment distribution prior template to obtain a deformation template (deformed lung segment distribution prior template) that is initially aligned with the patient's anatomical structure. Next, this deformation template is fused with the predicted correction map using pixel-level addition to generate fine-grained distribution features. Finally, based on this fine distribution feature, the final lung segmentation result is output through the segmentation head of the segmentation module.

[0090] As can be seen, the parallel output of the deformation field branch network and the calibration map branch network breaks through the limitation of traditional registration models that rely solely on geometric deformation. The deformation field maintains the topological rationality and spatial continuity of the anatomical structure, ensuring the physical credibility of the deformation. The calibration map focuses on the intensity adaptation and detail restoration of local tissues, compensating for the shortcomings of simple geometric deformation in matching tissue characteristics. Both branches are trained and generated based on the same prior enhancement features (prior templates), ensuring the spatial consistency between deformation and calibration.

[0091] The deformation field approach introduces a multi-template selection mechanism, dynamically selecting the most suitable category-specific template based on prior knowledge. The deformation field is then applied to the template space using a differentiable bilinear mesh, achieving end-to-end differential transformation, with gradient backpropagation providing the complete path. For example, the differentiable bilinear mesh predicts the deformation field of a patient image based on sampling point features, and applies this deformation field to the selected prior template, predicting where the points of the prior template should be moved to match the actual anatomical structures in the patient image. This approach retains the anatomical structural rationality of template-based methods while incorporating the flexible adaptability of deep learning, achieving intelligent adaptation from general templates to patient-specific images.

[0092] The calibration map prediction network predicts calibration maps based on sampling point features. By fusing the deformed prior template with the learned calibration map through pixel-by-pixel addition, it compensates for the shortcomings of pure geometric deformation in intensity distribution and local detail matching. Macroscopically, it achieves anatomical structure alignment through the deformation field, and microscopically, it uses the calibration map as a constraint to complete detail enhancement for individualized features. It achieves accurate matching with the input image while maintaining anatomical accuracy.

[0093] This dual-branch fusion mechanism decouples and organically combines macroscopic geometric deformation (deformation field) with microscopic local intensity adaptation and detail restoration (calibration map), ensuring both the anatomical rationality of the deformation (topological integrity) and enhancing the ability to match individualized tissue characteristics. Compared to methods that rely solely on geometric registration, this application significantly improves the accuracy and robustness of postoperative lung segmentation.

[0094] In another example, during model inference, lung segment variation information in the lung image to be identified is determined based on sampling point features. Feature fusion is then performed based on the lung segment variation information and a prior template of the target lung segment distribution to segment the distribution area of ​​the lung segments in the lung image to obtain lung segmentation results. This includes: inputting the sampling point features and the prior template of the target lung segment distribution into the segmentation model; the segmentation model then identifies lung segment variation information in the lung image to be identified based on the sampling point features; and performs feature fusion based on the lung segment variation information and the prior template of the target lung segment distribution to segment the distribution area of ​​the lung segments in the lung image to obtain lung segmentation results. The segmentation model and the classification model are jointly trained and optimized. The model training process is as follows: Figure 7 or Figure 8 As shown.

[0095] Figure 7 and Figure 8 A schematic diagram illustrating model training provided for an embodiment of this application.

[0096] and Figure 2 , Figure 3 The difference is, Figure 7 To and Figure 2 The corresponding model training process, Figure 8 To and Figure 3 The corresponding model training process involves inputting lung image samples to be identified, which correspond to a gold standard (label). The model parameters are adjusted based on the loss value between the model's prediction results for the lung image samples to be identified and the gold standard (label), thereby obtaining a trained model.

[0097] In one example, such as Figure 7 As shown, the segmentation model and classification model are jointly trained end-to-end. During training, the overall loss function is a weighted sum of the segmentation loss and the classification loss. Through backpropagation, the parameters of the two models are updated collaboratively in the same optimization process, enabling the features extracted from the classification task to effectively guide the segmentation task, and conversely, fine-grained segmentation information can also help to make the classification more accurate.

[0098] Through joint optimization, classification and segmentation tasks are no longer isolated but form a mutually reinforcing positive feedback loop. The global semantics (segmentectomy type) learned by the classification network provides the segmentation network with accurate template selection criteria; the local details (vascular ends, tissue displacement) learned by the segmentation network also provide the classification network with stronger discriminative features. This multi-task learning paradigm improves the overall generalization ability and prediction accuracy of the model.

[0099] The overall training process is as follows. It should be noted that the network parameters obtained through this training method are compatible with both postoperative Cls resection and preoperative lung segmentation.

[0100] (1) Randomly initialize the parameters of the neural network, including weights and biases.

[0101] (2) Input the image and several randomly generated sampling points, calculate the loss function and the segmentation branch loss function by comparing the network output with the corresponding gold standard. Examples include cross-entropy and the Dice loss function; and the loss function for classification branches. For example, the cross-entropy loss function. The two loss functions are weighted and summed, where... and The corresponding weights.

[0102]

[0103] (3) Starting from the output layer, calculate the gradient of the loss function for each parameter in reverse order, and update the parameters according to the gradient direction. (4) When the cycle described above has been completed for the entire training set, one round of training is completed.

[0104] (5) Repeat multiple rounds until the loss function no longer changes significantly, and after the loss function and Dice evaluation index are stable on the validation set, select the network parameters corresponding to the round with the highest Dice evaluation index and save them as the final overall neural network model parameters.

[0105] In another example, such as Figure 8 As shown, with Figure 7 The difference is, Figure 8 The classification module can be removed and replaced by obtaining the resection procedure from the patient's medical record to obtain the lung segment resection status category. By removing the classification module and replacing it with direct input of the lung segment resection status category from the medical record, the loss function of the classification network can be removed during model training, while other parts remain unchanged.

[0106] After model training is complete, inference applications are performed. The inference process is described differently from the training process: (1) The prior template is fixed and no longer updated during the reasoning process; (2) Sampling point strategy: Taking into account actual clinical needs, users can perform targeted sampling for specific regions of interest, that is, only input one or more coordinate points within the image range, and the segmentation module only calculates the segmentation results of the features corresponding to the sampling points. (3) The loss function is no longer calculated during the reasoning process.

[0107] In another example, the lung image samples to be identified for joint training of the segmentation model and the classification model are obtained by: acquiring real postoperative image samples and / or real preoperative image samples, wherein a first lung segment in the real postoperative image sample is removed and a second lung segment in the real preoperative image sample is to be removed; performing random offset simulation processing on the real postoperative image samples and / or real preoperative image samples to generate simulated postoperative image samples and / or simulated preoperative image samples, wherein a third lung segment in the simulated postoperative image samples is simulated to be removed and a fourth lung segment in the simulated preoperative image samples is to be removed; and obtaining the lung image samples to be identified based on the real postoperative image samples and simulated postoperative image samples, and / or based on the real preoperative image samples and simulated preoperative image samples.

[0108] Specifically, addressing the challenges of data acquisition, high annotation costs, and limited real sample sizes in clinical practice, this application proposes a simulated data generation mechanism based on anatomical priors. This mechanism aims to expand the training dataset and enhance the model's generalization ability across various lung segmentectomy types. It can generate alternative training sets independently or serve as a supplementary enhancement to real datasets.

[0109] For postoperative image samples, given the current difficulties in collecting postoperative data, this application proposes a scheme to simulate and generate postoperative data. This scheme can serve as an alternative to training data or as a supplementary dataset based on existing real postoperative datasets. For example, in a real postoperative image sample where the first lung segment has been removed, but a real postoperative image sample where the third lung segment has been removed is lacking, a simulated displacement deformation can be applied to the real postoperative image sample where the first lung segment has been removed. For instance, the deformation can be simulated to the point where the third lung segment has been removed, thus simulating a postoperative image sample where the third lung segment has been removed. Taking a real postoperative image sample as an example, assuming that the lung segment actually removed is the first lung segment (such as the apical segment of the right upper lobe), by performing random displacement simulation processing based on the deformation field on this sample, such as simulating the expansion and displacement of the surrounding tissues (such as the posterior and anterior segments) and the morphological changes of blood vessels and bronchial stumps, a new simulated postoperative image sample can be generated, where the simulated lung segment removed is the third lung segment (such as the posterior segment of the right upper lobe). The simulation process preserves the tissue texture and noise distribution in the real images, while changing the type of lung segment removed, thereby effectively expanding the category coverage of postoperative samples without increasing the burden of real acquisition.

[0110] Similarly, for a preoperative image sample, if a real preoperative image sample shows a lesion in the second lung segment (such as the apical segment of the right upper lobe) awaiting resection, but lacks a real preoperative image sample showing a lesion in the fourth lung segment (such as the posterior segment of the right upper lobe) awaiting resection, the real preoperative image sample showing the second lung segment awaiting resection can be simulated by shifting and deforming it. For example, based on the situation of the second lung segment to be resected, the simulation can be transformed to the situation of the fourth lung segment to be resected, thereby simulating a preoperative image sample showing the fourth lung segment to be resected.

[0111] Specifically, each lung segment region is pre-labeled manually before segmentectomy, and simulated pseudo-labels are generated based on the adjacent relationships of medical anatomical structures. For example, the apical segment of the right upper lobe is adjacent to the posterior and anterior segments of the right upper lobe. When the apical segment is removed, the trachea and blood vessels contained within it will be severed. At the same time, the posterior and anterior segments, as well as the trachea and blood vessels within them, will shift, expand, and displace. Based on the preoperative lung segment region contour, random displacement deformation is simulated, that is, the distance between the simulated resection site and other preserved lung segments is calculated, and the points are assigned to different lung segments according to the distance, thus obtaining pseudo-labels for the lung segment regions and pseudo-deformed trachea and blood vessels. In this way, each case of preoperative data can simulate the resection of 18 lung segments, generating 18 pseudo-data points, expanding the data volume to 19 times the original.

[0112] This approach alleviates the problem of "scarce labeled data" commonly faced in the field of medical image segmentation, especially for complex, diverse, and difficult-to-collect scenarios such as postoperative care. By generating a large amount of high-fidelity pseudo-data through physical simulation based on anatomical knowledge, it not only significantly expands the training set size but also forces the model to learn the general patterns of anatomical structural changes after different lung segment resections, thereby greatly improving the model's generalization ability and robustness in practical applications. The core advantage of this simulation generation method lies in its full utilization of the spatial adjacency and tissue displacement patterns of lung segment anatomy to construct a postoperative deformation model that conforms to medical logic, rather than simple random perturbations. Compared to directly collecting real data, this method significantly reduces data acquisition costs and labeling difficulties while greatly increasing the diversity of training samples.

[0113] During training, the model's output is compared with the gold standard to calculate the loss function; during inference, the model's output is used as the final output. Simultaneously, the resected lung segment category serves as input to the prior module (template generation model) and the sampling point feature concatenation module. The classification module outputs the resected lung segment category, representing the predicted preoperative and postoperative surgical procedures. The segmentation module outputs the lung segment segmentation region, containing segmentation results with a maximum of Class C1 foreground values ​​and Class 1 background values.

[0114] Figure 9 This is a schematic diagram of the lung segmentation results provided in the embodiments of this application.

[0115] like Figure 9As shown, the white arrow represents the posterior segment of the right upper lobe, and the red arrow represents the anterior segment of the right upper lobe. The output surgical procedure is apical resection of the right upper lobe. The segmentation result correctly identifies the extent of the posterior and anterior segments, and there is no apical segmentation result.

[0116] In addition, the template generation model that generates the prior template can be trained independently. After obtaining the prior template, the template generation model does not participate in the overall model training in order to ensure the stability of the training. At the same time, doctors can manually correct and modify the prior template.

[0117] Alternatively, the method of generating prior templates based on template generation models can be replaced by manually annotating prior templates. Or, the template generation model can be trained simultaneously with the overall model, such as the sampling point feature splicing module, classification model, and segmentation model, instead of being trained separately in advance.

[0118] In summary, this application specifically targets the identification of anatomical structures in CT images after lung segmentectomy. It uses finely annotated CT images after lung segmentectomy for training, allowing the model to learn various postoperative change patterns. A deformable knowledge graph is embedded in the model as prior knowledge to guide the segmentation of postoperative anatomical structures. The surgical procedure type (i.e., the name of the lung segment removed) is used as a parallel task for multi-task prediction, guiding it to perform targeted identification. It takes into account the variability of postoperative patients and can be applied to both preoperative and postoperative cases, expanding the applicability to cases with unknown medical history.

[0119] By employing prior fusion and multi-task mechanisms, an automatic prediction framework integrating prior knowledge was constructed. This framework organically combines a multi-template-driven strategy with a deep learning non-rigid deformation field and introduces a category-based prior embedding mechanism, enabling personalized deformation learning guided by anatomical semantics. This effectively addresses the registration challenges arising from postoperative anatomical structural variations. Joint optimization of surgical procedure type classification (i.e., identification of resected lung segments) and lung segment region segmentation not only enhances the model's understanding of complex postoperative anatomical structures but also significantly improves the system's inference efficiency through feature sharing. The system can quickly and accurately identify the anatomical affiliation of each lung segment, providing reliable anatomical evidence for postoperative functional assessment, residual lung function compensation analysis, and reoperation planning, effectively supporting the entire process of precise diagnosis and treatment in thoracic surgery. Testing on 160 cases showed a surgical procedure identification accuracy of 96.6%, a segmentation Dice (similarity between segmentation results and actual results) of 81.1%, and a false positive rate of 0.2%.

[0120] Figure 10 A schematic diagram of a lung anatomy recognition device provided in an embodiment of this application.

[0121] This application provides a lung anatomy structure recognition device 1000, comprising: The acquisition module 1010 is used to acquire the lung image to be identified and to extract multi-scale features from the lung image to obtain multi-scale features.

[0122] The first determining module 1020 is used to determine the lung segment resection status category in the lung image to be identified.

[0123] The second determining module 1030 is used to determine the sampling point features of the lung spatial sampling points based on the lung segment resection status category and multi-scale features.

[0124] The third determining module 1040 is used to determine the target lung segment distribution prior template corresponding to the lung segment resection status category from multiple candidate lung segment distribution prior templates based on the lung segment resection status category.

[0125] The recognition module 1050 is used to identify lung segment change information in the lung image to be identified based on the features of sampling points, and to perform feature fusion based on the lung segment change information and the target lung segment distribution prior template, so as to identify and segment the distribution area of ​​the lung segment in the lung image to be identified and obtain the lung segment segmentation result.

[0126] In other embodiments, the recognition module 1050 is further configured to: perform deformation recognition on lung segments in the lung image to be recognized based on sampling point features to obtain a lung segment deformation field; and perform local tissue change recognition on lung segments in the lung image to be recognized based on sampling point features to obtain a local tissue change correction map; perform deformation processing on the target lung segment distribution prior template based on the lung segment deformation field to obtain a deformed lung segment distribution prior template; fuse the local tissue change correction map and the deformed lung segment distribution prior template to obtain fine distribution features of the lung segments; and obtain lung segment segmentation results based on the fine distribution features.

[0127] In other implementations, the multiple candidate lung segment distribution prior templates include Cls lung segment resection prior templates and one lung segment unresection prior template, where Cls is an integer greater than or equal to 1. The multiple candidate lung segment distribution prior templates are obtained as follows: Cls+1 training datasets are acquired, where each Cls-group lung segment resection dataset includes a Cls-group lung segment resection dataset and one lung segment unresection dataset, with each lung segment resection dataset in the Cls-group lung segment resection dataset corresponding to a lung segment resection category; multiple batches of data are determined from the Cls+1-group training dataset and sequentially input into the template generation model for processing. Multiple rounds of training are conducted to identify the resected lung segments, yielding a loss function value for each round. This loss function value includes a similarity loss value and a cross-entropy loss value. The similarity loss value evaluates the similarity between the trained identification results and the true labels, while the cross-entropy loss value evaluates the difference between the distribution of the trained identification results and the true distribution. After multiple rounds of training until the loss function value meets the convergence condition, the model parameters of the template generation model are determined based on the evaluation metrics. Based on the model parameters of the template generation model, Cls+1 training identification results corresponding to the Cls+1 training datasets are determined as prior templates for the Cls+1 candidate lung segment distributions.

[0128] In other embodiments, the second determining module 1030 is further configured to: perform interpolation processing on the multi-scale features to obtain high-level features of the lung spatial sampling points; determine the spatial location of the lung spatial sampling points; and concatenate the high-level features, spatial location, and lung segment resection status category to obtain sampling point features.

[0129] In other embodiments, interpolation processing is performed on multi-scale features to obtain high-level features of lung spatial sampling points, including: receiving a specified instruction for a local lung region; determining lung spatial sampling points from a voxel grid constituting the lung space based on the specified instruction, wherein the lung spatial sampling points include discrete voxel points of the voxel grid and / or non-voxel points in continuous space; and interpolating the multi-scale features based on the lung spatial sampling points to obtain high-level features corresponding to the lung spatial sampling points, wherein the high-level features corresponding to the non-voxel points are obtained through interpolation.

[0130] In other embodiments, the first determining module 1020 is also used to: input multi-scale features into a classification model for identification and output the lung segment resection status category; or determine the lung segment resection status category in the lung image to be identified based on medical record data.

[0131] In other embodiments, the identification module 1050 is further configured to: input the sampling point features and the target lung segment distribution prior template into the segmentation model, identify the lung segment change information in the lung image to be identified based on the sampling point features through the segmentation model, and perform feature fusion based on the lung segment change information and the target lung segment distribution prior template, so as to identify and segment the distribution area of ​​the lung segment in the lung image to be identified to obtain the lung segment segmentation result, wherein the segmentation model and the classification model are jointly trained and optimized.

[0132] In other embodiments, the lung image samples to be identified for jointly training the segmentation model and the classification model are obtained by: acquiring real postoperative image samples and / or real preoperative image samples, wherein a first lung segment in the real postoperative image sample is removed and a second lung segment in the real preoperative image sample is to be removed; performing random offset simulation processing on the real postoperative image samples and / or real preoperative image samples to generate simulated postoperative image samples and / or simulated preoperative image samples, wherein a third lung segment in the simulated postoperative image samples is simulated to be removed and a fourth lung segment in the simulated preoperative image samples is to be removed; and obtaining the lung image samples to be identified based on the real postoperative image samples and simulated postoperative image samples, and / or based on the real preoperative image samples and simulated preoperative image samples.

[0133] It is understandable that a detailed description of the lung anatomy structure identification method can be found in the description of the lung anatomy structure identification method above, and will not be repeated here.

[0134] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method in any of the above embodiments.

[0135] One embodiment of this application provides a computer program product including instructions that, when executed by a processor of a computer device, enable the computer device to perform the steps of the method described in any of the above embodiments.

[0136] Figure 11 A block diagram of an electronic device provided in an embodiment of this application.

[0137] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method in any of the above embodiments.

[0138] like Figure 11 As shown, for ease of understanding, an embodiment of this application illustrates a specific electronic device 1100.

[0139] Electronic device 1100 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 1100 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0140] like Figure 11 As shown, device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1102 or a computer program loaded into random access memory (RAM) 1103 from storage unit 1108. The RAM 1103 may also store various programs and data required for the operation of electronic device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0141] Multiple components in electronic device 1100 are connected to I / O interface 1105. These components include: input unit 1106, such as a keyboard or mouse; output unit 1107, such as various types of displays or speakers; storage unit 1108, such as a disk or optical disk; and communication unit 1109, such as a network interface card (NIC), modem, or wireless transceiver. Communication unit 1109 allows electronic device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0142] The computing unit 1101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods described above. For example, in some embodiments, any one or more of the methods described above can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of any one or more of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 1101 can be configured to perform any one or more of the methods described above by any other suitable means (e.g., by means of firmware).

[0143] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this application, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0144] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0145] In the description of this application, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0146] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0147] Furthermore, the terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this application can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this application, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly and specifically defined in the embodiments.

[0148] In this application, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing" appearing in the embodiments should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication between two components, or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific implementation.

[0149] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

Claims

1. A method for identifying lung anatomical structures, characterized in that, The method includes: A lung image to be identified is acquired, and multi-scale features are extracted from the lung image to obtain multi-scale features; Determine the lung segment resection status category in the lung image to be identified; Based on the lung segment resection status category and the multi-scale features, the sampling point features of the lung spatial sampling points are determined; Based on the lung segment resection status category, a target lung segment distribution prior template corresponding to the lung segment resection status category is determined from multiple candidate lung segment distribution prior templates; Based on the features of the sampling points, lung segment change information in the lung image to be identified is identified, and feature fusion is performed based on the lung segment change information and the target lung segment distribution prior template to identify and segment the distribution area of ​​the lung segment in the lung image to be identified, thereby obtaining the lung segment segmentation result.

2. The method according to claim 1, characterized in that, The lung segment change information includes a lung segment deformation field and a local tissue change correction map; the step of identifying lung segment change information in the lung image to be identified based on the sampling point features, and performing feature fusion based on the lung segment change information and the target lung segment distribution prior template to identify and segment the distribution area of ​​the lung segment in the lung image to be identified, and obtaining the lung segment segmentation result, includes: Based on the sampling point features, deformation recognition is performed on the lung segments in the lung image to be identified to obtain the lung segment deformation field; and based on the sampling point features, local tissue change recognition is performed on the lung segments in the lung image to be identified to obtain the local tissue change correction map. Based on the lung segment deformation field, the target lung segment distribution prior template is deformed to obtain the deformed lung segment distribution prior template. The local tissue change correction map and the prior template of the deformed lung segment distribution are fused to obtain the fine distribution features of the lung segments; Based on the fine distribution features, the lung segmentation results are obtained.

3. The method according to claim 1, characterized in that, The multiple candidate lung segment distribution prior templates include Cls lung segment resection prior templates and 1 lung segment unresection prior template, where Cls is an integer greater than or equal to 1; the multiple candidate lung segment distribution prior templates are obtained in the following way: Obtain the Cls+1 group training dataset, wherein the Cls+1 group training dataset includes the Cls group lung segment resection dataset and 1 group lung segment unresected dataset, and each lung segment resection dataset in the Cls group lung segment resection dataset corresponds to a lung segment resection category; Multiple batches of data are selected from the Cls+1 training dataset and sequentially input into the template generation model for multiple rounds of training to identify the resected lung segments. The loss function value for each round is obtained, wherein the loss function value includes a similarity loss value and a cross-entropy loss value. The similarity loss value is used to evaluate the similarity between the training recognition result and the real label, and the cross-entropy loss value is used to evaluate the difference between the distribution of the training recognition result and the real distribution. After multiple rounds of training until the loss function value meets the convergence condition, the model parameters of the template generation model are determined based on the evaluation metric. Based on the model parameters of the template generation model, the Cls+1 training recognition results corresponding to the Cls+1 training datasets are determined as the Cls+1 candidate lung segment distribution prior templates.

4. The method according to claim 1, characterized in that, The determination of sampling point features for spatial sampling points of the lung based on the lung segment resection status category and the multi-scale features includes: Interpolation processing is performed on the multi-scale features to obtain the high-level features of the lung spatial sampling points; Determine the spatial location of the lung spatial sampling point; The high-level features, the spatial location, and the lung segment resection status category are concatenated to obtain the sampling point features.

5. The method according to claim 4, characterized in that, The interpolation process performed on the multi-scale features to obtain the high-level features of the lung spatial sampling points includes: Receive specified instructions for a local area of ​​the lungs; Based on the specified instructions, lung space sampling points are determined from a voxel grid constituting the lung space, wherein the lung space sampling points include discrete voxel points of the voxel grid and / or non-voxel points of the continuous space. Based on the lung spatial sampling points, the multi-scale features are interpolated to obtain the high-level features corresponding to the lung spatial sampling points, wherein the high-level features corresponding to the non-voxel points are obtained through interpolation.

6. The method according to any one of claims 1-5, characterized in that, Determining the lung segment resection status category in the lung image to be identified includes: The multi-scale features are input into a classification model for identification, and the lung segment resection status category is output; or Based on medical record data, the lung segment resection status category in the lung image to be identified is determined.

7. The method according to claim 6, characterized in that, The step of identifying lung segment variation information in the lung image to be identified based on the features of the sampling points, and performing feature fusion based on the lung segment variation information and the target lung segment distribution prior template to identify and segment the distribution area of ​​the lung segments in the lung image to be identified, and obtaining lung segment segmentation results, includes: The features of the sampling points and the prior template of the target lung segment distribution are input into the segmentation model. The segmentation model identifies lung segment variation information in the lung image to be identified based on the features of the sampling points, and performs feature fusion based on the lung segment variation information and the prior template of the target lung segment distribution to identify and segment the distribution area of ​​lung segments in the lung image to obtain the lung segment segmentation result. The segmentation model and the classification model are jointly trained and optimized.

8. The method according to claim 7, characterized in that, The lung image samples to be identified used for jointly training the segmentation model and the classification model are obtained in the following way: Obtain real postoperative image samples and / or real preoperative image samples, wherein a first lung segment in the real postoperative image sample is removed and a second lung segment in the real preoperative image sample is to be removed; Random offset simulation processing is performed on the real postoperative image sample and / or the real preoperative image sample to generate simulated postoperative image sample and / or simulated preoperative image sample, wherein the third lung segment in the simulated postoperative image sample is simulated to be removed, and the fourth lung segment in the simulated preoperative image sample is to be removed. The lung image sample to be identified is obtained based on the real postoperative image sample and the simulated postoperative image sample, and / or based on the real preoperative image sample and the simulated preoperative image sample.

9. A lung anatomical structure identification device, characterized in that, The device includes: The acquisition module is used to acquire an image of the lung to be identified and to extract multi-scale features from the image of the lung to be identified to obtain multi-scale features; The first determining module is used to determine the lung segment resection status category in the lung image to be identified; The second determining module is used to determine the sampling point features of the lung spatial sampling points based on the lung segment resection status category and the multi-scale features; The third determining module is used to determine, based on the lung segment resection status category, a target lung segment distribution prior template corresponding to the lung segment resection status category from multiple candidate lung segment distribution prior templates; The identification module is used to identify lung segment change information in the lung image to be identified based on the features of the sampling points, and to perform feature fusion based on the lung segment change information and the target lung segment distribution prior template, so as to identify and segment the distribution area of ​​the lung segment in the lung image to be identified to obtain the lung segment segmentation result.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-8.

Images