Lung airway marking method and device, medium and terminal

Abstract

The invention discloses a lung airway marking method and device, a medium and a terminal, relates to the technical field of image processing and the medical field, and mainly aims at solving the problem that existing lung airway marking accuracy is low. Comprising the steps that a target lung CT scanning image is acquired, and an airway branch attribute graph is generated according to the target lung CT scanning image; in the airway branch attribute graph, finding out airway branch nodes corresponding to each extrapulmonary airway branch to obtain a plurality of father branch nodes, and respectively determining a set of a plurality of child branch nodes with the same father branch nodes as hyperedges to construct an airway branch topological structure hypergraph; and on the basis of a lung airway marking model after model training is completed, according to the airway branch attribute graph and the airway branch topological structure hypergraph, determining an anatomical category classification result of each airway branch so as to complete lung airway marking.

Description

Technical Field

[0001] This application relates to the fields of image processing technology and medical technology, and in particular to a method, device, medium, and terminal for marking the airways of the lungs. Background Technology

[0002] Precise airway labeling is crucial for quantitative analysis, preoperative planning, and bronchoscopic navigation in lung diseases such as chronic obstructive pulmonary disease (COPD) and bronchiectasis. Advances in CT imaging technology have enabled the segmentation of finer terminal airways, but have also increased the complexity of airway labeling. Traditional airway labeling methods match the geometric spatial features of airway branches with pre-defined maps; however, this approach struggles to handle complex individual variability, resulting in low labeling accuracy.

[0003] With the development of artificial intelligence technology, deep learning technology has been applied to the field of airway labeling. Since airway trees naturally have a graph structure, most existing technologies model the airway labeling task as a node classification problem based on graph neural networks.

[0004] However, on the one hand, traditional graph neural networks have limited capabilities in feature extraction, making it difficult to fully capture the complex high-dimensional nonlinear features in airway nodes, resulting in the loss of detailed local information and thus reducing the accuracy of airway labeling; on the other hand, traditional graph neural networks typically acquire long-range information through multi-layer stacking, which can lead to over-smoothing or information dilution, resulting in the loss of extensive contextual data and further reducing the accuracy of airway labeling. Summary of the Invention

[0005] In view of this, this application provides a method, device, medium, and terminal for marking the lung airways, with the main purpose of improving the low accuracy of existing lung airway marking.

[0006] According to one aspect of this application, a method for marking the airways of the lungs is provided, comprising: Acquire a CT scan image of the target lung and generate an airway branch attribute map based on the target lung CT scan image. The airway branch attribute map contains multiple airway branch nodes, which are used to characterize airway branches and carry corresponding basic features and lung lobe category features. In the airway branch attribute graph, the airway branch nodes corresponding to each extrapulmonary airway branch are found, multiple parent branch nodes are obtained, and the sets of multiple child branch nodes with the same parent branch node are determined as hyperedges to construct the airway branch topology hypergraph. Based on the lung airway labeling model that has completed model training, the anatomical classification results of each airway branch are determined according to the airway branch attribute map and the airway branch topology hypergraph to complete the lung airway labeling. The lung airway labeling model includes a deep feature extraction sub-model for airway branch nodes, a local structural feature extraction sub-model, a global relational feature extraction sub-model, a feature fusion sub-model, and a classification sub-model.

[0007] Preferably, the lung airway labeling model based on the completed model training determines the anatomical classification results of each airway branch according to the airway branch attribute map and the airway branch topology hypergraph, including: The airway branch attribute map is input into the airway branch node feature depth extraction sub-model to perform nonlinear transformation processing on the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map, so as to obtain the depth features corresponding to each airway branch node. Each of the depth features and the airway branch topology hypergraph are input into the local structure feature extraction sub-model to aggregate the local topology neighborhood information of each airway branch node within its respective hyperedge, thereby obtaining the depth features of each airway branch node after fusing local structure features. The airway branch attribute map is input into the global relation feature extraction sub-model to perform global relation feature extraction processing on each airway branch node contained in the airway branch attribute map, so as to obtain the global relation features corresponding to each airway branch node. Based on the feature fusion sub-model, the depth features and global relationship features of the fused local structural features of each airway branch node are fused to obtain the fused features corresponding to each airway branch node. Each of the fused features is input into the classification sub-model to obtain the probability value of each airway branch node belonging to each anatomical category, and the anatomical category corresponding to the maximum probability value is determined as the anatomical category of the airway branch node.

[0008] Preferably, the step of inputting the airway branch attribute map into the airway branch node feature depth extraction sub-model to perform nonlinear transformation processing on the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map to obtain the depth features corresponding to each airway branch node includes: In the airway branch node feature depth extraction sub-model, the multivariate feature function corresponding to the airway branch attribute map is decomposed into a combination of multiple univariate feature functions by using the radial basis function approximation algorithm of the third-order B-spline basis function; Based on the combination of multiple univariate feature functions, the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map are subjected to nonlinear transformation processing to obtain the depth features corresponding to each airway branch node.

[0009] Preferably, the step of inputting each of the depth features and the airway branch topology hypergraph into the local structure feature extraction sub-model to aggregate the local topological neighborhood information of each airway branch node within its respective hyperedge, and obtaining the fused local structure features corresponding to each airway branch node, includes: In the local structural feature extraction sub-model, each of the depth features is multiplied with the airway branch topology hypergraph to obtain the aggregated features of each hyperedge contained in the airway branch topology hypergraph. The interaction features between each airway branch node and its corresponding superedge neighbor node are calculated to obtain the interaction features of each superedge. The aggregation features and interaction features of each hyperedge are fused to obtain the hypergraph features of the airway branch topology hypergraph. Each of the depth features and the corresponding hypergraph features are concatenated and similarity coefficients are filtered to obtain the local topological neighborhood information of each airway branch node within its respective hyperedge. Each of the local topological neighborhood information is added to its corresponding depth feature to obtain the depth feature after fusing local structural features for each of the airway branch nodes.

[0010] Preferably, the step of inputting the airway branch attribute map into the global relation feature extraction sub-model to perform global relation feature extraction processing on each airway branch node contained in the airway branch attribute map, and obtaining the global relation features corresponding to each airway branch node, includes: In the global relation feature extraction sub-model, the airway branch attribute map is converted into the first airway branch depth attribute vector, the second airway branch depth attribute vector, and the third airway branch depth attribute vector based on the first airway branch node feature depth extraction layer, the second airway branch node feature depth extraction layer, and the third airway branch node feature depth extraction layer, respectively. The first airway branch depth attribute vector and the second airway branch depth attribute vector are respectively subjected to a first normalization process to obtain the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector. Based on the first normalized first airway branch depth attribute vector, the first normalized second airway branch depth attribute vector, and the third airway branch depth attribute vector, calculate the attention weight between airway branch nodes. Based on a preset scaling factor, the attention weights between the airway branch nodes are optimized to obtain the optimized attention weights between the airway branch nodes. Based on the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector, an attention normalization denominator is constructed. Based on the attention normalization denominator, the attention weights between the optimized airway branch nodes are subjected to a second normalization process to obtain the global relationship features corresponding to each airway branch node.

[0011] Preferably, the step of acquiring a target lung CT scan image and generating an airway branch attribute map based on the target lung CT scan image includes: Acquire CT scan images of the target lungs; A three-dimensional airway tree structure model is extracted from the target lung CT scan image, and the three-dimensional airway tree structure model is skeletonized to obtain a two-dimensional airway tree structure diagram. Each airway branch is identified from the two-dimensional structure diagram of the airway tree, and the basic features of each airway branch are calculated respectively. The basic features include topological features, spatial features and geometric features. Based on the spatial features of each airway branch and the lung lobe segmentation results of the target lung CT scan image, the lung lobe category features of each airway branch are determined. Each airway branch is abstracted into an airway branch node to generate an airway branch attribute map, wherein the airway branch node carries basic features and lung lobe category features.

[0012] Preferably, in the airway branch attribute graph, finding the airway branch nodes corresponding to each extrapulmonary airway branch to obtain multiple parent branch nodes, and determining the sets of multiple child branch nodes with the same parent branch node as hyperedges to construct an airway branch topology hypergraph, includes: In the airway branch attribute graph, find the airway branch node corresponding to each extrapulmonary airway branch, determine it as the parent branch node, and store it in a pre-created double-ended queue; From the first end of the double-ended queue, obtain each of the parent branch nodes one by one as the target parent branch node; Add the target parent branch node to the hypergraph. In the airway branch attribute graph, identify all direct child branch nodes of the target parent branch node and establish a target hyperedge, wherein the target hyperedge contains the target parent branch node and all direct child branch nodes. The number of direct child branch nodes contained in the target parent branch node is counted, and when the number exceeds a preset threshold for the number of child nodes, each direct child branch node is stored from the second end of the double-ended queue into the double-ended queue to serve as a parent branch node for super-edge establishment. By integrating the hyperedges of each target, a hypergraph of the airway branch topology is obtained.

[0013] According to another aspect of this application, a marking device for the lung airways is provided, comprising: The airway branch attribute map generation module is used to acquire a target lung CT scan image and generate an airway branch attribute map based on the target lung CT scan image. The airway branch attribute map contains multiple airway branch nodes, which are used to characterize airway branches and carry corresponding basic features and lung lobe category features. The airway branch topology hypergraph construction module is used to find the airway branch nodes corresponding to each extrapulmonary airway branch in the airway branch attribute graph, obtain multiple parent branch nodes, and determine the set of multiple child branch nodes with the same parent branch node as the hyperedge to construct the airway branch topology hypergraph. The lung airway labeling module is used to determine the anatomical classification results of each airway branch based on the airway branch attribute map and the airway branch topology hypergraph, based on the lung airway labeling model that has been trained, so as to complete the lung airway labeling. The lung airway labeling model includes a deep feature extraction sub-model for airway branch nodes, a local structural feature extraction sub-model, a global relation feature extraction sub-model, a feature fusion sub-model, and a classification sub-model.

[0014] Preferably, the lung airway marking module includes: The deep feature extraction unit is used to input the airway branch attribute map into the airway branch node feature deep extraction sub-model, so as to perform nonlinear transformation processing on the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map to obtain the deep features corresponding to each airway branch node. The local structural feature extraction unit is used to input each of the depth features and the airway branch topology hypergraph into the local structural feature extraction sub-model, so as to aggregate the local topological neighborhood information of each airway branch node in its respective hyperedge and obtain the depth features after fusing local structural features corresponding to each airway branch node. The global relation feature extraction unit is used to input the airway branch attribute map into the global relation feature extraction sub-model to perform global relation feature extraction processing on each airway branch node contained in the airway branch attribute map, so as to obtain the global relation features corresponding to each airway branch node. The feature fusion unit is used to fuse the depth features and global relationship features of the fused local structural features of each airway branch node based on the feature fusion sub-model, so as to obtain the fused features corresponding to each airway branch node. The classification unit is used to input each of the fused features into the classification sub-model to obtain the probability value of each airway branch node belonging to each anatomical category, and to determine the anatomical category corresponding to the maximum probability value as the anatomical category of the airway branch node.

[0015] Preferably, the deep feature extraction unit is specifically used for: In the airway branch node feature depth extraction sub-model, the multivariate feature function corresponding to the airway branch attribute map is decomposed into a combination of multiple univariate feature functions by using the radial basis function approximation algorithm of the third-order B-spline basis function; Based on the combination of multiple univariate feature functions, the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map are subjected to nonlinear transformation processing to obtain the depth features corresponding to each airway branch node.

[0016] Preferably, the local structural feature extraction unit is specifically used for: In the local structural feature extraction sub-model, each of the depth features is multiplied with the airway branch topology hypergraph to obtain the aggregated features of each hyperedge contained in the airway branch topology hypergraph. The interaction features between each airway branch node and its corresponding superedge neighbor node are calculated to obtain the interaction features of each superedge. The aggregation features and interaction features of each hyperedge are fused to obtain the hypergraph features of the airway branch topology hypergraph. Each of the depth features and the corresponding hypergraph features are concatenated and similarity coefficients are filtered to obtain the local topological neighborhood information of each airway branch node within its respective hyperedge. Each of the local topological neighborhood information is added to its corresponding depth feature to obtain the depth feature after fusing local structural features for each of the airway branch nodes.

[0017] Preferably, the global relation feature extraction unit is specifically used for: In the global relation feature extraction sub-model, the airway branch attribute map is converted into the first airway branch depth attribute vector, the second airway branch depth attribute vector, and the third airway branch depth attribute vector based on the first airway branch node feature depth extraction layer, the second airway branch node feature depth extraction layer, and the third airway branch node feature depth extraction layer, respectively. The first airway branch depth attribute vector and the second airway branch depth attribute vector are respectively subjected to a first normalization process to obtain the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector. Based on the first normalized first airway branch depth attribute vector, the first normalized second airway branch depth attribute vector, and the third airway branch depth attribute vector, calculate the attention weight between airway branch nodes. Based on a preset scaling factor, the attention weights between the airway branch nodes are optimized to obtain the optimized attention weights between the airway branch nodes. Based on the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector, an attention normalization denominator is constructed. Based on the attention normalization denominator, the attention weights between the optimized airway branch nodes are subjected to a second normalization process to obtain the global relationship features corresponding to each airway branch node.

[0018] Preferably, the airway branch attribute map generation module is specifically used for: Acquire CT scan images of the target lungs; A three-dimensional airway tree structure model is extracted from the target lung CT scan image, and the three-dimensional airway tree structure model is skeletonized to obtain a two-dimensional airway tree structure diagram. Each airway branch is identified from the two-dimensional structure diagram of the airway tree, and the basic features of each airway branch are calculated respectively. The basic features include topological features, spatial features and geometric features. Based on the spatial features of each airway branch and the lung lobe segmentation results of the target lung CT scan image, the lung lobe category features of each airway branch are determined. Each airway branch is abstracted into an airway branch node to generate an airway branch attribute map, wherein the airway branch node carries basic features and lung lobe category features.

[0019] Preferably, the airway branch topology hypergraph construction module is specifically used for: In the airway branch attribute graph, find the airway branch node corresponding to each extrapulmonary airway branch, determine it as the parent branch node, and store it in a pre-created double-ended queue; From the first end of the double-ended queue, obtain each of the parent branch nodes one by one as the target parent branch node; Add the target parent branch node to the hypergraph. In the airway branch attribute graph, identify all direct child branch nodes of the target parent branch node and establish a target hyperedge, wherein the target hyperedge contains the target parent branch node and all direct child branch nodes. The number of direct child branch nodes contained in the target parent branch node is counted, and when the number exceeds a preset threshold for the number of child nodes, each direct child branch node is stored from the second end of the double-ended queue into the double-ended queue to serve as a parent branch node for super-edge establishment. By integrating the hyperedges of each target, a hypergraph of the airway branch topology is obtained.

[0020] According to another aspect of this application, a storage medium is provided that stores at least one executable instruction that causes a processor to perform an operation corresponding to the above-described lung airway marking method.

[0021] According to another aspect of this application, a terminal is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the above-described lung airway marking method.

[0022] By employing the above technical solutions, the technical solutions provided in the embodiments of this application have at least the following advantages: This application provides a method, device, medium, and terminal for labeling lung airways. First, a CT scan image of the target lung is acquired, and an airway branch attribute map is generated based on the target lung CT scan image. The airway branch attribute map contains multiple airway branch nodes, which represent airway branches and carry corresponding basic features and lung lobe category features. Second, in the airway branch attribute map, the airway branch nodes corresponding to each extrapulmonary airway branch are identified, resulting in multiple parent branch nodes. The sets of multiple child branch nodes with the same parent branch node are defined as hyperedges to construct an airway branch topology hypergraph. Finally, based on a lung airway labeling model that has completed model training, the anatomical classification results of each airway branch are determined according to the airway branch attribute map and the airway branch topology hypergraph to complete lung airway labeling. The lung airway labeling model includes a sub-model for deep feature extraction of airway branch nodes, a sub-model for local structural feature extraction, a sub-model for global relational feature extraction, a feature fusion sub-model, and a classification sub-model. Compared with existing technologies, the embodiments of this application construct an airway branch attribute graph carrying rich node information and an airway branch topology hypergraph integrating comprehensive topological relationships. Based on a pre-constructed lung airway labeling model, node depth features, local structural features, and global relationship features are extracted from the airway branch attribute graph and the airway branch topology hypergraph and fused together. This results in higher accuracy of node anatomical categories generated based on the fused features, thereby effectively improving the accuracy of airway labeling. Furthermore, by incorporating the prior knowledge of lung lobe category features into the node information, the generalization ability and robustness of the lung airway labeling model are improved.

[0023] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0024] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a lung airway marking method provided in an embodiment of this application is shown; Figure 2 A flowchart of another method for marking the lung airways provided in an embodiment of this application is shown; Figure 3This illustration shows a block diagram of a lung airway marking device according to an embodiment of this application; Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of this application is shown. Detailed Implementation

[0025] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0026] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0027] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this application and its application or use.

[0028] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0029] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0030] The embodiments of this application can be applied to computer systems / servers that can operate with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with computer systems / servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems, etc.

[0031] Computer systems / servers can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are performed by remote processing devices linked through a communication network. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0032] This application provides a method for marking the airways of the lungs, such as... Figure 1 As shown, the method includes: 101. Obtain the target lung CT scan image and generate an airway branch attribute map based on the target lung CT scan image.

[0033] The airway branch attribute map contains multiple airway branch nodes, which represent airway branches and carry corresponding basic features and lung lobe category features. Basic features represent features that can be calculated from CT scan images and are deterministic; lung lobe category features represent prior knowledge and are uncertain. In this embodiment, the current execution end can be the lung airway labeling module of a medical system.

[0034] 102. In the airway branch attribute graph, find the airway branch nodes corresponding to each extrapulmonary airway branch, obtain multiple parent branch nodes, and determine the set of multiple child branch nodes with the same parent branch node as the hyperedge to construct the airway branch topology hypergraph.

[0035] Among them, the extrapulmonary airway branches can be the central large airways such as the trachea, main bronchus, and intermediate trunk bronchus; the sub-branch nodes are abstracted from the next-level airway branches that directly branch off from the parent airway branch represented by the parent branch node; the hyperedge is used to represent a set of a parent branch node and all its direct sub-branches.

[0036] It should be noted that, since bronchi at different levels may connect to multiple regions simultaneously in the lung airway tree, resulting in a complex topological structure, this application constructs an airway branch topology hypergraph to make the integrated airway branch topological relationships more comprehensive.

[0037] 103. Based on the lung airway labeling model that has been trained, determine the anatomical classification results of each airway branch according to the airway branch attribute map and the airway branch topology hypergraph to complete the lung airway labeling.

[0038] The lung airway labeling model includes a deep feature extraction sub-model for airway branch nodes, a local structural feature extraction sub-model, a global relational feature extraction sub-model, a feature fusion sub-model, and a classification sub-model. The deep feature extraction sub-model for airway branch nodes is built based on the Kolmogorov-Arnold Network neural network and is used to perform nonlinear transformations on the features into deep features. The local structural feature extraction sub-model is built based on the adjacency topology network and is used to aggregate the local topological neighborhood information of airway branch nodes within their respective hyperedges. The global relational feature extraction sub-model is built based on the pairwise attention network and is used to extract the global relational features of airway branch nodes. The feature fusion sub-model is used for feature fusion. The classification sub-model is used to calculate the probability that airway branch nodes belong to each anatomical category.

[0039] It should be noted that the lung airway labeling model can be trained using labeled smoothed cross-entropy loss, with the label smoothing hyperparameter set to 0.02, the loss weight set to 1, and the Adam optimizer used for training. The learning rate can be set to 0.001, and the training time is 600 iterations. In addition, all features except topological features need to be normalized.

[0040] Compared with existing technologies, the embodiments of this application construct an airway branch attribute graph carrying rich node information and an airway branch topology hypergraph integrating comprehensive topological relationships. Based on a pre-constructed lung airway labeling model, node depth features, local structural features, and global relationship features are extracted from the airway branch attribute graph and the airway branch topology hypergraph and fused together. This results in higher accuracy of node anatomical categories generated based on the fused features, thereby effectively improving the accuracy of airway labeling. Furthermore, by incorporating the prior knowledge of lung lobe category features into the node information, the generalization ability and robustness of the lung airway labeling model are improved.

[0041] This application provides another method for marking the lung airways, such as... Figure 2 As shown, the method includes: 201. Obtain the target lung CT scan image and generate an airway branch attribute map based on the target lung CT scan image.

[0042] Accordingly, step 201 of the embodiment specifically includes: acquiring a CT scan image of the target lung; extracting a three-dimensional airway tree structure model from the CT scan image of the target lung, and performing skeletonization processing on the three-dimensional airway tree structure model to obtain a two-dimensional airway tree structure map; identifying each airway branch from the two-dimensional airway tree structure map, and calculating the basic features of each airway branch, wherein the basic features include topological features, spatial features, and geometric features; determining the lung lobe category features of each airway branch based on the spatial features of each airway branch and the lung lobe segmentation results of the CT scan image of the target lung; and abstracting each airway branch into airway branch nodes to generate an airway branch attribute map, wherein the airway branch nodes carry basic features and lung lobe category features.

[0043] The airway tree 3D structural model is used to characterize the 3D tubular airway structure and can be extracted from the target lung CT scan image using a segmentation method. Skeletonization processing simplifies the 3D airway tree structural model into a one-dimensional centerline skeleton structure to obtain a 2D airway tree structure diagram. Each airway branch is identified from the 2D airway tree structure diagram. First, all bifurcation points are identified in the centerline skeleton structure. Then, a continuous centerline segment between two bifurcation points and from the bifurcation point to the terminal point is identified as an airway branch. Basic features include topological features, spatial features, and geometric features. Topological features include generational features (derived from airway generation). The generational characteristics are determined by designating the central airway as the first generation, increasing with each branch; the number of offspring (characterizing the number of lower-level airway branches directly branching from the current airway branch); and the number of siblings (characterizing the number of sibling airway branches belonging to the same parent airway branch as the current airway branch). Spatial characteristics include global relative position (characterizing the relative position between the midpoint of an airway branch and the end point of the central airway, i.e., the coordinate difference in the X, Y, and Z directions) and local relative position (characterizing the relative position between the start and end points of an airway branch, i.e., the coordinate difference in the X, Y, and Z directions). The system uses standard deviation and absolute position (to characterize the position ranking value of the midpoint of the airway branch in each coordinate axis of three-dimensional space); geometric features include angle (to characterize the projection angle of the airway branch phasor on the X, Y, Z coordinate planes), length (to characterize the three-dimensional length of the airway branch), projected length (to characterize the projected length of the airway branch on the X, Y, Z coordinate axes), and airway branch similarity (to characterize the cosine similarity between the current airway branch and its parent airway branch); the lung lobe segmentation result is obtained by inputting the target lung CT scan image into a pre-trained lung lobe segmentation model to perform lung segmentation. A label map of the same size as the target lung CT scan image is provided, in which lung lobe regions are marked, such as upper right, middle right, lower right, upper left, lower left, etc. In this embodiment, the lung lobe category features are determined by mapping the spatial location of airway branches to the lung lobe segmentation results. It should be noted that when training the lung lobe segmentation model, data augmentation methods can be used. For example, with a 5% probability, the lung lobe category feature values ​​of some nodes are randomly set as masks. For child nodes, with a 25% probability, their lung lobe category is forcibly modified to the lung lobe category of their parent node, and with a 75% probability, their original true value is retained.

[0044] 202. In the airway branch attribute graph, find the airway branch nodes corresponding to each extrapulmonary airway branch, obtain multiple parent branch nodes, and determine the set of multiple child branch nodes with the same parent branch node as the hyperedge to construct the airway branch topology hypergraph.

[0045] Accordingly, step 202 of the embodiment specifically includes: in the airway branch attribute graph, finding the airway branch nodes corresponding to each extrapulmonary airway branch, determining them as parent branch nodes, and storing them in a pre-created double-ended queue; from the first end of the double-ended queue, obtaining each parent branch node one by one as the target parent branch node; adding the target parent branch node to the hypergraph, identifying all direct child branch nodes of the target parent branch node in the airway branch attribute graph, and establishing a target hyperedge, wherein the target hyperedge contains the target parent branch node and all direct child branch nodes; counting the number of direct child branch nodes contained in the target parent branch node, and when the number exceeds a preset threshold for the number of child nodes, storing each direct child branch node from the second end of the double-ended queue into the double-ended queue as a parent branch node for hyperedge establishment; integrating each target hyperedge to obtain the airway branch topology hypergraph.

[0046] The double-ended queue consists of a first end and a second end. The first end can store and retrieve parent branch nodes, while the second end can only store parent branch nodes.

[0047] It should be noted that if the number of direct child branches contained in the target parent branch node exceeds the preset threshold for the number of child branches, these direct child branches will be regarded as terminal units and no new exploration tasks will be triggered, thus eliminating the need for a double-ended queue.

[0048] 203. Input the airway branch attribute map into the airway branch node feature depth extraction sub-model to perform nonlinear transformation processing on the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map to obtain the depth features corresponding to each airway branch node.

[0049] Accordingly, step 203 of the embodiment specifically includes: in the airway branch node feature depth extraction sub-model, the multivariate feature function corresponding to the airway branch attribute map is decomposed into a combination of multiple univariate feature functions by using the radial basis function approximation of the third-order B-spline basis function algorithm; based on the combination of multiple univariate feature functions, the basic features of each airway branch node and the lung lobe category features contained in the airway branch attribute map are subjected to nonlinear transformation processing to obtain the depth features corresponding to each airway branch node.

[0050] In this embodiment, the radial basis function approximation algorithm for a third-order B-spline basis function can be expressed as the following formula. , in, Represents multivariate characteristic functions, Indicates airway branches The Middle One characteristic, Indicates the inner unary function. Indicates the output dimension. This indicates inner summation. Indicates the outer unary function. This indicates summation of the outermost layer.

[0051] Furthermore, the following formula is a concrete mathematical implementation of the radial basis function approximation of the third-order B-spline basis function: , , in, Represents a univariate characteristic function ( or ), Indicates the number of radial basis functions. Indicates the first The learnable weights corresponding to each radial basis function Indicates the first The center point of each radial basis function Indicates distance, Represents radial basis functions. Indicates distance (i.e.) ), This represents the smoothing parameter.

[0052] 204. Input each depth feature and the airway branch topology hypergraph into the local structure feature extraction sub-model to aggregate the local topology neighborhood information of each airway branch node within its respective hyperedge, and obtain the depth features of each airway branch node after fusing local structure features.

[0053] Accordingly, step 204 of the embodiment specifically includes: in the local structural feature extraction sub-model, multiplying each depth feature with the airway branch topology hypergraph to obtain the aggregated features of each hyperedge contained in the airway branch topology hypergraph; calculating the interaction features between each airway branch node and its corresponding neighboring nodes within the hyperedge to obtain the interaction features of each hyperedge; fusing the aggregated features and interaction features of each hyperedge to obtain the hypergraph features of the airway branch topology hypergraph; concatenating each depth feature with the corresponding hypergraph features and performing similarity coefficient filtering to obtain the local topological neighborhood information of each airway branch node within its respective hyperedge; and adding each local topological neighborhood information with the corresponding depth feature to obtain the depth feature of each airway branch node after fusing the local structural features.

[0054] In this embodiment of the application, the aggregation feature is represented as follows: , in, Indicates aggregation features, Representing depth features, This represents a hypergraph representing the topological structure of airway branches.

[0055] Interaction features are represented as follows: , in, Indicates interactive features, Indicates the neighbor node inside the superedge The set, Table 1 Characteristics of each airway branch node This represents the characteristics of the neighboring nodes within the superedge. This represents a sub-model for deep feature extraction of airway branch nodes.

[0056] Hypergraph features are represented as follows:

[0057] in, Represents hypergraph features. It can take the value 0.1.

[0058] The splicing process can be achieved based on the following formula: , in, This indicates splicing / joining. This indicates the features after splicing.

[0059] The similarity coefficient filtering process can be implemented based on the following formula: , in, Represents local topological neighborhood information. This indicates an index selection operation. Indicates the features after splicing Connecting information with hyperedge The similarity coefficient between them.

[0060] The deep features after fusing local structural features are represented as follows: , in, This represents the depth features after fusing local structural features.

[0061] 205. Input the airway branch attribute map into the global relation feature extraction sub-model to perform global relation feature extraction processing on each airway branch node contained in the airway branch attribute map, and obtain the global relation features corresponding to each airway branch node.

[0062] Accordingly, step 205 of the embodiment specifically includes: in the global relation feature extraction sub-model, the airway branch attribute map is converted into the first airway branch depth attribute vector, the second airway branch depth attribute vector, and the third airway branch depth attribute vector based on the first airway branch node feature depth extraction layer, the second airway branch node feature depth extraction layer, and the third airway branch node feature depth extraction layer, respectively. The first airway branch depth attribute vector and the second airway branch depth attribute vector are respectively subjected to the first normalization process to obtain the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector. Based on the first normalized first airway branch depth attribute vector, the first normalized second airway branch depth attribute vector, and the third airway branch depth attribute vector, calculate the attention weight between airway branch nodes. Based on a preset scaling factor, the attention weights between airway branch nodes are optimized to obtain the optimized attention weights between airway branch nodes. Based on the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector, an attention normalization denominator is constructed. Based on the attention normalization denominator, a second normalization process is performed on the attention weights between the optimized airway branch nodes to obtain the global relationship features corresponding to each airway branch node.

[0063] In this embodiment, the depth attribute vector of the first airway branch is represented as follows: , in, This represents the depth attribute vector of the first airway branch. This represents the feature depth extraction layer for the first airway branch node. This represents a diagram showing the attributes of airway branches.

[0064] The depth attribute vector of the second airway branch is represented as follows: , in, This represents the depth attribute vector of the second airway branch. This represents the feature depth extraction layer for the second airway branch node.

[0065] The depth attribute vector of the third airway branch is represented as follows: , in, This represents the depth attribute vector of the third airway branch. This represents the feature depth extraction layer for the third airway branch node.

[0066] The first normalized depth attribute vector of the first airway branch is represented as follows: , in, This represents the first normalized airway branch depth attribute vector.

[0067] The first normalized second airway branch depth attribute vector is represented as follows: , in, This represents the second airway branch depth attribute vector after the first normalization.

[0068] The attention weights between airway branch nodes can be calculated using the following formula. , , in, Indicates the weighted value. Indicates the number of airway branch nodes. This represents the attention weight between airway branch nodes.

[0069] The optimized attention weights between airway branch nodes are represented as follows: , in, This represents the optimized attention weights between airway branch nodes. This indicates the preset scaling factor.

[0070] The attention normalization denominator is represented as follows: , in, This represents the attention normalization denominator.

[0071] The global relational features are represented as follows: , in, This indicates global relational characteristics.

[0072] 206. Based on the feature fusion sub-model, the deep features and global relational features of the fused local structural features of each airway branch node are fused to obtain the fused features corresponding to each airway branch node.

[0073] The fusion process can be performed by addition.

[0074] 207. Input each fusion feature into the classification sub-model to obtain the probability value of each airway branch node belonging to each anatomical category, and determine the anatomical category corresponding to the maximum probability value as the anatomical category of the airway branch node.

[0075] This application provides a method for labeling lung airways. First, a CT scan image of the target lung is acquired, and an airway branch attribute map is generated based on the target lung CT scan image. The airway branch attribute map contains multiple airway branch nodes, which are used to represent airway branches and carry corresponding basic features and lung lobe category features. Second, in the airway branch attribute map, the airway branch nodes corresponding to each extrapulmonary airway branch are found, resulting in multiple parent branch nodes. The sets of multiple child branch nodes with the same parent branch node are determined as hyperedges to construct an airway branch topology hypergraph. Finally, based on a lung airway labeling model that has completed model training, the anatomical classification results of each airway branch are determined according to the airway branch attribute map and the airway branch topology hypergraph to complete lung airway labeling. The lung airway labeling model includes a sub-model for deep feature extraction of airway branch nodes, a sub-model for local structural feature extraction, a sub-model for global relational feature extraction, a feature fusion sub-model, and a classification sub-model. Compared with existing technologies, the embodiments of this application construct an airway branch attribute graph carrying rich node information and an airway branch topology hypergraph integrating comprehensive topological relationships. Based on a pre-constructed lung airway labeling model, node depth features, local structural features, and global relationship features are extracted from the airway branch attribute graph and the airway branch topology hypergraph and fused together. This results in higher accuracy of node anatomical categories generated based on the fused features, thereby effectively improving the accuracy of airway labeling. Furthermore, by incorporating the prior knowledge of lung lobe category features into the node information, the generalization ability and robustness of the lung airway labeling model are improved.

[0076] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this application provides a lung airway marking device, such as... Figure 3 As shown, the device includes: Airway branch attribute graph generation module 31, airway branch topology hypergraph construction module 32, lung airway marking module 33; The airway branch attribute map generation module 31 is used to acquire a target lung CT scan image and generate an airway branch attribute map based on the target lung CT scan image. The airway branch attribute map contains multiple airway branch nodes, which are used to characterize airway branches and carry corresponding basic features and lung lobe category features. The airway branch topology hypergraph construction module 32 is used to find the airway branch nodes corresponding to each extrapulmonary airway branch in the airway branch attribute graph, obtain multiple parent branch nodes, and determine the set of multiple child branch nodes with the same parent branch node as the hyperedge to construct the airway branch topology hypergraph. The lung airway labeling module 33 is used to determine the anatomical classification results of each airway branch based on the airway branch attribute map and the airway branch topology hypergraph, based on the lung airway labeling model that has been trained, so as to complete the lung airway labeling. The lung airway labeling model includes a deep feature extraction sub-model for airway branch nodes, a local structural feature extraction sub-model, a global relation feature extraction sub-model, a feature fusion sub-model, and a classification sub-model.

[0077] In specific application scenarios, the lung airway marking module includes: The deep feature extraction unit is used to input the airway branch attribute map into the airway branch node feature deep extraction sub-model, so as to perform nonlinear transformation processing on the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map to obtain the deep features corresponding to each airway branch node. The local structural feature extraction unit is used to input each of the depth features and the airway branch topology hypergraph into the local structural feature extraction sub-model, so as to aggregate the local topological neighborhood information of each airway branch node in its respective hyperedge and obtain the depth features after fusing local structural features corresponding to each airway branch node. The global relation feature extraction unit is used to input the airway branch attribute map into the global relation feature extraction sub-model to perform global relation feature extraction processing on each airway branch node contained in the airway branch attribute map, so as to obtain the global relation features corresponding to each airway branch node. The feature fusion unit is used to fuse the depth features and global relationship features of the fused local structural features of each airway branch node based on the feature fusion sub-model, so as to obtain the fused features corresponding to each airway branch node. The classification unit is used to input each of the fused features into the classification sub-model to obtain the probability value of each airway branch node belonging to each anatomical category, and to determine the anatomical category corresponding to the maximum probability value as the anatomical category of the airway branch node.

[0078] In specific application scenarios, the deep feature extraction unit is specifically used for: In the airway branch node feature depth extraction sub-model, the multivariate feature function corresponding to the airway branch attribute map is decomposed into a combination of multiple univariate feature functions by using the radial basis function approximation algorithm of the third-order B-spline basis function; Based on the combination of multiple univariate feature functions, the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map are subjected to nonlinear transformation processing to obtain the depth features corresponding to each airway branch node.

[0079] In specific application scenarios, the local structural feature extraction unit is specifically used for: In the local structural feature extraction sub-model, each of the depth features is multiplied with the airway branch topology hypergraph to obtain the aggregated features of each hyperedge contained in the airway branch topology hypergraph. The interaction features between each airway branch node and its corresponding superedge neighbor node are calculated to obtain the interaction features of each superedge. The aggregation features and interaction features of each hyperedge are fused to obtain the hypergraph features of the airway branch topology hypergraph. Each of the depth features and the corresponding hypergraph features are concatenated and similarity coefficients are filtered to obtain the local topological neighborhood information of each airway branch node within its respective hyperedge. Each of the local topological neighborhood information is added to its corresponding depth feature to obtain the depth feature after fusing local structural features for each of the airway branch nodes.

[0080] In specific application scenarios, the global relation feature extraction unit is specifically used for: In the global relation feature extraction sub-model, the airway branch attribute map is converted into the first airway branch depth attribute vector, the second airway branch depth attribute vector, and the third airway branch depth attribute vector based on the first airway branch node feature depth extraction layer, the second airway branch node feature depth extraction layer, and the third airway branch node feature depth extraction layer, respectively. The first airway branch depth attribute vector and the second airway branch depth attribute vector are respectively subjected to a first normalization process to obtain the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector. Based on the first normalized first airway branch depth attribute vector, the first normalized second airway branch depth attribute vector, and the third airway branch depth attribute vector, calculate the attention weight between airway branch nodes. Based on a preset scaling factor, the attention weights between the airway branch nodes are optimized to obtain the optimized attention weights between the airway branch nodes. Based on the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector, an attention normalization denominator is constructed. Based on the attention normalization denominator, the attention weights between the optimized airway branch nodes are subjected to a second normalization process to obtain the global relationship features corresponding to each airway branch node.

[0081] In specific application scenarios, the airway branch attribute map generation module is specifically used for: Acquire CT scan images of the target lungs; A three-dimensional airway tree structure model is extracted from the target lung CT scan image, and the three-dimensional airway tree structure model is skeletonized to obtain a two-dimensional airway tree structure diagram. Each airway branch is identified from the two-dimensional structure diagram of the airway tree, and the basic features of each airway branch are calculated respectively. The basic features include topological features, spatial features and geometric features. Based on the spatial features of each airway branch and the lung lobe segmentation results of the target lung CT scan image, the lung lobe category features of each airway branch are determined. Each airway branch is abstracted into an airway branch node to generate an airway branch attribute map, wherein the airway branch node carries basic features and lung lobe category features.

[0082] In specific application scenarios, the airway branch topology hypergraph construction module is specifically used for: In the airway branch attribute graph, find the airway branch node corresponding to each extrapulmonary airway branch, determine it as the parent branch node, and store it in a pre-created double-ended queue; From the first end of the double-ended queue, obtain each of the parent branch nodes one by one as the target parent branch node; Add the target parent branch node to the hypergraph. In the airway branch attribute graph, identify all direct child branch nodes of the target parent branch node and establish a target hyperedge, wherein the target hyperedge contains the target parent branch node and all direct child branch nodes. The number of direct child branch nodes contained in the target parent branch node is counted, and when the number exceeds a preset threshold for the number of child nodes, each direct child branch node is stored from the second end of the double-ended queue into the double-ended queue to serve as a parent branch node for super-edge establishment. By integrating the hyperedges of each target, a hypergraph of the airway branch topology is obtained.

[0083] This application provides a lung airway labeling device. First, a CT scan image of the target lung is acquired, and an airway branch attribute map is generated based on the target lung CT scan image. The airway branch attribute map contains multiple airway branch nodes, which are used to represent airway branches and carry corresponding basic features and lung lobe category features. Second, in the airway branch attribute map, the airway branch nodes corresponding to each extrapulmonary airway branch are found, resulting in multiple parent branch nodes. The sets of multiple child branch nodes with the same parent branch node are determined as hyperedges to construct an airway branch topology hypergraph. Finally, based on a lung airway labeling model that has completed model training, the anatomical classification results of each airway branch are determined according to the airway branch attribute map and the airway branch topology hypergraph to complete lung airway labeling. The lung airway labeling model includes a sub-model for deep feature extraction of airway branch nodes, a sub-model for local structural feature extraction, a sub-model for global relational feature extraction, a feature fusion sub-model, and a classification sub-model. Compared with existing technologies, the embodiments of this application construct an airway branch attribute graph carrying rich node information and an airway branch topology hypergraph integrating comprehensive topological relationships. Based on a pre-constructed lung airway labeling model, node depth features, local structural features, and global relationship features are extracted from the airway branch attribute graph and the airway branch topology hypergraph and fused together. This results in higher accuracy of node anatomical categories generated based on the fused features, thereby effectively improving the accuracy of airway labeling. Furthermore, by incorporating the prior knowledge of lung lobe category features into the node information, the generalization ability and robustness of the lung airway labeling model are improved.

[0084] According to one embodiment of this application, a storage medium is provided, the storage medium storing at least one executable instruction that can execute the lung airway marking method in any of the above method embodiments.

[0085] Based on this understanding, the technical solution of this application can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0086] Figure 4 The diagram shows a structural schematic of a terminal according to one embodiment of the present application. The specific embodiments of the present application do not limit the specific implementation of the terminal.

[0087] like Figure 4As shown, the terminal may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0088] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0089] Communication interface 404 is used to communicate with other network elements such as clients or other servers.

[0090] The processor 402 is used to execute program 410, specifically to perform the relevant steps in the above-described lung airway marking method embodiment.

[0091] Specifically, program 410 may include program code that includes computer operation instructions.

[0092] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The computer device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0093] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0094] Specifically, program 410 can be used to cause processor 402 to perform the following operations: Acquire a CT scan image of the target lung and generate an airway branch attribute map based on the target lung CT scan image. The airway branch attribute map contains multiple airway branch nodes, which are used to characterize airway branches and carry corresponding basic features and lung lobe category features. In the airway branch attribute graph, the airway branch nodes corresponding to each extrapulmonary airway branch are found, multiple parent branch nodes are obtained, and the sets of multiple child branch nodes with the same parent branch node are determined as hyperedges to construct the airway branch topology hypergraph. Based on the lung airway labeling model that has completed model training, the anatomical classification results of each airway branch are determined according to the airway branch attribute map and the airway branch topology hypergraph to complete the lung airway labeling. The lung airway labeling model includes a deep feature extraction sub-model for airway branch nodes, a local structural feature extraction sub-model, a global relational feature extraction sub-model, a feature fusion sub-model, and a classification sub-model.

[0095] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device for the aforementioned lung airway labeling method, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0096] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0097] The methods and systems of this application may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this application are not limited to the order specifically described above, unless otherwise specifically stated. Furthermore, in some embodiments, this application may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this application. Thus, this application also covers recording media storing programs for performing the methods according to this application.

[0098] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0099] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for marking lung airways, characterized in that, include: Acquire a CT scan image of the target lung and generate an airway branch attribute map based on the target lung CT scan image. The airway branch attribute map contains multiple airway branch nodes, which are used to characterize airway branches and carry corresponding basic features and lung lobe category features. In the airway branch attribute graph, the airway branch nodes corresponding to each extrapulmonary airway branch are found, multiple parent branch nodes are obtained, and the sets of multiple child branch nodes with the same parent branch node are determined as hyperedges to construct the airway branch topology hypergraph. Based on the lung airway labeling model that has completed model training, the anatomical classification results of each airway branch are determined according to the airway branch attribute map and the airway branch topology hypergraph to complete the lung airway labeling. The lung airway labeling model includes a deep feature extraction sub-model for airway branch nodes, a local structural feature extraction sub-model, a global relational feature extraction sub-model, a feature fusion sub-model, and a classification sub-model.

2. The method according to claim 1, characterized in that, The lung airway labeling model, based on the completed model training, determines the anatomical classification results of each airway branch according to the airway branch attribute map and the airway branch topology hypergraph, including: The airway branch attribute map is input into the airway branch node feature depth extraction sub-model to perform nonlinear transformation processing on the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map, so as to obtain the depth features corresponding to each airway branch node. Each of the depth features and the airway branch topology hypergraph are input into the local structure feature extraction sub-model to aggregate the local topology neighborhood information of each airway branch node within its respective hyperedge, thereby obtaining the depth features of each airway branch node after fusing local structure features. The airway branch attribute map is input into the global relation feature extraction sub-model to perform global relation feature extraction processing on each airway branch node contained in the airway branch attribute map, so as to obtain the global relation features corresponding to each airway branch node. Based on the feature fusion sub-model, the depth features and global relationship features of the fused local structural features of each airway branch node are fused to obtain the fused features corresponding to each airway branch node. Each of the fused features is input into the classification sub-model to obtain the probability value of each airway branch node belonging to each anatomical category, and the anatomical category corresponding to the maximum probability value is determined as the anatomical category of the airway branch node.

3. The method according to claim 2, characterized in that, The step of inputting the airway branch attribute map into the airway branch node feature depth extraction sub-model, and performing nonlinear transformation processing on the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map, to obtain the depth features corresponding to each airway branch node, including: In the airway branch node feature depth extraction sub-model, the multivariate feature function corresponding to the airway branch attribute map is decomposed into a combination of multiple univariate feature functions by using the radial basis function approximation algorithm of the third-order B-spline basis function; Based on the combination of multiple univariate feature functions, the basic features and lung lobe category features of each airway branch node contained in the airway branch attribute map are subjected to nonlinear transformation processing to obtain the depth features corresponding to each airway branch node.

4. The method according to claim 2, characterized in that, The step of inputting each of the depth features and the airway branch topology hypergraph into the local structure feature extraction sub-model to aggregate the local topological neighborhood information of each airway branch node within its respective hyperedge, and obtaining the depth features fused with the local structure features corresponding to each airway branch node, includes: In the local structural feature extraction sub-model, each of the depth features is multiplied with the airway branch topology hypergraph to obtain the aggregated features of each hyperedge contained in the airway branch topology hypergraph. The interaction features between each airway branch node and its corresponding superedge neighbor node are calculated to obtain the interaction features of each superedge. The aggregation features and interaction features of each hyperedge are fused to obtain the hypergraph features of the airway branch topology hypergraph. Each of the depth features and the corresponding hypergraph features are concatenated and similarity coefficients are filtered to obtain the local topological neighborhood information of each airway branch node within its respective hyperedge. Each of the local topological neighborhood information is added to its corresponding depth feature to obtain the depth feature after fusing local structural features for each of the airway branch nodes.

5. The method according to claim 2, characterized in that, The step of inputting the airway branch attribute map into the global relation feature extraction sub-model to perform global relation feature extraction processing on each airway branch node contained in the airway branch attribute map, and obtaining the global relation features corresponding to each airway branch node, includes: In the global relation feature extraction sub-model, the airway branch attribute map is converted into the first airway branch depth attribute vector, the second airway branch depth attribute vector, and the third airway branch depth attribute vector based on the first airway branch node feature depth extraction layer, the second airway branch node feature depth extraction layer, and the third airway branch node feature depth extraction layer, respectively. The first airway branch depth attribute vector and the second airway branch depth attribute vector are respectively subjected to a first normalization process to obtain the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector. Based on the first normalized first airway branch depth attribute vector, the first normalized second airway branch depth attribute vector, and the third airway branch depth attribute vector, calculate the attention weight between airway branch nodes. Based on a preset scaling factor, the attention weights between the airway branch nodes are optimized to obtain the optimized attention weights between the airway branch nodes. Based on the first normalized first airway branch depth attribute vector and the first normalized second airway branch depth attribute vector, an attention normalization denominator is constructed. Based on the attention normalization denominator, the attention weights between the optimized airway branch nodes are subjected to a second normalization process to obtain the global relationship features corresponding to each airway branch node.

6. The method according to claim 1, characterized in that, The step of acquiring a target lung CT scan image and generating an airway branch attribute map based on the target lung CT scan image includes: Acquire CT scan images of the target lungs; A three-dimensional airway tree structure model is extracted from the target lung CT scan image, and the three-dimensional airway tree structure model is skeletonized to obtain a two-dimensional airway tree structure diagram. Each airway branch is identified from the two-dimensional structure diagram of the airway tree, and the basic features of each airway branch are calculated respectively. The basic features include topological features, spatial features and geometric features. Based on the spatial features of each airway branch and the lung lobe segmentation results of the target lung CT scan image, the lung lobe category features of each airway branch are determined. Each airway branch is abstracted into an airway branch node to generate an airway branch attribute map, wherein the airway branch node carries basic features and lung lobe category features.

7. The method according to claim 1, characterized in that, In the airway branch attribute graph, the airway branch nodes corresponding to each extrapulmonary airway branch are found to obtain multiple parent branch nodes, and the sets of multiple child branch nodes with the same parent branch node are determined as hyperedges to construct an airway branch topology hypergraph, including: In the airway branch attribute graph, find the airway branch node corresponding to each extrapulmonary airway branch, determine it as the parent branch node, and store it in a pre-created double-ended queue; From the first end of the double-ended queue, obtain each of the parent branch nodes one by one as the target parent branch node; Add the target parent branch node to the hypergraph. In the airway branch attribute graph, identify all direct child branch nodes of the target parent branch node and establish a target hyperedge, wherein the target hyperedge contains the target parent branch node and all direct child branch nodes. The number of direct child branch nodes contained in the target parent branch node is counted, and when the number exceeds a preset threshold for the number of child nodes, each direct child branch node is stored from the second end of the double-ended queue into the double-ended queue to serve as a parent branch node for super-edge establishment. By integrating the hyperedges of each target, a hypergraph of the airway branch topology is obtained.

8. A marking device for the lung airways, characterized in that, include: The airway branch attribute map generation module is used to acquire a target lung CT scan image and generate an airway branch attribute map based on the target lung CT scan image. The airway branch attribute map contains multiple airway branch nodes, which are used to characterize airway branches and carry corresponding basic features and lung lobe category features. The airway branch topology hypergraph construction module is used to find the airway branch nodes corresponding to each extrapulmonary airway branch in the airway branch attribute graph, obtain multiple parent branch nodes, and determine the set of multiple child branch nodes with the same parent branch node as the hyperedge to construct the airway branch topology hypergraph. The lung airway labeling module is used to determine the anatomical classification results of each airway branch based on the airway branch attribute map and the airway branch topology hypergraph, based on the lung airway labeling model that has been trained, so as to complete the lung airway labeling. The lung airway labeling model includes a deep feature extraction sub-model for airway branch nodes, a local structural feature extraction sub-model, a global relation feature extraction sub-model, a feature fusion sub-model, and a classification sub-model.

9. A storage medium storing at least one executable instruction, characterized in that, The executable instructions cause the processor to perform operations corresponding to the lung airway marking method as described in any one of claims 1-7.

10. A terminal, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, characterized in that the executable instruction causes the processor to perform an operation corresponding to the lung airway marking method as described in any one of claims 1-7.

Images