Ultrasound four-dimensional reconstruction method and system based on graph convolution network embedding alignment fusion

Abstract

The ultrasound four-dimensional reconstruction method and system based on graph convolution network embedding alignment fusion can avoid the interference of abnormal values while ensuring the accurate extraction of the respiratory signal, and improve the accuracy of the alignment and matching of the slice sequence respiratory signals between different positions. The method comprises the following steps: (1) collecting 2D slice sequences of different parts of a target organ of a patient; (2) constructing a respiratory motion related graph; (3) embedding and aligning the respiratory motion graph; and (4) matching 4D reconstruction slices.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to an ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion, and an ultrasound four-dimensional reconstruction system based on graph convolutional network embedding alignment fusion. Background Technology

[0002] In recent years, four-dimensional (4D) medical imaging has received widespread attention due to its ability to provide three-dimensional (3D) anatomical information on how a patient's organs change with motion. A 4D image is a three-dimensional image sequence (3D+t) with a time / motion dimension, which can better assist doctors in understanding the patterns of organ movement. Currently, the mainstream 4D medical imaging technology is two-dimensional (2D) slice stacking reconstruction technology. This technology stacks 2D slices extracted from different anatomical locations based on the similarity of respiratory motion states to reconstruct 3D volume data under multiple motion states. Matching slices with similar respiratory motion states is the key to 4D reconstruction technology based on slice stacking.

[0003] 4D reconstruction techniques based on slice stacking often utilize respiratory signals to match slices with similar respiratory motion states. Respiratory signals are numerical representations of the respiratory state of a slice; the more similar the slice motion states, the closer the respiratory signal values. Respiratory signals can be obtained by tracking external markers or by using algorithms to track feature points within the slice. However, the accuracy of tracking affects the matching of respiratory signals. Furthermore, a limited number of markers or feature points cannot accurately represent the respiratory state.

[0004] The aforementioned problems can be addressed by estimating respiratory signals through learning the slice distribution in a low-dimensional manifold space. Manifold learning methods such as ISOMAP and Laplacian Eigenmaps estimate respiratory signals by embedding slices into a low-dimensional manifold space. However, since 2D slice sequences at different locations reside in different manifold spaces, the respiratory signals obtained from dimensionality reduction of each sequence need to be aligned for matching. Existing manifold alignment processes perform one-to-one alignment of all samples between manifolds, and anomalous samples introduced by uneven breathing reduce the accuracy of the alignment. Furthermore, traditional manifold learning methods reduce slices with similar grayscale information to near-similarity positions, leading to mismatches between inspiratory and expiratory slices in the respiratory signal. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the technical problem to be solved by this invention is to provide an ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion, which can ensure accurate extraction of respiratory signals while avoiding interference from outliers and improving the accuracy of respiratory signal alignment and matching between slice sequences at different locations.

[0006] The technical solution of this invention is: a method for four-dimensional ultrasound reconstruction based on graph convolutional network embedding alignment fusion, which includes the following steps:

[0007] (1) Collect 2D slice sequences of different parts of the patient's target organ;

[0008] (2) Each 2D slice sequence is converted into a graph data containing respiratory motion features. Each slice in the sequence is regarded as a node in the graph data, and the one-dimensional expansion of the slice gray value is regarded as the high-dimensional feature of the node. Based on the prior knowledge that nodes with similar respiratory states have similar high-dimensional features, connections are established through feature similarity between nodes. In order to avoid misconnection between nodes in the inhalation and exhalation stages, each node is classified according to its respiratory state and a subgraph is established. Finally, the subgraphs are merged to complete the construction of the respiratory motion related graph.

[0009] (3) A graph alignment model based on embedding is constructed using a graph convolutional neural network to extract and align respiratory signals from respiratory motion maps at adjacent acquisition locations. The alignment process is trained using a hybrid self-supervised and semi-supervised approach, with only a subset of nodes used to supervise the alignment process. The alignment model combines graph alignment and manifold embedding to design a loss function suitable for respiratory motion map alignment and embedding. During training, embedding and alignment are combined to obtain accurate alignment results while also making the embedded signals have respiratory motion characteristics.

[0010] (4) Use the aligned signal to match slices with similar breathing states at different locations. Find slices with similar breathing states in adjacent locations by searching for the signal with the smallest distance in the aligned signal. In addition to calculating the distance between the aligned breathing signals, the phase ratio difference between the slices is also introduced during the matching process to suppress breathing phase drift.

[0011] The above steps are repeated position by position. Finally, a set of slices with similar respiratory signals are collected position by position and used for stacking and reconstruction of 3D volume data. The above slice matching steps are repeated for multiple respiratory phases to obtain the final 4D image sequence.

[0012] This invention represents 2D slice sequences at different locations as respiratory motion correlation maps, transforming the four-dimensional reconstruction problem into an alignment problem between different maps. Graph alignment is achieved using an embedding-based alignment model built from graph convolutional networks, and a sparse alignment strategy is introduced. While ensuring accurate extraction of respiratory signals, it can avoid interference from outliers and improve the accuracy of respiratory signal alignment and matching between slice sequences at different locations.

[0013] A system for ultrasound four-dimensional reconstruction based on graph convolutional network embedding alignment fusion is also provided, which includes:

[0014] The acquisition module is configured to acquire 2D slice sequences of different parts of the patient's target organ;

[0015] The respiratory motion correlation map construction module is configured to convert each 2D slice sequence into a graph data containing respiratory motion features. Each slice in the sequence is regarded as a node in the graph data, and the one-dimensional expansion of the slice grayscale value is regarded as the high-dimensional feature of the node. Based on the prior knowledge that nodes with similar respiratory states have similar high-dimensional features, connections are established between nodes through feature similarity. To avoid misconnection between nodes in the inspiratory and expiratory phases, each node is classified according to its respiratory state and a subgraph is built for each node. Finally, the subgraphs are merged to complete the construction of the respiratory motion correlation map.

[0016] The respiratory motion map embedding and alignment module is configured to use a graph convolutional neural network to construct an embedding-based graph alignment model to extract and align respiratory signals from respiratory motion maps at adjacent acquisition locations. The alignment process is trained using a hybrid self-supervised and semi-supervised approach, with only a subset of nodes used for supervision. The alignment model combines graph alignment and manifold embedding concepts to design a loss function suitable for respiratory motion map alignment and embedding. During training, embedding and alignment are combined to obtain accurate alignment results while also ensuring that the embedded signals possess respiratory motion characteristics.

[0017] The 4D reconstruction slice matching module is configured to match slices with similar breathing states at different locations using aligned signals. It finds slices with similar breathing states in adjacent locations by searching for the signal with the smallest distance among the aligned signals. During the slice matching process, in addition to calculating the distance between aligned breathing signals, it also introduces the phase ratio difference between slices to suppress breathing phase drift. The above steps are repeated position by position, and finally, a set of slices with similar breathing signals are collected position by position and used for stacking and reconstructing 3D volume data.

[0018] The above slice matching steps are repeated for multiple respiratory phases to obtain the final 4D image sequence. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the ultrasonic four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to the present invention.

[0020] Figure 2 This is a flowchart illustrating step (3) of the ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to the present invention.

[0021] Figure 3 This is a flowchart illustrating step (4) of the ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to the present invention. Detailed Implementation

[0022] like Figure 1 As shown, this ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion includes the following steps:

[0023] (1) Collect 2D slice sequences of different parts of the patient's target organ;

[0024] (2) Each 2D slice sequence is transformed into a graph data containing respiratory motion features. Each slice in the sequence is regarded as a node in the graph data, and the one-dimensional expansion of the slice gray value is regarded as the high-dimensional feature of the node. Based on the prior knowledge that nodes with similar respiratory states have similar high-dimensional features, connections are established between nodes through feature similarity.

[0025] Next, to avoid misconnections between nodes in the inhalation and exhalation phases, each node is classified according to its respiratory state, and subgraphs are created for each node. Finally, the subgraphs are merged to complete the construction of the respiratory motion correlation graph.

[0026] (3) A graph alignment model based on embedding is constructed using a graph convolutional neural network to extract and align respiratory signals from respiratory motion maps at adjacent acquisition locations. The alignment process is trained using a hybrid self-supervised and semi-supervised approach, with only a subset of nodes used to supervise the alignment process. The alignment model combines graph alignment and manifold embedding to design a loss function suitable for respiratory motion map alignment and embedding. During training, embedding and alignment are combined to obtain accurate alignment results while also making the embedded signals have respiratory motion characteristics.

[0027] (4) Use the aligned signal to match slices with similar respiratory states at different locations.

[0028] The matching process involves finding slices with similar breathing states in adjacent locations by searching for the signal with the smallest distance among the alignment signals. In addition to calculating the distance between the aligned breathing signals, the phase ratio difference between the slices is also introduced during the matching process to suppress breathing phase drift. The above steps are repeated position by position. Finally, a set of slices with similar breathing signals are collected position by position and used for stacking and reconstructing 3D volume data. The above slice matching steps are repeated for multiple breathing phases to obtain the final 4D image sequence.

[0029] This invention represents 2D slice sequences at different locations as respiratory motion correlation maps, transforming the four-dimensional reconstruction problem into an alignment problem between different maps. Graph alignment is achieved using an embedding-based alignment model built from graph convolutional networks, and a sparse alignment strategy is introduced. While ensuring accurate extraction of respiratory signals, it can avoid interference from outliers and improve the accuracy of respiratory signal alignment and matching between slice sequences at different locations.

[0030] Preferably, in step (1), 2D slice sequences of different parts of the target organ are acquired under the condition of free breathing of the patient. Each sequence contains 3 to 4 respiratory cycles, and the slice planes of each sequence are kept parallel. This module is completed using a medical image acquisition device. Assume that a total of N sets of 2D slice sequences are acquired, and each set of sequences has M frames. The m-th frame at position n is denoted as... m = 1, ..., M n , n=1,…,N.

[0031] Preferably, in step (2), a respiratory motion correlation map is constructed for each group of 2D slice sequences, and each slice in the sequence is... Viewed as nodes in graph data The one-dimensional expansion of the slice grayscale values ​​is used as the high-dimensional feature of the node. Compared to the nearest neighbor graph that is simply built by using the similarity of node features, the respiratory motion graph nodes are only connected to nodes related to their own respiratory state, thus expressing the lag of breathing and avoiding misconnection between inhalation and exhalation nodes due to similar features.

[0032] Preferably, step (2) includes the following sub-steps:

[0033] (2.1) Treat the slices as nodes in the respiratory correlation graph and form a node set. After appropriate downsampling and grayscale normalization, the slices are expanded into high-dimensional vectors in one dimension to form the feature matrix. To avoid misconnections between inhalation and exhalation nodes in the diagram, the nodes are divided into four categories: inhalation phase nodes, exhalation phase nodes, end-inhalation nodes, and end-exhalation nodes.

[0034] (2.2) Construct the nearest neighbor graph for each of the four categories of nodes;

[0035] (2.3) The above four sub-graphs are merged to obtain the respiratory motion correlation graph used in this method;

[0036] (2.4) Calculate the weights of the edges of the respiratory motion correlation graph using the features of each node, and set the feature similarity between nodes as the edge weights. The similarity is defined as follows:

[0037]

[0038] in, and They are nodes and Node characteristics, For nodes k g The nearest neighbor set, σ gIndicates the strength of the adjacency relationships in the graph.

[0039] The respiratory motion correlation diagram constructed from the position n sequence is denoted as... The respiratory motion diagrams at all locations constitute a set.

[0040] Preferably, in step (2.2), the subgraph of the inhalation / exhalation phase nodes is constructed as follows: first, each node in the inhalation / exhalation category is associated with k nodes in the set consisting of inhalation / exhalation phase nodes, end-inhalation nodes, and end-exhalation nodes. g The nearest neighbor nodes are connected; then only the connections between nodes in the inspiratory / expiratory phases are retained to form the inspiratory / expiratory phase subgraph; simultaneously, the end-inspiratory / expiratory subgraph connects each node in the end-inspiratory / expiratory category with k nodes in the set consisting of inspiratory phase nodes, end-inspiratory / expiratory nodes, and expiratory phase nodes. g Connect the nearest neighbor nodes to form the end-inspiratory / end-expiratory subgraph.

[0041] Preferably, in step (3), a pseudo-twin network is formed by using two structurally identical but independent respiratory signal embedding models to realize respiratory motion maps at adjacent locations. and Respiratory signal extraction and alignment between them and The number of nodes may be the same or different.

[0042] Preferably, such as Figure 2 As shown, step (3) includes the following sub-steps:

[0043] (3.1) A two-layer graph convolutional network is used to build a graph respiratory signal embedding model to extract respiratory signals. The input features of the first layer are the feature matrix X. n The output of the second graph convolutional layer is used as a breathing signal embedded in two-dimensional space. For the nodes in the graph Graph convolutional layers integrate the features of neighboring nodes into the node's features during the learning process. Generate new features, defined as follows:

[0044]

[0045] Where σ is the activation function, and ReLU is used in this paper; This represents the adjacency matrix for adding self-joins, where I is the identity matrix. H (n,l) and H (n,l+1) W represents the node features output from layer l and layer (l+1), respectively. (n,l) This is the weight matrix for node feature transformation;

[0046] The embedding is supervised using a contrastive learning loss function, which aims to make nodes with similar or identical breathing states as close as possible in the two-dimensional space, and nodes with different breathing states as far apart as possible. The specific definition of the embedding loss function is as follows:

[0047]

[0048] in Indicates respiratory signal and The Euclidean distance between nodes, if nodes It is a node Neighbors Otherwise The constant margin > 0 is a marginal value that limits the node embedding distance;

[0049] (3.2) Two structurally identical but independent respiratory signal embedding models are combined to form a pseudo-twin network as a graph alignment model, and input respectively. and The node feature matrix and adjacency matrix are used to extract and align respiratory signals between adjacent graphs;

[0050] (3.3) Obtain matching node pairs for supervision during graph alignment model training. Assume that slices with similar breathing states at adjacent locations have similar gray levels, and use this as a basis to find matching pairs between nodes. Given the node feature matrices X of two graphs... n and X n+1 Set nodes and nodes The similarity between them is:

[0051]

[0052] By pairwise combining nodes from different graphs and calculating their similarity using the formula above, a fully connected similarity matrix is ​​obtained. Then, the Hungarian algorithm is used to sparse it to obtain one-to-one matching node pairs;

[0053] (3.4) Removing outlier nodes from matched node pairs: First, the node pairs sparsed by the Hungarian algorithm are sorted from high to low similarity; then, during alignment training, only the top t% of node pairs are retained, and mismatches between inhalation and exhalation caused by slice similarity are also removed from the top t% of matched pairs; finally, the one-to-one matching between nodes is expanded to many-to-one matching to increase the training samples, i.e., node... Not only with Perform a match, and also with the node. In the figure k in matchMatch each neighbor node, and denote the final set of all matched node pairs as a collection.

[0054] (3.5) The embedding-based graph alignment model is trained, and contrastive learning is also used for alignment training. The alignment loss function is defined as follows:

[0055]

[0056] in, Respiratory signals and The Euclidean distance between them To align the tags, define them as follows:

[0057]

[0058] Preferably, in step (3), in order to preserve the original manifold of the two respiratory motion maps during the alignment process, the embedding loss function and the alignment loss function are weighted and summed to obtain the final loss function used for training the respiratory signal alignment model:

[0059]

[0060] Where μ1 and μ2 are the alignment loss weight and embedding loss weight, respectively.

[0061] Preferably, such as Figure 3 As shown, step (4) includes the following sub-steps:

[0062] (4.1) Select the respiratory state t to be reconstructed, and select a location n with significant respiratory motion as a reference location;

[0063] (4.2) Establish k based on the dimensionality-reduced respiratory signal at position n r -Nearest neighbor graph g n Meanwhile, in Figure g n Add connections on the timeline to connect nodes to their preceding and following neighbors on the timeline, g n All edge weights are 1; select the last node of the air intake process. and end-expiratory node As a reference node, to calculate The phase ratio is calculated as follows:

[0064]

[0065] in, For nodes and In Figure g n The shortest path length in the, For nodes and In Figure g n The shortest path lengths in the table are calculated using Dijkstra's algorithm, with the node at reference position n being used as the reference. The phase ratio is denoted as p reference ;

[0066] (4.3) Respiratory signals Calculate the distance to the respiratory signals of nodes in adjacent positions n+1, and only calculate the distance between them in adjacent positions. The signal distance between nodes at the same respiratory stage; if If it originates from the intake node, then only the following calculations are performed. Distance between respiratory signals from all inspiratory phase nodes in n+1;

[0067] (4.4) Along the timeline of the slice extraction order, find the position n+1 that is related to... The nodes with the local minimum distance form a set. Where m * express The index in its slice sequence, for each slice in the set. Expand it into a subset along the time axis

[0068] (4.5) Form a candidate slice set from all extended subsets, and calculate the phase ratio for all slices in the candidate slice set. The phase ratio is calculated in the same way as in step (4.2);

[0069] (4.6) The phase ratio of all slices in the candidate slice set is compared with p respectively. reference The comparison is performed, and then the matching slice is selected by searching the optional set for the slice that minimizes the distance d′. d′ is defined as follows:

[0070]

[0071] Where d e Respiratory signals and The Euclidean distance between them, where α represents the weight of the phase ratio difference term, used to balance the phase ratio difference value with the Euclidean distance value of the low-dimensional features;

[0072] (4.7) A set of slices They are collected position by position, starting from reference position n, and stacked. Reconstruct the three-dimensional volume of the respiratory state t.

[0073] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium. When executed, the program includes the steps of the methods of the above embodiments. The storage medium can be ROM / RAM, magnetic disk, optical disk, memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also includes an ultrasound four-dimensional reconstruction system based on graph convolutional network embedding alignment fusion. This system is typically represented in the form of functional modules corresponding to the steps of the method. The system includes:

[0074] The acquisition module is configured to acquire 2D slice sequences of different parts of the patient's target organ;

[0075] The respiratory motion correlation map construction module is configured to convert each 2D slice sequence into a graph data containing respiratory motion features. Each slice in the sequence is considered a node in the graph data, and the one-dimensional expansion of the slice's grayscale value is used as the high-dimensional feature of the node. Based on the prior knowledge that nodes with similar respiratory states have similar high-dimensional features, connections are established between nodes through feature similarity. To avoid misconnections between nodes in the inspiratory and expiratory phases, each node is classified according to its respiratory state, and subgraphs are created for each subgraph. Finally, the subgraphs are merged to complete the construction of the respiratory motion correlation map. The respiratory motion map embedding and alignment module...

[0076] The block is configured to use a graph convolutional neural network to construct an embedding-based graph alignment model to extract and align respiratory signals from respiratory motion maps at adjacent acquisition locations. The alignment process is trained using a hybrid self-supervised and semi-supervised approach, with only a subset of nodes used for supervision. The alignment model combines graph alignment and manifold embedding to design a loss function suitable for respiratory motion map alignment and embedding. During training, embedding and alignment are combined to obtain accurate alignment results while also ensuring that the embedded signals have respiratory motion characteristics.

[0077] The 4D reconstruction slice matching module is configured to match slices with similar breathing states at different locations using aligned signals. It finds slices with similar breathing states in adjacent locations by searching for the signal with the smallest distance among the aligned signals. During the slice matching process, in addition to calculating the distance between aligned breathing signals, it also introduces the phase ratio difference between slices to suppress breathing phase drift. The above steps are repeated position by position, and finally, a set of slices with similar breathing signals are collected position by position and used for stacking and reconstructing 3D volume data.

[0078] The above slice matching steps are repeated for multiple respiratory phases to obtain the final 4D image sequence.

[0079] Compared with existing intraoperative liver respiratory motion compensation systems based on ultrasound imaging, the advantages of this invention are:

[0080] 1. The 4D image reconstruction process is transformed into a graph matching process. The respiratory correlation between slices is modeled as graph data, and a graph convolutional network is used to learn the precise alignment between the graphs modeled at different locations.

[0081] 2. During the training of graph convolutional networks, the embedding and alignment processes are integrated, so that the two complement each other during training, achieving accurate alignment while preserving the breathing cycle features of the manifold;

[0082] 3. A hybrid strategy of self-supervised and semi-supervised learning is proposed, which can achieve sparse alignment between respiratory motion correlation maps and reduce the error caused by outliers in the maps.

[0083] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for four-dimensional ultrasound reconstruction based on graph convolutional network embedding alignment and fusion, characterized in that: It includes the following steps: (1) Collect 2D slice sequences of different parts of the patient's target organ; (2) Convert each 2D slice sequence into a graph data containing respiratory motion features, regard each slice in the sequence as a node in the graph data, and treat the one-dimensional expansion of the slice gray value as the high-dimensional feature of the node. Based on the prior knowledge that nodes with similar respiratory states have similar high-dimensional features, connections are established through feature similarity between nodes. To avoid misconnections between nodes in the inhalation and exhalation phases, each node is classified into respiratory states and subgraphs are created. Finally, the subgraphs are merged to complete the construction of the respiratory motion correlation graph. (3) A graph alignment model based on embedding is constructed using graph convolutional neural networks to extract and align respiratory signals from respiratory motion maps at adjacent acquisition locations. The alignment process is trained using a hybrid self-supervised and semi-supervised approach, and only some nodes are used to supervise the alignment process. The alignment model combines graph alignment and manifold embedding ideas to design a loss function suitable for respiratory motion graph alignment embedding. During training, embedding and alignment are combined to obtain accurate alignment results while also making the embedded signal have respiratory motion characteristics. (4) Match slices with similar breathing states between different positions using the aligned signals. Find slices with similar breathing states in adjacent positions by searching for the signal with the smallest distance in the aligned signals. In addition to calculating the distance between the aligned breathing signals, the phase ratio difference between the slices is also introduced during the matching process to suppress breathing phase drift. Repeat the above steps position by position. Finally, a set of slices with similar breathing signals are collected in sequence and used for stacking and reconstruction of 3D volume data. Repeat the above slice matching steps for multiple breathing phases to obtain the final 4D image sequence.

2. The ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to claim 1, characterized in that: In step (1), 2D slice sequences of different parts of the target organ are acquired under the patient's free breathing condition. Each sequence contains 3-4 respiratory cycles, and the slice planes of each sequence are kept parallel. This is done using a medical image acquisition device. Assume that a total of N sets of 2D slice sequences are acquired, and each set of sequences has M frames. The m-th frame at position n is denoted as... , , .

3. The ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to claim 2, characterized in that: In step (2), a respiratory motion correlation map is constructed for each group of 2D slice sequences, and each slice in the sequence is... Viewed as nodes in graph data The one-dimensional expansion of the slice grayscale values ​​is used as the high-dimensional feature of the node. Compared to the nearest neighbor graph that is simply built by using the similarity of node features, the respiratory motion graph nodes are only connected to nodes related to their own respiratory state, thus expressing the lag of breathing and avoiding misconnection between inhalation and exhalation nodes due to similar features.

4. The ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to claim 3, characterized in that: Step (2) includes the following sub-steps: (2.1) Treat the slices as nodes in the respiratory correlation graph and form a node set. After appropriate downsampling and grayscale normalization, the slices are expanded into high-dimensional vectors in one dimension to form the feature matrix. To avoid misconnections between inhalation and exhalation nodes in the diagram, the nodes are divided into four categories: inhalation phase nodes, exhalation phase nodes, end-inhalation nodes, and end-exhalation nodes. (2.2) Construct the nearest neighbor graph for each of the four categories of nodes; (2.3) The above four sub-graphs are merged to obtain the respiratory motion correlation graph used in this method; (2.4) Calculate the weights of the edges of the respiratory motion correlation graph using the features of each node, and set the feature similarity between nodes as the edge weights. The similarity is defined as follows: （1） in, and They are nodes and Node characteristics, For nodes of The set of nearest neighbor nodes, Indicates the strength of the adjacency relationships in the graph. The respiratory motion correlation diagram constructed from the position n sequence is denoted as... The respiratory motion diagrams at all locations constitute a set. .

5. The ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to claim 4, characterized in that: In step (2.2), the subgraph of the inhalation / exhalation phase nodes is constructed as follows: First, each node in the inhalation / exhalation category is associated with its set consisting of inhalation / exhalation phase nodes, end-inhalation nodes, and end-exhalation nodes. The nearest neighbor nodes are connected; then only the connections between nodes in the inspiratory / expiratory phases are retained to form the inspiratory / expiratory phase subgraph; simultaneously, the end-inspiratory / expiratory subgraph connects each node in the end-inspiratory / expiratory category with its corresponding node in the set consisting of inspiratory phase nodes, end-inspiratory / expiratory nodes, and expiratory phase nodes. Connect the nearest neighbor nodes to form the end-inspiratory / end-expiratory subgraph.

6. The ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to claim 5, characterized in that: In step (3), a pseudo-twin network is formed by using two structurally identical but independent respiratory signal embedding models to realize respiratory motion maps at adjacent locations. and Respiratory signal extraction and alignment between them and The number of nodes may be the same or different.

7. The ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to claim 6, characterized in that: Step (3) includes the following sub-steps: (3.1) A two-layer graph convolutional network is used to build a graph respiratory signal embedding model to extract respiratory signals. The input features of the first layer are the feature matrix. The output of the second graph convolutional layer is used as a breathing signal embedded in two-dimensional space. For the nodes in the graph The graph convolutional layer integrates the features of neighboring nodes into the node's features during the learning process. Generate new features, defined as follows: （2） in The activation function is ReLU; , This represents the adjacency matrix for adding self-joins, where I is the identity matrix. ; and These are the node features output from the l-th and (l+1)-th layers, respectively. This is the weight matrix for node feature transformation; The embedding is supervised using a contrastive learning loss function, which aims to make nodes with similar or identical breathing states as close as possible in the two-dimensional space, and nodes with different breathing states as far apart as possible. The specific definition of the embedding loss function is as follows: （3） in Indicates respiratory signal and The Euclidean distance between nodes, if nodes It is a node Neighbors ,otherwise ,constant >0 is a marginal value that limits the node embedding distance; (3.2) Two structurally identical but independent respiratory signal embedding models are combined to form a pseudo-twin network as a graph alignment model, and input respectively. and The node feature matrix and adjacency matrix are used to extract and align respiratory signals between adjacent graphs; (3.3) Obtain matching node pairs for supervision during graph alignment model training. Assuming that slices with similar breathing states at adjacent positions have similar gray levels, find matching pairs between nodes based on this. Given the node feature matrices of two graphs. and Set nodes and nodes The similarity between them is: （4） By pairwise combining nodes from different graphs and calculating their similarity using the formula above, a fully connected similarity matrix is ​​obtained. Then, the Hungarian algorithm is used to sparse it to obtain one-to-one matching node pairs; (3.4) Removing outlier nodes from matched node pairs: First, the node pairs sparsed by the Hungarian algorithm are sorted from high to low similarity; then, during alignment training, only the top t% of node pairs are retained, and mismatches between inhalation and exhalation caused by slice similarity are also removed from the top t% of matched pairs; finally, the one-to-one matching between nodes is expanded to many-to-one matching to increase the training samples, i.e., node... Not only with Perform a match, and also with the node. In the figure In Match each neighbor node, and denote the final set of all matched node pairs as a collection. ; (3.5) The embedding-based graph alignment model is trained, and contrastive learning is also used for alignment training. The alignment loss function is defined as follows: （5） in, Respiratory signals and The Euclidean distance between them To align the tags, define them as follows: （6）。 8. The ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to claim 7, characterized in that: In step (3), in order to preserve the original manifold of the two respiratory motion maps during the alignment process, the embedding loss function and the alignment loss function are weighted and summed to obtain the final loss function used for training the respiratory signal alignment model: （7） in and These are the alignment loss weights and the embedding loss weights, respectively.

9. The ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to claim 8, characterized in that: Step (4) includes the following sub-steps: (4.1) Select the respiratory state t to be reconstructed, and select a location n with significant respiratory motion as a reference location; (4.2) Establishment of respiratory signals after dimensionality reduction at position n -Nearest neighbor graph Meanwhile, in the figure Add connections along the timeline to link nodes to their preceding and following neighbors on the timeline. All edge weights are 1; select the last node of the air intake process. and end-expiratory node As a reference node The phase ratio is calculated as follows: （8） in, For nodes and In the figure The shortest path length in the, For nodes and In the figure The shortest path lengths in the table are calculated using Dijkstra's algorithm, with the node at reference position n being used as the reference. The phase ratio is denoted as ; (4.3) Respiratory signals Calculate the distance to the respiratory signals of nodes in adjacent positions n+1, and only calculate the distance between them in adjacent positions. The signal distance between nodes at the same respiratory stage; if If it originates from the intake node, then only the following calculations are performed. Distance between respiratory signals from all inspiratory phase nodes in n+1; (4.4) Locate the position along the timeline of slice extraction order. Zhongyu The nodes with the local minimum distance form a set. ,in express The index in its slice sequence, for each slice in the set. Expand it into subsets along the time axis ; (4.5) Form a candidate slice set from all extended subsets, and calculate the phase ratio for all slices in the candidate slice set. The phase ratio is calculated in the same way as in step (4.2); (4.6) Compare the phase ratios of all slices in the candidate slice set with... Compare, and then search the optional set for items that make the distance... The smallest slice is used to select the matching slice. , The definition is as follows: （9） in Respiratory signals and The Euclidean distance between them This represents the weight of the phase ratio difference term, used to balance the phase ratio difference value with the Euclidean distance value of the low-dimensional features; (4.7) A set of slices They are collected position by position, starting from reference position n, and stacked. Reconstruct the three-dimensional volume of the respiratory state t.

10. The system of the ultrasound four-dimensional reconstruction method based on graph convolutional network embedding alignment fusion according to claim 1, characterized in that: It includes: The acquisition module is configured to acquire 2D slice sequences of different parts of the patient's target organ; The respiratory motion correlation graph construction module is configured to convert each set of 2D slice sequences into a graph data containing respiratory motion features, treat each slice in the sequence as a node in the graph data, and treat the one-dimensional expansion of the slice grayscale value as the high-dimensional feature of the node. Based on the prior knowledge that nodes with similar respiratory states have similar high-dimensional features, connections are established through feature similarity between nodes. To avoid misconnections between nodes in the inhalation and exhalation phases, each node is classified into respiratory states and subgraphs are created. Finally, the subgraphs are merged to complete the construction of the respiratory motion correlation graph. The respiratory motion map embedding and alignment module is configured to use a graph convolutional neural network to construct an embedding-based graph alignment model to extract and align respiratory signals from respiratory motion maps at adjacent acquisition locations. The alignment process is trained using a hybrid self-supervised and semi-supervised approach, and only a subset of nodes are used to supervise the alignment process. The alignment model combines graph alignment and manifold embedding ideas to design a loss function suitable for respiratory motion graph alignment embedding. During training, embedding and alignment are combined to obtain accurate alignment results while also making the embedded signal have respiratory motion characteristics. The 4D reconstruction slice matching module is configured to match slices with similar breathing states at different locations using aligned signals. It finds slices with similar breathing states in adjacent locations by searching for the signal with the smallest distance among the aligned signals. In addition to calculating the distance between aligned breathing signals, the phase ratio difference between slices is also introduced during the slice matching process to suppress breathing phase drift. The above steps are repeated position by position. Finally, a set of slices with similar breathing signals are collected position by position and used for stacking and reconstruction of 3D volume data. The above slice matching steps are repeated for multiple breathing phases to obtain the final 4D image sequence.

Images