Ultrasonic cardiogram video semi-supervised segmentation method and system based on transform domain space-time tensor low-rank network

Abstract

The invention discloses an echocardiogram video semi-supervised segmentation method and system based on a transform domain space-time tensor low-rank network, and relates to the technical field of ultrasonic video data processing. The invention aims to solve the problem of left ventricle segmentation in an ultrasonic video. According to the technical key points, the method comprises the following steps: (1) preprocessing collected echocardiogram video data; (2) establishing an echocardiogram video semi-supervised segmentation model; (3) constructing an adaptive space-time tensor semantic perception module; (4) constructing a memory flow module; (5) training the network by using the existing data to obtain a network model; and (6) segmenting the echocardiogram video by using the trained model. According to the method, the left ventricle in the echocardiogram video can be automatically segmented by using fewer label training models under the condition of partial labeling of the video data, and the segmentation processing time is greatly shortened.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasound video data processing, and in particular to a method and system for semi-supervised segmentation of ultrasound cardiogram video based on transform domain spatiotemporal tensor low-rank network. Background Art

[0002] Ultrasound imaging[1] is a non-invasive medical imaging technology that uses the physical properties of ultrasound and the reflection characteristics of different tissues to obtain information about the internal tissue structure and function of the human body. Compared with computed tomography, magnetic resonance imaging and other technologies, ultrasound imaging is fast and has high temporal resolution. It can observe the structure, function and movement of organs and tissues in real time. It is widely used in the diagnosis of heart disease and has broad application prospects. However, traditional ultrasound examinations require doctors to manually outline the contours of the patient's ventricles, myocardium and other tissues for subsequent parameter calculations, which are subject to subjective differences and errors, and are time-consuming and labor-intensive, affecting the efficiency and accuracy of clinical diagnosis and treatment[2].

[0003] To this end, researchers have proposed many automatic echocardiographic tissue segmentation algorithms. Traditional model-based algorithms usually have high computational complexity, slow segmentation speed and low accuracy[3]. Existing deep learning-based algorithms mostly process echocardiographic video data in a frame-by-frame segmentation manner[4]. They do not utilize rich temporal information and perform poorly in video segmentation tasks. In recent years, some researchers have begun to focus on how to use the temporal correlation of cardiac motion to improve the accuracy of echocardiographic video segmentation.

[0004] In addition, most methods use methods such as optical flow or inter-frame attention [5], and rarely consider the inherent characteristics of echocardiographic video data. Therefore, it is necessary to design a segmentation method that utilizes the spatiotemporal correlation of echocardiographic videos to improve the accuracy and efficiency of cardiac tissue segmentation. Summary of the invention

[0005] The technical problems to be solved by the present invention are:

[0006] The purpose of the present invention is to provide a method and system for semi-supervised segmentation of ultrasound cardiogram video based on transform domain spatiotemporal tensor low-rank network, in order to solve the problem of automatic segmentation of left ventricle in ultrasound video.

[0007] The technical solution adopted by the present invention to solve the above problems is:

[0008] (1) Preprocessing the collected echocardiographic video data;

[0009] (2) Establish a semi-supervised segmentation model for echocardiography videos;

[0010] (3) Construct an adaptive spatiotemporal tensor semantic perception module;

[0011] (4) Construct memory flow module;

[0012] (5) Use existing data to train the network and obtain a network model;

[0013] (6) Use the trained model to segment echocardiography videos.

[0014] Furthermore, in the above step (1), ultrasound imaging equipment is used to collect echocardiographic data, and the data is preprocessed, including unifying the data size, randomly augmenting the data, normalizing and standardizing the data, and sampling the data.

[0015] Specifically, the Cubic interpolation method is used to unify the echocardiogram frames and labels to 256×256, and the data is randomly flipped horizontally, randomly scaled, randomly rotated, randomly contrasted, and gamma enhanced to avoid overfitting. The data is normalized to [0, 1] using the maximum-minimum value, as shown in the following formula:

[0016]

[0017] Among them, x is the original data, x max and x min are the maximum and minimum values ​​of the data respectively.

[0018] Use z-score standardization to transform the data into a standard normal distribution with a mean of 0 and a standard deviation of 1, as shown in the following formula:

[0019]

[0020] Among them, μ is the mean of the data and σ is the standard deviation of the data.

[0021] In each sequence, we sampled a total of 10 evenly distributed frames, starting from the diastolic frame and ending with the systolic frame.

[0022] Furthermore, the segmentation model construction in the above step (2) can be carried out according to the following steps:

[0023] Given an echocardiogram video of size H×W×T, where H and W represent the spatial resolution of each frame and T is the number of frames in the video, a shared encoder is used to extract semantic features of each frame t Where h and w are the spatial resolution of the feature, and C is the number of channels. Then, F t Stacking to get a high-dimensional spatiotemporal feature tensor And feed it into the proposed Adaptive Spatiotemporal Tensor Semantic Awareness (ASTSA) module. For the first frame, the output feature F1′ of the ASTSA module is sent to the shared decoder to generate the prediction M1 of the segmentation result. Starting from the second frame, in order to finely handle the frame-by-frame changes of dynamic objects, this paper uses the Memory Flow (MF) module to transfer valuable information between adjacent frames. Specifically, the feature F1 of the previous frame and the current frame is t-1 , F t , F t ′ and the prediction M of the previous frame segmentation result t-1 As the input of the MF module, the output features of MF are sent to the decoder to obtain the prediction of the segmentation result of the current frame.

[0024] In addition, in the encoder, a multi-scale fusion module (MSF) module is used to integrate the features of the last two scales. Specifically, the features of each scale are divided into multiple groups along the channel dimension, and convolutional layers with different receptive fields (e.g., 1×1, 3×3, and 5×5) are used to fully fuse the information in these features.

[0025] Furthermore, in the above step (3), an adaptive spatiotemporal tensor semantic perception module is constructed, which mainly uses the transform domain tensor singular value decomposition algorithm to extract the video feature tensor χ in The main semantic components in . First, the definition of the transform domain tensor is given: given a three-dimensional tensor χ in Indicates the nth forward slice Given a transformation and the corresponding inverse transform Can get and

[0026] definition Given a tensor Tensor and transformation It can be expressed as follows,

[0027]

[0028] Here, the operator × represents standard matrix multiplication, Representing a tensor The forward slicing block diagonal matrix of is defined as follows,

[0029]

[0030] Fold represents the expansion operator. Re-transformed into a tensor, as shown below,

[0031]

[0032] According to formulas (15), (16) and (17), we can obtain:

[0033]

[0034] This means Through and Each forward slice of is obtained by performing a two-dimensional matrix multiplication operation in the transform domain.

[0035] For a three-dimensional tensor The adaptive tensor singular value decomposition can be expressed as follows,

[0036]

[0037] in, and is an orthogonal tensor. is a diagonal tensor, each forward slice of which is a diagonal matrix.

[0038] According to formulas (18) and (19), we can get: At this point, the adaptive tensor singular value decomposition can be performed by computing the tensor in the transform domain The singular value decomposition of each forward slice matrix is ​​conveniently obtained. In SAMF-Net, the semantic features extracted from each frame are stacked into a high-dimensional feature tensor in the order of the time dimension. And the adaptive tensor singular value threshold algorithm is applied in the transform domain for processing. The specific calculation is as follows:

[0039]

[0040] Where σ is the singular value threshold,

[0041] To perceive spatiotemporal semantics, this method uses 3D convolution to learn the transformation based on echocardiography video data. and inverse transform The singular value threshold σ of each slice is set as a parameter that can be automatically learned in the network. The singular values ​​of the tensor are filtered using the learned optimal threshold to retain the most important semantic components and obtain a low-rank approximate representation of the feature tensor.

[0042] It is expressed as follows,

[0043]

[0044] in, is a composite operator. Finally, the fusion module is further used to fuse the low-rank approximate feature tensors and the input feature tensor Get the final output

[0045] Furthermore, in step (4), a memory flow module is constructed, given the segmented frame feature F of the previous frame t-1 and the prediction mask M t-1 , and the feature F of the current frame to be segmented t First, generate Key (K) to calculate the pixel-level affinity coefficient, and Value (V) to store the detailed semantic information for segmentation. Then use the MaxPool operator to pool K t-1 ,K t and V t-1 Downsampling to different scales, we get and At each scale l, the affinity coefficient is calculated separately as shown in the following formula:

[0046]

[0047] in, yes and The normalized affinity coefficient at the h,j-th pixel position on the l-th scale, c(·,·) is the simplified L2 similarity.

[0048] Based on the affinity matrix A l ,from The effective features can be obtained from

[0049]

[0050] In order to obtain all the Finally, the adaptive aggregation weights are calculated to fuse them and obtain the output features of the final MF module.

[0051] Furthermore, in the above step (5), the preprocessed data set is used for training to obtain optimal network parameters for the subsequent left ventricular tissue segmentation of the echocardiography video data; to ensure the segmentation performance of the network, the data set used contains 1000 sets of echocardiography video data.

[0052] Furthermore, in the above step (6), the collected data is input into the trained network to obtain the corresponding segmentation result. The process can be expressed as:

[0053] y=f net (x) (24)

[0054] Among them, f netrepresents the constructed echocardiography video segmentation network, x and y represent the input echocardiography video data and the output left ventricle segmentation result, respectively.

[0055] The present invention has the following beneficial technical effects:

[0056] The present invention comprises the following steps: (1) preprocessing the collected ultrasound video data; (2) establishing a semi-supervised segmentation model for ultrasound video; (3) constructing an adaptive spatiotemporal tensor semantic perception module; (4) constructing a memory stream module; (5) training a network using existing data to obtain a network model; (6) segmenting the ultrasound video using the trained model. The present invention can automatically segment the left ventricle in the ultrasound video using a training model with fewer labels under the condition that the video data is partially labeled, and greatly shorten the processing time of segmentation, thereby effectively solving the problem of left ventricle segmentation in ultrasound video.

[0057] The present invention is a method for semi-supervised segmentation of an echocardiogram video based on a transform domain spatiotemporal tensor low-rank network. The method for semi-supervised segmentation of an echocardiogram video based on a transform domain spatiotemporal tensor low-rank constraint provided by the present invention can automatically segment the left ventricular tissue area of ​​the heart from echocardiogram video data. The algorithm proposed by the present invention has high computational efficiency and can be conveniently accelerated using a GPU, and is an efficient algorithm. Experiments show that compared with the existing optical flow-based method, the present invention can significantly improve the segmentation accuracy of the video, and achieve the purpose of rapid and automatic segmentation of the echocardiogram video. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The figure is a flow chart of the method of the present invention.

[0059] Figure 2 Schematic diagram of the overall structure of the network proposed in the present invention (schematic diagram of the overall structure of the method for semi-supervised segmentation of echocardiogram video based on transform domain spatiotemporal tensor low-rank network proposed in the present invention);

[0060] Figure 3 This is a schematic diagram of the structure of the adaptive spatiotemporal tensor semantic perception module;

[0061] Figure 4 This is a schematic diagram of the memory stream module structure;

[0062] Figure 5 It is a schematic diagram of the structure of the multi-scale fusion module;

[0063] Figure 6 A comparison diagram of the segmentation results of the method of the present invention and other currently advanced methods; DETAILED DESCRIPTION

[0064] The present invention is described in detail below with reference to the accompanying drawings and examples.

[0065] The process flow chart of the present invention is shown in Figure 1 , the specific implementation steps are as follows:

[0066] (1) Preprocessing the collected echocardiographic video data;

[0067] (2) Establish a semi-supervised segmentation model for echocardiography videos;

[0068] (3) Construct an adaptive spatiotemporal tensor semantic perception module;

[0069] (4) Construct memory flow module;

[0070] (5) Use existing data to train the network and obtain a network model;

[0071] (6) Use the trained model to segment echocardiography videos.

[0072] The above step (1) is performed as follows:

[0073] 3. Collect echocardiographic data using ultrasound imaging equipment and preprocess the data, including unifying the data size, randomly augmenting the data, normalizing and standardizing the data, and sampling the data.

[0074] Specifically, we use the Cubic interpolation method to unify the echocardiogram frames and labels to 256×256, and perform random horizontal flipping, random scaling, random rotation, random contrast, and gamma enhancement on the data to avoid overfitting. The data is normalized to [0, 1] using the maximum-minimum value, as shown in the following formula:

[0075]

[0076] Among them, x is the original data, x max and x min are the maximum and minimum values ​​of the data respectively.

[0077] Use z-score standardization to transform the data into a standard normal distribution with a mean of 0 and a standard deviation of 1, as shown in the following formula:

[0078]

[0079] Among them, μ is the mean of the data and σ is the standard deviation of the data.

[0080] In each sequence, we sampled a total of 10 evenly distributed frames, starting from the diastolic frame and ending with the systolic frame.

[0081] The above step (2) is performed as follows:

[0082] Given an echocardiogram video of size H×W×T, where H and W represent the spatial resolution of each frame and T is the number of frames in the video, a shared encoder is used to extract semantic features of each frame t Where h and w are the spatial resolution of the feature, and C is the number of channels. Then, F t Stacking to get a high-dimensional spatiotemporal feature tensor And feed it into the proposed Adaptive Spatiotemporal Tensor Semantic Awareness (ASTSA) module. For the first frame, the output feature F1′ of the ASTSA module is sent to the shared decoder to generate the prediction M1 of the segmentation result. Starting from the second frame, in order to finely handle the frame-by-frame changes of dynamic objects, this paper uses the Memory Flow (MF) module to transfer valuable information between adjacent frames. Specifically, the feature F1 of the previous frame and the current frame is t-1 , F t , F t ′ and the prediction M of the previous frame segmentation result t-1 As the input of the MF module, the output features of MF are sent to the decoder to obtain the prediction of the segmentation result of the current frame.

[0083] In addition, in the encoder, a multi-scale fusion module (MSF) module is used to integrate the features of the last two scales. Specifically, the features of each scale are divided into multiple groups along the channel dimension, and convolutional layers with different receptive fields (e.g., 1×1, 3×3, and 5×5) are used to fully fuse the information in these features.

[0084] The above step (3) is performed as follows:

[0085] In the ASTSA module, the transform domain tensor singular value decomposition algorithm is mainly used to extract the video feature tensor χ in The main semantic components in . First, the definition of the transform domain tensor is given: given a three-dimensional tensor χ in Indicates the nth forward slice Given a transformation and the corresponding inverse transform Can get and

[0086] definition Given a tensor Tensor and transformation It can be expressed as follows,

[0087]

[0088] Here, the operator × represents standard matrix multiplication, Representing a tensor The forward slicing block diagonal matrix of is defined as follows,

[0089]

[0090] Fold represents the expansion operator. Re-transformed into a tensor, as shown below,

[0091]

[0092] According to formulas (27), (28) and (29), we can get:

[0093]

[0094] This means Through and Each forward slice of is obtained by performing a two-dimensional matrix multiplication operation in the transform domain.

[0095] For a three-dimensional tensor The adaptive tensor singular value decomposition can be expressed as follows,

[0096]

[0097] in, and is an orthogonal tensor. is a diagonal tensor, each forward slice of which is a diagonal matrix.

[0098] According to formulas (30) and (31), we can get: At this point, the adaptive tensor singular value decomposition can be performed by computing the tensor in the transform domain The singular value decomposition of each forward slice matrix is ​​conveniently obtained. In SAMF-Net, the semantic features extracted from each frame are stacked into a high-dimensional feature tensor in the order of the time dimension. And the adaptive tensor singular value threshold algorithm is applied in the transform domain for processing. The specific calculation is as follows:

[0099]

[0100] Where σ is the singular value threshold,

[0101] To perceive spatiotemporal semantics, this method uses 3D convolution to learn the transformation based on echocardiography video data. and inverse transform The singular value threshold σ of each slice is set as a parameter that can be automatically learned in the network. The singular values ​​of the tensor are filtered using the learned optimal threshold to retain the most important semantic components and obtain a low-rank approximate representation of the feature tensor. It is expressed as follows,

[0102]

[0103] in, is a composite operator. Finally, the fusion module is further used to fuse the low-rank approximate feature tensors and the input feature tensor Get the final output

[0104] The above step (4) is performed as follows:

[0105] Given the segmentation frame feature F of the previous frame t-1 and the prediction mask M t-1 , and the feature F of the current frame to be segmented t First, generate Key (K) to calculate the pixel-level affinity coefficient, and Value (V) to store the detailed semantic information for segmentation. Then use the MaxPool operator to pool K t-1 ,K t and V t-1 Downsampling to different scales, we get and At each scale l, the affinity coefficient is calculated separately as shown in the following formula:

[0106]

[0107] in, yes and The normalized affinity coefficient at the h,j-th pixel position on the l-th scale, c(·,·) is the simplified L2 similarity.

[0108] Based on the affinity matrix A l ,from The effective features can be obtained from

[0109]

[0110] In order to obtain all the Finally, the adaptive aggregation weights are calculated to fuse them and obtain the output features of the final MF module.

[0111] The above step (5) is performed as follows:

[0112] The constructed network can be trained using the preprocessed data set to obtain the best network parameters for the subsequent left ventricular tissue segmentation of echocardiography video data; to ensure the segmentation performance of the network, the data set used contains 1000 sets of echocardiography video data.

[0113] The above step (6) is performed as follows:

[0114] By inputting the collected data into the trained network, the corresponding segmentation results can be obtained. The process can be expressed as:

[0115] y=f net (x) (36)

[0116] Among them, f net represents the constructed echocardiography video segmentation network, x and y represent the input echocardiography video data and the output left ventricle segmentation result, respectively.

[0117] In order to quantitatively evaluate the performance of the method proposed in this invention, three quantitative evaluation indicators, namely Dice Similarity Coefficient (DC), Sensitivity (Sen) and Average Surface Distance (ASD), were used in the experiment to evaluate the effect of echocardiography video segmentation. The Dice Similarity Coefficient is defined as follows:

[0118]

[0119] Among them, G is the given true value label and P is the prediction result of the network. Sen is defined as follows:

[0120]

[0121] Among them, TP represents the number of correctly segmented pixels belonging to the segmentation target, and FN represents the number of pixels belonging to the segmentation target but incorrectly segmented. The average surface distance is calculated as follows:

[0122]

[0123] Where M is the number of rows in the image, N is the number of columns in the image, (i,j) represents the coordinates of the foreground pixel in the ground-truth label. When calculating d(x,y), the minimum distance is found by traversing the foreground pixels in G. When computing d′(x,y), the minimum distance is found by traversing the foreground pixels in P.

[0124] Figure 6The comparison diagram of the segmentation results of the method of the present invention and other currently advanced methods is shown. From left to right, the echocardiogram frame, label, and the segmentation results of DAEF, LV-Net, SSCF, CLAS, and the method proposed by the present invention are shown. It can be seen from the figure that the most accurate left ventricle segmentation result can be obtained by using the method proposed by the present invention for echocardiogram segmentation, which is closest to the shape of the label.

[0125] Table 1 Quantitative comparison results of left ventricle segmentation by different methods on the test dataset

[0126]

[0127] Table 1 lists the quantitative comparison results of the segmentation of the method of the present invention and other currently advanced methods, including Dice similarity coefficient, sensitivity Sen and average surface distance ASD. The quantitative comparison results show that compared with the existing most advanced methods, the present invention can significantly improve the accuracy of echocardiographic segmentation.

[0128] The references cited in the present invention are listed as follows:

[0129] [1]Antico, Maria, et al. "Ultrasound guidance in minimally invasiverobotic procedures."

[0130] Medical image analysis 54(2019):149-167.

[0131] [2] Leclerc, Sarah, et al. "Deep learning for segmentation using an openlarge-scaledataset in 2Dechocardiography." IEEE transactions on medical imaging 38.9 (2019): 2198-2210.

[0132] [3]Bosch, Johan G., et al. "Automatic segmentation of echocardiographic sequences by activeappearance motion models." IEEE transactions on medical imaging 21.11(2002):1374-1383.

[0134] [4]Awasthi,Navchetan,et al."LVNet:Lightweight model for leftventricle segmentation forshort axis views in echocardiographic imaging."IEEETransactions on Ultrasonics,

[0135] Ferroelectrics,and Frequency Control 69.6(2022):2115-2128.

[0136] [5]Wei,Hongrong,et al."Co-learning of appearance and shape forprecise ejection fractionestimation from echocardiographic sequences."MedicalImage Analysis 84(2023):102686.

Claims

1. A semi-supervised segmentation method for echocardiography video based on transform domain spatiotemporal tensor low-rank network, characterized in that: The method improves the accuracy of left ventricle segmentation by using the spatiotemporal correlation of echocardiographic video features, the method comprising the following steps: (1) Preprocessing the collected echocardiographic video data; (2) Establish a semi-supervised segmentation model for echocardiography videos; (3) Construct an adaptive spatiotemporal tensor semantic perception module; (4) Construct memory flow module; (5) Use existing data to train the network and obtain a network model; (6) Use the trained model to segment echocardiography videos.

2. The method according to claim 1, characterized in that In step (1), ultrasound imaging equipment is used to collect echocardiographic data, and the data is preprocessed, including unifying the data size, randomly augmenting the data, normalizing and standardizing the data, and sampling the data. The specific process is: using the Cubic interpolation method to unify the image frames and labels of the echocardiogram to 256×256, and performing random horizontal flipping, random scaling, random rotation, random contrast, and gamma enhancement on the data to avoid overfitting; the data is normalized to [0, 1] using the maximum-minimum value, as shown in the following formula: Among them, x is the original data, x max and x min are the maximum and minimum values ​​of the data respectively; Use z-score normalization to convert the image into a standard normal distribution with a mean of 0 and a standard deviation of 1, as shown in the following formula: Among them, μ is the mean of the data, σ is the standard deviation of the data; In each sequence, a total of 10 evenly distributed frames were sampled, starting from the diastolic frame and ending with the systolic frame.

3. The method according to claim 1 or 2, characterized in that The step (2) is carried out as follows: Given an echocardiogram video of size H×W×T, where H and W represent the spatial resolution of each frame and T is the number of frames in the video, a shared encoder is used to extract semantic features of each frame t Where h and w are the spatial resolutions of the features, and C is the number of channels; then F t Stacking to get a high-dimensional spatiotemporal feature tensor And send it to the proposed Adaptive Spatiotemporal Tensor Semantic Awareness (ASTSA) module; for the first frame, the output feature F1′ of the ASTSA module is sent to the shared decoder to generate the prediction M1 of the segmentation result; starting from the second frame, the memory flow (MF) module is used to transfer valuable information between adjacent frames to handle the frame-by-frame changes of dynamic targets, specifically: the features F1′ of the previous frame and the current frame are combined into a single feature vector. t-1 , F t , F t ′ and the prediction M of the previous frame segmentation result t-1 As the input of the MF module, the output features of MF are sent to the decoder to obtain the prediction of the segmentation result of the current frame; In the encoder, a multi-scale fusion module (MSF) module is used to integrate the features of the last two scales, divide the features of each scale into multiple groups along the channel dimension, and use convolutional layers with different receptive fields to fully fuse the information in these features.

4. The method according to claim 3, characterized in that The step (3) is carried out as follows: In the ASTSA module, the transform domain tensor singular value decomposition algorithm is mainly used to extract the video feature tensor χ in The main semantic components in the transform domain tensor are: χ in Indicates the nth forward slice Given a transformation and the corresponding inverse transform Can get and definition Given a tensor Tensor and transformation It can be expressed as follows, Here, the operator × represents standard matrix multiplication, Representing a tensor The forward slicing block diagonal matrix of is defined as follows, Fold represents the expansion operator. Re-transformed into a tensor, as shown below, According to formulas (3), (4) and (5), we can get: This means Through and Each forward slice of is obtained by performing a two-dimensional matrix multiplication operation in the transform domain; For a three-dimensional tensor The adaptive tensor singular value decomposition can be expressed as follows, in, and is an orthogonal tensor; is a diagonal tensor, each forward slice of which is a diagonal matrix; According to formulas (6) and (7), we can get At this point, the adaptive tensor singular value decomposition can be performed by computing the tensor in the transform domain The singular value decomposition of each forward slice matrix of is conveniently obtained; In SAMF-Net, the semantic features extracted from each frame are stacked into a high-dimensional feature tensor in the order of the time dimension. And the adaptive tensor singular value threshold algorithm is applied in the transform domain for processing; the specific calculation is as follows: Where σ is the singular value threshold, Using 3D convolution to learn transformations based on echocardiography video data for spatiotemporal semantics and inverse transform The singular value threshold σ of each slice is set as a parameter that can be automatically learned in the network; the singular values ​​of the tensor are filtered using the learned optimal threshold to retain the most important semantic components and obtain a low-rank approximate representation of the feature tensor It is expressed as follows, Among them, ° is a composite operator; finally, the fusion module is further used to fuse the low-rank approximate feature tensor and the input feature tensor Get the final output 5. The method according to claim 4, characterized in that The step (4) is performed as follows: Given the segmentation frame feature F of the previous frame t-1 and the prediction mask M t-1 , and the feature F of the current frame to be segmented t First, generate Key (K) to calculate the pixel-level affinity coefficient, and Value (V) to store the detailed semantic information for segmentation; then use the MaxPool operator to pool K t-1 ,K t and V t-1 Downsampling to different scales, we get and At each scale l, the affinity coefficient is calculated separately as shown in the following formula: in, yes and The normalized affinity coefficient at the h,jth pixel position on the lth scale, c(·,·) is the simplified L2 similarity; Based on the affinity matrix A l ,from The effective features can be obtained from In order to obtain all the Finally, the adaptive aggregation weights are calculated to fuse them and obtain the output feature F of the final MF module. t out .

6. The method according to claim 5, characterized in that The step (5) is performed as follows: The constructed network can be trained using the preprocessed data set to obtain the best network parameters for the subsequent left ventricular tissue segmentation of echocardiography video data; to ensure the segmentation performance of the network, the data set used contains 1000 sets of echocardiography video data.

7. The method according to claims 2 and 6, characterized in that The step (6) is performed as follows: By inputting the collected data into the trained network, the corresponding segmentation results can be obtained. The process can be expressed as: y=f net (x) (12) Among them, f net represents the constructed echocardiography video segmentation network, x and y represent the input echocardiography video data and the output left ventricle segmentation result, respectively.

8. A semi-supervised segmentation system for echocardiography video based on transform domain spatiotemporal tensor low-rank network, characterized by: The system has a program module corresponding to the steps of any one of claims 1 to 7, and executes the steps in the above-mentioned semi-supervised segmentation method of echocardiography video based on low-rank constraints of transform domain spatiotemporal tensor when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of a semi-supervised segmentation method for echocardiography video based on a transform domain spatiotemporal tensor low-rank network as described in any one of claims 1 to 7 when called by a processor.

Images