Lightweight ultrasonic medical dynamic image segmentation method based on Mama

Abstract

The invention discloses a lightweight ultrasonic medical dynamic image segmentation method based on Mama, which comprises the following steps: acquiring cardiac ultrasound image information, and analyzing and preprocessing the cardiac ultrasound image information to obtain a cardiac ultrasound dynamic image set; performing parallel processing on the cardiac hyperdynamic image set by using a deep convolutional neural network, and extracting a deep semantic feature set of the cardiac structure; performing parallel processing on the cardiac hyperdynamic image set by using a time sequence tracking module of the bidirectional state space model, and extracting a time sequence change feature set of the dynamic change of the heart; performing cross-modal feature fusion on the deep semantic feature set and the time sequence change feature set to obtain a unified fusion feature set; generating and outputting heart structure segmentation information according to the unified fusion feature set; accurate and comprehensive scientific data support is provided for medical diagnosis.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of medical image processing, and particularly relates to a lightweight medical dynamic ultrasound image segmentation method based on Mamba. BACKGROUND

[0002] Cardiac ultrasound (echocardiography) is a safe, non-invasive and widely used important imaging technology for cardiovascular disease diagnosis. Through echocardiography, doctors can evaluate the structure, function and blood flow dynamics of the heart. Heart dynamic image segmentation technology is crucial for early diagnosis and treatment of heart-related diseases.

[0003] However, the existing echocardiographic dynamic image segmentation technology cannot guarantee image segmentation accuracy while meeting the real-time requirements of the medical diagnosis process, resulting in poor echocardiographic dynamic image segmentation results and limited auxiliary effect on medical diagnosis. SUMMARY

[0004] The purpose of the application is to provide a lightweight medical dynamic ultrasound image segmentation method based on Mamba, which solves the problem that the existing technology cannot achieve a good balance between segmentation accuracy and segmentation speed, resulting in poor echocardiographic dynamic image segmentation results.

[0005] Technical scheme: The lightweight medical dynamic ultrasound image segmentation method based on Mamba comprises the following steps:

[0006] (1) Obtain echocardiographic image information and analyze and preprocess the echocardiographic image information to obtain an echocardiographic dynamic image set;

[0007] (2) Use a deep convolutional neural network to process the echocardiographic dynamic image set in parallel to extract a deep semantic feature set of the heart structure;

[0008] (3) Use a time series tracking module of a bidirectional state space model to process the echocardiographic dynamic image set in parallel to extract a time series change feature set of the heart dynamic change;

[0009] (4) Cross-modal feature fusion is performed on the deep semantic feature set and the time series change feature set to obtain a unified fusion feature set;

[0010] (5) According to the unified fusion feature set, generate and output the heart structure segmentation information.

[0011] Further, step (1) is specifically as follows: real-time frame extraction is performed on the echocardiogram information to obtain an ultrasound frame image set; denoising and normalization processing are performed on the ultrasound frame image set to obtain a normalized image set; image enhancement is performed on each frame image in the normalized image set, and a time sequence mark is added to constitute the echocardiogram dynamic image set.

[0012] Further, step (2) is specifically as follows: low-level local features of each image in the echocardiogram dynamic image set are extracted by using a shallow convolution parameter; high-level structure features are extracted by further abstracting the low-level local features by using a deep convolution parameter; the low-level local features and the high-level structure features are weighted and fused in a nonlinear manner to generate a fused convolution feature map of each image; and the fused convolution feature map of all images constitutes a deep semantic feature set.

[0013] Further, step (3) is specifically as follows: image feature vectors of each image in the echocardiogram dynamic image set at corresponding time points are extracted; based on a preset state transition matrix and the image feature vectors, in combination with state transition noise, a heart state feature vector at each time point is estimated; based on a preset observation matrix and the heart state feature vector, in combination with observation noise, an observation feature vector at each time point is obtained; a forward state feature reflecting historical and current states is calculated by using a forward recursion algorithm in combination with the heart state feature vector and the observation feature vector; a backward state feature reflecting future state influence is calculated by using a backward recursion algorithm in combination with the heart state feature vector and the observation feature vector; the forward state feature and the backward state feature are fused to obtain a time sequence state feature at each time point, and the time sequence state feature at all time points constitutes a time sequence change feature set.

[0014] Further, step (4) is specifically as follows: the features in the deep semantic feature set are mapped to a first unified feature space to obtain a unified semantic feature set; the features in the time sequence change feature set are mapped to a second unified feature space to obtain a unified time sequence feature set; the unified semantic feature set and the unified time sequence feature set are subjected to feature interaction and weighted mixing based on an attention mechanism; and the mixed features are mapped to a new target feature space to form a unified fusion feature set.

[0015] The lightweight echocardiogram dynamic image segmentation system based on Mamba provided by the application comprises:

[0016] An image extraction module is configured to acquire echocardiogram information, analyze and preprocess the echocardiogram information, and obtain an echocardiogram dynamic image set.

[0017] A semantic feature analysis module is configured to utilize a deep convolutional neural network to perform parallel processing on the echocardiogram dynamic image set and extract a deep semantic feature set of a heart structure.

[0018] The time sequence feature analysis module is configured to utilize a time sequence tracking module based on a bidirectional state space model to perform parallel processing on the echocardiogram dynamic image set and extract a time sequence change feature set of the dynamic change of the heart;

[0019] The feature fusion module is configured to perform cross-modal feature fusion on the deep semantic feature set and the time sequence change feature set to obtain a unified fusion feature set.

[0020] The output module is configured to generate and output the cardiac structure segmentation information according to the unified fusion feature set.

[0021] Further, the image extraction module is specifically configured as follows: real-time frame extraction is performed on the echocardiogram image information to obtain an ultrasound frame image set; denoising and normalization are performed on the ultrasound frame image set to obtain a normalized image set; image enhancement is performed on each frame of the normalized image set, and a time sequence label is added to form the echocardiogram dynamic image set.

[0022] Further, the semantic feature analysis module is specifically configured as follows: shallow convolution parameters are used to extract low-level local features of each image in the echocardiogram dynamic image set; deep convolution parameters are used to further abstract the low-level local features to extract high-level structure features; the low-level local features and the high-level structure features are fused by a nonlinear method to generate a fusion convolution feature map of each image; and the deep semantic feature set is formed by the fusion convolution feature maps of all images.

[0023] Further, the time sequence feature analysis module is specifically configured as follows: image feature vectors of each image in the echocardiogram dynamic image set at corresponding time points are extracted; based on a preset state transition matrix and the image feature vectors, in combination with state transition noise, a heart state feature vector at each time point is estimated; based on a preset observation matrix and the heart state feature vector, in combination with observation noise, an observation feature vector at each time point is obtained; a forward state feature reflecting historical and current states is calculated by using a forward recursion algorithm in combination with the heart state feature vector and the observation feature vector; a backward state feature reflecting future state influences is calculated by using a backward recursion algorithm in combination with the heart state feature vector and the observation feature vector; the forward state feature and the backward state feature are fused to obtain a time sequence state feature at each time point, and the time sequence change feature set is formed by the time sequence state features at all time points.

[0024] Further, the feature fusion module is specifically configured as follows: the features in the deep semantic feature set are mapped to a first unified feature space to obtain a unified semantic feature set; the features in the time sequence change feature set are mapped to a second unified feature space to obtain a unified time sequence feature set; feature interaction and weighted mixing are performed on the unified semantic feature set and the unified time sequence feature set based on an attention mechanism; and the mixed features are mapped to a new target feature space to form the unified fusion feature set.

[0025] Beneficial effects: Compared with the prior art, the present application has the following remarkable advantages: the present application effectively improves the segmentation efficiency of dynamic images of echocardiography by parallel processing of different models, realizes a good balance between segmentation accuracy and segmentation speed, and through feature fusion of the deep semantic feature set and the time sequence change feature set, the unified fusion features can reflect the dynamic changes of different dimensions of cardiac features in the cardiac cycle from two core aspects of semantic depth and time sequence breadth, and generate and output cardiac structure information accordingly, providing accurate and comprehensive scientific data support for medical diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 An application scenario diagram is provided for the present application.

[0027] Figure 2 A flowchart of the present application is provided. DETAILED DESCRIPTION

[0028] The technical solutions of the present application will be further described below in conjunction with the drawings.

[0029] The embodiment of the present application provides a lightweight ultrasonic medical dynamic image segmentation method based on Mamba, which comprises the following steps: Figure 1 An application scenario diagram is provided for the present application. In the process of segmenting dynamic images of echocardiography, the method provided by the present application is applied to realize a good balance between segmentation accuracy and segmentation speed. The method of the present application is applied to any server, which communicates with a cardiac ultrasound device, acquires and analyzes echocardiographic image information provided by the cardiac ultrasound device to obtain a set of dynamic images of echocardiography, performs multi-dimensional semantic feature extraction processing on the set of dynamic images of echocardiography according to a set of deep convolution parameters provided by the cardiac ultrasound device to obtain a set of deep semantic features, and performs time sequence feature change tracking on the set of dynamic images of echocardiography according to a set of bidirectional state parameters provided by the cardiac ultrasound device to obtain a set of time sequence change features. Through feature fusion of the set of deep semantic features and the set of time sequence change features, the unified fusion features can reflect the dynamic changes of different dimensions of cardiac features in the cardiac cycle from two core aspects of semantic depth and time sequence breadth, and generate and output cardiac structure information accordingly, providing accurate and comprehensive scientific data support for medical diagnosis. The specific implementation mode can refer to the following embodiments.

[0030] Figure 2 A flowchart of a lightweight ultrasonic medical dynamic image segmentation method based on Mamba is provided for an embodiment of the present application. The method of the present embodiment can be applied to the server in the above scenario. As shown in the figure, Figure 2 the method comprises:

[0031] S201. Acquire echocardiographic information, analyze echocardiographic information, and determine the set of dynamic echocardiographic images.

[0032] Echocardiographic information can be ultrasound images of the heart obtained using ultrasound imaging technology, and can be provided by echocardiography equipment. A dynamic echocardiographic image set can be a collection of image data reflecting cardiac characteristics extracted from echocardiographic information.

[0033] Specifically, before segmenting and extracting features from echocardiogram images, it is necessary to first analyze the echocardiogram image information obtained by doctors using professional echocardiogram equipment. Since there is some basic noise in the echocardiogram image information, by preprocessing the echocardiogram image information to filter out the basic noise, and by normalizing the echocardiogram images corresponding to different states, a dynamic echocardiogram image set can be obtained. This can improve the performance of the segmentation algorithm, reduce errors in subsequent processing, and lay a good foundation for accurate feature extraction.

[0034] S202. Obtain the depth convolution parameter set. Based on the depth convolution parameter set, perform multi-dimensional semantic feature extraction processing on the cardiac ultrasound dynamic image set to determine the depth semantic feature set.

[0035] The depth convolution parameter set can be the set of parameters required for feature extraction of the same image at different depths using CNN (Convolutional Neural Network) on a dynamic echocardiogram image set. It can include parameters such as convolution kernel and convolution bias term. The depth convolution parameter set can be obtained from the training results of convolution models on existing echocardiogram image datasets, such as EchoNet-Dynamic dataset and EchoNet-LVH dataset, and stored in the echocardiogram equipment.

[0036] Multidimensional semantic feature extraction processing can be the process of extracting features of the heart in different dimensions, such as shape features and edge features, from the same image in an echocardiogram dynamic image set.

[0037] A deep semantic feature set can be a dataset containing heart features of different dimensions within different images.

[0038] Specifically, since the echocardiogram image set contains images of the heart from different angles and at different levels, the heart features reflected in the corresponding dynamic echocardiogram image set contain multiple dimensions, such as edge features and internal features. By using a CNN model and based on the deep convolution parameter set, mathematical analysis can be used to effectively analyze the different dimensions of the heart in different echocardiogram images, thereby deriving the corresponding deep semantic feature set and improving the segmentation accuracy of the heart structure from the semantic depth level.

[0039] S203. Obtain the bidirectional state parameter set, and based on the bidirectional state parameter set, track the temporal feature changes of the echocardiogram dynamic image set to determine the temporal change feature set.

[0040] The bidirectional state parameter set can be a set of parameters required to analyze the temporal changes of cardiac features between different images in a dynamic echocardiogram image set using the bidirectional state space technique in the Vision-Mamba model. It can include state transition matrices, observation matrices, etc. The bidirectional state parameter set can be obtained by training the Vision-Mamba model on existing echocardiogram image datasets, such as the EchoNet-Dynamic dataset and the EchoNet-LVH dataset, and stored in the echocardiogram equipment.

[0041] Temporal feature change tracking can be a process of tracking and analyzing the changes in cardiac features between different images within a dynamic echocardiogram image set during the corresponding temporal sequence of the cardiac cycle.

[0042] A temporal variation feature set can be a dataset containing temporal variation features corresponding to different cardiac characteristics.

[0043] Specifically, since the heart is in real-time motion, if only CNNs are used to extract and analyze the multidimensional features of each echocardiogram image, the accuracy of analyzing the changes in the heart's features during motion will be reduced. This will have a significant negative impact on medical diagnosis based on echocardiogram images. While using CNNs to extract multidimensional semantic features, the bidirectional state space technique in the Vision-Mamba model is used in parallel to analyze the current and past states of different cardiac features during motion. At the same time, the future state is combined to provide comprehensive contextual information, deriving a set of temporal change features. This allows for more accurate capture of the dynamic features of the heart during contraction and relaxation, effectively extracting smooth cardiac motion trajectories and complex temporal patterns, thus better reflecting the physiological activities and functional state of the heart and providing significant assistance for subsequent medical diagnosis. At the same time, the parallel processing of CNNs and Vision-Mamba effectively improves the segmentation efficiency of dynamic echocardiogram images.

[0044] S204. Perform feature fusion on the deep semantic feature set and the temporal change feature set to determine a unified fused feature set.

[0045] A unified fusion feature set can be a set of feature data obtained by fusing cardiac feature information reflected in a deep semantic feature set and a temporal variation feature set.

[0046] Specifically, after obtaining the deep semantic feature set and the temporal variation feature set, since the feature information structures corresponding to the deep semantic feature set and the temporal variation feature set are different, directly segmenting the heart structure based on the deep semantic feature set and the temporal variation feature set will negatively affect the segmentation accuracy. Therefore, it is necessary to fuse the deep semantic feature set and the temporal variation feature set so that the two are mapped to the same spatial dimension. The resulting unified fused feature can comprehensively reflect the characteristics of different dimensions of heart features during the dynamic changes in the cardiac cycle at the two core levels of semantic depth and temporal breadth, thereby comprehensively reflecting the changes in heart features.

[0047] S205. Generate and output cardiac structure information based on the unified fusion feature set.

[0048] Cardiac structural information can be a collection of images containing specific structural markers of the various parts of the heart.

[0049] Specifically, after fusing the deep semantic feature set and the temporal change feature set to obtain a unified fused feature set, the performance of different features in the corresponding echocardiogram dynamic images is segmented and labeled. The specific structural markers of different parts of the heart are uniformly mapped to the corresponding echocardiogram dynamic images to generate cardiac structural information that can intuitively reflect changes in the heart's state. The cardiac structural information is then output in real time for doctors to make medical diagnoses, providing accurate and comprehensive scientific data support for medical diagnosis.

[0050] This scheme analyzes echocardiogram information to obtain a dynamic echocardiogram image set, ensuring the timeliness and continuity of data processing and laying the foundation for subsequent feature extraction. Based on the deep convolution parameter set, the dynamic echocardiogram image set undergoes multi-dimensional semantic feature extraction processing to obtain a deep semantic feature set, enabling effective analysis of different dimensional features of the heart in different echocardiogram images. This improves the segmentation accuracy of heart structure from the semantic depth level. Based on the bidirectional state parameter set, the dynamic echocardiogram image set is tracked for temporal feature changes to obtain a temporal change feature set, accurately capturing the dynamic features of the heart during contraction and relaxation. At the same time, the parallel processing of different models effectively improves the segmentation efficiency of dynamic echocardiogram images, achieving a good balance between segmentation accuracy and segmentation speed. Through feature fusion of the deep semantic feature set and the temporal change feature set, the resulting unified fused feature can comprehensively reflect the characteristics of different dimensions of heart features during dynamic changes within the cardiac cycle at two core levels: semantic depth and temporal breadth. Based on this, heart structure information is generated and output, providing accurate and comprehensive scientific data support for medical diagnosis.

[0051] In some embodiments, echocardiogram information is analyzed, real-time frame extraction processing is performed on the echocardiogram information to determine the ultrasound frame image set; the ultrasound frame image set is analyzed, noise reduction and normalization processing is performed on the ultrasound frame image set to determine the normalized image set; enhancement processing and temporal labeling are performed on each image in the normalized image set to determine the dynamic echocardiogram image set.

[0052] An ultrasound frame image set can be a collection of data containing each frame of echocardiogram image extracted from continuous echocardiogram image information.

[0053] A normalized image set can be an image dataset obtained by denoising and normalizing all images within an ultrasound frame image set.

[0054] Enhancement processing can be a process that improves the visibility of echocardiogram images, including techniques such as increasing contrast and sharpness.

[0055] Time stamping can be a process of adding timestamps to each frame of echocardiogram images to reflect the position of different images in time sequence.

[0056] Specifically, in the preprocessing of echocardiogram images, computer vision tools such as OpenCV (Open Source Computer Vision Library) are used to extract each time frame from the echocardiogram images to obtain an echocardiogram frame image set. Then, Gaussian filtering and histogram equalization algorithms are used to denoise and normalize each echocardiogram image in the set to reduce image noise and ensure uniform brightness and contrast across different images, providing consistent basic image data support for subsequent image segmentation. Furthermore, adaptive histogram equalization is used to enhance the structural texture contrast within the images, and corresponding timestamp metadata is added to each echocardiogram image to achieve enhancement and temporal labeling of each image, resulting in a dynamic echocardiogram image set. Image enhancement can improve the recognition rate of important structures, while temporal labeling helps in subsequent analysis of cardiac motion processes and changes.

[0057] This approach extracts frames from echocardiogram images to obtain an ultrasound frame image set, laying the foundation for capturing dynamic changes in the heart. By denoising and normalizing the ultrasound frame image set, a normalized image set is obtained, ensuring the consistency and clarity of the echocardiogram images. Enhancement processing of the echocardiogram images improves the visibility of key structures, and temporal labeling provides important temporal dimension information for subsequent image analysis. This enables the resulting dynamic echocardiogram image set to support the subsequent construction of an efficient dynamic echocardiogram image segmentation model, thereby improving the accuracy and efficiency of echocardiogram image analysis.

[0058] In some embodiments, based on a shallow convolution parameter set, shallow feature convolution kernels and corresponding shallow feature bias terms corresponding to several different types of cardiac structural features are extracted; based on a shallow convolution parameter set, deep feature convolution kernels and corresponding deep feature bias terms corresponding to several different types of cardiac structural features are extracted; the echocardiogram dynamic image set is analyzed to determine the image tensor corresponding to each image; based on the image tensor, shallow features corresponding to each image in the echocardiogram dynamic image set are determined according to several shallow feature convolution kernels and shallow feature bias terms; based on the image tensor, deep features corresponding to each image in the echocardiogram dynamic image set are determined according to several deep feature convolution kernels and deep feature bias terms; nonlinear fusion processing is performed on the shallow features and deep features to determine the convolution feature map corresponding to each image, thereby constructing a deep semantic feature set.

[0059] The depthwise convolution parameter set includes the shallow convolution parameter set and the deep convolution parameter set.

[0060] Shallow feature convolution kernels can be filters that extract local features from echocardiogram images in the shallow layers of the network, typically capturing basic features such as edges and textures.

[0061] Shallow feature bias terms can be parameters that shift the output in shallow convolution operations, which can improve the shallow expressive power of the model.

[0062] Deep feature convolution kernels can be filters in the shallow layers of a network that extract local features from echocardiogram images, typically capturing high-level features such as shape and object.

[0063] Deep feature bias terms can be parameters that shift the output during deep convolution operations, thereby improving the model's deep expressive power.

[0064] Image tensors can be matrices in the form of multidimensional arrays corresponding to echocardiogram images, which are then used for deep learning operations.

[0065] Shallow features can be the basic features that reflect the state of the heart in an echocardiogram, such as edges and textures.

[0066] Deep features can be high-level features within an echocardiogram that reflect the state of the heart, such as shape and objects.

[0067] A convolutional feature map can be a two-dimensional array generated by sliding convolution operations on an input image using one or more convolutional kernels. Each element in this array represents the feature response within a corresponding region, reflecting the characteristics of the echocardiogram image under a specific convolutional kernel.

[0068] Specifically, in the process of analyzing and extracting multidimensional features from echocardiogram images using CNNs, the features are hierarchically divided through shallow and deep convolution parameter sets. Shallow features reflect the basic components of the echocardiogram image, capturing low-level visual information and providing preliminary structural information, which helps enhance the accuracy and robustness of the image segmentation process. Deep features reflect more complex cardiac structural features in the echocardiogram image, such as blood vessels and ventricles, providing high-level feature representation for the image segmentation process, making it more sensitive and accurate in distinguishing different cardiac structures. Image processing libraries, such as NumPy (Numerical), are used to further enhance the segmentation process. Using Python (a numerical computation library), each echocardiogram image in the dynamic echocardiogram image set is numerically transformed to obtain the corresponding image tensor. Based on the image tensor, through mathematical analysis, shallow and deep features in each echocardiogram image are quantized and extracted according to shallow feature convolution kernels, shallow feature bias terms, deep feature convolution kernels, and deep feature bias terms. Further mathematical processing is used to nonlinearly fuse the shallow and deep features to generate a convolutional feature map corresponding to each echocardiogram image. This process integrates multi-level information and avoids the inability to fully utilize the interrelationships between features due to linear splicing. Finally, a deep semantic feature set is constructed based on the convolutional feature map corresponding to each echocardiogram image.

[0069] This scheme hierarchically divides the features within echocardiogram images using shallow and deep convolution parameter sets. Based on the image tensor, shallow and deep features within each echocardiogram image are quantized and extracted according to shallow feature convolution kernels, shallow feature bias terms, deep feature convolution kernels, and deep feature bias terms. The shallow and deep features are then nonlinearly fused to generate a convolutional feature map corresponding to each echocardiogram image. This constructs a deep semantic feature set, enabling the convolutional feature map to achieve a better balance in representing the details and global features of the heart structure, providing a more accurate and efficient feature representation for subsequent segmentation tasks of echocardiogram images.

[0070] In some embodiments, shallow features and deep features are nonlinearly fused to determine the convolutional feature map corresponding to each image, specifically as shown in the following formula (1):

[0071] (1)

[0072] in, For convolutional feature maps, It is a composite nonlinear mapping function. To preset the fusion coefficient, For image tensors, These are shallow features. For deep features; the composite nonlinear mapping function is obtained by a weighted combination of the ReLU function, the Sigmoid function and the Tanh function.

[0073] The preset fusion coefficient can be used to adjust the contribution ratio of shallow and deep features in the final feature map, and its range is usually between [0,1]. When biased towards 0, the results emphasize deeper features; when... When biased towards 1, the results focus more on shallow features, and the preset fusion coefficient can be obtained by fitting historical data.

[0074] Composite nonlinear functions can be functions used to perform nonlinear transformations on the fused features.

[0075] Specifically, image tensors are extracted using mathematical analysis techniques and shallow convolutional neural networks. The shallow features in the corresponding echocardiogram images are obtained. Then, using shallow features as input, deep features are extracted through mathematical analysis using deep convolutional neural networks. Based on the preset fusion coefficient, through We perform weighted linear fusion of shallow and deep features to obtain basic fused features. Then, we use a composite nonlinear mapping function to perform a nonlinear transformation on the basic fused features to obtain a convolutional feature map. The ReLU function is used to perform non-negative activation on the basic fused features, introducing sparsity. The Sigmoid function is used to compress the basic fused features, making the output range between (0,1), which helps to determine the importance of the basic fused features. The Tanh function is used to standardize the basic fused features, compressing the output to the range of (-1,1), improving the expressiveness of the basic fused features at different scales. The influence weights of different functions in the composite nonlinear function can be obtained by fitting experimental data.

[0076] This scheme utilizes mathematical analysis methods, combining shallow and deep features corresponding to echocardiogram images, to design mathematical formulas for quantitative analysis of convolutional feature maps that reflect the multi-layered features of the heart within echocardiogram images. This enables the convolutional feature maps to express the rich semantic information corresponding to echocardiogram images from multiple perspectives, which helps improve the accuracy of subsequent segmentation work. At the same time, by processing nonlinear changes during the feature fusion process, the convolutional feature maps are smoothed, improving their expressive power.

[0077] In some embodiments, based on the image tensor, the shallow features corresponding to each image in the echocardiogram dynamic image set are determined according to several shallow feature convolution kernels and shallow feature bias terms, specifically as follows: (2)

[0078] (2)

[0079] in, These are shallow features. For max pooling function, It is a shallow activation function. For image tensors, For the first A shallow feature convolution kernel, For the first One shallow feature bias term;

[0080] Based on the image tensor, the deep features corresponding to each image in the echocardiogram dynamic image set are determined according to several deep feature convolution kernels and deep feature bias terms, specifically as follows (3):

[0081] (3)

[0082] in, For deep features, These are shallow features. For average pooling function, For deep activation functions, For the first A deep feature convolution kernel, For the first A deep feature bias term.

[0083] Max pooling functions can be used to extract the most salient features from an image.

[0084] Shallow activation functions can be functions used to introduce non-linearity into shallow features, and the ReLU function is commonly used.

[0085] Average pooling functions can be used to average shallow features.

[0086] Deep activation functions can be functions used to introduce non-linearity into shallow features, typically the ReLU function.

[0087] Specifically, through formula (2) For image tensors All shallow feature convolution kernels are applied to the corresponding echocardiogram image. Each convolution kernel performs element-wise multiplication with the image tensor. During this process, a corresponding shallow bias term is introduced, and the results of all the above shallow convolution operations are summed. The shallow activation function is used to introduce nonlinearity into the summation result, enabling the model to learn more complex features. The max pooling function is used to perform max pooling on the summation result after activation to obtain shallow features, reduce the scale of shallow features, retain important shallow feature information, and improve the efficiency of subsequent processing. Through formula (3) All deep feature convolution kernels are applied to the shallow features, a deep bias term is introduced, and the results of all deep convolutions are summed. Nonlinearity is introduced using a deep activation function to extract higher-level abstract features from the echocardiogram to represent the complex structure within the echocardiogram. Then, the activated feature values ​​are downsampled using an average pooling function to obtain the corresponding deep features. Unlike the max pooling function mentioned above, the purpose of the average pooling function is to take the mean of the region to average the deep features, helping to retain more deep feature information.

[0088] This scheme utilizes mathematical analysis to quantify the shallow and deep features within echocardiogram images. Through mathematical formulas, it reduces the scale of shallow features, retains important shallow feature information, and improves subsequent processing efficiency. Simultaneously, it extracts higher-level abstract features from the echocardiogram images to characterize the complex structures within the images. This allows the shallow and deep features to comprehensively reflect the semantic features of the heart at different levels within the echocardiogram images.

[0089] In some embodiments, an echocardiogram dynamic image set is analyzed to extract the image feature vector of each image at its corresponding time point; based on noise information, the state transition noise and observation noise corresponding to different time points are determined; based on the state transition matrix, the state feature vector of each image at its corresponding time point is determined according to the image feature vector and the state transition noise; based on the observation matrix, the state feature vector and observation noise are analyzed to determine the observation feature vector of each image at its corresponding time point; based on the state feature vector and observation feature vector, forward state recursion processing and reverse state recursion processing are performed on each image respectively to determine the forward state features and reverse state features; based on the forward state features and reverse state features, the temporal state features of each image at its corresponding time point are determined, thereby constructing a temporal change feature set.

[0090] The bidirectional state parameter set includes the state transition matrix, the observation matrix, and noise information.

[0091] A state transition matrix can be a matrix that describes how the heart's state evolves from the current state to the next state within a time period.

[0092] An observation matrix can be a matrix that models the relationship between the cardiac state reflected in an echocardiogram and the actual observed values.

[0093] Noise information can include information about the cardiac state transition process and the random noise model during the observation process.

[0094] Image feature vectors are numerical representations extracted from images that can summarize the key features of the image.

[0095] State transition noise can be used to describe random noise during the state transition of the heart at different time points in echocardiogram images. State transition noise can be modeled using a Gaussian noise model.

[0096] Observation noise can be used to describe the random noise in an echocardiogram as the heart transitions from its actual state to observational data. Observation noise can be modeled using a Gaussian noise model.

[0097] The state feature vector can be a vector information that reflects the state of the heart at its corresponding time point in each echocardiogram.

[0098] The observed feature vector can be the actual observed vector information of the corresponding state feature vector.

[0099] Forward state features can be derived by predicting the cardiac state at each time point from the initial state.

[0100] Reverse state features can start from the final state and gradually correct the feature information of the heart state at each time point.

[0101] Temporal state features can be a combination of forward and reverse state features, which can comprehensively reflect the characteristic information of the changes in the state of the heart at different time points in echocardiogram images.

[0102] Specifically, in the process of analyzing the temporal variation features of cardiac state in echocardiogram images using the Vision-Mamba model, the bidirectional state-space technique can analyze and extract the variation features of cardiac state at different time points based on the image time sequence. Through the convolutional layer in the Vision-Mamba model, feature vectors are extracted from the images in the dynamic echocardiogram image set to determine the corresponding image feature vectors. Based on these image feature vectors, mathematical analysis is used to obtain state feature vectors that reflect the cardiac state at the time point in the image, based on the state transition mean and combined with state transition noise. On this basis, mathematical analysis is used to quantize the observation feature vectors, combined with the observation matrix and observation noise. Then, forward and backward state recursive processing is performed on each image accordingly. Based on the features reflected by the changes in cardiac state in the image and the observed changes in the temporal variation process, forward and backward state features that reflect the changes in cardiac state before and after the corresponding time point are obtained. By fusing the forward and backward state features, the temporal state features corresponding to each image are obtained, thereby constructing a temporal variation feature set.

[0103] This approach significantly enhances the accuracy and robustness of echocardiogram image segmentation by extracting feature vectors from dynamic echocardiogram images, determining state and observation noise, and employing forward and backward recursive processing. It effectively captures dynamic changes in the heart and surrounding structures, improves adaptability to complex backgrounds and noise, and enhances the efficiency of recognizing fine cardiac structures. Furthermore, the method of constructing temporal variation feature sets provides in-depth cardiac function analysis for clinical use, supporting precise disease detection and diagnosis.

[0104] In some embodiments, based on the state transition matrix, the state feature vector of each image at its corresponding time point is determined according to the image feature vector and the state transition noise, specifically as shown in the following formula (4):

[0105] (4)

[0106] in, For the current image at its corresponding time point The state feature vector under the following conditions For time points Image feature vector at that location, State transition matrix, For time points The corresponding state transition noise; based on the observation matrix, analyze the state feature vector and observation noise, and determine the observation feature vector of each image at its corresponding time point, specifically as shown in the following formula (5):

[0107] (5)

[0108] in, For the current image at its corresponding time point The observed feature vectors, For the observation matrix, For the current image at its corresponding time point The state feature vector under the following conditions Time point The corresponding observation noise at that location.

[0109] Specifically, formula (4) can be viewed as a cardiac state update equation, which uses past states as the basis for the equation. Based on this, a linear transformation is performed on the state transition matrix, and state transition noise is added as a random variable in the state transition process. This yields a state feature vector that reflects the state of the heart in the image at its corresponding time point. In a dynamic system, a linear model is usually used to describe the relationship between the object state and the observation. The state feature vector is linearly transformed by the observation matrix through formula (5). However, since noise is objectively present in the actual observation process, the observation cannot perfectly reflect the state of the heart. Therefore, observation noise needs to be introduced as a random influence, and the state feature vector is then mapped to the observation space through the observation matrix.

[0110] This scheme utilizes mathematical analysis based on the state transition matrix and observation matrix. By designing corresponding formulas according to image feature vectors, state transition noise, and observation noise, it achieves quantitative analysis of state feature vectors and observation feature vectors, enabling the tracking of cardiac state changes. This significantly improves the segmentation accuracy of echocardiogram images and provides a scientific data foundation for further analysis of the temporal changes in cardiac state.

[0111] In some embodiments, forward state recursion processing is performed on each image based on the state feature vector and the observation feature vector to determine the forward state features, specifically as shown in the following formula (6):

[0112] (6)

[0113] in, For the current image at its corresponding time point The forward state features below, This represents the state feature vector. The observed corresponding eigenvectors The probability, This represents the state feature vector corresponding to the previous time point. Transform to the current time point corresponding to the state feature vector The probability; based on the state feature vector and the observation feature vector, perform reverse state recursive processing on each image to determine the reverse state features, specifically as follows (7):

[0114] (7)

[0115] in, For the current image at its corresponding time point The reverse state features under, Indicates the state feature vector The observed feature vectors of the future are observed below. The probability, This represents the state feature vector corresponding to the current time point. Transformation to future state feature vectors The probability; based on the forward state features and the reverse state features, determine the temporal state features of each image at its corresponding time point, specifically as follows (8):

[0116] (8)

[0117] in, For the current image at its corresponding time point The temporal state characteristics below, For the current image at its corresponding time point The forward state features below, For the current image at its corresponding time point The reverse state characteristics below.

[0118] Specifically, formula (6) quantifies the forward state features of the current state feature vector given the observed feature vector at the current time point. Formula (6) is based on a hidden Markov model and uses a forward recursive algorithm to analyze the transformation relationship between the observed feature vector and the current state feature vector, and between the current state feature vector and the historical state vector, thereby realizing the quantification of the forward state features. The probability function in the formula is... Neural network models, such as the long short-term memory collaborative model, can be used to analyze different features. Similarly, the reverse recursive algorithm can be used to analyze the transformation relationship between the future observation feature vector and the current state feature vector, and between the current state feature vector and the future state feature vector, through formula (7), so as to quantify the reverse state features. On the basis of the above, the forward state features and the reverse state features can be fused through formula (8) to obtain the temporal state features that can comprehensively reflect the temporal change features of the heart state before and after its corresponding time point.

[0119] This scheme utilizes mathematical analysis to process each echocardiogram image using forward and reverse recursive state features based on state and observation feature vectors and mathematical formulas. This yields forward and reverse state features, which are then fused to create temporal state features that comprehensively reflect the temporal changes in cardiac state before and after the corresponding time point. This approach enables precise tracking and feature extraction of cardiac state changes, reflecting the characteristics of cardiac state changes over time and significantly improving the segmentation accuracy of echocardiogram images.

[0120] In some embodiments, different structural features within the deep semantic feature set and the temporal change feature set are subjected to unified feature space mapping to determine a unified semantic feature set and a unified temporal feature set; based on an attention mechanism, the features within the unified semantic feature set and the unified temporal feature set are mixed, and the mixed processing result is mapped to a new feature space to determine a unified fusion feature set.

[0121] Unified feature space mapping can be the process of mapping data from different structured datasets to the same feature space.

[0122] A unified semantic feature set can be a data set containing all semantic features mapped to the same feature space.

[0123] A unified temporal feature set can be a data set containing all temporal features mapped to the same feature space.

[0124] A unified fusion feature set can be a data set containing all fusion features mapped to the same feature space after the unified semantic feature set and the unified temporal feature set have been mixed and processed.

[0125] Specifically, after obtaining the deep semantic feature set and the temporal variation feature set, since there are structural differences between data in the same set and different sets, in order to reduce the image segmentation error caused by the differences in data structure, data dimensionality reduction techniques, such as principal component analysis algorithms, are used to achieve a unified spatial mapping of different data in the same set, so as to obtain the corresponding unified semantic feature set and unified temporal feature set. Then, the attention mechanism built into the model in the aforementioned embodiment is used to perform weighted mixing of different features to improve the model's sensitivity to key features, and the mixed feature information is mapped to a new feature space again to obtain a unified fused feature set.

[0126] This scheme performs unified spatial mapping processing on deep semantic feature sets and temporal variation feature sets, unifying data with structural differences within the same set to obtain unified semantic feature sets and unified temporal feature sets. This reduces image segmentation errors caused by differences in data structure. Furthermore, based on an attention mechanism, features within the unified semantic feature sets and unified temporal feature sets are mixed to obtain a unified fused feature set, improving the sensitivity of the image processing process to key features.

Claims

1. A lightweight ultrasound medical dynamic image segmentation method based on Mamba, characterized in that, Includes the following steps: (1) Acquire echocardiographic information, analyze and preprocess the echocardiographic information to obtain a set of dynamic echocardiographic images; (2) The deep convolutional neural network is used to process the cardiac ultrasound dynamic image set in parallel to extract the deep semantic feature set of cardiac structure; (3) Using the time-tracking module of the bidirectional state-space model, the echocardiogram dynamic image set is processed in parallel to extract the time-series change feature set of cardiac dynamic changes; (4) The deep semantic feature set and the temporal change feature set are fused across modalities to obtain a unified fused feature set; (5) Generate and output heart structure segmentation information based on the unified fusion feature set.

2. The lightweight ultrasound medical dynamic image segmentation method based on Mamba according to claim 1, characterized in that, Step (1) is as follows: real-time frame extraction is performed on the echocardiogram image information to obtain an ultrasound frame image set; the ultrasound frame image set is denoised and normalized to obtain a normalized image set; image enhancement is performed on each frame image in the normalized image set and time sequence markers are added to form the echocardiogram dynamic image set.

3. The lightweight ultrasound medical dynamic image segmentation method based on Mamba according to claim 1, characterized in that, Step (2) is as follows: use shallow convolution parameters to extract low-level local features of each image in the cardiac dynamic image set; use deep convolution parameters to further abstract the low-level local features and extract high-level structural features; use non-linear weighted fusion of low-level local features and high-level structural features to generate a fused convolution feature map of each image; and use the fused convolution feature maps of all images to form a deep semantic feature set.

4. The lightweight ultrasound medical dynamic image segmentation method based on Mamba according to claim 1, characterized in that, Step (3) is as follows: Extract the image feature vectors of each image in the echocardiogram dynamic image set at the corresponding time point; estimate the cardiac state feature vectors at each time point based on the preset state transition matrix and image feature vectors, combined with state transition noise; obtain the observation feature vectors at each time point based on the preset observation matrix and cardiac state feature vectors, combined with observation noise; calculate the forward state features reflecting the historical and current states using a forward recursive algorithm, combined with cardiac state feature vectors and observation feature vectors; calculate the reverse state features reflecting the influence of future states using a reverse recursive algorithm, combined with cardiac state feature vectors and observation feature vectors; fuse the forward state features and reverse state features to obtain the temporal state features at each time point, and construct a temporal change feature set from the temporal state features of all time points.

5. A lightweight ultrasound medical dynamic image segmentation method based on Mamba according to claim 1, characterized in that, Step (4) is as follows: Map the features in the deep semantic feature set to the first unified feature space to obtain the unified semantic feature set; map the features in the temporal change feature set to the second unified feature space to obtain the unified temporal feature set; perform feature interaction and weighted mixing on the unified semantic feature set and the unified temporal feature set based on the attention mechanism; map the mixed features to the new target feature space to form a unified fusion feature set.

6. A lightweight ultrasound medical dynamic image segmentation system based on Mamba, characterized in that, include: Image extraction module: used to acquire echocardiogram image information, and to analyze and preprocess the echocardiogram image information to obtain a set of dynamic echocardiogram images; Semantic feature analysis module: used to perform parallel processing on the set of echocardiogram dynamic images using a deep convolutional neural network to extract a set of deep semantic features of the heart structure; Temporal Feature Analysis Module: This module utilizes a temporal tracking module based on a bidirectional state-space model to perform parallel processing on a dynamic echocardiogram image set and extract a temporal feature set of dynamic cardiac changes. Feature fusion module: used to perform cross-modal feature fusion of deep semantic feature set and temporal variation feature set to obtain unified fused feature set; Output module: Used to generate and output heart structure segmentation information based on the unified fusion feature set.

7. A lightweight ultrasound medical dynamic image segmentation system based on Mamba according to claim 6, characterized in that, The image extraction module specifically performs the following steps: real-time frame extraction of echocardiogram image information to obtain an ultrasound frame image set; denoising and normalization processing of the ultrasound frame image set to obtain a normalized image set; image enhancement and time-series markers are added to each frame image in the normalized image set to form the dynamic echocardiogram image set.

8. A lightweight ultrasound medical dynamic image segmentation system based on Mamba according to claim 6, characterized in that, In the semantic feature analysis module, the following steps are taken: shallow convolution parameters are used to extract low-level local features from each image in the cardiac ultrasound dynamic image set; deep convolution parameters are used to further abstract the low-level local features and extract high-level structural features; the low-level local features and high-level structural features are weighted and fused in a non-linear manner to generate a fused convolutional feature map for each image; and the fused convolutional feature maps of all images constitute a deep semantic feature set.

9. A lightweight ultrasound medical dynamic image segmentation system based on Mamba according to claim 6, characterized in that, The temporal feature analysis module specifically performs the following steps: Extracting image feature vectors from each image in the echocardiogram dynamic image set at corresponding time points; estimating cardiac state feature vectors at each time point based on a preset state transition matrix and image feature vectors, combined with state transition noise; obtaining observation feature vectors at each time point based on a preset observation matrix and cardiac state feature vectors, combined with observation noise; calculating forward state features reflecting historical and current states using a forward recursive algorithm, combining cardiac state feature vectors and observation feature vectors; calculating reverse state features reflecting the impact of future states using a backward recursive algorithm, combining cardiac state feature vectors and observation feature vectors; fusing the forward and reverse state features to obtain the temporal state features at each time point, and constructing a temporal change feature set from the temporal state features of all time points.

10. A lightweight ultrasound medical dynamic image segmentation system based on Mamba according to claim 6, characterized in that, In the feature fusion module, the following steps are taken: features in the deep semantic feature set are mapped to the first unified feature space to obtain a unified semantic feature set; features in the temporal variation feature set are mapped to the second unified feature space to obtain a unified temporal feature set; feature interaction and weighted mixing of the unified semantic feature set and the unified temporal feature set are performed based on an attention mechanism; and the mixed features are mapped to a new target feature space to form a unified fused feature set.

Images