Echocardiography analysis method, system, terminal and computer readable storage medium

Abstract

The application discloses an echocardiogram analysis method, system, terminal and computer readable storage medium, and the method comprises the steps of: acquiring a target gray-scale image and a target Doppler image; constructing a target model, wherein the target model comprises a frequency decomposition module, a cross-Transformer fusion module and a cross-convolution fusion module; obtaining gray-scale high-frequency features, gray-scale low-frequency features, color high-frequency features and color low-frequency features according to the target gray-scale image and the target Doppler image; inputting the gray-scale low-frequency features and the color low-frequency features into the cross-Transformer fusion module for fusion to obtain fused low-frequency information, and inputting the gray-scale high-frequency features and the color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information; and inputting the low-frequency information and the high-frequency information into a classifier to obtain a classification result of the target echocardiogram. The application overcomes the defects of slow artificial analysis efficiency and easy errors, and can efficiently and accurately analyze the echocardiogram.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an echocardiography analysis method, system, terminal, and computer-readable storage medium. Background Technology

[0002] Echocardiography is a cardiac ultrasound examination that uses pulsed or continuous ultrasound waves to accurately measure the velocity of blood and myocardial tissue at any location, and displays a two-dimensional image of the heart using standard ultrasound techniques or Doppler ultrasound. Echocardiography is the most important method for examining the heart. It has no side effects on young children, the equipment is relatively inexpensive, and it is convenient for large-scale screening and data acquisition.

[0003] However, current echocardiography requires professional personnel for processing and analysis. Given the shortage of professionals and the ever-increasing amount of image data to be processed, professionals have to increase their workload and extend their working hours in order to complete their tasks, making errors in echocardiography analysis unavoidable. Furthermore, existing automated auxiliary analysis tools do not take into account the use of different frequency characteristics to express specific information, resulting in inaccurate echocardiography analysis results.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide an echocardiography analysis method, system, terminal, and computer-readable storage medium, aiming to solve the problems in the prior art where echocardiography analysis relies on professional personnel, resulting in low efficiency, unavoidable errors, and inaccurate analysis results from existing automated assisted echocardiography analysis tools.

[0006] To achieve the above objectives, the present invention provides an echocardiographic analysis method, which includes the following steps:

[0007] Obtain the target echocardiogram, and obtain the target grayscale image and the target Doppler image based on the target echocardiogram;

[0008] An echocardiographic analysis model is constructed, and the echocardiographic analysis model is trained and tested to obtain a target model. The target model includes: a frequency decomposition module, a cross-Transformer fusion module, and a cross-convolution fusion module.

[0009] The target grayscale image and the target Doppler image are input into the frequency decomposition module for a mixed filtering operation to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features.

[0010] The grayscale low-frequency features and the color low-frequency features are input into the cross-transformer fusion module for fusion to obtain fused low-frequency information. The grayscale high-frequency features and the color high-frequency features are input into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information.

[0011] The low-frequency information and the high-frequency information are input into a classifier to obtain the classification result of the target echocardiogram.

[0012] Optionally, the echocardiographic analysis method, wherein training and testing the echocardiographic analysis model to obtain the target model specifically includes:

[0013] Historical video sequences are acquired, and the historical video sequences are converted into image data. The image data is preprocessed to obtain sample data, and the sample data is divided into training set and test set according to a preset ratio.

[0014] Based on the training set, the echocardiogram analysis model is trained by performing a preset number of cross-validations using ADAM, and the loss function is calculated. When the loss function reaches a preset convergence condition, the trained echocardiogram analysis model is obtained.

[0015] The trained echocardiography analysis model was evaluated using the test set to obtain the target model that meets the preset requirements.

[0016] Optionally, in the echocardiographic analysis method, the frequency decomposition module includes a forward Fourier transform unit, a hybrid filter unit, and an inverse Fourier transform unit.

[0017] The cross-Transformer fusion module includes: a group fusion unit, a feature dimensionality reduction unit, a selection attention unit, a Norm unit, and an Mlp unit.

[0018] Optionally, in the echocardiographic analysis method, the step of inputting the target grayscale image and the target Doppler image into the frequency decomposition module for Fourier transform and hybrid filtering operations to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features specifically includes:

[0019] The target grayscale image G and the target Doppler image D are input into the Fourier forward transform unit for Fourier transform to obtain the first spectrum G of the grayscale image. s The first phase map G of the grayscale image p And the second spectrogram D of the Doppler image sThe second phase map D of the Doppler image p ;

[0020] The first spectrum G is respectively s and the second spectrum D s The input is fed into the hybrid filter unit for hybrid filtering to obtain the first low-frequency filter G. filterLow Second low-frequency filter D filterLow and the first high-frequency filter G filterHigh Second high frequency filter D filterHigh ;

[0021] The first low-frequency filter G filterLow The low-frequency characteristics and the first high-frequency filter G filterHigh The high-frequency characteristics in the first phase map G p The input is fed into the inverse Fourier transform unit to obtain the grayscale high-frequency feature G. h Gray-scale low-frequency characteristics G l ;

[0022] The second low-frequency filter D filterLow The low-frequency characteristics and the second high-frequency filter D filterHigh The high-frequency characteristics in the second phase diagram D p The input is fed into the inverse Fourier transform unit to obtain the color high-frequency feature D. h Color low-frequency characteristics D l .

[0023] Optionally, in the echocardiographic analysis method, the step of inputting the grayscale low-frequency features and the color low-frequency features into the cross-Transformer fusion module for fusion to obtain the fused low-frequency information specifically includes:

[0024] The grayscale low-frequency feature G l and the color low-frequency feature D l The inputs are respectively fed into the group fusion unit for channel-dimensional segmentation, resulting in four feature vectors. Three different sized convolution kernels are then used to convolve the four feature vectors along the channel dimension to obtain the grayscale low-frequency feature G. l The corresponding first query feature G q and the color low-frequency feature D l The corresponding second query feature D q ;

[0025] The grayscale low-frequency feature G l and the color low-frequency feature D l The grayscale low-frequency feature G is obtained by performing dimensionality reduction operations on the feature dimensionality reduction unit. lThe corresponding first key feature G k and the first value feature G v and the color low-frequency feature D l The corresponding second key feature D k and second-valued feature D v ;

[0026] The grayscale low-frequency features G are respectively l and the color low-frequency feature D l The corresponding query features, key features, and value features are input into the selection attention unit for feature extraction to obtain the first output feature T. G Second output feature T D ;

[0027] The first output feature T G and the second output feature T D The results are input into the Norm unit and the Mlp unit respectively for calculation, resulting in a first calculation result and a second calculation result.

[0028] The first calculation result is compared with the first output feature T. G The two features are then fused to obtain the first fused feature. The second calculation result is then combined with the second output feature T. D The fusion process yields the second fusion feature.

[0029] The first fusion feature and the second fusion feature are fused to obtain the fused low-frequency information T.

[0030] Optionally, in the echocardiographic analysis method, the grayscale low-frequency feature G is respectively... l and the color low-frequency feature D l The corresponding query features, key features, and value features are input into the selection attention unit for feature extraction to obtain the first output feature T. G Second output feature T D Specifically, it includes:

[0031] The first query feature G is respectively q The first key feature G k and the first value feature G v The data is divided into three parts: the first query vector Q1, the first key vector K1, and the first value vector V1. R represents the numerical value of the image, P represents the number of regions to be divided, and H, W, and C represent the height, width, and number of channels of the feature, respectively.

[0032] Normalize the first query vector Q1 and the first key vector K1 to obtain the normalized Q1′ and K1′. Calculate the first relation matrix A1 between the first query vector Q1 and the first key vector K1 based on the normalization results:

[0033]

[0034] Where M represents the transpose of the matrix;

[0035] Select a preset number of tokens from the first relation matrix A1, and transform the first key vector K1 and the first value vector V1 according to the tokens to obtain the transformed first key vector K1″ and the transformed first value vector.

[0036] Using the first query vector Q1, the transformed first key vector K1″, and the transformed first value vector V1′ as the query term, key term, and value of the selected attention unit, respectively, the first attention SA1 is calculated:

[0037]

[0038] Where SA(.) represents the attention function, D represents the feature embedding dimension, and Softmax(.) represents the normalization function;

[0039] Calculate the first computational attention SA1 and the first query feature G q The sum of these values ​​yields the first output feature T. G :T G =SA1+G q ;

[0040] The second query feature D is respectively q The second key feature D k and the second value feature D v The data is divided into two parts: a second query vector Q2, a second key vector K2, and a second value vector V2.

[0041] Normalize the second query vector Q2 and the second key vector K2 to obtain the normalized Q2′ and Calculate the second relation matrix A2 between the second query vector Q2 and the second key vector K2 based on the normalization results:

[0042]

[0043] Select a preset number of tokens from the second relation matrix A2, and transform the second key vector K2 and the second value vector V2 according to the tokens to obtain the transformed second key vector K2″ and the transformed second value vector.

[0044] Using the second query vector Q2, the transformed second key vector K2″, and the transformed second value vector V2′ as the query term, key term, and value of the selected attention unit, respectively, the second attention SA2 is calculated:

[0045]

[0046] Calculate the second computational attention SA2 and the second query feature D q The sum of these values ​​yields the second output feature T. D :T D =SA2+D q .

[0047] Optionally, in the echocardiographic analysis method, the step of inputting the grayscale high-frequency features and the color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information specifically includes:

[0048] High-frequency features of grayscale G h The input to the convolutional residual block in the cross-convolution fusion module yields a first calculation result, which is then compared with the color high-frequency feature D. h Summation yields the color high-frequency feature D′. h ;

[0049] The multimodal high-frequency feature D′ h The input to the convolutional residual block in the cross-convolution fusion module yields a second calculation result, which is then combined with the grayscale high-frequency feature G. h Multiplying them together yields the grayscale high-frequency feature G′. h ;

[0050] The color high-frequency feature D′ h and the grayscale high-frequency feature G′ h The fused multimodal high-frequency information is obtained by connecting the components using the cat operation.

[0051] Furthermore, to achieve the above objectives, the present invention also provides an echocardiographic analysis system, wherein the echocardiographic analysis system comprises:

[0052] The target image acquisition module is used to acquire the target echocardiogram and obtain the target grayscale image and the target Doppler image based on the target echocardiogram;

[0053] The model building and training module is used to build an echocardiogram analysis model, train and test the echocardiogram analysis model to obtain a target model, wherein the target model includes: a frequency decomposition module, a cross-Transformer fusion module and a cross-convolution fusion module;

[0054] The high- and low-frequency decomposition module is used to input the target grayscale image and the target Doppler image into the frequency decomposition module to perform Fourier transform and hybrid filtering operations to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features;

[0055] The feature fusion module is used to input the grayscale low-frequency features and the color low-frequency features into the cross-Transformer fusion module for fusion to obtain fused low-frequency information, and to input the grayscale high-frequency features and the color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information.

[0056] The classification result acquisition module is used to input the low-frequency information and the high-frequency information into the classifier to obtain the classification result of the target echocardiogram.

[0057] In addition, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and an echocardiography analysis program stored in the memory and executable on the processor, wherein when the echocardiography analysis program is executed by the processor, it implements the steps of the echocardiography analysis method as described above.

[0058] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an echocardiography analysis program, which, when executed by a processor, implements the steps of the echocardiography analysis method as described above.

[0059] In this invention, a target grayscale image and a target Doppler image are acquired; a target model is constructed, comprising a frequency decomposition module, a cross-transformer fusion module, and a cross-convolution fusion module; grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features are obtained from the target grayscale image and the target Doppler image; the grayscale low-frequency features and color low-frequency features are input into the cross-transformer fusion module for fusion to obtain fused low-frequency information, and the grayscale high-frequency features and color high-frequency features are input into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information; the low-frequency information and high-frequency information are input into a classifier to obtain the classification result of the target echocardiogram. This invention overcomes the shortcomings of slow efficiency and error susceptibility of manual analysis, and can efficiently and accurately analyze echocardiograms. Attached Figure Description

[0060] Figure 1 This is a flowchart of a preferred embodiment of the echocardiographic analysis method of the present invention;

[0061] Figure 2 This is a framework diagram of the target model in the echocardiographic analysis method of the present invention;

[0062] Figure 3 This is an architecture diagram of the cross-Transformer fusion module in the echocardiography analysis method of the present invention;

[0063] Figure 4 This is a schematic diagram of the group fusion unit in the echocardiographic analysis method of the present invention;

[0064] Figure 5 This is a schematic diagram of the selection of the attention unit in the echocardiographic analysis method of the present invention;

[0065] Figure 6 This is an architectural diagram of the cross-convolution fusion framework in the echocardiography analysis method of the present invention;

[0066] Figure 7 This is a schematic diagram of a preferred embodiment of the echocardiography analysis system of the present invention;

[0067] Figure 8 This is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0068] This application provides an echocardiographic analysis method and related equipment. To make the purpose, technical solution, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0069] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0070] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0071] The preferred embodiment of the echocardiographic analysis method of the present invention, such as... Figure 1 and Figure 2 As shown, the echocardiographic analysis method includes the following steps:

[0072] Step S100: Obtain the target echocardiogram, and obtain the target grayscale image and the target Doppler image based on the target echocardiogram.

[0073] Specifically, this invention acquires the target echocardiogram of the user to be processed, and obtains two modalities of images based on the target echocardiogram: a target grayscale image and a target Doppler image. Echocardiograms contain a wealth of information in their two-dimensional grayscale image and color Doppler image. However, existing algorithms only consider a single modal image, or fail to consider the advantages of both modalities, simply stitching them together as input to the network. But the two modalities express different information; that is, most current algorithms only perform the same operations on features, without considering that different frequency features express different information. Therefore, this invention acquires the target grayscale image and the target Doppler image as input, using different frequency features to express specific information.

[0074] Step S200: Construct an echocardiographic analysis model, train and test the echocardiographic analysis model to obtain a target model, wherein the target model includes: a frequency decomposition module, a cross-Transformer fusion module and a cross-convolution fusion module.

[0075] Specifically, historical video sequences are acquired, the historical video sequences are converted to obtain image data, the image data is preprocessed to obtain sample data, and the sample data is divided into training set and test set according to a preset ratio.

[0076] It is understood that the present invention collects multimodal five-chamber view heart data as analysis data, and the analysis data is a historical video sequence, which needs to be converted. Therefore, the historical video sequence is converted to obtain image data.

[0077] Further, the image data is preprocessed to obtain sample data. Since the image data contains two sizes—1226 frames of normal images and 789 frames of VSD (ventricular septal defect) images—it cannot be directly input into the echocardiographic analysis model for training. Therefore, the image data is cropped to a uniform size of 224×224 to obtain uniformly sized sample data. This sample data is then divided into a training set and a test set according to a preset ratio (e.g., 7:3). The training set is used to train the echocardiographic analysis model. The test set is used to evaluate the performance of the trained echocardiographic analysis model.

[0078] Furthermore, based on the training set, the echocardiogram analysis model is trained by performing a preset number of cross-validations using ADAM, and a loss function is calculated. When the loss function reaches a preset convergence condition, the trained echocardiogram analysis model is obtained.

[0079] ADAM was used as the optimizer during training, and the network was trained using 5 cross-validations. The initial learning rate was set to 1e-4, the momentum to 0.99, and the image size to 224×224. Focal loss was chosen as the loss function, defined as follows:

[0080] Focal Loss=-α t (1-p t ) γ log(p t );

[0081] Where α and γ are adjustable parameters, p t It predicts probabilities. When the Focal Loss function reaches the preset convergence condition, the trained echocardiographic analysis model is obtained.

[0082] Furthermore, the trained echocardiography analysis model is evaluated using the test set to obtain the target model that meets the preset requirements.

[0083] The target model includes: a frequency decomposition module, a cross-transformer fusion module, and a cross-convolution fusion module.

[0084] Step 300: Input the target grayscale image and the target Doppler image into the frequency decomposition module for a mixed filtering operation to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features.

[0085] The frequency decomposition module includes a forward Fourier transform unit, a hybrid filter unit, and an inverse Fourier transform unit.

[0086] Specifically, the target grayscale image G and the target Doppler image D are input into the Fourier forward transform unit for Fourier transform to obtain the first spectrum G of the grayscale image. s The first phase map G of the grayscale image p And the second spectrogram D of the Doppler image s The second phase map D of the Doppler image p The calculation formula is:

[0087] (D s D p ) = FFT(D); (G s G p ) = FFT(G); where FFT(·) is the forward Fourier transform.

[0088] The first spectrum G is respectively s and the second spectrum D s The input is fed into the hybrid filter unit for hybrid filtering to obtain the first low-frequency filter G. filterLow Second low-frequency filter D filterLow and the first high-frequency filter G filterHigh Second high frequency filter D filterHigh The calculation formula is:

[0089] D filterLow D filterHigh =D s ×HF(H D );

[0090] G filterLow G filterHigh =G s ×HF(R G );

[0091] Where HF(·) is a circular filter, R D and R G These are the radii of circular filters of different sizes.

[0092] The first low-frequency filter G filterLow The low-frequency characteristics and the first high-frequency filter G filterHigh The high-frequency characteristics in the first phase map G p The input is fed into the inverse Fourier transform unit to obtain the grayscale high-frequency feature G. h Gray-scale low-frequency characteristics G l The calculation formula is:

[0093] G h =IFFT(G filterHigh G p);

[0094] G l =IFFT(G filterLow G p );

[0095] IFFT(·) is the inverse Fourier transform.

[0096] The second low-frequency filter D filterLow The low-frequency characteristics and the second high-frequency filter D filterHigh The high-frequency characteristics in the second phase diagram D p The input is fed into the inverse Fourier transform unit to obtain the color high-frequency feature D. h Color low-frequency characteristics D l The calculation formula is:

[0097] D h =IFFT(D filterHigh D p );

[0098] D L =IFFT(D filterLow D p ).

[0099] It is understandable that, such as Figure 2 As shown, see details. Figure 2 On the left, this invention constructs an image decomposition module based on Fourier transform and hybrid filters to extract low-frequency global features and high-frequency local features. First, Fourier transform is performed on the input gray image G and the color Doppler image D to obtain the spectrum diagrams (G) corresponding to different modes. s D s ) and phase diagram (G p D p The spectrogram reflects the intensity of different spatial frequency components in the image. The phase diagram reflects the phase shift of each frequency component. Subsequently, a hybrid filtering operation is performed on the spectrograms of different modes to obtain the low-frequency filters (D) corresponding to different modes. filterLow G filterLow ) and high-frequency filters (D filterHigh G filterHigh The process involves using low-pass filtering to extract the low-frequency components of the original data and high-pass filtering to extract the high-frequency components. Finally, the different frequency features and phase maps are used as inputs to the inverse Fourier transform to obtain the high and low frequency information corresponding to the original data, i.e., the first spectrum G of the grayscale image. s The first phase map G of the grayscale image p And the second spectrogram D of the Doppler image s The second phase map D of the Doppler image p .

[0100] Step 400: Input the grayscale low-frequency features and the color low-frequency features into the cross-transformer fusion module for fusion to obtain fused low-frequency information. Input the grayscale high-frequency features and the color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information.

[0101] like Figure 3 As shown, it can be understood that, in order to reduce the computational complexity of the transformer model and enhance the interactive learning capability of multimodal low-frequency features, this invention constructs a cross-transformer fusion module (CFT). The cross-transformer fusion module includes: a group fusion unit, a feature dimensionality reduction unit, a selection attention unit, a Norm unit, and an Mlp unit.

[0102] In the cross-Transformer fusion module, the low-frequency features D of the two modes are combined. L G L The features after passing through the GroupMix unit are used as query features D for Selective Attention (SA). q G q =GroupMix(D L GroupMix (G) L The features of the two modalities after convolutional fusion (DownCony) are used as the key features D for Selective Attention (SA). k G k =DownConv(D L G L ), value feature D v G v =DownConv(D L G L This enables direct interaction between different modalities.

[0103] The step of inputting the grayscale low-frequency features and the color low-frequency features into the cross-Transformer fusion module for fusion to obtain fused low-frequency information specifically includes:

[0104] The grayscale low-frequency feature G l and the color low-frequency feature D l The inputs are respectively fed into the group fusion unit for channel-dimensional segmentation, resulting in four feature vectors. Three different sized convolution kernels are then used to convolve the four feature vectors along the channel dimension to obtain the grayscale low-frequency feature G. l The corresponding first query feature G q and the color low-frequency feature Dl The corresponding second query feature D q .

[0105] Understandably, in order to extract more comprehensive and refined features, capture the correlation between tokens and groups, and obtain higher representation capabilities, this invention constructs a group fusion unit, such as... Figure 3 As shown, D L G L Q∈R are respectively used as inputs to the group fusion module. H×W×C H, W, and C represent the height, width, and number of channels of the input feature, respectively.

[0106] The specific implementation process of the group fusion module is as follows: Figure 4 As shown, Q is first divided along the channel dimension to obtain... Then, three different convolution kernel operators are used to convolve the feature vector along the channel dimension to aggregate adjacent token features in the feature vector to obtain Q′∈R. H×W×C The calculation process is as follows:

[0107] Q1, Q2, Q3, Q4=Split(Q);

[0108] Q′=CAT(Conv 7×7 (Q1), Conv 5×5 (Q2), Conv 3×3 (Q3), Q4);

[0109] Here, Split(·) represents the splitting operation, CAT(·) represents the aggregation operation, and Conv(·) represents the convolution operation. D L G L After group fusion, the first query feature G after aggregation is obtained. q Second query feature D q .

[0110] Furthermore, the grayscale low-frequency feature G l and the color low-frequency feature D l The grayscale low-frequency feature G is obtained by performing dimensionality reduction operations on the feature dimensionality reduction unit. l The corresponding first key feature G k and the first value feature G v and the color low-frequency feature D l The corresponding second key feature D k and second-valued feature D v .

[0111] The grayscale low-frequency features G are respectively l and the color low-frequency feature D lThe corresponding query features, key features, and value features are input into the selection attention unit for feature extraction to obtain the first output feature T. G Second output feature T D .

[0112] like Figure 5 As shown, it can be understood that, in order to achieve more refined feature extraction and to focus on more effective information, this invention designs a selective attention unit to replace the traditional attention unit.

[0113] The attention unit selects tokens that are relevant to the query vector from the top k key vectors by calculating the relationship graph between the query vector and the key vectors. This allows for dynamic attention to useful information. The attention operation needs to be performed on the features of the Doppler image and the grayscale image separately. The specific process is as follows:

[0114] The first query feature G is respectively q The first key feature G k and the first value feature G v The data is divided into three parts: the first query vector Q1, the first key vector K1, and the first value vector V1. R represents the numerical value of the image, P represents the number of regions to be divided, and H, W, and C represent the height, width, and number of channels of the feature, respectively.

[0115] Normalize the first query vector Q1 and the first key vector K1 to obtain the normalized Q1′ and Calculate the first relation matrix A1 between the first query vector Q1 and the first key vector K1 based on the normalization results:

[0116]

[0117] Where M represents the transpose of the matrix.

[0118] Select a preset number of tokens from the first relation matrix A1, and transform the first key vector K1 and the first value vector V1 according to the tokens to obtain the transformed first key vector K1″ and the transformed first value vector.

[0119] Using the first query vector Q1, the transformed first key vector K1″, and the transformed first value vector V1′ as the query term, key term, and value of the selected attention unit, respectively, the first attention SA1 is calculated:

[0120]

[0121] Where SA(·) represents the attention function, D represents the feature embedding dimension, and Softmax(·) represents the normalization function.

[0122] Calculate the first computational attention SA1 and the first query feature G q The sum of these values ​​yields the first output feature T. G :T G =SA1+G a .

[0123] The second query feature D is respectively q The second key feature D k and the second value feature D v The data is divided into two parts: a second query vector Q2, a second key vector K2, and a second value vector V2.

[0124] Normalize the second query vector Q2 and the second key vector K2 to obtain the normalized Q2′ and Calculate the second relation matrix A2 between the second query vector Q2 and the second key vector K2 based on the normalization results:

[0125]

[0126] Select a preset number of tokens from the second relation matrix A2, and transform the second key vector K2 and the second value vector V2 according to the tokens to obtain the transformed second key vector K2″ and the transformed second value vector.

[0127] Using the second query vector Q2, the transformed second key vector K2″, and the transformed second value vector V2′ as the query term, key term, and value of the selected attention unit, respectively, the second attention SA2 is calculated:

[0128]

[0129] Calculate the second computational attention SA2 and the second query feature D q The sum of these values ​​yields the second output feature T. D :T D =SA2+D q .

[0130] Furthermore, the first output feature T G and the second output feature T D The results are input into the Norm unit and the Mlp unit respectively for calculation, and the first calculation result is obtained. Second calculation result

[0131] Furthermore, the first calculation result With the first output feature T G The fusion is performed to obtain the first fusion feature, and the second calculation result is then used. With the second output feature T D The first and second fusion features are then fused to obtain the second fusion feature. The first and second fusion features are then fused together to obtain the fused low-frequency information T. The fusion process is as follows:

[0132]

[0133]

[0134]

[0135] Where norm(·) is the layer normalization operation, mlp(·) is the mlp function in vit, and SA(·) represents the selection attention function.

[0136] Further, the step of inputting the grayscale high-frequency features and the color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain the fused high-frequency information specifically includes:

[0137] High-frequency features of grayscale G h The input to the convolutional residual block in the cross-convolution fusion module yields a first calculation result, which is then compared with the color high-frequency feature D. h Summation yields the color high-frequency feature D′. h ;

[0138] The multimodal high-frequency feature D′ h The input to the convolutional residual block in the cross-convolution fusion module yields a second calculation result, which is then combined with the grayscale high-frequency feature G. h Multiplying them together yields the grayscale high-frequency feature G′. h ;

[0139] The color high-frequency feature D′ h and the grayscale high-frequency feature G′ h The fused multimodal high-frequency information is obtained by connecting the components using the cat operation.

[0140] like Figure 6 As shown, it can be understood that the cross-convolution fusion module utilizes convolution to extract local detail features, using multimodal high-frequency features G. h D h ∈R H×W×C / 2As input, the cardiac structural features of the ultrasound image are first fused into the Doppler blood flow features using convolutional residual blocks (CRBs) to enhance the high-frequency color features; the high-frequency Doppler features D′ combining structural features and blood flow information are then analyzed. h Then it propagates back to the grayscale features to enhance the high-frequency grayscale features G′. h The calculation process is as follows:

[0141] D′ h =CRB(G h )+D h ;

[0142] G′ h =G h ⊙exp(CRB(D′ h ));

[0143] Here, ⊙ represents the Hadamard product.

[0144] Step 500: Input the low-frequency information and the high-frequency information into the classifier to obtain the classification result of the target echocardiogram.

[0145] Specifically, the low-frequency information is summed with the high-frequency information, and the summation result is used as the input of the classification head to obtain the classification result of the target echocardiogram.

[0146] As can be seen, this invention designs an echocardiographic analysis method from the perspective of frequency and multimodality. This invention combines the spectral analysis characteristics of Fourier transform with the complementary advantages of convolution and transformer. First, a frequency decomposition module based on Fourier transform is constructed to extract high-frequency and low-frequency features corresponding to different modes. Second, a cross-Transformer fusion module is designed for multimodal low-frequency features, achieving the fusion of global multimodal information while reducing computational costs. Furthermore, a cross-convolution fusion module is designed for multimodal high-frequency features, realizing the fusion of anatomical structural details in two-dimensional grayscale images with blood flow information in color Doppler images. This improves the automation capability and accuracy of echocardiographic analysis.

[0147] Furthermore, such as Figure 7 As shown, based on the above-described echocardiographic analysis method, the present invention also provides an echocardiographic analysis system, wherein the echocardiographic analysis system includes:

[0148] The target image acquisition module 51 is used to acquire the target echocardiogram and obtain the target grayscale image and the target Doppler image based on the target echocardiogram.

[0149] The model building and training module 52 is used to build an echocardiogram analysis model, train and test the echocardiogram analysis model to obtain a target model, wherein the target model includes: a frequency decomposition module, a cross-Transformer fusion module and a cross-convolution fusion module;

[0150] The high- and low-frequency decomposition module 53 is used to input the target grayscale image and the target Doppler image into the frequency decomposition module for a mixed filtering operation to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features and color low-frequency features;

[0151] The feature fusion module 54 is used to input the grayscale low-frequency features and the color low-frequency features into the cross-Transformer fusion module for fusion to obtain fused low-frequency information, and to input the grayscale high-frequency features and the color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information.

[0152] The classification result acquisition module 55 is used to input the low-frequency information and the high-frequency information into the classifier to obtain the classification result of the target echocardiogram.

[0153] Furthermore, such as Figure 8 As shown, based on the above-mentioned echocardiography analysis method and system, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 8 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0154] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. Further, the memory 20 may include both internal and external storage units. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores an echocardiogram analysis program 40, which can be executed by the processor 10 to implement the echocardiogram analysis method of this application.

[0155] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the echocardiography analysis method.

[0156] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0157] In one embodiment, when the processor 10 executes the echocardiography analysis program 40 in the memory 20, the following steps are performed:

[0158] Obtain the target echocardiogram, and obtain the target grayscale image and the target Doppler image based on the target echocardiogram;

[0159] An echocardiographic analysis model is constructed, and the echocardiographic analysis model is trained and tested to obtain a target model. The target model includes: a frequency decomposition module, a cross-Transformer fusion module, and a cross-convolution fusion module.

[0160] The target grayscale image and the target Doppler image are input into the frequency decomposition module for a mixed filtering operation to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features.

[0161] The grayscale low-frequency features and the color low-frequency features are input into the cross-transformer fusion module for fusion to obtain fused low-frequency information. The grayscale high-frequency features and the color high-frequency features are input into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information.

[0162] The low-frequency information and the high-frequency information are input into a classifier to obtain the classification result of the target echocardiogram.

[0163] The step of training and testing the echocardiographic analysis model to obtain the target model specifically includes:

[0164] Historical video sequences are acquired, and the historical video sequences are converted into image data. The image data is preprocessed to obtain sample data, and the sample data is divided into training set and test set according to a preset ratio.

[0165] Based on the training set, the echocardiogram analysis model is trained by performing a preset number of cross-validations using ADAM, and the loss function is calculated. When the loss function reaches a preset convergence condition, the trained echocardiogram analysis model is obtained.

[0166] The trained echocardiography analysis model was evaluated using the test set to obtain the target model that meets the preset requirements.

[0167] The frequency decomposition module includes a forward Fourier transform unit, a hybrid filter unit, and an inverse Fourier transform unit.

[0168] The cross-Transformer fusion module includes: a group fusion unit, a feature dimensionality reduction unit, a selection attention unit, a Norm unit, and an Mlp unit.

[0169] Specifically, the step of inputting the target grayscale image and the target Doppler image into the frequency decomposition module for Fourier transform and hybrid filtering operations to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features includes:

[0170] The target grayscale image G and the target Doppler image D are input into the Fourier forward transform unit for Fourier transform to obtain the first spectrum G of the grayscale image. s The first phase map G of the grayscale image p And the second spectrogram D of the Doppler image s The second phase map D of the Doppler image p ;

[0171] The first spectrum G is respectively s and the second spectrum D s The input is fed into the hybrid filter unit for hybrid filtering to obtain the first low-frequency filter G. filterLow Second low-frequency filter D filterLow and the first high-frequency filter G filterHigh Second high frequency filter D filterHigh ;

[0172] The first low-frequency filter G filterLow The low-frequency characteristics and the first high-frequency filter G filterHigh The high-frequency characteristics in the first phase map G p The input is fed into the inverse Fourier transform unit to obtain the grayscale high-frequency feature G. h Gray-scale low-frequency characteristics G l ;

[0173] The second low-frequency filter D filterLow The low-frequency characteristics and the second high-frequency filter DfilterHigh The high-frequency characteristics in the second phase diagram D p The input is fed into the inverse Fourier transform unit to obtain the color high-frequency feature D. h Color low-frequency characteristics D l .

[0174] Specifically, the step of inputting the grayscale low-frequency features and the color low-frequency features into the cross-Transformer fusion module for fusion to obtain the fused low-frequency information includes:

[0175] The grayscale low-frequency feature G l and the color low-frequency feature D l The inputs are respectively fed into the group fusion unit for channel-dimensional segmentation, resulting in four feature vectors. Three different sized convolution kernels are then used to convolve the four feature vectors along the channel dimension to obtain the grayscale low-frequency feature G. l The corresponding first query feature G q and the color low-frequency feature D l The corresponding second query feature D q ;

[0176] The grayscale low-frequency feature G l and the color low-frequency feature D l The grayscale low-frequency feature G is obtained by performing dimensionality reduction operations on the feature dimensionality reduction unit. l The corresponding first key feature G k and the first value feature G v and the color low-frequency feature D l The corresponding second key feature D k and second-valued feature D v ;

[0177] The grayscale low-frequency features G are respectively l and the color low-frequency feature D l The corresponding query features, key features, and value features are input into the selection attention unit for feature extraction to obtain the first output feature T. G Second output feature T D ;

[0178] The first output feature T G and the second output feature T D The results are input into the Norm unit and the Mlp unit respectively for calculation, resulting in a first calculation result and a second calculation result.

[0179] The first calculation result is compared with the first output feature T. GThe two features are then fused to obtain the first fused feature. The second calculation result is then combined with the second output feature T. D The fusion process yields the second fusion feature.

[0180] The first fusion feature and the second fusion feature are fused to obtain the fused low-frequency information T.

[0181] Wherein, the grayscale low-frequency features G are respectively l and the color low-frequency feature D l The corresponding query features, key features, and value features are input into the selection attention unit for feature extraction to obtain the first output feature T. G Second output feature T D Specifically, it includes:

[0182] The first query feature G is respectively q The first key feature G k and the first value feature G v The data is divided into three parts: the first query vector Q1, the first key vector K1, and the first value vector V1. R represents the numerical value of the image, P represents the number of regions to be divided, and H, W, and C represent the height, width, and number of channels of the feature, respectively.

[0183] Normalize the first query vector Q1 and the first key vector K1 to obtain the normalized Q1′ and Calculate the first relation matrix A1 between the first query vector Q1 and the first key vector K1 based on the normalization results:

[0184]

[0185] Where M represents the transpose of the matrix;

[0186] Select a preset number of tokens from the first relation matrix A1, and transform the first key vector K1 and the first value vector V1 according to the tokens to obtain the transformed first key vector K1″ and the transformed first value vector.

[0187] Using the first query vector Q1, the transformed first key vector K1″, and the transformed first value vector V1′ as the query term, key term, and value of the selected attention unit, respectively, the first attention SA1 is calculated:

[0188]

[0189] Where SA(.) represents the attention function, D represents the feature embedding dimension, and Softmax(.) represents the normalization function;

[0190] Calculate the first computational attention SA1 and the first query feature G q The sum of these values ​​yields the first output feature T. G :T G =SA1+G q ;

[0191] The second query feature D is respectively q The second key feature D k and the second value feature D v The data is divided into two parts: a second query vector Q2, a second key vector K2, and a second value vector V2.

[0192] Normalize the second query vector Q2 and the second key vector K2 to obtain the normalized Q2′ and Calculate the second relation matrix A2 between the second query vector Q2 and the second key vector K2 based on the normalization results:

[0193]

[0194] Select a preset number of tokens from the second relation matrix A2, and transform the second key vector K2 and the second value vector V2 according to the tokens to obtain the transformed second key vector K2″ and the transformed second value vector.

[0195] Using the second query vector Q2, the transformed second key vector K2″, and the transformed second value vector V2′ as the query term, key term, and value of the selected attention unit, respectively, the second attention SA2 is calculated:

[0196]

[0197] Calculate the second computational attention SA2 and the second query feature D q The sum of these values ​​yields the second output feature T. D :T D =SA2+D q .

[0198] Specifically, the step of inputting the grayscale high-frequency features and the color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain the fused high-frequency information includes:

[0199] High-frequency features of grayscale G h The input to the convolutional residual block in the cross-convolution fusion module yields a first calculation result, which is then compared with the color high-frequency feature D. hSummation yields the color high-frequency feature D′. h ;

[0200] The multimodal high-frequency feature D′ h The input to the convolutional residual block in the cross-convolution fusion module yields a second calculation result, which is then combined with the grayscale high-frequency feature G. h Multiplying them together yields the grayscale high-frequency feature G′. h ;

[0201] The color high-frequency feature D′ h and the grayscale high-frequency feature G′ h The fused multimodal high-frequency information is obtained by connecting the components using the cat operation.

[0202] In summary, this invention provides an echocardiographic analysis method and related equipment. The method includes: acquiring a target grayscale image and a target Doppler image; constructing a target model, which includes a frequency decomposition module, a cross-Transformer fusion module, and a cross-convolution fusion module; obtaining grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features based on the target grayscale image and the target Doppler image; inputting the grayscale low-frequency features and color low-frequency features into the cross-Transformer fusion module for fusion to obtain fused low-frequency information; inputting the grayscale high-frequency features and color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information; and inputting the low-frequency information and high-frequency information into a classifier to obtain the classification result of the target echocardiogram. This invention combines the spectral analysis characteristics of Fourier transform with the complementary advantages of convolution and transformer, overcoming the shortcomings of slow efficiency and error susceptibility of manual analysis, and enabling efficient and accurate analysis of echocardiograms.

[0203] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0204] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0205] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for echocardiographic analysis, characterized in that, The aforementioned echocardiographic analysis method includes: Obtain the target echocardiogram, and obtain the target grayscale image and the target Doppler image based on the target echocardiogram; An echocardiographic analysis model is constructed, and the echocardiographic analysis model is trained and tested to obtain a target model. The target model includes: a frequency decomposition module, a cross-Transformer fusion module, and a cross-convolution fusion module. The target grayscale image and the target Doppler image are input into the frequency decomposition module for a mixed filtering operation to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features. The grayscale low-frequency features and the color low-frequency features are input into the cross-transformer fusion module for fusion to obtain fused low-frequency information. The grayscale high-frequency features and the color high-frequency features are input into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information. The low-frequency information and the high-frequency information are input into a classifier to obtain the classification result of the target echocardiogram.

2. The echocardiographic analysis method according to claim 1, characterized in that, The process of training and testing the echocardiographic analysis model to obtain the target model specifically includes: Historical video sequences are acquired, and the historical video sequences are converted into image data. The image data is preprocessed to obtain sample data, and the sample data is divided into training set and test set according to a preset ratio. Based on the training set, the echocardiogram analysis model is trained by performing a preset number of cross-validations using ADAM, and the loss function is calculated. When the loss function reaches a preset convergence condition, the trained echocardiogram analysis model is obtained. The trained echocardiography analysis model was evaluated using the test set to obtain the target model that meets the preset requirements.

3. The echocardiographic analysis method according to claim 1, characterized in that, The frequency decomposition module includes a forward Fourier transform unit, a hybrid filter unit, and an inverse Fourier transform unit. The cross-Transformer fusion module includes: a group fusion unit, a feature dimensionality reduction unit, a selection attention unit, a Norm unit, and an Mlp unit.

4. The echocardiographic analysis method according to claim 3, characterized in that, The step of inputting the target grayscale image and the target Doppler image into the frequency decomposition module for a mixed filtering operation to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features specifically includes: inputting the target gray-scale image G and the target Doppler image D into the Fourier inverse transformation unit to perform Fourier transformation, to obtain a first spectrum image G of the gray-scale image s and a first phase image G of the gray-scale image p a second spectrum image D of the Doppler image s and a second phase image D of the Doppler image p ; The first spectrum G is respectively s and the second spectrum D s The input is fed into the hybrid filter unit for hybrid filtering to obtain the first low-frequency filter G. filterLow Second low-frequency filter D filterLow and the first high-frequency filter G filterHigh Second high frequency filter D filterHigh ; The first low-frequency filter G filterLow The low-frequency characteristics and the first high-frequency filter G filterHigh The high-frequency characteristics in the first phase map G p The input is fed into the inverse Fourier transform unit to obtain the grayscale high-frequency feature G. h Gray-scale low-frequency characteristics G l ; The second low-frequency filter D filterLow The low-frequency characteristics and the second high-frequency filter D filterHigh The high-frequency characteristics in the second phase diagram D p The input is fed into the inverse Fourier transform unit to obtain the color high-frequency feature D. h Color low-frequency characteristics D l .

5. The echocardiographic analysis method according to claim 4, characterized in that, The step of inputting the grayscale low-frequency features and the color low-frequency features into the cross-Transformer fusion module for fusion to obtain fused low-frequency information specifically includes: The grayscale low-frequency feature G l and the color low-frequency feature D l The inputs are respectively fed into the group fusion unit for channel-dimensional segmentation, resulting in four feature vectors. Three different sized convolution kernels are then used to convolve the four feature vectors along the channel dimension to obtain the grayscale low-frequency feature G. l The corresponding first query feature G q and the color low-frequency feature D l The corresponding second query feature D q ; The grayscale low-frequency feature G l and the color low-frequency feature D l The grayscale low-frequency feature G is obtained by performing dimensionality reduction operations on the feature dimensionality reduction unit. l The corresponding first key feature G k and the first value feature G v and the color low-frequency feature D l The corresponding second key feature D k and second-valued feature D v ; The grayscale low-frequency features G are respectively l and the color low-frequency feature D l The corresponding query features, key features, and value features are input into the selection attention unit for feature extraction to obtain the first output feature T. G Second output feature T D ; The first output feature T G and the second output feature T D The results are input into the Norm unit and the Mlp unit respectively for calculation, resulting in a first calculation result and a second calculation result. The first calculation result is compared with the first output feature T. G The two features are then fused to obtain the first fused feature. The second calculation result is then combined with the second output feature T. D The fusion process yields the second fusion feature. The first fusion feature and the second fusion feature are fused to obtain the fused low-frequency information T.

6. The echocardiographic analysis method according to claim 5, characterized in that, The grayscale low-frequency features G are respectively... l and the color low-frequency feature D l The corresponding query features, key features, and value features are input into the selection attention unit for feature extraction to obtain the first output feature T. G Second output feature T D Specifically, it includes: The first query feature G is respectively q The first key feature G k and the first value feature G v The data is divided into three parts: the first query vector Q1, the first key vector K1, and the first value vector V1. R represents the numerical value of the image, P represents the number of regions to be divided, and H, W, and C represent the height, width, and number of channels of the feature, respectively. Normalize the first query vector Q1 and the first key vector K1 to obtain the normalized Q1′ and K1′. Calculate the first relation matrix A1 between the first query vector Q1 and the first key vector K1 based on the normalization results: Where M represents the transpose of the matrix; A predetermined number of tokens are selected from the first relation matrix A1. Based on the tokens, the first key vector K1 and the first value vector V1 are transformed to obtain the transformed first key vector K1″ and the transformed first value vector V1′. Using the first query vector Q1, the transformed first key vector K1″, and the transformed first value vector V1′ as the query term, key term, and value of the selected attention unit, respectively, the first attention SA1 is calculated: Where SA(·) represents the attention function, D represents the feature embedding dimension, and Softmax(·) represents the normalization function; Calculate the first attention SA1 and the first query feature G q The sum of these values ​​yields the first output feature T. G :T G =SA1+G q ; The second query feature D is respectively q The second key feature D k and the second value feature D v The data is divided into two parts: a second query vector Q2, a second key vector K2, and a second value vector V2. Normalize the second query vector Q2 and the second key vector K2 to obtain the normalized Q2′ and K2′. Calculate the second relation matrix A2 between the second query vector Q2 and the second key vector K2 based on the normalization results: A predetermined number of tokens are selected from the second relation matrix A2. Based on these tokens, the second key vector K2 and the second value vector V2 are transformed to obtain the transformed second key vector K2″ and the transformed second value vector V2′. Using the second query vector Q2, the transformed second key vector K2″, and the transformed second value vector V2′ as the query term, key term, and value of the selected attention unit, respectively, the second attention SA2 is calculated: Calculate the second attention SA2 and the second query feature D q The sum of these values ​​yields the second output feature T. D :T D =SA2+D q .

7. The echocardiographic analysis method according to claim 4, characterized in that, The step of inputting the grayscale high-frequency features and the color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information specifically includes: High-frequency features of grayscale G h The input to the convolutional residual block in the cross-convolution fusion module yields a first calculation result, which is then compared with the color high-frequency feature D. h Summing yields the high-frequency feature D' h ; The high-frequency feature D' h The input to the convolutional residual block in the cross-convolution fusion module yields a second calculation result, which is then combined with the grayscale high-frequency feature G. h Multiplying yields the high-frequency feature G' h ; The high-frequency feature D' h and the high-frequency feature G' h The fused multimodal high-frequency information is obtained by connecting the components using the cat operation.

8. An echocardiographic analysis system, characterized in that, The echocardiographic analysis system includes: The target image acquisition module is used to acquire the target echocardiogram and obtain the target grayscale image and the target Doppler image based on the target echocardiogram; The model building and training module is used to build an echocardiogram analysis model, train and test the echocardiogram analysis model to obtain a target model, wherein the target model includes: a frequency decomposition module, a cross-Transformer fusion module and a cross-convolution fusion module; The high- and low-frequency decomposition module is used to input the target grayscale image and the target Doppler image into the frequency decomposition module for a mixed filtering operation to obtain grayscale high-frequency features, grayscale low-frequency features, color high-frequency features, and color low-frequency features. The feature fusion module is used to input the grayscale low-frequency features and the color low-frequency features into the cross-Transformer fusion module for fusion to obtain fused low-frequency information, and to input the grayscale high-frequency features and the color high-frequency features into the cross-convolution fusion module for feature extraction and fusion to obtain fused high-frequency information. The classification result acquisition module is used to input the low-frequency information and the high-frequency information into the classifier to obtain the classification result of the target echocardiogram.

9. A terminal, characterized in that, The terminal includes: a memory, a processor, and an echocardiography analysis program stored in the memory and executable on the processor, wherein the echocardiography analysis program, when executed by the processor, implements the steps of the echocardiography analysis method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an echocardiography analysis program, which, when executed by a processor, implements the steps of the echocardiography analysis method as described in any one of claims 1-7.

Images