A method and device for judging pericardial effusion by using cardiac ultrasound video responsibility frame

Abstract

The application discloses a method and device for judging pericardial effusion by using a heart ultrasound video responsibility frame, and the method comprises the following steps: collecting a heart ultrasound standard section video, unifying a format, and training a static image classification neural network model by using the heart ultrasound standard section video; building a pericardial effusion judgment model based on the static image classification neural network model, inputting a heart ultrasound standard section video to be identified into the pericardial effusion judgment model; the pericardial effusion judgment model outputs a feature matrix of the video, a random forest model is used to calculate the feature matrix of the video to obtain a feature importance matrix of the video, a responsibility frame used for video benign and malignant judgment is selected based on information entropy reduction and the feature importance matrix of the video; and the static image classification neural network model outputs whether the heart ultrasound video has pericardial effusion based on the responsibility frame used for video benign and malignant judgment. The application can realize automatic identification of whether there is pericardial effusion in an ultrasound heartbeat video.

Description

A method and apparatus for determining pericardial effusion using cardiac ultrasound video responsibility frames. Technical Field

[0001] This invention relates to the field of image recognition, and more particularly to a method and apparatus for determining pericardial effusion using responsible frames of cardiac ultrasound video. 。 Background Technology

[0002] Pericardial effusion is defined as an increase in fluid within the pericardial cavity, which can manifest as transudate, exudate, hemocardiosis, or empyema. Normally, only a small amount of fluid is present in the pericardial cavity, while an accumulation of more than 50 mL is considered abnormal. Its causes are highly diverse. Besides diseases that selectively affect the pericardium, other common medical conditions such as tuberculosis, hypothyroidism, end-stage renal disease, viral infections, and postoperative complications can all cause pericardial effusion. Furthermore, pericardial effusion can provide substantial evidence for the etiological analysis of various diseases.

[0003] The impact of pericardial effusion on cardiac hemodynamics is complex, but once the effusion volume exceeds a certain threshold, even small changes in fluid volume can lead to significant changes in intracardiac pressure, potentially causing cardiac tamponade and threatening life. On the other hand, asymptomatic pericardial effusions are often discovered incidentally during examinations for unrelated conditions, such as in patients with chest pain due to acute pericarditis.

[0004] Currently, echocardiography is the preferred imaging method for assessing the pericardium, due to its ease of use, low cost, and strong ability to assess cardiac hemodynamics. In a typical ultrasound examination, the sonographer follows clinical guidelines to manipulate the ultrasound probe to cover the lesion site, ultimately selecting a few characteristic images to write the ultrasound report. The entire scan video is replaced by responsibility frame images on a few representative responsibility sections. These responsibility frame images should reflect the unique morphological information of the lesion site and can indicate whether it is benign or malignant. However, ultrasound diagnosis heavily relies on the sonographer's clinical experience. In clinical practice, there are numerous cases of misdiagnosis and missed diagnosis due to the omission of important responsibility sections.

[0005] In summary, there is an urgent need for an automated method to identify pericardial effusion in echocardiograms and to extract the responsible section and frame for pericardial effusion. However, there is currently no artificial intelligence method for intelligent identification of pericardial effusion. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method and apparatus for determining pericardial effusion using cardiac ultrasound video responsibility frames, which can automatically identify pericardial effusion in echocardiograms.

[0007] A first aspect of the present invention provides a method for determining pericardial effusion using a responsible frame of cardiac ultrasound video, comprising:

[0008] Acquire standard-section videos of cardiac ultrasound and standardize their format; use these standard-section videos to train a static image classification neural network model.

[0009] A pericardial effusion identification model is built based on the static image classification neural network model. The standard cross-sectional video of the cardiac ultrasound to be identified is input into the pericardial effusion identification model; the feature matrix of the video is obtained.

[0010] The pericardial effusion judgment model outputs the feature matrix of the video. The feature matrix of the video is calculated using a random forest model to obtain the feature importance matrix of the video. Based on the information entropy reduction and the feature importance matrix of the video, the responsible frame for judging the benignity or malignancy of the video is selected.

[0011] The static image classification neural network model outputs whether there is pericardial effusion in the cardiac ultrasound video based on the responsibility frame used for determining the benignity or malignancy of the video.

[0012] Optionally, the acquisition of standard-section video of cardiac ultrasound, and the standardization of the format, includes:

[0013] Videos of the parasternal acoustic window PSAX section, apical acoustic window A4C section, parasternal acoustic window PLAX section, and subxiphoid acoustic window SC4C section are acquired, and the frame rate, width, height, and number of channels of the video are standardized.

[0014] Optionally, training the static image classification neural network model using the standard cross-sectional video of the cardiac ultrasound includes:

[0015] One frame is extracted from the standard cross-sectional video of cardiac ultrasound at fixed intervals as the input image;

[0016] The input image is randomly subjected to data augmentation processing including rotation, translation, flipping, and Gaussian noise.

[0017] A static image classification neural network model is trained using stochastic gradient descent. The static image classification neural network model includes a skeleton part for extracting image features and a fully connected layer part for classification. The feature matrix obtained from the last convolutional layer of the static image classification neural network model is added to the image's positional bias code to obtain the image's spatial position information, which is used to evaluate the region of responsibility.

[0018] Optionally, the step of using a random forest model to calculate the feature matrix of the video to obtain the feature importance matrix of the video includes:

[0019] The feature importance matrix of the video is obtained by multiplying the eigenvalue of each feature dimension in the feature matrix by the importance value of that feature dimension in the random forest.

[0020] Optionally, the step of multiplying the feature value of each feature dimension in the video's feature matrix by the importance value of that feature dimension in the random forest to obtain the video's feature importance matrix includes:

[0021] The target image in the feature matrix of the video is selected by the information entropy reduction method. Max pooling is performed only on the target image in the column direction to obtain the feature matrix contributed by the Top N images. The feature matrix contributed by the Top N images is then fed into the random forest model to obtain the responsibility frame used for video benignity / malignancy judgment.

[0022] Optionally, the selection of the responsible frame for determining the goodness or badness of the video based on information entropy reduction and the feature importance matrix of the video includes:

[0023] The value of each channel in the video feature importance matrix is ​​taken as the total information contributed by the entire video in that feature dimension, and the value of each channel in the feature importance matrix of each frame image is taken as the single frame information contributed by that frame image in the feature dimension.

[0024] Subtracting the feature importance matrix of a single frame image from the feature importance matrix of the video yields the remaining feature importance matrix; summing the information content of each feature dimension of the remaining feature importance matrix yields the remaining information entropy after subtracting the feature importance matrix of a single frame image from the video.

[0025] The frame with the minimum residual entropy is used as the responsible frame for determining the benignity or malignancy of the video.

[0026] Optionally, the method further includes:

[0027] Use class activation graphs to generate dynamic heatmaps of responsibility regions.

[0028] A second aspect of the present invention provides an apparatus for determining pericardial effusion using cardiac ultrasound video responsibility frames, comprising:

[0029] The training module is used to acquire standard cross-sectional videos of cardiac ultrasound and standardize their format, and to train a static image classification neural network model using the standard cross-sectional videos of cardiac ultrasound.

[0030] The matrix acquisition module is used to build a pericardial effusion judgment model based on the static image classification neural network model, and input the standard cross-sectional video of the cardiac ultrasound to be identified into the pericardial effusion judgment model.

[0031] The identification module is used to output the feature matrix of the video from the pericardial effusion judgment model, calculate the feature matrix of the video using a random forest model to obtain the feature importance matrix of the video, and select the responsible frame for judging the benignity or malignancy of the video based on information entropy reduction and the feature importance matrix of the video.

[0032] A judgment module is used by the static image classification neural network model to output whether there is pericardial effusion in the cardiac ultrasound video based on the responsibility frame used for video benignity / malignancy judgment. A third aspect of the present invention provides an electronic device, comprising:

[0033] At least one processor; and at least one memory communicatively connected to the processor, wherein the memory stores program instructions executable by the processor, and the processor invokes the program instructions to perform the method as described in the first aspect of the present invention.

[0034] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a computer, performs the method described in the first aspect of the embodiments of the present invention.

[0035] This invention uses a static image classification neural network model, a random forest model, and information entropy reduction to automatically identify whether there is pericardial effusion in echocardiogram videos, and can extract the responsible frame from the echocardiogram video to assist in the judgment of pericardial effusion. Attached Figure Description

[0036] Figure 1 is a flowchart illustrating a method for determining pericardial effusion using cardiac ultrasound video responsibility frames in an embodiment of the present invention.

[0037] Figure 2 is a schematic diagram of the network structure of the echocardiogram video pericardial effusion judgment model in an embodiment of the present invention.

[0038] Figure 3 is a schematic diagram of the feature importance matrix of the generated video and each frame image in an embodiment of the present invention.

[0039] Figure 4 is a schematic diagram of the remaining feature importance matrix generated for a certain frame object in an embodiment of the present invention.

[0040] Figure 5 shows the video benignity / malignancy judgment model combined with information entropy in an embodiment of the present invention.

[0041] Figure 6 is a schematic diagram of a device for determining pericardial effusion using cardiac ultrasound video responsibility frames in an embodiment of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0044] Please refer to Figure 1. This invention provides a method for determining pericardial effusion using a responsible frame of cardiac ultrasound video, comprising:

[0045] Step 100: Acquire standard cross-sectional videos of cardiac ultrasound, standardize the format, and use the standard cross-sectional videos of cardiac ultrasound to train a static image classification neural network model.

[0046] Since the original videos come from different devices, the input videos need to be standardized to a uniform size: the video format example is (number of frames x width x height x number of channels), and the specific values ​​can be selected.

[0047] In this embodiment of the invention, videos of the parasternal acoustic window PSAX section, the apical acoustic window A4C section, the parasternal acoustic window PLAX section, and the subxiphoid acoustic window SC4C section are acquired, and the number of frames, width, height, and number of channels of the videos are unified.

[0048] Step 200: Build a pericardial effusion judgment model based on the static image classification neural network model, and input the standard cross-sectional video of the cardiac ultrasound to be identified into the pericardial effusion judgment model.

[0049] In this step, a classification neural network model is built to learn whether pericardial effusion is present in standard cross-sectional videos of echocardiography. The neural network model can be divided into a skeleton part for extracting image features and a fully connected layer part for classification. Different convolutional neural networks can be used for the skeleton part. For the feature matrix obtained from the last convolutional layer of the skeleton part, a max-pooling operation is performed to obtain a high-dimensional feature map of size K×4×4, which is then added to the positional bias code. This makes the resulting one-dimensional vector after dimensionality reduction has more accurate spatial location information, which is used to evaluate the region of responsibility in subsequent steps. After dimensionality reduction by convolution on the high-dimensional feature map, it is finally transformed into a 1×16K vector. The expanded vector features thus carry the positional information of the original image.

[0050] In one embodiment of the present invention, the static image classification neural network model includes a skeleton part for extracting image features and a fully connected layer part for classification. The skeleton network in the pericardial effusion judgment model inherits the weights of the static image benign / malignant judgment network trained in the static image classification neural network model. The feature matrix obtained from the last convolutional layer of the static image classification neural network model is added to the image's positional bias code to obtain image spatial location information, which is used to evaluate the region of responsibility.

[0051] The specific training steps for the static image classification neural network model are as follows:

[0052] One frame is extracted from a standard cross-sectional video of cardiac ultrasound at fixed intervals as the input image; for example, one frame is extracted from every x frames of the video as the input image.

[0053] To enhance the generalization ability of the neural network model and simulate the image displacement, scaling, and deflection that occur during actual clinical examinations of nodules, the input image is randomly augmented using methods including rotation, translation, flipping, and Gaussian noise.

[0054] A static image classification neural network model is trained using stochastic gradient descent. The main training process involves iteratively training and updating the weight and bias parameters using stochastic gradient descent, and learning the features of pericardial effusion in the image during the training process.

[0055] The skeleton network in the pericardial effusion assessment model inherits the weights of the pre-trained static image benign / malignant assessment network. After passing through the skeleton network, a selected frame image from the input video generates a 1×16k feature matrix. The resulting feature matrix, obtained by maximizing the values ​​in the column directions (i.e., the directions of the feature dimensions), integrates the important information contributed by each frame and represents the video's features; this matrix is ​​called the video's feature matrix. The structure of the echocardiogram video pericardial effusion assessment model is shown in Figure 2.

[0056] Step 300: The pericardial effusion judgment model outputs the feature matrix of the video. The feature matrix of the video is calculated using a random forest model to obtain the feature importance matrix of the video. Based on the information entropy reduction and the feature importance matrix of the video, the responsible frame for judging the benignity or malignancy of the video is selected.

[0057] Specifically, the feature importance matrix of the video is obtained by multiplying the feature value of each feature dimension (1×k dimensions) in the video's feature matrix by the importance value of that feature dimension in the random forest. The importance of each feature in the random forest is defined by the random forest model and is always a positive number. See Figure 3 for details.

[0058] Furthermore, the importance matrix is ​​processed by combining information entropy and the pericardial volume judgment model to form a video benignity / malignancy judgment model. As shown in Figure 4, the value of each channel in the video feature importance matrix is ​​taken as the total information contributed by the entire video in that feature dimension, and the value of each channel in the feature importance matrix of each frame image is taken as the single-frame information contributed by that frame image in the feature dimension; the feature importance matrix of the video is subtracted from the feature importance matrix of the single frame image to obtain the remaining feature importance matrix; the information of each feature dimension of the remaining feature importance matrix is ​​summed to obtain the remaining information entropy after subtracting the feature importance matrix of the single frame image from the video; the frame image with the minimum remaining information entropy is taken as the responsible frame for video benignity / malignancy judgment.

[0059] For example, the value of each channel in the feature importance matrix is ​​considered as the amount of information, and the value of each channel in the video feature importance matrix is ​​considered as the total amount of information contributed by the entire video in that feature dimension. Correspondingly, the value of each channel in the feature importance matrix of each frame is considered as the amount of information contributed by that frame in that feature dimension. For each frame, the feature importance matrix of that frame is subtracted from the video feature importance matrix to generate the residual feature importance matrix, as shown in Figure 5. The information values ​​of each feature dimension of the residual feature importance matrix are summed to obtain the residual entropy of the video after subtracting that frame. The frame that produces the minimum residual entropy is found, i.e., the most important responsible frame is found. After finding the most important responsible frame, the corresponding residual feature importance matrix is ​​considered as the new video feature importance matrix, and the next most important responsible frame is found using the same method.

[0060] Because after subtracting the most important responsible frame from the video feature importance matrix, the feature dimensions that previously contributed a large amount of information lose the significant information contribution of that responsible frame, resulting in a very small remaining amount of information in those feature dimensions. Therefore, when selecting the next most important responsible frame, the feature dimensions selected that contribute a large amount of information will differ from the first selected most important responsible frame. This allows for the acquisition of responsible frames with diverse features without the need for manually defining the frame-by-frame extraction distance.

[0061] Further, step 300, which involves multiplying the eigenvalue of each feature dimension in the video's feature matrix by the importance value of that feature dimension in the random forest to obtain the video's feature importance matrix, includes:

[0062] The target image in the feature matrix of the video is selected by the information entropy reduction method. Max pooling is performed only on the target image in the column direction to obtain the feature matrix contributed by the Top N images. The feature matrix contributed by the Top N images is then fed into the random forest model to obtain the responsibility frame used for video benignity / malignancy judgment.

[0063] This approach uses information entropy reduction to extract a certain number of key responsibility frames to replace the entire video for beneficence judgment, thereby further improving model performance. After the skeleton network (feature extractor) extracts the feature matrix of each frame, the information entropy reduction method is used to select the most important N frames. Max pooling is then performed only on these images in the column direction to generate a feature matrix contributed by the Top N images. This feature matrix is ​​then fed into a random forest to make the final beneficence judgment and output the Top N responsibility frames selected by the model.

[0064] Step 400: The static image classification neural network model outputs whether there is pericardial effusion in the cardiac ultrasound video based on the responsible frame used for video benignity / malignancy determination.

[0065] After step 400, the method may further include: generating a dynamic heatmap of the responsibility region using a class activation graph.

[0066] The class activation map is a weighted linear sum of visual patterns present at different spatial locations. By upsampling the class activation map to the size of the input image, the image region most relevant to a specific category can be identified; in this invention, the specific category refers to the category of positive pericardial effusion. Before obtaining the video feature matrix in step 200, a K×4×4 feature map obtained through max-pooling can be used to infer the region that contributes most to the classification result by fusing positional encoding information. This invention retains the relationship between corresponding frames in the original video in the final feature matrix, and can generate a dynamic activation map that activates the category of pericardial effusion presence. Based on this, this invention can realize the responsible frames and responsible regions regarding the presence or absence of pericardial effusion through inverse feature contribution solving and information entropy reduction iteration.

[0067] This invention enables rapid assessment of the presence of pericardial effusion from echocardiograms, reducing the possibility of missed diagnoses by physicians. It utilizes a reverse feature contribution solution and information entropy reduction iteration method to determine the responsible frame and region for the presence or absence of pericardial effusion. It exhibits good applicability and transferability, providing assistance to physicians with limited clinical experience. The training scheme of this invention is relatively simple and does not require complex annotation processes.

[0068] As shown in Figure 6, the present invention provides a device for determining pericardial effusion using cardiac ultrasound video responsibility frames, comprising:

[0069] Training module 61 is used to acquire standard cross-sectional videos of cardiac ultrasound and standardize their format, and to train a static image classification neural network model using the standard cross-sectional videos of cardiac ultrasound.

[0070] For example, videos of the parasternal acoustic window PSAX section, the apical acoustic window A4C section, the parasternal acoustic window PLAX section, and the subxiphoid acoustic window SC4C section are acquired, and the number of frames, width, height, and number of channels of the videos are unified.

[0071] The matrix acquisition module 62 is used to build a pericardial effusion judgment model based on the static image classification neural network model, and inputs the standard cross-sectional video of the cardiac ultrasound to be identified into the pericardial effusion judgment model.

[0072] The training process of the static image classification neural network model is as follows:

[0073] A frame is extracted from the standard cross-sectional video of cardiac ultrasound at fixed intervals as the input image; the input image is then randomly subjected to data augmentation processing including rotation, translation, flipping, and Gaussian noise.

[0074] A static image classification neural network model is trained using stochastic gradient descent. The static image classification neural network model includes a skeleton part for extracting image features and a fully connected layer part for classification. The feature matrix obtained from the last convolutional layer of the static image classification neural network model is added to the image's positional bias code to obtain the image's spatial position information, which is used to evaluate the region of responsibility.

[0075] The identification module 63 is used to output the feature matrix of the video from the pericardial effusion judgment model, calculate the feature matrix of the video using a random forest model to obtain the feature importance matrix of the video, and select the responsible frame for judging the benignity or malignancy of the video based on information entropy reduction and the feature importance matrix of the video.

[0076] The feature importance matrix of the video is obtained by multiplying the eigenvalue of each feature dimension in the feature matrix by the importance value of that feature dimension in the random forest.

[0077] Specifically, the target image in the feature matrix of the video is selected by the information entropy reduction method, and max pooling is performed only on the target image in the column direction to obtain the feature matrix contributed by the Top N images; the feature matrix contributed by the Top N images is fed into the random forest model to obtain the responsibility frame used for video benignity / malignancy judgment.

[0078] Furthermore, the value of each channel in the video feature importance matrix is ​​taken as the total information contributed by the entire video in that feature dimension, and the value of each channel in the feature importance matrix of each frame is taken as the single-frame information contributed by that frame in the feature dimension.

[0079] Subtracting the feature importance matrix of a single frame image from the feature importance matrix of the video yields the remaining feature importance matrix; summing the information content of each feature dimension of the remaining feature importance matrix yields the remaining information entropy after subtracting the feature importance matrix of a single frame image from the video.

[0080] The frame with the minimum residual entropy is used as the responsible frame for determining the benignity or malignancy of the video.

[0081] The judgment module 64 is used by the static image classification neural network model to output whether there is pericardial effusion in the cardiac ultrasound video based on the responsibility frame used for video benignity / malignancy judgment.

[0082] The device for determining pericardial effusion using cardiac ultrasound video responsibility frames also includes a heatmap module, which is used to generate a dynamic heatmap of the responsibility region using a class activation graph.

[0083] This invention can provide artificial intelligence to determine whether there is pericardial effusion in echocardiogram videos, and can provide the responsible frame and responsible area, which helps to further reduce the missed diagnosis rate of pericardial effusion-related diseases in the process of ultrasound screening at the grassroots level or to monitor pericardial effusion information in real time during surgery.

[0084] The present invention also provides an electronic device, comprising:

[0085] At least one processor; and at least one memory communicatively connected to the processor, wherein the memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the above-described method for determining pericardial effusion using cardiac ultrasound video responsibility frames.

[0086] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for determining pericardial effusion using cardiac ultrasound video responsibility frames.

[0087] It is understood that computer-readable storage media can include: any entity or device capable of carrying computer programs, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. Computer programs include computer program code. Computer program code can be in the form of source code, object code, executable files, or certain intermediate forms, etc. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0088] In some embodiments of the present invention, the device may include a controller, which is a microcontroller chip integrating a processor, memory, communication module, etc. The processor may refer to the processor included in the controller. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0089] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0090] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0091] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining pericardial effusion using cardiac ultrasound video responsibility frames, characterized in that, include: Acquire standard-section videos of cardiac ultrasound and standardize the format; use the standard-section videos of cardiac ultrasound to train a static image classification neural network model. A pericardial effusion assessment model is built based on the static image classification neural network model. The standard section video of the cardiac ultrasound to be identified is input into the pericardial effusion assessment model. The pericardial effusion assessment model outputs the feature matrix of the video. A random forest model is used to calculate the feature matrix of the video to obtain the feature importance matrix of the video. Based on information entropy reduction and the feature importance matrix of the video, a responsibility frame is selected for determining the benignity or malignancy of the video. The static image classification neural network model outputs whether there is pericardial effusion in the cardiac ultrasound video based on the responsibility frame used for determining the benignity or malignancy of the video. The acquisition of standard section videos of cardiac ultrasound, in a standardized format, includes: acquiring the parasternal acoustic window PSAX section, the apical acoustic window A4C section, and the parasternal acoustic window PLAX section. The video, specifically the subxiphoid 4C transect, is used to unify the number of frames, width, height, and number of channels. The training of a static image classification neural network model using this standard transect video includes: extracting one frame from the standard transect video at fixed intervals as input; randomly applying data augmentation processing to the input image, including rotation, translation, flipping, and Gaussian noise; and training the static image classification neural network model using stochastic gradient descent. The static image classification neural network model includes a skeleton for extracting image features and a fully connected layer for classification. The feature matrix obtained from the last convolutional layer of the static image classification neural network model is added to the image's positional bias code to obtain the image space... The location information is used to assess the area of ​​responsibility. The step of calculating the feature matrix of the video using a random forest model to obtain the video's feature importance matrix includes: multiplying the eigenvalue of each feature dimension in the video's feature matrix by the importance value of that feature dimension in the random forest to obtain the video's feature importance matrix; multiplying the eigenvalue of each feature dimension in the video's feature matrix by the importance value of that feature dimension in the random forest to obtain the video's feature importance matrix includes: selecting target images in the video's feature matrix using information entropy reduction, performing maximum pooling only on the target images in the column direction to obtain the feature matrix contributed by the Top N images; and feeding the feature matrix contributed by the Top N images into the random forest model. The method involves obtaining a responsible frame for determining the benignity or malignancy of a video. The selection of the responsible frame based on information entropy reduction and the video's feature importance matrix includes: using the value of each channel in the video feature importance matrix as the total information contributed by the entire video in that feature dimension; using the value of each channel in the feature importance matrix of each frame as the single-frame information contributed by that frame in that feature dimension; subtracting the feature importance matrix of a single frame from the video feature importance matrix to obtain the remaining feature importance matrix; summing the information values ​​of each feature dimension of the remaining feature importance matrix to obtain the remaining information entropy after subtracting the single-frame image feature importance matrix from the video; and selecting the frame with the minimum remaining information entropy as the responsible frame for determining the benignity or malignancy of the video.

2. The method for determining pericardial effusion using cardiac ultrasound video responsibility frames according to claim 1, characterized in that, The method further includes generating a dynamic heatmap of the responsibility region using a class activation graph.

3. A device for determining pericardial effusion using cardiac ultrasound video responsibility frames, characterized in that, include: The training module is used to acquire standard cross-sectional videos of cardiac ultrasound and standardize their format, and to train a static image classification neural network model using the standard cross-sectional videos of cardiac ultrasound. The matrix acquisition module is used to build a pericardial effusion judgment model based on the static image classification neural network model, and inputs the standard cross-sectional video of the cardiac ultrasound to be identified into the pericardial effusion judgment model; the recognition module is used to calculate the feature matrix of the video output by the pericardial effusion judgment model, obtain the feature importance matrix of the video by using a random forest model, and select the responsible frame for judging the benignity or malignancy of the video based on information entropy reduction and the feature importance matrix of the video. The judgment module is used by the static image classification neural network model to output whether there is pericardial effusion in the echocardiogram video based on the responsibility frame used for video benignity / malignancy judgment; the acquisition of standard echocardiogram video and standardization of format includes: acquiring videos of the parasternal acoustic window PSAX section, apical acoustic window A4C section, parasternal acoustic window PLAX section, and subxiphoid acoustic window SC4C section, and standardizing the number of frames, width, height, and number of channels of the video; the training of the static image classification neural network model using the standard echocardiogram video includes: extracting one frame image from the standard echocardiogram video every fixed frame as the input image; for The input image is randomly augmented with rotation, translation, flipping, and Gaussian noise. A static image classification neural network model is trained using stochastic gradient descent. The static image classification neural network model includes a skeleton part for extracting image features and a fully connected layer part for classification. The feature matrix obtained from the last convolutional layer of the static image classification neural network model is added to the image's positional bias code to obtain image spatial location information, which is used to evaluate the responsibility region. The step of calculating the feature matrix of the video using a random forest model to obtain the video's feature importance matrix includes: multiplying the feature value of each feature dimension in the video's feature matrix by the importance value of that feature dimension in the random forest to obtain the video's feature importance matrix. The step of multiplying the feature value of each feature dimension in the video's feature matrix by the importance value of that feature dimension in the random forest to obtain the video's feature importance matrix includes: selecting target images in the video's feature matrix using information entropy reduction, performing maximum pooling only on the target images in the column direction to obtain the feature matrix contributed by the Top N images; and feeding the feature matrix contributed by the Top N images into the random forest model to obtain the responsibility frame used for video benignity / malignancy judgment. The step of calculating the feature matrix based on information entropy reduction and the video's feature matrix includes: selecting target images in the video's feature matrix using information entropy reduction, performing maximum pooling only on the target images in the column direction to obtain the feature matrix contributed by the Top N images; and feeding the feature matrix contributed by the Top N images into the random forest model to obtain the responsibility frame used for video benignity / malignancy judgment. The selection of the responsible frame for determining the benignity or malignancy of a video using the feature importance matrix includes: taking the value of each channel in the video feature importance matrix as the total information contributed by the entire video in that feature dimension, and taking the value of each channel in the feature importance matrix of each frame as the single-frame information contributed by that frame in the feature dimension; subtracting the feature importance matrix of a single frame from the feature importance matrix of the video to obtain the remaining feature importance matrix; summing the information of each feature dimension of the remaining feature importance matrix to obtain the remaining information entropy after subtracting the feature importance matrix of a single frame from the video; and selecting the frame with the minimum remaining information entropy as the responsible frame for determining the benignity or malignancy of the video.

4. An electronic device, characterized in that, include: At least one processor; And at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor can execute the method as described in claim 1 by invoking the program instructions.

5. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a computer, performs the method as described in claim 1.

Images