A Deep Learning-Based Method for Enhancing the Display of Key Areas in PDAs

Abstract

This invention relates to the fields of medical image analysis and artificial intelligence, and in particular to a deep learning-based method for enhancing the display of key regions in a PDA. The method includes: acquiring a target Doppler ultrasound video; inputting the target Doppler ultrasound video into a pre-constructed enhancement display model; and outputting a dynamically superimposed attention heatmap of the enhanced video segments. The enhancement display model employs a TimeSformer architecture to identify and locate the parasternal short-axis section in each frame of the video, and generates the enhanced display through weighted fusion. This invention can automatically identify Doppler ultrasound videos and enhance the display of key regions in a PDA, assisting doctors in making judgments.

Description

Technical Field

[0001] This invention relates to the fields of medical image analysis and artificial intelligence technology, and in particular to a method for enhancing the display of key areas of a PDA based on deep learning. Background Technology

[0002] Patent ductus arteriosus (PDA) is a pathological condition in which the ductus arteriosus fails to close as expected after birth and remains open. PDA is a common cardiovascular malformation in newborns and infants. Currently, echocardiography is the main non-invasive examination method for PDA. One of the most commonly used views for observing PDA is the parasternal short-axis view at the level of great vessels. The accuracy of this view is highly dependent on the operator's experience and ability to interpret the images.

[0003] In recent years, with the rapid development of deep learning technology, its application in medical image analysis has become increasingly widespread and in-depth. Traditional computer-aided diagnostic systems often rely on manually designed features, while deep learning models, especially convolutional neural networks (CNNs), can automatically learn high-dimensional, discriminative features from image data. However, echocardiography is usually recorded in video format, containing rich temporal information, which static image analysis methods often overlook. To address this issue, models capable of processing video or continuous time-series data have emerged. TimeSformer is a time-series model based on the Transformer architecture, which effectively captures long-range dependencies in video data through a self-attention mechanism. This characteristic makes it highly suitable for analyzing dynamic echocardiographic videos. Based on the above, this invention proposes a deep learning-based method for enhancing the display of key regions in a PDA. Summary of the Invention

[0004] The purpose of this invention is to provide a deep learning-based method for enhancing the display of key areas in a PDA, which can automatically identify Doppler ultrasound videos and enhance the display of key areas in the PDA to assist doctors in making judgments.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A deep learning-based method for enhancing the display of key areas in a PDA includes:

[0007] Acquire target Doppler ultrasound video;

[0008] The target Doppler ultrasound video is input into a pre-constructed enhanced display model, and the enhanced video segments are dynamically overlaid with attention heatmaps. The enhanced display model adopts the TimeSformer architecture to identify the parasternal short axis section of each frame in the video and locate the region. The enhanced display is generated after weighted fusion.

[0009] Optionally, the enhanced display model includes:

[0010] The section recognition module is used to analyze the target Doppler ultrasound video using the first model, identify the parasternal short axis section in each frame, and determine the key frame based on the preset section recognition confidence threshold.

[0011] The contrastive learning module is used to extract features by using a second model to calculate the similarity scores between the keyframe sequence and preset positive and negative samples, and to locate key regions by assigning attention scores.

[0012] The enhanced display module is used to overlay the key frames and corresponding key regions to output an enhanced video clip with dynamically overlaid attention heatmap.

[0013] Optionally, the first model is used to analyze the target Doppler ultrasound video to identify the parasternal short-axis section in each frame, including:

[0014] ;

[0015] in, Represents the first model The parameters, It is the first model The output classification result determines whether it is a parasternal short-axis section. For input video.

[0016] Optionally, determining keyframes based on a preset section recognition confidence threshold includes:

[0017] When the first model identifies the parasternal short-axis section, it outputs a confidence score for each frame in the target Doppler ultrasound video. When the confidence score corresponding to the video frame is greater than or equal to the preset section identification confidence threshold, the video frame is determined to be a key frame and added to the key frame sequence for enhanced display.

[0018] Optionally, the similarity scores between the keyframe sequence and preset positive and negative samples are calculated, including:

[0019] The second model maps keyframe sequences into feature vectors, calculates the distance between the feature vectors and preset positive sample feature centers and negative sample feature centers, and obtains the similarity score, wherein the positive samples are a set of labeled positive patent ductus arteriosus (PDA) sample videos, and the negative samples are a set of normal sample videos.

[0020] Optionally, the second model is trained using a contrastive learning strategy. During the training process, anchor samples are set, and the video set of positive patent ductus arteriosus (PDA) samples is set as positive samples, while the video set of normal samples is set as negative samples. The contrastive learning loss function is used for optimization, which brings the feature vector distance of positive sample pairs closer in the high-dimensional feature space, while pushing the feature vector distance of negative sample pairs further apart.

[0021] Optionally, key regions can be located by assigning attention scores, including:

[0022] When the second model extracts features by calculating similarity, it assigns an attention score to each image patch in each frame, determines the key region by the attention score, and constructs an attention map by combining the attention scores of all image patches, and generates an attention heatmap using pseudo-color rendering.

[0023] Optionally, the first model and the second model adopt a weight sharing mechanism during training, sharing the same weight parameters in a preset part of the network.

[0024] Optionally, the enhanced video clip dynamically overlaid with an attention heatmap adjusts the heatmap display intensity and spatial focus range by setting joint display control parameters.

[0025] The beneficial effects of this invention are as follows:

[0026] This invention enables automatic identification of Doppler ultrasound videos and enhances the display of key frames and key regions in the PDA (Programmable Analyzer), providing precise visual guidance and significantly improving the efficiency and accuracy of image interpretation. Through comparative learning, this invention provides quantitative similarity between the video to be examined and typical symptoms, transforming subjective impressions into objective data and offering a new and reliable reference dimension for clinical diagnosis, assisting doctors in making more accurate judgments. A core advantage of this invention is its weight-sharing mechanism. By sharing weights between the two model backbones, it not only significantly improves the overall performance of the model but also effectively reduces the number of parameters requiring training, accelerating the model's convergence process. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a deep learning-based method for enhancing the display of key areas in a PDA, according to an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] This embodiment provides a deep learning-based method for enhancing the display of key areas in a PDA, such as... Figure 1 As shown, it includes:

[0032] Acquire target Doppler ultrasound video;

[0033] The target Doppler ultrasound video is input into a pre-constructed enhanced display model, and the enhanced video segments are dynamically overlaid with attention heatmaps. The enhanced display model adopts the TimeSformer architecture to identify the parasternal short axis section of each frame in the video and locate the region. The enhanced display is generated after weighted fusion.

[0034] Specifically, this embodiment can automatically identify Doppler ultrasound videos and enhance the display of key frames and key areas of the PDA, providing precise visual guidance and significantly improving the efficiency and accuracy of image interpretation. This embodiment also provides quantitative similarity between the video to be examined and typical symptoms through comparative learning, transforming subjective impressions into objective data and providing a new and reliable reference dimension for clinical diagnosis, assisting doctors in making more accurate judgments. A core advantage of this embodiment is its weight-sharing mechanism. By sharing weights between the two model backbones, it not only significantly improves the overall performance of the model but also effectively reduces the number of parameters that need to be trained, accelerating the model's convergence process.

[0035] Furthermore, the enhanced display model includes:

[0036] The section recognition module is used to analyze the target Doppler ultrasound video using the first model, identify the parasternal short axis section in each frame, and determine the key frame based on the preset section recognition confidence threshold.

[0037] The contrastive learning module is used to extract features by using a second model to calculate the similarity scores between the keyframe sequence and preset positive and negative samples, and to locate key regions by assigning attention scores.

[0038] The enhanced display module is used to overlay the key frames and corresponding key regions to output an enhanced video clip with dynamically overlaid attention heatmap.

[0039] Furthermore, the section recognition module:

[0040] The goal is to automatically identify the parasternal short-axis (PSA) section from the input raw echocardiogram video.

[0041] Input: A raw Doppler echocardiogram video, represented as The video may contain multiple different cross-sections of the heart.

[0042] Objective: To accurately identify the parasternal short-axis view (PSA).

[0043] Processing: First Model For input video Analysis is performed. This model, based on the TimeSformer architecture, is tasked with performing cross-sectional classification along the video dimension. The processing can be formally represented as follows:

[0044] ;

[0045] in, Represents the first model The parameters, It is the first model The output classification results determine whether the video is a PSA section.

[0046] Output: Only when When the video was identified as a "PSA section", Only then will it be passed on to the next stage.

[0047] In addition, to ensure that enhanced display is performed only on high-quality, high-confidence cross-sectional images, a threshold-based keyframe selection strategy is introduced.

[0048] First Model When identifying the cross-section, it will be done for each frame of the video. Output a confidence score The confidence score represents the degree of confidence the model has in identifying the frame as a standard PSA section (typically ranging from [0,1]). A section identification confidence threshold is set. Only when its confidence score Greater than or equal to the threshold Only then will that frame be selected into the final sequence of keyframes used to enhance the display. , represented as:

[0049] ;

[0050] in, For the first model Recognized video All video frames in the PSA section.

[0051] Furthermore, comparing the learning modules:

[0052] Using a contrastive learning approach, the model is trained to effectively distinguish the features of positive samples (including PDA) and negative samples (normal).

[0053] Input: Receive video frame data identified as PSA sections by the section recognition module. In addition, there are a small set of positive (PDA) sample videos and a set of negative (normal) sample videos labeled by doctors.

[0054] Objective: To process positive and negative sample pairs using the same encoder, extract feature representations, and increase the distance between paired samples in the feature space.

[0055] deal with:

[0056] a) Model training phase (contrastive learning): A feature extraction model based on the TimeSformer architecture is used. As a second model, during training, given an anchor sample (e.g., a PDA video), other PDA videos are considered positive samples, and all normal videos are considered negative samples. The model is optimized using a contrastive learning loss function (such as InfoNCELoss), with the goal of narrowing the distance between the feature vectors of positive sample pairs (embedded) in a high-dimensional feature space, while widening the distance between the feature vectors of negative sample pairs.

[0057] ;

[0058] in, It is a contrastive learning loss. This is a query sample. It is a positive sample. These are negative samples, and N is the total number of negative samples. It is the cosine similarity function. It is a temperature hyperparameter.

[0059] b) Model Application Stage (Similarity Calculation): When the video data processed by the section recognition module is input, the trained second model... It is mapped to a feature vector; by calculating the distance between this vector and the preset feature centers of positive and negative samples, a quantified similarity score is obtained.

[0060] The calculation process for quantified similarity scores is as follows: First, the new video to be inspected is mapped into a high-dimensional feature vector using the model trained in the above process. Then, the distance between this vector and two preset "feature centers" is calculated. These two center points are pre-constructed by averaging the feature vectors of a large number of known positive (PDA) and negative (normal) samples. Finally, based on the principle that the closer the distance, the higher the similarity, these two distance values ​​are converted into an intuitive score between 0 and 1 through normalization.

[0061] Output: A quantified similarity score or distance relationship with a typical sample.

[0062] Using the second model Attention mechanisms are used to locate key areas related to lesions. Includes ,Model In the During feature extraction, each keyframe will be... Each image patch, i.e., each keyframe Each pixel in Assign an attention score The higher the attention score, the greater the contribution of that region to the model's judgment. The set of attention scores of all image patches constitutes an attention map, which can be generated into an intuitive attention heatmap through pseudo-color rendering.

[0063] Furthermore, enhance the display module:

[0064] The enhanced video clips are dynamically overlaid with attention heatmaps, and the intensity and spatial focus range of the heatmap display can be adjusted by setting joint display control parameters.

[0065] Specifically, set a joint display control parameter. Simultaneously adjust the heatmap display intensity and spatial focus range to determine the final display intensity of the heatmap overlay. Calculated using the following set of formulas:

[0066] ;

[0067] in, For normalization function, To overlay pixels in the attention heatmap Attention score The maximum attention score, For threshold parameters, It is an exponential function used to construct a Gaussian distribution. To overlay pixels in the attention heatmap With attention peak pixels The physical distance between them σ represents the standard deviation of the Gaussian distribution. The larger the value of σ, the wider the range of the heatmap; the smaller the value of σ, the narrower the range. This is the proportionality coefficient. For minor corrections, ensure Even when set to 1, there is still a display area.

[0068] By adjusting a single parameter This allows control over the overall "focus intensity" of the heatmap. A lower... A higher value will display the heatmap with a wider range and a lower threshold, suitable for a global overview; The value will narrow the display range and raise the display threshold, highlighting only the most critical signal area, which is suitable for precise focusing.

[0069] Furthermore, the first model and the second model employ a weight-sharing mechanism during training, sharing the same weight parameters in a pre-defined portion of the network.

[0070] Specifically, in order to improve the overall performance and training efficiency of the model, a weight-sharing strategy was adopted between the models in the two stages.

[0071] Two models ( and The backbone network of a network shares the same set of weight parameters, which can be represented as... .

[0072] Two models ( and It has an independent, task-specific classifier head, whose parameters are represented as follows: and .

[0073] Therefore, the total parameters of the two-stage model can be expressed as:

[0074] Stage 1 model parameters: =[ ];

[0075] Stage 2 model parameters: =[ ];

[0076] This design significantly improves model performance by sharing learned features across different but related tasks, while effectively reducing the number of trainable parameters and accelerating the model convergence process.

[0077] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for enhancing the display of key areas in a PDA based on deep learning, characterized in that, include: Acquire target Doppler ultrasound video; The target Doppler ultrasound video is input into a pre-constructed enhanced display model, and the enhanced video segments are dynamically overlaid with attention heatmaps. The enhanced display model adopts the TimeSformer architecture to identify the parasternal short axis section of each frame in the video and locate the region. The enhanced display is generated after weighted fusion. The enhanced display model includes: The section recognition module is used to analyze the target Doppler ultrasound video using the first model, identify the parasternal short axis section in each frame, and determine the key frame based on the preset section recognition confidence threshold. The contrastive learning module is used to extract features by calculating the similarity scores between the keyframe sequence and preset positive and negative samples using a second model, and to locate key regions by allocating attention scores. The positive samples are a video set of labeled positive patent ductus arteriosus samples, and the negative samples are a video set of normal samples. An enhanced display module is used to overlay the key frames and corresponding key regions, and output an enhanced video clip dynamically overlaid with an attention heatmap.

2. The deep learning-based PDA key area enhancement display method according to claim 1, characterized in that, The first model is used to analyze the target Doppler ultrasound video to identify the parasternal short-axis section in each frame, including: ； in, Represents the first model The parameters, It is the first model The output classification result determines whether it is a parasternal short-axis section. For input video.

3. The deep learning-based PDA key area enhancement display method according to claim 1, characterized in that, Determining keyframes based on a preset section recognition confidence threshold includes: When the first model identifies the parasternal short-axis section, it outputs a confidence score for each frame in the target Doppler ultrasound video. When the confidence score corresponding to the video frame is greater than or equal to the preset section identification confidence threshold, the video frame is determined to be a key frame and added to the key frame sequence for enhanced display.

4. The deep learning-based PDA key area enhancement display method according to claim 1, characterized in that, Calculate the similarity scores between the keyframe sequence and preset positive and negative samples, including: The second model maps keyframe sequences into feature vectors, calculates the distance between the feature vectors and preset positive sample feature centers and negative sample feature centers, and obtains the similarity score.

5. The deep learning-based PDA key area enhancement display method according to claim 4, characterized in that, The second model is trained using a contrastive learning strategy. During training, anchor samples are set, and the video set of positive patent ductus arteriosus is set as positive samples, while the video set of normal samples is set as negative samples. The contrastive learning loss function is used for optimization, which brings the feature vector distance of positive sample pairs closer in the high-dimensional feature space, while pushing the feature vector distance of negative sample pairs further apart.

6. The deep learning-based method for enhancing the display of key areas in a PDA according to claim 1, characterized in that, Key regions are located by assigning attention scores, including: When the second model extracts features by calculating similarity scores, it assigns an attention score to each image patch in each frame, determines the key region by the attention scores, and constructs an attention map by combining the attention scores of all image patches. Then, it generates an attention heatmap using pseudo-color rendering.

7. The deep learning-based method for enhancing the display of key areas in a PDA according to claim 1, characterized in that, The first model and the second model adopt a weight sharing mechanism during training, sharing the same weight parameters in a preset part of the network.

8. The deep learning-based method for enhancing the display of key areas in a PDA according to claim 1, characterized in that, The enhanced video clip dynamic overlay attention heatmap adjusts the heatmap display intensity and spatial focus range by setting joint display control parameters.

Images