Carotid artery ultrasonic anatomical structure detection method based on real-time video
Through a real-time video-based carotid artery ultrasound anatomical structure detection method, a deep learning model is used to expand the receptive field of temporal information, thereby achieving automatic recognition and labeling of the carotid artery anatomical structure. This solves the shortcomings of junior ultrasound physicians in detection accuracy and stability, and improves detection accuracy and inspection efficiency.
Patent Information
- Application Number
- CN202510820063.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-18
- Publication Date
- 2025-09-26
AI Technical Summary
Junior ultrasound physicians in primary hospitals find it difficult to accurately obtain standard carotid artery sections and identify atherosclerotic plaques. Existing technology requires an auxiliary means to improve detection accuracy and stability.
A real-time video-based carotid artery ultrasound anatomical structure detection method is adopted. Through the real-time video target detection algorithm, the temporal information receptive field of the deep learning model is expanded, and the ability to extract the temporal features of dynamic ultrasound images is enhanced. Combined with the backbone network, short-term and long-term temporal feature aggregation modules, neck network and detection head, the automatic recognition and labeling of the carotid artery anatomical structure is realized.
The accuracy and stability of carotid artery anatomical structure detection are improved, human operation errors are reduced, inspection efficiency is improved, and the generalization ability of the model is enhanced under conditions of limited data.
Smart Images
Figure CN120707953A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image computer processing, and in particular to a method for detecting the ultrasonic anatomical structure of a carotid artery based on real-time video. Background Art
[0002] Diabetes mellitus is a chronic disease that poses a serious threat to human health and is often accompanied by cardiovascular complications such as coronary heart disease and stroke. Its primary pathological feature is atherosclerosis, which is one of the leading causes of death in diabetic patients. Carotid ultrasound, a widely used, non-invasive, accurate, and high-resolution cardiovascular diagnostic tool, has been shown to effectively assess vascular wall morphological changes (e.g., the degree of intimal thickening) in the common, internal, and external carotid arteries, thereby identifying the presence of atherosclerotic plaques.
[0003] Existing studies have shown that junior ultrasound physicians in primary care hospitals often have difficulty accurately acquiring standard carotid artery sections and identifying atherosclerotic plaques. Therefore, a supplementary method is needed to help junior ultrasound physicians improve the scanning accuracy of standard sections and the ability to detect atherosclerotic plaques. Summary of the Invention
[0004] This invention aims to address the shortcomings of existing technologies by providing a real-time video-based method for detecting the anatomical structure of the carotid artery using ultrasound. This method utilizes a real-time video object detection algorithm to improve the model's detection accuracy in dynamic video. This method expands the receptive field of temporal information within the deep learning model, enhancing the model's ability to extract temporal features from dynamic ultrasound images, thereby improving the accuracy and stability of carotid artery anatomical structure detection.
[0005] In order to solve the problems existing in the prior art, the present invention proposes a method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video, comprising the following steps:
[0006] (1) Real-time acquisition of images from ultrasonic inspection equipment and transmission of video signals to the data processing module for preprocessing;
[0007] (2) The processed image is fed into the target detection model to identify the anatomical structures of the common carotid artery, internal and external carotid arteries, jugular vein, and plaque;
[0008] (3) Output the position coordinates and mark them on the real-time ultrasound image in the form of a bounding box;
[0009] (4) The system displays the recognition results in real time, providing doctors with visual auxiliary diagnosis information.
[0010] The present invention proposes a method for detecting the anatomical structure of the carotid artery using ultrasound in real-time video. The target detection model includes a backbone network, a short-term temporal feature aggregation module, a long-term temporal feature aggregation module, a neck network, and a detection head. The data flow of the target monitoring model includes the following steps:
[0011] Step 1: Input an image sequence of 3 frames in length, and extract spatial features of each frame through the backbone network;
[0012] Step 2: splicing the spatial features of the three frames of images to construct a short-term feature sequence, and inputting the short-term feature sequence into a short-term temporal feature aggregation module to model the short-term spatiotemporal features within the feature sequence range;
[0013] Step 3: Input the short-term spatiotemporal features from step 2 into the long-term temporal feature aggregation module. The short-term spatiotemporal features will be fused with the working memory features and long-term memory features in the memory buffer for cross-frame feature fusion to generate long-term spatiotemporal features with a larger receptive field.
[0014] In step 4, the long-term spatiotemporal features generated in step 3 and the shallow spatial feature map extracted by the backbone network in step 1 are input into the neck network and the detection head. After confidence threshold screening and maximum suppression to remove redundant frames, the edge frame of the foreground target in the current frame and the target category in each frame are output.
[0015] In the aforementioned method for detecting the anatomical structure of the carotid artery using ultrasound in real-time video, the fusion result in step 3 reversely updates the memory buffer, maintains semantic information of continuity across the time domain, and provides a temporal prior for subsequent frames.
[0016] In the aforementioned method for detecting the anatomical structure of the carotid artery using real-time video ultrasound, the data preprocessing includes data set annotation and data enhancement;
[0017] The annotation of the dataset includes the use of DarkLabel software to annotate the key anatomical structures and plaque areas in the carotid ultrasound images for the training dataset, manually drawing target detection frames for the common carotid artery, internal carotid artery, external carotid artery, jugular vein, and suspicious plaques, and assigning corresponding labels to each type of target. LabelMe software is used for plaque segmentation and annotation, and the plaque area is finely depicted for data enhancement of the plaque enhancement dataset. To ensure the quality of the annotation, the entire annotation process is carried out under the guidance of the hospital's ultrasound physician. For images with controversial or uncertain annotations, a secondary review mechanism is used to review them to minimize mislabeling and omissions, providing an accurate, reliable, and high-quality data foundation for model training.
[0018] Data augmentation methods are categorized into spatial, temporal, and plaque enhancement strategies. Spatial enhancement includes random scaling, random center cropping, and random rotation. Temporal enhancement uses a sliding window with random step sizes of 1, 2, or 4, sampling 16 frames of the ultrasound sequence at equal intervals to enrich the dataset. If the number of sequence frames is less than the sampling size, the index is looped multiple times to meet the requirement. During the plaque data augmentation process, a transverse or longitudinal section containing the carotid artery structure is first captured from the ultrasound image to ensure that the selected region clearly displays the vascular lumen. Subsequently, the image is grayscale binarized using the Otsu method to separate the vascular lumen from the surrounding tissue. Gaussian low-pass filtering is then used to smooth the image to reduce noise, improve the stability of edge detection, and ensure accurate extraction of the vascular lumen contour.
[0019] In the aforementioned real-time video-based carotid artery ultrasound anatomical structure detection method, the timing enhancement first selects a fixed sampling step size to ensure the continuity and regularity of the time series; then, a set of plaque sample sequences is randomly extracted from the acquired plaque video data, so that it contains multiple plaque frames with time correlation; for each selected time series sample sequence, the corresponding plaque frame is sequentially pasted into the corresponding ultrasound image frame to ensure that the spatial position and morphology of the plaque in the time series sample changes over time in accordance with physiological characteristics and ultrasound imaging laws.
[0020] The aforementioned real-time video-based carotid artery ultrasound anatomical structure detection method uses a backbone network using CSPDarkNet. CSPDarkNet introduces a Cross Stage Partial structure to divide the input feature map into a main path and a residual path in the channel dimension, respectively retaining shallow information and extracting deep features. Finally, through channel splicing and fusion, it alleviates the loss of deep information and enhances gradient flow and feature expression capabilities.
[0021] Define the current moment frame t The video sequence composed of the first two frames is Clip t , Clip t The first two video sequences are Clip t-3 and Clip t-6 ; Clip t ={frame t-2 ,frame t-1 ,frame t} is input to CSPDarkNet, and the corresponding feature sequence output is and Feat5 represents the Clip generated by Backbone's Stage 5 t The feature sequence corresponding to each frame image, Featt Represents the frames generated by the last three stages (Stages 3 to 5) of Backbone t The feature map sequence The side lengths are
[0022] In the aforementioned real-time video-based carotid artery ultrasound anatomical structure detection method, the short-term temporal feature aggregation module adopts a Transformer-like architecture, consisting of an encoder and an optimized decoder. Its workflow is divided into the following two parts:
[0023] Feat5 is spliced in the time series dimension and input into the Encoder. After the Encoder calculates the global self-attention of Feat5, the encoded features are output. where c, h and w are The number of channels, the height and width of the feature map;
[0024] Decoder uses spatial features As Query, and let it be generated by Encoder Calculate cross attention. In this way, Decoder can use from Retrieve the frame t Related features, thereby outputting short-term spatial and temporal features
[0025] The aforementioned method for detecting the anatomical structure of the carotid artery using ultrasound in real-time video, in which the long-term temporal feature aggregation module constructs a working memory buffer and a long-term memory buffer, As input, the long-term spatiotemporal features stored in the working memory buffer and and long-term memory features stored in the long-term memory buffer The features are concatenated and input into the long-term temporal feature aggregation module Encoder. The Encoder aggregates the spatiotemporal features of the working memory buffer, the long-term memory buffer and the current moment, and uses a 4×3×3 3D convolution kernel to compress the temporal dimension and output and Derived from Clip t-3 and Clip t-6 The long-term spatiotemporal features obtained by inference are thus generated Compared to It has richer and denser temporal feature information.
[0026] In the aforementioned method for detecting the anatomical structure of the carotid artery using ultrasound in real-time video, the neck network uses the long-term spatiotemporal features of the current frame and the shallow feature map for multi-scale fusion, and outputs the edge boxes of the carotid artery anatomical structure and plaque through the detection head; the three fused multi-scale feature maps are input into the detection head, and the target box and category are predicted respectively through two 3×3 convolutional layers with different weights, thereby obtaining the positioning box and category results of the object.
[0027] Compared with the existing technology, the present invention integrates image processing technology and deep learning models, which can analyze ultrasound images in real time and automatically identify and mark the carotid artery and its diseased areas. The system adopts a target detection algorithm based on real-time video to improve the detection accuracy of the model in dynamic videos. The receptive field of temporal information of the deep learning model has been expanded to enhance the model's ability to extract temporal features of dynamic ultrasound images, thereby improving the accuracy and stability of carotid artery anatomical structure detection. With the help of real-time navigation and intelligent prompt functions, the system can assist doctors in optimizing the probe placement angle, improve inspection efficiency, and reduce human operation errors. At the same time, under the condition of limited data volume, the system effectively improves the generalization ability of the model through data enhancement technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 This is the overall network framework structure of this embodiment.
[0029] Figure 2 This is a schematic diagram of the backbone network process of this embodiment.
[0030] Figure 3 This is a flow chart of the short-term time series feature aggregation module in this embodiment.
[0031] Figure 4 This is a flow chart of the long-term temporal feature aggregation module of this embodiment.
[0032] Figure 5 This is a schematic diagram of the neck network and detection head process in this embodiment.
[0033] Figure 6 This is a schematic diagram of the Conv3 and Convs Block structures of this embodiment.
[0034] Figure 7 This is the final prediction result diagram of this embodiment. DETAILED DESCRIPTION
[0035] The technical solution of the present invention is described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.
[0036] This embodiment discloses a method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video, comprising the following steps:
[0037] (1) Real-time acquisition of images from ultrasonic inspection equipment and transmission of video signals to the data processing module for preprocessing;
[0038] (2) The processed image is input into the target detection model to identify the anatomical structures of the common carotid artery, internal and external carotid arteries, jugular vein, and plaque; the location coordinates are output and annotated on the real-time ultrasound image in the form of a bounding box;
[0039] (3) The system displays the recognition results in real time, providing doctors with visual auxiliary diagnosis information.
[0040] Data acquisition and preprocessing
[0041] In this embodiment, a carotid artery target detection dataset is obtained, which is sourced from Philips Medical Systems, GE Vingmed Ultrasound, Mindary Premium Ultrasound and other equipment in the Ultrasound Department of a Class III hospital. The dataset consists of 143 randomly selected carotid ultrasound data, each of which contains approximately 133 to 537 frames of ultrasound images, and covers five types of anatomical structures including the longitudinal section of the common carotid artery, the longitudinal section of the internal and external carotid arteries, the longitudinal section of the jugular vein, the cross-section of the carotid artery and plaques. In addition, another 20 carotid artery plaque data were randomly selected for data enhancement processing of plaque features. The ultrasound image data was converted from the original DICOM format to a video file in AVI format and randomly arranged during the data processing process. Finally, the processed dataset was divided into a training set and a test set in a ratio of 8:2 to ensure the balance and generalization ability of the model training.
[0042] The training dataset uses DarkLabel software to annotate key anatomical structures and plaque areas in carotid ultrasound images. Target detection frames for the common carotid artery, internal carotid artery, external carotid artery, jugular vein, and suspicious plaques are manually drawn one by one, and corresponding labels are assigned to each type of target. The enhanced dataset uses LabelMe software for plaque segmentation and annotation, and finely depicts the plaque area for data enhancement of the plaque. To ensure the quality of the annotation, the entire annotation process is carried out under the guidance of the hospital's ultrasound physician. For images with controversial or uncertain annotations, a secondary review mechanism is used to review them to minimize mislabeling and omissions, providing an accurate, reliable, and high-quality data foundation for model training.
[0043] Data augmentation methods include spatial augmentation, temporal augmentation, and a newly designed patch augmentation strategy. Spatial augmentation includes random scaling, random center cropping, and random rotation. Temporal augmentation uses a sliding window with random step sizes of 1, 2, or 4 to sample 16 frames of ultrasound sequences at equal intervals to enrich the dataset. If the number of sequence frames is less than the sampling size, the index is looped multiple times to meet the requirement.
[0044] During the plaque data enhancement process, a cross-section or longitudinal section containing the carotid artery structure is first captured from the ultrasound image to ensure that the selected area clearly displays the vascular lumen. Subsequently, the image is subjected to grayscale binarization using Otsu's Thresholding method to separate the vascular lumen from the surrounding tissue. Gaussian low-pass filtering is then used to smooth the image to reduce noise, improve edge detection stability, and ensure accurate extraction of the vascular lumen contour.
[0045] After acquiring the vessel lumen region, two points are randomly selected from the vessel wall as reference locations for plaque attachment, ensuring that the generated data matches the lesion distribution seen in actual ultrasound imaging. Within the selected plaque region, scaling is performed based on the distance between the selected lumen wall points to ensure that the plaque size aligns with the vascular anatomy and imaging characteristics, thereby enhancing the authenticity and diversity of the data.
[0046] During the plaque fitting process, a column-by-column splicing method is used to paste the scaled plaque area into the target image column by column along the vascular lumen wall to ensure that the pasted lesion area conforms to the structural characteristics of ultrasound imaging.
[0047] During the data augmentation process for time series samples, a fixed sampling step size is first selected to ensure continuity and regularity of the time series. Subsequently, a set of plaque sample sequences is randomly extracted from the acquired plaque video data, containing multiple temporally correlated plaque frames. For each selected time series sample sequence, the corresponding plaque frames are sequentially pasted into the corresponding ultrasound image frames, ensuring that the spatial position and morphology of the plaques in the time series samples change over time in accordance with physiological characteristics and ultrasound imaging patterns.
[0048] This method preserves and simulates the temporal variation characteristics of plaques during data augmentation, enabling deep learning models to learn the dynamic evolution of plaques in ultrasound videos during training, thereby improving their ability to perceive temporal features. Furthermore, this augmentation strategy further optimizes the model's learning of plaque evolution patterns, making it more adaptable to real ultrasound data with inter-frame variations and enhancing the temporal robustness of plaque detection and recognition.
[0049] Object detection model
[0050] The target detection model of this embodiment includes a backbone network, a short-term temporal feature aggregation module (STA), a long-term temporal feature aggregation module (LTA), a neck network and a detection head; the target detection model can effectively aggregate short-term and long-term temporal features, thereby improving the target detection performance in dynamic video streaming scenarios and ensuring the sensitivity and accuracy of detection of anatomical structures and plaques in the carotid artery.
[0051] Considering the advantages of the classic SSD (Single Shot MultiBox Detector) algorithm in terms of simple structure, high inference efficiency, and good generalization ability for small sample data, this embodiment makes structural improvements based on SSD to better meet the dual requirements of real-time and robustness in actual clinical environments. In the classic SSD architecture, the VGG network used suffers from serious gradient vanishing and significant parameter redundancy, which limits the model's expressiveness in deeper feature learning. To this end, this embodiment replaces its backbone network with CSPDarkNet to improve the network's feature extraction capabilities and training stability while maintaining computational efficiency. By introducing the Cross Stage Partial (CSP) structure, CSPDarkNet divides the input feature map into a backbone path and a residual path in the channel dimension, respectively retaining shallow information and extracting deep features. Finally, through channel splicing and fusion, it effectively alleviates the loss of deep information and enhances gradient flow and feature expression capabilities. In addition, the CSP structure only performs residual calculation operations on half of the channels, significantly reducing computational complexity and resource consumption while ensuring modeling capability, helping to achieve efficient and lightweight network deployment. The overall structure of CSPDarkNet is as follows Figure 2 shown.
[0052] For the convenience of description, this embodiment defines the current time frame frame t The video sequence composed of the first two frames is Clip t , Clip t The first two video sequences are Clip t-3 and Clip t-6 .like Figure 2 As shown, Clip t ={frame t-2 ,frame t-1 ,frame t} is input to CSPDarkNet, and the corresponding feature sequence output is and Feat5 represents the Clip generated by Backbone's Stage 5t The feature sequence corresponding to each frame image, Feat t Represents the frames generated by the last three stages (Stages 3 to 5) of Backbone t The feature map sequence The side lengths are
[0053] The short-term temporal feature aggregation (STA) module is used to model the short-term spatiotemporal features in the range of [t-2, t]. Its structure is as follows: Figure 3 As shown in the figure. The subscript st in the figure represents short-term, which specifically refers to the short spatiotemporal features generated after aggregating short feature sequences. The main task of STA is to aggregate short-term temporal features to form a short-term temporal receptive field. By encoding the spatial features of the previous two frames and the current frame, it captures local temporal dependencies, thereby enhancing the model's ability to focus on key targets such as anatomical structures and plaques. STA adopts a Transformer-like architecture, consisting of an encoder and an optimized decoder. The calculation process of the encoder is the same as that of the Transformer Encoder, which is responsible for modeling the relationship between frames. It is optimized. Considering that the features input to the decoder already have a high level of semantic feature abstraction, the decoder omits the self-attention module (MSA) compared to the Transformer Decoder, and only retains the cross attention calculation (MCA) to reduce computational redundancy and improve computational efficiency.
[0054] Different from the TDTD module in TransVOD, STA does not use randomly initialized target queries as cross-attention queries, but directly uses the current frame spatial features. The core of this improvement is to avoid using random queries that rely on large amounts of data and complex initialization strategies. Due to the limited size of medical image datasets, directly using the spatial features of the current frame As a query item of cross attention, it can reduce the difficulty of training while improving the convergence speed and detection accuracy of the model.
[0055] The workflow of the STA module is divided into the following parts:
[0056] Feat5 is spliced in the time series dimension and input into the Encoder. After the Encoder calculates the global self-attention of Feat5, the encoded features are output. where c, h and w are The number of channels, height and width of the feature map. In this process, the encoder models the relationship between spatial features through self-attention calculation, thereby generating temporal features;
[0057] Decoder uses spatial features As Query, and let it be generated by Encoder Calculate cross attention. In this way, Decoder can use from Retrieve the frame t Related features, thereby outputting short-term spatial and temporal features This feature covers the temporal receptive field in the range of [t-2, t], providing contextual information of short sequences for further expanding the receptive field in a longer range.
[0058] The long-term temporal feature aggregation module will use short-term spatiotemporal features As input, further expand the temporal receptive field and generate long-term spatiotemporal features In order to efficiently model long-term spatiotemporal dependencies, two memory buffers are constructed: Working Memory Buffer (WMB) and Long-term Memory Buffer (LMB). The LTA module aims to retrieve spatiotemporal information related to the current frame from historical features of a longer time span, and to mine foreground target features under long-term dependencies, thereby improving the model's ability to recognize key areas that require long-range context support, such as anatomical structures and plaques. This module models the short-term spatiotemporal features of the current frame. The relationship between the features stored in the memory buffer is used to expand the temporal receptive field and generate long-term spatiotemporal features. The memory buffer is used to store and dynamically update the long-term spatiotemporal and long-term memory features extracted from historical frames, ensuring that the model continues to obtain rich temporal context information during inference.
[0059] The specific structure and reasoning process of LTA are as follows Figure 4 As shown, this module receives the output of STA Long-term spatiotemporal features stored in WMB and and long-term memory features stored in the LMB These features are concatenated and input into the LTA Encoder. The Encoder aggregates the spatiotemporal features of WMB, LMB and the current moment, and uses a 4×3×3 3D convolution kernel to compress the time series dimension and output and Derived from Clip t-3 and Clip t-6 The long-term spatiotemporal features obtained by inference are thus generated Compared to It has richer and denser temporal feature information.
[0060] The design of Decoder is similar to that of STADecoder. As Query, and As the Key and Value for cross attention calculation, the final output is the long-term spatiotemporal features That is, it has the temporal receptive field of the entire memory buffer. WMB and LMB are used to store the long-term spatiotemporal features and long-term memory features of the previous stage, respectively, to ensure that the spatiotemporal features of the memory buffer cover the range of [0, t-3]. The capacity of WMB is 2 long-term spatiotemporal features, and its update strategy is the same as that of the queue: Inference Clip t When WMB saves Clip t-3 and Clip t-6 ;Clip t Reasoning completed, the latest Enter the queue and replace the oldest feature The capacity of LMB is 1 long memory feature. t After the inference is completed, the LMB Replace with the latest
[0061] When managing long-term spatiotemporal features, the design of WMB and LMB instead of simply caching the long-term spatiotemporal features of all previous frames is mainly based on the following two considerations: 1. Memory efficiency and computational complexity: if the features of all previous frames are saved, the memory or video memory usage will be significantly increased; and since the computational complexity of MSA and MCA is O(n 2 ), where n = t × h × w. Longer feature sequences significantly increase the computational overhead of the LTA module, hindering training and deployment. 2. Feature Focus Decoupling: During temporal aggregation, different types of features have different emphases. Long-term spatiotemporal features emphasize the expression of features in the current frame, while long-term memory features focus on the overall information of the entire video sequence. By decoupling these two types of features, each can be assigned specific functions, improving expression quality and facilitating rapid model convergence.
[0062] In summary, the LTA module expands the temporal receptive field by aggregating short-term spatiotemporal features and features stored in the memory buffer, thereby generating long-term spatiotemporal features with long-term temporal receptive field, providing rich spatiotemporal context expression for the subsequent generation of target boxes.
[0063] The neck network uses the long-term spatiotemporal features of the current frame and the shallow feature map to perform multi-scale fusion, and outputs the edge box of the carotid artery anatomical structure and plaque through the detection head. Figure 5 As shown, the feature map and frame t The spatial feature sequence In order to improve the detection capability of small targets (such as plaques), this embodiment adopts the PAN structure to fuse multi-scale features. and shallow features, which can transfer rich spatiotemporal feature expressions to shallow features and Therefore, the steps of performing STA and LTA calculations on the shallow layer are omitted. and The width and height of the MSA and MCA are relatively large, and the relationship between the computational complexity of the MSA and MCA and the length of the feature sequence is O(n 2 )(n=hw), so performing temporal feature aggregation only on the feature map with the smallest size can save a lot of computing resources without significantly affecting the accuracy of target detection.
[0064] Figure 5 In the figure, the feature maps and arrows of different colors represent the process of transferring temporal features to shallow features. The green arrows represent the information flow containing temporal features, and the yellow arrows represent the spatial feature flow. The PAN structure realizes the feature fusion of multi-scale features by constructing bottom-up and top-down feature flows. In the figure, Conv3 is used to reduce the dimension of the spliced features, thereby retaining the effective features between different feature maps and removing redundant features. Its structure is as follows Figure 6 As shown, the Conv+BN+SiLU operation is a set of operators, that is, the input data is sequentially passed through a convolution layer, a BN (Batch Normalization) layer and a SiLU (Sigmoid Weighted Linear Unit) activation function operation. The parameter k of the convolution layer is the size of the convolution kernel, and s is the step size of each movement of the convolution kernel. Conv3 contains two feature operation paths: the left path passes through 1 CBS and 3 ConvsBlock operations in sequence, and the right path passes through only 1 CBS. Finally, the two features are spliced and reduced in dimension through 1 CBS operation. Through the feature channel integration and nonlinear expression functions provided by the Conv3 structure, The spatiotemporal features in the image are fused with the shallow features to remove feature redundancy and retain effective features, so as to enhance the shallow features and make them have rich spatiotemporal information.
[0065] This example uses the classic SSD detection head. The three fused multi-scale feature maps are input into the detection head. The target box and category are predicted through two 3×3 convolutional layers with different weights respectively. The positioning box and category of the object can be obtained. The results are as follows: Figure 7 shown.
[0066] It should be noted that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.
Claims
1. A method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video, characterized in that The following steps are involved: (1) Real-time acquisition of images from ultrasonic inspection equipment and transmission of video signals to the data processing module for preprocessing; (2) The processed image is fed into the target detection model to identify the anatomical structures of the common carotid artery, internal and external carotid arteries, jugular vein, and plaque; (3) Output the position coordinates and mark them on the real-time ultrasound image in the form of a bounding box; (4) The system displays the recognition results in real time, providing doctors with visual auxiliary diagnosis information.
2. The method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video according to claim 1, wherein: The target detection model includes a backbone network, a short-term temporal feature aggregation module, a long-term temporal feature aggregation module, a neck network, and a detection head; the data flow of the target monitoring model includes the following steps: Step 1: Input an image sequence of 3 frames in length, and extract spatial features of each frame through the backbone network; Step 2: splicing the spatial features of the three frames of images to construct a short-term feature sequence, and inputting the short-term feature sequence into a short-term temporal feature aggregation module to model the short-term spatiotemporal features within the feature sequence range; Step 3: Input the short-term spatiotemporal features from step 2 into the long-term temporal feature aggregation module. The short-term spatiotemporal features will be fused with the working memory features and long-term memory features in the memory buffer for cross-frame feature fusion to generate long-term spatiotemporal features with a larger receptive field. In step 4, the long-term spatiotemporal features generated in step 3 and the shallow spatial feature map extracted by the backbone network in step 1 are input into the neck network and the detection head. After confidence threshold screening and maximum suppression to remove redundant frames, the edge frame of the foreground target in the current frame and the target category in each frame are output.
3. The method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video according to claim 2, wherein: The fusion result in step 3 reversely updates the memory buffer, maintains the semantic information of continuity across the time domain, and provides a temporal prior for subsequent frames.
4. The method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video according to claim 1, wherein: The data preprocessing includes data set annotation and data enhancement; The dataset annotation includes using DarkLabel software to annotate the key anatomical structures and plaque areas in the carotid artery ultrasound image as a training dataset, and using LabelMe software to perform plaque segmentation and annotation, and finely depict the plaque area for data enhancement of the plaque enhancement dataset; Data enhancement methods are divided into spatial enhancement, temporal enhancement and patch enhancement strategies.
5. The method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video according to claim 4, wherein: The temporal enhancement method first selects a fixed sampling step size to ensure the continuity and regularity of the time series. Then, a set of plaque sample sequences is randomly extracted from the acquired plaque video data, so that it contains multiple temporally correlated plaque frames. For each selected temporal sample sequence, the corresponding plaque frames are sequentially pasted into the corresponding ultrasound image frames to ensure that the spatial position and morphology of the plaque in the temporal sample change over time in accordance with physiological characteristics and ultrasound imaging laws.
6. The method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video according to claim 2, wherein: The backbone network adopts CSPDarkNet. By introducing the CrossStagePartial structure, CSPDarkNet divides the input feature map into a trunk path and a residual path in the channel dimension, respectively retaining shallow information and extracting deep features. Finally, through channel splicing and fusion, it alleviates the loss of deep information and enhances gradient flow and feature expression capabilities. Define the current moment frame t The video sequence composed of the first two frames is Clip t , Clip t The first two video sequences are Clip t-3 and Clip t-6 ; Clip t =frame t-2 ,frame t-1 ,frame t Input to CSPDarkNet, the corresponding feature sequence output is and Feat5 represents the Clip generated by Backbone's Stage5 t The feature sequence corresponding to each frame image, Feat t Represents the frames generated by the last three stages (Stage3 to 5) of Backbone t The feature map sequence The side lengths are and 7. The method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video according to claim 6, characterized in that: The short-term temporal feature aggregation module adopts a Transformer-like architecture, consisting of an encoder and an optimized decoder. Its workflow is divided into the following two parts: Feat5 is spliced in the time series dimension and input into the Encoder. After the Encoder calculates the global self-attention of Feat5, the encoded features are output. where c, h and w are The number of channels, the height and width of the feature map; Decoder uses spatial features As Query, and let it be generated by Encoder Calculate cross attention. In this way, Decoder can use from Retrieve the frame t Related features, thereby outputting short-term spatial and temporal features 8. The method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video according to claim 7, characterized in that: The long-term temporal feature aggregation module constructs a working memory buffer and a long-term memory buffer to As input, the long-term spatiotemporal features stored in the working memory buffer and and long-term memory features stored in the long-term memory buffer The features are concatenated and input into the long-term temporal feature aggregation module Encoder. The Encoder aggregates the spatiotemporal features of the working memory buffer, the long-term memory buffer and the current moment, and uses a 4×3×3 3D convolution kernel to compress the temporal dimension and output and Derived from Clip t-3 and Clip t-6 The long-term spatiotemporal features obtained by inference are thus generated Compared to It has richer and denser temporal feature information.
9. The method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video according to claim 2, wherein: The neck network uses the long-term spatiotemporal features of the current frame and the shallow feature map to perform multi-scale fusion, and outputs the edge box of the carotid artery anatomical structure and plaque through the detection head.
10. The method for detecting the anatomical structure of the carotid artery using ultrasound based on real-time video according to claim 2, wherein: The three fused multi-scale feature maps are input into the detection head, and the target frame and category are predicted through two 3×3 convolutional layers with different weights respectively, so as to obtain the positioning frame and category results of the object.