Ultrasonic video ovary lesion analysis method fusing anatomical prior and spatial-temporal characteristics

Abstract

The invention discloses an ultrasonic video ovary lesion analysis method fusing anatomical prior and spatial-temporal characteristics. Extracting features of three scales from each frame of image in the ultrasonic video sequence through the backbone network, and stacking the features of the same scale along the time dimension to obtain multi-scale frame-level features; the features of different scales are optimized and fused through a feature pyramid module, and scale enhancement features are obtained; through an anatomical prior enhancement module, generating a prediction segmentation map of the ovary and the focus according to the scale enhancement features; through a gating mechanism, obtaining anatomical prior enhancement features according to the prediction segmentation map and the scale enhancement features; performing space-time fusion on the anatomical prior enhancement features through a pixel-level time sequence attention mechanism through a space-time feature fusion module to obtain fusion features; and through the target detection head, generating a focus bounding box set and a confidence set of each frame of image based on the fusion features. The problems that a traditional single-frame detection method lacks time sequence association and background interference in a dynamic video is large can be solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for analyzing ovarian lesions using ultrasound video that integrates anatomical priors and spatiotemporal features. Background Technology

[0002] Ultrasound, with its unique advantages such as being non-invasive, real-time, and providing dynamic visualization, has become the preferred imaging method for screening and following up on ovarian lesions. Compared with static images, ultrasound video can provide richer spatial structural changes and temporal clues, such as the deformation patterns of lesions as the probe moves, their relative positional changes with the ovary and adnexa, and the dynamic manifestations of tissue or fluid-filled areas.

[0003] With the development of deep learning technology, related research has begun to explore the use of artificial intelligence to assist in the detection of ovarian lesions in order to make up for the shortcomings of traditional manual diagnosis. Current AI-assisted detection methods are mainly based on single-frame static images, and can be divided into two categories: one is static image-based target detection methods, which use detection networks such as YOLO and Faster R-CNN to locate lesion bounding boxes in single-frame images. These methods have clear implementation paths and easy-to-control training samples, but their detection performance is highly dependent on the quality of single-frame images. When there is noise, shadow interference, blurred lesion boundaries, or changes in probe angle, the detection accuracy is easily reduced, and it is difficult to capture the continuous structural changes of lesions as they move with the scan and their dynamic relationship with surrounding anatomical structures, which easily leads to false detections or missed detections. The other category is segmentation-assisted target detection methods. These methods attempt to segment the ovary or lesion first, and then use the segmentation map as a spatial constraint for detection. However, they still have the limitations of static frames and are highly dependent on segmentation accuracy. Once the segmentation is affected by noise and deviates from the real structure, it will directly lead to missed detections or incorrect localization of lesions. At the same time, due to the variable location and complex shape of lesions, static segmentation-assisted strategies are difficult to maintain stable performance and robustness under actual scanning conditions.

[0004] In summary, existing technologies have significant limitations: Firstly, current methods are all based on single-frame static images, failing to fully utilize the continuous structural and temporal information in ultrasound videos. They can only acquire local, instantaneous information, making it difficult to capture the morphological changes of lesions as the probe moves and their relative relationship with surrounding anatomical structures. Furthermore, they cannot learn inter-frame correlations and stable lesion patterns, leading to jumps, flickering, or missed detections in the video. Secondly, existing technologies lack effective anatomical prior constraints. Single-frame detection methods rely solely on image appearance features, making it difficult to accurately distinguish lesions from surrounding structures or lesions inside or outside the ovary. Even when some methods introduce segmentation as an auxiliary prior, the reliance on static segmentation results and limitations in segmentation accuracy prevent them from providing stable structural guidance for detection, hindering stable lesion localization. These deficiencies make it difficult for existing technologies to achieve stability and consistency in ovarian lesion detection under complex scanning conditions, failing to fully leverage the dynamic advantages of ultrasound videos and ultimately failing to meet the needs of actual clinical diagnosis and treatment.

[0005] Therefore, existing technologies still need improvement and development. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide an ultrasound video ovarian lesion analysis method that integrates anatomical priors and spatiotemporal features, in order to address the above-mentioned deficiencies of the prior art. This method aims to solve the problems of existing technologies that rely on static images for ovarian lesion detection, are susceptible to noise interference, and are difficult to deal with complex tissue structures.

[0007] The technical solution adopted by this invention to solve the problem is as follows: In a first aspect, embodiments of the present invention provide a method for analyzing ovarian lesions using ultrasound video that integrates anatomical priors and spatiotemporal features, the method comprising: Through the backbone network, features at three scales are extracted from each frame of the ultrasound video sequence. Features at the same scale from each frame are stacked along the time dimension to obtain multi-scale frame-level features. The three scales correspond to different resolutions. The feature pyramid module optimizes and fuses features of different scales in the multi-scale frame-level features to obtain scale-enhanced features for each scale. The anatomical prior enhancement module generates a predicted segmentation map of the ovary and lesions based on the scale enhancement features at each scale using an auxiliary segmentation head; the anatomical prior enhancement features at each scale are obtained based on the predicted segmentation map and the scale enhancement features using a gating mechanism. The spatiotemporal feature fusion module performs spatiotemporal fusion on the anatomical prior enhancement features at each scale through a pixel-level temporal attention mechanism to obtain the fused features at each scale. Using the target detection head, a set of lesion bounding boxes and a set of confidence scores are generated for each frame of the image based on the fusion features at each scale.

[0008] In one implementation, the step of optimizing and fusing features of different scales in the multi-scale frame-level features to obtain scale-enhanced features at each scale includes: For each scale of the multi-scale frame-level features, the features of the other two scales are aligned with the features of that scale in terms of spatial resolution. Based on the features of each aligned scale, element-wise weighted fusion is performed to obtain the scale-enhanced features corresponding to that scale.

[0009] In one implementation, the step of obtaining the scale-enhanced features at each scale is followed by: The scale-enhanced features from the three scales are combined to obtain the multi-scale pyramid features; The multi-scale pyramid features are used as input to the anatomical prior enhancement module.

[0010] In one implementation, the step of obtaining the anatomical prior enhancement features for each scale based on the predicted segmentation map and the scale enhancement features through a gating mechanism includes: For each scale, gating weights are generated using a gating mechanism and the predicted segmentation map for that scale; The anatomical prior enhancement features for that scale are generated based on the scale enhancement features and the gating weights.

[0011] In one implementation, the step of obtaining the anatomical prior enhancement features at each scale further includes: By summing up the anatomical prior enhancement features at all scales, we obtain the multi-scale features of anatomical prior enhancement. The multi-scale features enhanced by the anatomical prior are used as input to the spatiotemporal feature fusion module.

[0012] In one implementation, the step of spatiotemporally fusing the anatomical prior enhancement features at each scale using a pixel-level temporal attention mechanism to obtain the fused features at each scale includes: Perform spatial convolution on each frame to obtain spatial features; perform temporal convolution along the time dimension based on the spatial features to obtain temporal features; Pixel-level temporal attention weights are generated based on the temporal features of each frame; Based on the temporal features of each frame and the pixel-level temporal attention weights, fused features are generated; all fused features are summarized to obtain a multi-scale feature pyramid sequence. In one implementation, the step of obtaining the fusion features at each scale is followed by: The fused features from all scales are combined to obtain a multi-scale feature pyramid sequence; The multi-scale feature pyramid sequence is used as the input to the target detection head.

[0013] In one implementation, the step of generating a set of lesion bounding boxes and a set of confidence scores for each frame of image based on the fusion features at each scale includes: For each scale of the fused features, channels are compressed and features are fused through a convolutional layer, and then a prediction result is generated through a convolutional head; the prediction result includes: the bounding box regression value and confidence probability of each potential ovarian lesion; Based on the prediction results at the three scales, the set of lesion bounding boxes and the set of confidence scores corresponding to each frame of the image are output.

[0014] Secondly, embodiments of the present invention also provide an ultrasound video ovarian lesion analysis system that integrates anatomical priors and spatiotemporal features, the system comprising: The backbone network is used to extract features at three scales from each frame of an ultrasound video sequence. Features at the same scale from each frame are stacked along the time dimension to obtain multi-scale frame-level features. The three scales correspond to different resolutions. The feature pyramid module is used to optimize and fuse features of different scales in the multi-scale frame-level features to obtain scale-enhanced features for each scale. The anatomical prior enhancement module is used to generate a predicted segmentation map of the ovary and lesions based on the scale enhancement features at each scale using an auxiliary segmentation head; and to obtain the anatomical prior enhancement features at each scale based on the predicted segmentation map and the scale enhancement features through a gating mechanism. The spatiotemporal feature fusion module is used to perform spatiotemporal fusion of the anatomical prior enhancement features at each scale through a pixel-level temporal attention mechanism to obtain the fused features at each scale. The target detection head is used to generate a set of lesion bounding boxes and a set of confidence scores for each frame of image based on the fusion features at each scale.

[0015] Thirdly, embodiments of the present invention also provide a computer-readable storage medium having stored thereon a plurality of instructions adapted to be loaded and executed by a processor to implement the steps of the ultrasound video ovarian lesion analysis method that integrates anatomical priors and spatiotemporal features as described above.

[0016] The beneficial effects of this invention are as follows: In this embodiment, a backbone network is used to extract features at three scales from each frame of an ultrasound video sequence. Features at the same scale are stacked along the time dimension to obtain multi-scale frame-level features. A feature pyramid module optimizes and fuses features at different scales to obtain scale-enhanced features. An anatomical prior enhancement module generates predicted segmentation maps of the ovary and lesions based on the scale-enhanced features. A gating mechanism is used to obtain anatomical prior enhancement features based on the predicted segmentation maps and scale-enhanced features. A spatiotemporal feature fusion module performs spatiotemporal fusion of the anatomical prior enhancement features using a pixel-level temporal attention mechanism to obtain fused features. A target detection head generates a set of lesion bounding boxes and a set of confidence scores for each frame based on the fused features. This invention, through the anatomical prior enhancement module and the spatiotemporal feature fusion module, achieves high-quality optimization of video-level features with dual enhancements of spatial constraints and temporal calibration, solving the problems of lack of temporal correlation and large background interference in dynamic videos in traditional single-frame detection methods. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the steps of the ultrasound video ovarian lesion analysis method that integrates anatomical priors and spatiotemporal features provided in the embodiments of the present invention.

[0019] Figure 2 This is a flowchart of the anatomical prior enhancement module provided in an embodiment of the present invention.

[0020] Figure 3 This is a flowchart of the spatiotemporal feature fusion module provided in an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of the module of the ultrasound video ovarian lesion analysis method system that integrates anatomical priors and spatiotemporal features provided in the embodiments of the present invention.

[0022] Figure 5 This is a schematic diagram of the terminal provided in the embodiment of the present invention. Detailed Implementation

[0023] This invention discloses a method for analyzing ovarian lesions using ultrasound video that integrates anatomical priors and spatiotemporal features. To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention.

[0024] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0025] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0026] To address the aforementioned shortcomings of existing technologies, this invention provides a method for analyzing ovarian lesions using ultrasound video that integrates anatomical priors and spatiotemporal features, such as... Figure 1 As shown, the method specifically includes the following steps: Step S100: Through the backbone network, extract features of three scales from each frame of the ultrasound video sequence, stack the features of the same scale of each frame along the time dimension to obtain multi-scale frame-level features; the three scales correspond to different resolutions.

[0027] Specifically, the backbone network, also known as the backbone network feature extraction module, provides high-quality basic features for the entire ovarian lesion detection process. The backbone network takes the original continuous ultrasound video sequence as input and extracts multi-scale spatial features from each frame of the sequence. As the foundational layer of the entire detection network, the backbone network ensures the full expression of lesion information in the spatial dimension, providing basic feature input for subsequent feature enhancement and temporal fusion. In specific processing, the backbone network independently extracts three different resolution features from each frame of the ultrasound video sequence. These three different scale features can respectively capture the detailed information of the lesion, balance spatial and semantic information, and global contextual information, thus fully representing the lesion structures of varying sizes and shapes. After extracting the features of each frame at the three scales, features of the same scale from all frames are stacked along the temporal dimension to form multi-scale frame-level features. This preserves the spatial attributes of features at each scale while maintaining the temporal correlation between video frames, providing input with both spatial detail and temporal information for further processing by subsequent modules.

[0028] For example, the key data processing steps of the backbone network are as follows: 1. Input data: Receive raw ultrasound video sequence Each frame is sized as , forming a video tensor ,in For frame number, , , These represent the number of channels, height, and width, respectively.

[0029] 2. Multi-scale feature extraction: for each frame Input the convolutional neural network (CNN) backbone and extract three layers of multi-scale features: ； in, High-resolution features preserve detailed information for small lesion detection; It provides medium-resolution features, balancing spatial and semantic information; These are low-resolution features used to capture global context.

[0030] 3. Frame sequence stacking: Features of the same scale from each frame are stacked along the time dimension to form multi-scale frame-level features (also known as video-level multi-scale feature tensors or multi-scale frame sequence features): ; Through the above steps, the backbone network can effectively capture lesion information of different sizes and shapes at different spatial scales, while maintaining complete frame-level time series features, providing basic data for stable localization and dynamic analysis of ovarian lesions.

[0031] Step S200: Through the feature pyramid module, the features of different scales in the multi-scale frame-level features are optimized and fused to obtain the scale-enhanced features of each scale.

[0032] Specifically, the feature pyramid module fuses the multi-scale frame-level features extracted by the backbone network to construct a pyramid feature representation. This enables the network to effectively capture lesion information of varying sizes and complex shapes at different spatial scales, providing a multi-scale foundation for subsequent anatomical prior enhancement and temporal feature fusion. The multi-scale frame-level features output by the backbone network contain features at three scales corresponding to different resolutions, each reflecting different dimensions of information such as lesion details, semantics, or global context. Since single-scale feature representation is relatively limited, the feature pyramid module optimizes and fuses these features at different scales, generating corresponding scale-enhanced features for each scale. These features contain information from the original scale as well as information from other scales.

[0033] In one implementation, the step of optimizing and fusing features of different scales in the multi-scale frame-level features to obtain scale-enhanced features for each scale includes: For each scale of the multi-scale frame-level features, the features of the other two scales are aligned with the features of that scale in terms of spatial resolution. Based on the features of each aligned scale, element-wise weighted fusion is performed to obtain the scale-enhanced features corresponding to that scale.

[0034] Furthermore, after obtaining the scale-enhanced features at each scale, the process also includes: The scale-enhanced features from the three scales are combined to obtain the multi-scale pyramid features; The multi-scale pyramid features are used as input to the anatomical prior enhancement module.

[0035] Specifically, for features at each scale, convolutional processing, combined with upsampling or downsampling operations, is used to adjust the spatial resolution of the features, achieving dimensional alignment of features at different scales. Then, weighted fusion is used to make the information from each scale complementary. Ultimately, this fusion process generates corresponding scale-enhanced features for each original scale. These features retain the key information from the original scale while incorporating effective information from other scales, becoming a more robust multi-scale foundation required for subsequent dissection prior enhancement and temporal feature fusion. Summarizing the scale-enhanced features from the three scales yields the multi-scale pyramid features (also known as the fused pyramid features or multi-scale video features).

[0036] For example, the key data processing steps of the feature pyramid module are as follows: 1. Input data: Receive multi-scale frame-level features output from the backbone network: ; in, For frame number, , , The first The number of channels, height, and width of the layer features.

[0037] 2. Context Enhancement and Scale Fusion: Each layer of features is first processed by convolution and upsampling, or convolution and downsampling, to align features of different scales in spatial resolution. Then, element-wise weighted fusion is performed to obtain scale-enhanced features. ; in, Bilinear interpolation is used to improve resolution. Compressing the spatial size through stride convolution, and The operation uses channel-weighted summation to integrate multi-layer semantic information. This operation enhances cross-scale information transfer, enabling the simultaneous expression of detailed information from small lesions and global contextual information. Compared to the traditional FPN structure that only uses upsampling, this bidirectional fusion method further strengthens the bidirectional transfer of cross-scale information, making the multi-scale feature expression more complete and stable.

[0038] 3. Output data: Multi-scale pyramid features: ; Through the processing of the feature pyramid module, the network can effectively capture lesion information of different sizes and shapes at different spatial scales, while maintaining complete frame-level time series features. This provides a robust multi-scale input foundation for the anatomical prior enhancement module, thereby supporting the stable localization and dynamic analysis of ovarian lesions.

[0039] Step S300: Using the anatomical prior enhancement module, a predicted segmentation map of the ovary and lesion is generated by the auxiliary segmentation head based on the scale enhancement features at each scale; using the gating mechanism, the anatomical prior enhancement features at each scale are obtained based on the predicted segmentation map and the scale enhancement features.

[0040] Specifically, such as Figure 2 As shown, the anatomical prior enhancement module first generates predicted segmentation maps of the ovary and lesions for each scale enhancement feature using an auxiliary segmentation head. This step can be seen as introducing anatomical topological constraints on top of the multi-scale pyramid features, allowing the model to capture the spatial structural patterns of the ovary and lesions, providing clear anatomical semantic references for subsequent feature enhancement. Subsequently, a gating mechanism is used to dynamically adjust the generated predicted segmentation map and the original scale enhancement features. The gating mechanism adaptively adjusts the contribution of segmentation features (i.e., features in the predicted segmentation map) to the original features (i.e., scale enhancement features), avoiding negative impacts on detection when segmentation results are affected by noise, ultimately achieving precise enhancement of features at each scale. Throughout the process, the anatomical prior enhancement module integrates anatomical prior information about the ovary and its surrounding structures to perform structural guidance and region enhancement on the multi-scale pyramid features output by the feature pyramid module, highlighting potential lesion areas and suppressing background noise. This allows the network to more stably focus on potential lesion areas, thereby improving the stability and accuracy of the network's localization of real lesion areas.

[0041] In one implementation, the step of obtaining the anatomical prior enhancement features for each scale based on the predicted segmentation map and the scale enhancement features through a gating mechanism includes: For each scale, gating weights are generated using a gating mechanism and the predicted segmentation map for that scale; The anatomical prior enhancement features for that scale are generated based on the scale enhancement features and the gating weights.

[0042] Furthermore, after obtaining the anatomical prior enhancement features at each scale, the process also includes: By summing up the anatomical prior enhancement features at all scales, we obtain the multi-scale features of anatomical prior enhancement. The multi-scale features enhanced by the anatomical prior are used as input to the spatiotemporal feature fusion module.

[0043] For example, the key data processing steps of the anatomy prior enhancement module are as follows: 1. Input data: Receive multi-scale pyramid features output by the feature pyramid module: ; in For frame number, , , The first The number of channels, height, and width of layer features 2. Segmented supervision and enhanced gating: (1) Segmentation supervision: For each layer of features Predicted ovarian and lesion segmentation maps are generated using an auxiliary segmentation head: ; Supervised training using real-world annotations ensures that segmentation features preserve global spatial information about the ovaries and lesions.

[0044] (2) Gating mechanism: In order to avoid the negative impact of segmentation failure on detection, gating weights are introduced. ; The segmentation features, which implicitly contain anatomical semantics, are processed through a 1×1 convolution + The mapping yields an automatically adjusted contribution of segmentation features to the original features, resulting in the final enhanced features: ; The gating mechanism ensures that the original features are enhanced only when the segmentation is reliable, suppresses unreliable segmentation regions, and improves feature stability.

[0045] 3. Output Data: Output multi-scale features enhanced by anatomical priors: ; This feature not only preserves frame-level time series information but also enhances the spatial representation of lesions, providing a robust input foundation for the spatiotemporal feature fusion module.

[0046] In summary, the anatomical prior enhancement module includes two key technologies: (1) Topological constraint mechanism: the anatomical structure prior of the ovary and lesion is introduced by the auxiliary segmentation head, which forces the network to focus on potential physiological structure regions during the feature extraction stage. (2) Adaptive gating enhancement: a gating weight based on nonlinear mapping operation is constructed to automatically adjust the contribution of segmented semantic features to the original features, realize soft attention enhancement, and effectively suppress irrelevant background noise.

[0047] Through the processing of the anatomical prior enhancement module, the network can make full use of anatomical information at different spatial scales, highlight potential lesion areas and suppress background interference, while ensuring the positive role of segmentation supervision and gating mechanisms, providing reliable, multi-scale input features for the spatiotemporal feature fusion module, thereby improving the stability and accuracy of ovarian lesion detection.

[0048] In one implementation, the anatomy prior enhancement module can be combined with fully supervised segmentation (relying on Mask) or weakly supervised strategies (relying on Box to generate pseudomasks) to flexibly adapt to training data with different annotation accuracies.

[0049] Step S400: The spatiotemporal feature fusion module performs spatiotemporal fusion on the anatomical prior enhancement features at each scale through a pixel-level temporal attention mechanism to obtain the fused features at each scale.

[0050] Specifically, the spatiotemporal feature fusion module fully utilizes the temporal information of ultrasound video while preserving spatial structural features. It performs spatiotemporal fusion of multi-scale features enhanced by anatomical priors to generate stable frame-level features, providing reliable input for the target detection head. Unlike simple feature overlay or global pooling, the spatiotemporal feature fusion module in this embodiment employs a strategy combining frame-by-frame convolution and temporal-dimensional convolution, supplemented by a pixel-level temporal attention mechanism to adaptively aggregate important features from the video sequence. For example... Figure 3 As shown, the specific processing logic of the spatiotemporal feature fusion module is as follows: For the anatomical prior enhancement features at each scale, inter-frame correlation modeling is carried out through a pixel-level temporal attention mechanism combined with temporal convolution. This automatically captures the morphological changes of lesions as the probe moves and their temporal stable patterns, achieving comprehensive feature representation for video. This process not only makes the network's understanding of the spatial structure and dynamic evolution of lesions more consistent with the clinical observation process, but also effectively reduces interference from single-frame noise, artifacts, and probe movement, significantly improving robustness and consistency while ensuring detection accuracy. Finally, by spatiotemporally fusing the anatomical prior enhancement features at each scale, the fused features at the corresponding scale are obtained, thereby helping to achieve continuous lesion detection for video.

[0051] In one implementation, the step of spatiotemporally fusing the anatomical prior enhancement features at each scale using a pixel-level temporal attention mechanism to obtain the fused features at each scale includes: Perform spatial convolution on each frame to obtain spatial features; perform temporal convolution along the time dimension based on the spatial features to obtain temporal features; Pixel-level temporal attention weights are generated based on the temporal features of each frame; Based on the temporal features of each frame and the pixel-level temporal attention weights, fused features are generated; all fused features are summarized to obtain a multi-scale feature pyramid sequence. Furthermore, after obtaining the fusion features at each scale, the process also includes: The fused features from all scales are combined to obtain a multi-scale feature pyramid sequence; The multi-scale feature pyramid sequence is used as the input to the target detection head.

[0052] For example, the key data processing steps of the spatiotemporal feature fusion module are as follows: 1. Input data: Receive anatomical prior augmented multi-scale features: ; in The number of frames in the video sequence. This represents the feature hierarchy. The feature map still retains its complete spatial dimensions. In order to preserve the geometric information of the lesion.

[0053] 2. Frame-by-frame spatial convolution and temporal convolution: (1) Spatial feature extraction (frame-by-frame 2D convolution): Perform convolution operations independently on each frame (i.e., the features at each scale in the multi-scale features enhanced by anatomical priors) to extract morphological features while preserving the spatial details of small lesions: .

[0054] (2) Temporal feature extraction (convolution along the time dimension): In the time dimension Convolution is performed on the top layer to fuse contextual information from adjacent frames, enhancing the temporal consistency of features: ; in, and All include convolution, and Activation function, with output dimension consistent with input: .

[0055] 3. Pixel-level Temporal Self-Attention Aggregation: To address lesion morphology changes caused by probe movement, the spatiotemporal feature fusion module introduces a pixel-level temporal attention mechanism, adaptively weighting and recalibrating each pixel in each frame of the video sequence. Unlike methods that simply compress features from multiple frames into a single frame, this embodiment preserves the spatial information of each frame and automatically learns the confidence weight of each pixel through the network, thereby enhancing clear and discriminative lesion features while suppressing background noise and motion blur. The specific calculation is as follows: ; The frame-level features are weighted and summed using an attention weight map to generate fused frame-level features (i.e., fused features): ; in, This indicates element-wise multiplication. The formula employs a residual connection structure, which preserves the original feature information while also incorporating attention weights. High-value areas are highlighted. At this point, the output features are... Still maintains a complete time dimension .

[0056] 4. Output Data: The spatiotemporal feature fusion module outputs a multi-scale feature pyramid sequence that has undergone time-series calibration and enhancement. ; In summary, the spatiotemporal feature fusion module includes the following key technologies: (1) Decompositional spatiotemporal modeling: A strategy combining frame-by-frame spatial 2D convolution and temporal 1D convolution is adopted to efficiently fuse temporal context without destroying spatial details. (2) Pixel-level temporal self-calibration: A pixel-level temporal attention mechanism is introduced to calculate inter-frame correlation weights and use residual connections to recalibrate features, automatically combating interference caused by probe movement and lesion deformation. (3) Non-collapsed output: Unlike traditional temporal pooling, the output of the spatiotemporal feature fusion module in this embodiment maintains completeness. The frame-time dimension supports end-to-end frame-by-frame detection.

[0057] Through the spatiotemporal feature fusion module, the network retains the complete temporal length of the video while eliminating interference from unstable features along the time dimension. This enables adaptive optimization of the feature quality for each frame, significantly improving the localization accuracy of subsequent target detection heads in dynamic video streams. The multi-scale feature pyramid sequence output by the spatiotemporal feature fusion module can be directly input into the target detection head for frame-by-frame lesion localization and confidence prediction.

[0058] Step S500: Using the target detection head, generate a set of lesion bounding boxes and a set of confidence scores for each frame of image based on the fusion features at each scale.

[0059] The target detection head generates bounding boxes and confidence scores for ovarian lesions based on the multi-scale feature pyramid sequence output by the spatiotemporal feature fusion module. It is important to emphasize that these bounding boxes and confidence scores are not diagnostic information, but rather supplementary reference information for clinical analysis. Medical personnel need to use this information for further confirmation. In other words, the bounding box indicates the possible presence of an ovarian lesion in the area, while the confidence score reflects the probability (usually not 100%) that it is an ovarian lesion, not a 100% certainty.

[0060] The target detection head is the final output of the entire detection process. It directly receives a multi-scale feature pyramid sequence from the spatiotemporal feature fusion module, which contains fused features at various scales. These features have fully integrated the spatial structural information and temporally stable patterns of the ultrasound video and are high-quality features optimized through multiple rounds. In specific processing, the target detection head uses lightweight convolutional layers or standard detection head structures (such as YOLO or RetinaNet Head) for target regression and classification prediction: target regression calculates the location and size of regions where lesions may exist, forming bounding boxes; classification prediction determines the probability that a region is a lesion, generating confidence scores. Based on this, the target detection head further integrates the prediction results of the fused features at various scales, generating a set of lesion bounding boxes and a set of confidence scores for each frame of the image. These output results can be directly used for quantitative localization results in clinical analysis, risk assessment, and treatment decisions. Through the synergistic effect of various modules such as the backbone network, feature pyramid module, anatomical prior enhancement module, and spatiotemporal feature fusion module, the spatial and temporal information in the ultrasound video can be fully extracted, ultimately achieving high-precision and robust detection of ovarian lesions.

[0061] In one implementation, the step of generating a set of lesion bounding boxes and a set of confidence scores for each frame of image based on the fusion features at each scale includes: For each scale of the fused features, channels are compressed and features are fused through a convolutional layer, and then a prediction result is generated through a convolutional head; the prediction result includes: the bounding box regression value and confidence probability of each potential ovarian lesion; Based on the prediction results at the three scales, the set of lesion bounding boxes and the set of confidence scores corresponding to each frame of the image are output.

[0062] For example, the key data processing steps of the target detection head are as follows: 1. Input data: Receive the multi-scale feature pyramid sequence (also known as single-frame-level multi-scale features) output by the spatiotemporal feature fusion module: ; in, , , These are the number of frames, the number of feature channels, the height, and the width, respectively.

[0063] 2. Feature Convolution and Prediction: For each layer of features The convolutional layers compress channels and perform feature fusion, then use convolutional prediction to generate bounding box regressions for each potential lesion. and confidence probability : ; in, This indicates the bounding box offset relative to the anchor frame or a predefined grid. This represents the probability of lesion presence in each prediction box.

[0064] 3. Output Data: Integrate the prediction results from multiple scales to obtain the final bounding box and confidence score set for each lesion frame: ; in, For the first The set of lesion bounding boxes in the frame. This represents the corresponding confidence level set.

[0065] 4. Training and Testing: During the training phase, the target detection head is supervised using regression loss and classification loss, based on the lesion bounding boxes and category information labeled in each frame, enabling the network to learn accurate lesion localization and confidence prediction. During the testing phase, the network generates predicted bounding boxes and corresponding probabilities for each frame, and redundant overlapping boxes are removed using non-maximum suppression (NMS) to obtain the final lesion detection results for each frame.

[0066] While maintaining high spatial resolution, the target detection head can achieve precise localization and confidence assessment of lesions in each frame, ensuring that the detection results can be used for clinical analysis, O-RADS grading, and subsequent treatment decisions.

[0067] The advantages of this invention are: 1. Dual Enhancement: This invention achieves high-quality video-level feature optimization through two innovative modules: an anatomical prior enhancement module and a spatiotemporal feature fusion module. This dual enhancement of "spatial constraints + temporal calibration" addresses the key issues of traditional single-frame detection methods, such as lack of temporal correlation and significant background interference in dynamic videos. The anatomical prior enhancement module introduces anatomical topological constraints and, through a segmentation-supervised gating mechanism, effectively suppresses noise in non-lesion areas in the spatial dimension, performing spatial denoising on features and filtering out background noise and irrelevant tissue interference. This allows the network to more accurately focus on potential lesion areas and strengthens the spatial representation of lesion regions. The spatiotemporal feature fusion module, through a pixel-level spatiotemporal attention mechanism, recalibrates features in the temporal dimension to address lesion deformation caused by probe movement. It captures the morphological changes and temporal stability patterns of lesions as the probe moves, ensuring that detection results are unaffected by dynamic interference. This comprehensive optimization of video-level features allows the network's understanding of the spatial structure and dynamic evolution of lesions to better align with clinical observation, significantly improving the robustness and accuracy of lesion detection in dynamic ultrasound environments.

[0068] 2. End-to-end frame-level detection process ensures continuous and stable localization: This invention constructs a complete end-to-end detection process from ultrasound video input to frame-by-frame lesion localization output, without the need for additional manual intervention, realizing continuous monitoring and stable localization of lesions in dynamic video.

[0069] 3. Modular expansion design balances flexibility and practicality: The core module of this invention has plug-and-play characteristics and supports flexible combination with different backbone networks (such as ResNet, EfficientNet) and detection heads (such as YOLO, RetinaNet). It can select the appropriate network structure according to the accuracy requirements of the actual application scenario, while also taking into account the computing efficiency, and can flexibly meet the usage needs of different hardware environments and diagnostic and treatment scenarios.

[0070] Based on the above embodiments, the present invention also provides an ultrasound video ovarian lesion analysis system that integrates anatomical priors and spatiotemporal features, such as... Figure 4 As shown, the system includes: The backbone network is used to extract features at three scales from each frame of an ultrasound video sequence. Features at the same scale from each frame are stacked along the time dimension to obtain multi-scale frame-level features. The three scales correspond to different resolutions. The feature pyramid module is used to optimize and fuse features of different scales in the multi-scale frame-level features to obtain scale-enhanced features for each scale. The anatomical prior enhancement module is used to generate a predicted segmentation map of the ovary and lesions based on the scale enhancement features at each scale using an auxiliary segmentation head; and to obtain the anatomical prior enhancement features at each scale based on the predicted segmentation map and the scale enhancement features through a gating mechanism. The spatiotemporal feature fusion module is used to perform spatiotemporal fusion of the anatomical prior enhancement features at each scale through a pixel-level temporal attention mechanism to obtain the fused features at each scale. The target detection head is used to generate a set of lesion bounding boxes and a set of confidence scores for each frame of image based on the fusion features at each scale.

[0071] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 5 As shown, the terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for analyzing ovarian lesions using ultrasound video that integrates anatomical priors and spatiotemporal features. The display screen can be an LCD screen or an e-ink screen.

[0072] Those skilled in the art will understand that Figure 5The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0073] In one implementation, the terminal's memory stores one or more programs, and these programs are configured to be executed by one or more processors. The programs include instructions for performing an ultrasound video ovarian lesion analysis method that integrates anatomical priors and spatiotemporal features.

[0074] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0075] In summary, this invention discloses a method for analyzing ovarian lesions in ultrasound video by fusing anatomical priors and spatiotemporal features. The method extracts features at three scales from each frame of an ultrasound video sequence using a backbone network. Features at the same scale from each frame are stacked along the time dimension to obtain multi-scale frame-level features. Each of the three scales corresponds to a different resolution. A feature pyramid module optimizes and fuses the features at different scales within the multi-scale frame-level features to obtain scale-enhanced features for each scale. An anatomical prior enhancement module, using an auxiliary segmentation head, generates a predicted segmentation map of the ovary and lesions based on the scale-enhanced features at each scale. A gating mechanism is used to obtain anatomical prior enhancement features at each scale based on the predicted segmentation map and the scale-enhanced features. A spatiotemporal feature fusion module performs spatiotemporal fusion of the anatomical prior enhancement features at each scale using a pixel-level temporal attention mechanism to obtain fused features at each scale. Finally, a target detection head generates a set of lesion bounding boxes and a set of confidence scores for each frame based on the fused features at each scale. This invention achieves high-quality optimization of video-level features with dual enhancements of spatial constraints and temporal calibration through a priori dissection enhancement module and a spatiotemporal feature fusion module. This can solve the problems of lack of temporal correlation and large background interference in dynamic videos in traditional single-frame detection methods.

[0076] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for analyzing ovarian lesions using ultrasound video that integrates anatomical priors and spatiotemporal features, characterized in that, The method includes: Through the backbone network, features at three scales are extracted from each frame of the ultrasound video sequence. Features at the same scale from each frame are stacked along the time dimension to obtain multi-scale frame-level features. The three scales correspond to different resolutions. The feature pyramid module optimizes and fuses features of different scales in the multi-scale frame-level features to obtain scale-enhanced features for each scale. The anatomical prior enhancement module generates a predicted segmentation map of the ovary and lesions based on the scale enhancement features at each scale using an auxiliary segmentation head; the anatomical prior enhancement features at each scale are obtained based on the predicted segmentation map and the scale enhancement features using a gating mechanism. The spatiotemporal feature fusion module performs spatiotemporal fusion on the anatomical prior enhancement features at each scale through a pixel-level temporal attention mechanism to obtain the fused features at each scale. Using the target detection head, a set of lesion bounding boxes and a set of confidence scores are generated for each frame of the image based on the fusion features at each scale.

2. The method for analyzing ovarian lesions using ultrasound video by fusing anatomical priors and spatiotemporal features according to claim 1, characterized in that, The steps for optimizing and fusing features at different scales in the multi-scale frame-level features to obtain scale-enhanced features at each scale include: For each scale of the multi-scale frame-level features, the features of the other two scales are aligned with the features of that scale in terms of spatial resolution. Based on the features of each aligned scale, element-wise weighted fusion is performed to obtain the scale-enhanced features corresponding to that scale.

3. The method for analyzing ovarian lesions using ultrasound video by fusing anatomical priors and spatiotemporal features according to claim 2, characterized in that, After obtaining the scale-enhanced features for each scale, the process also includes: The scale-enhanced features from the three scales are combined to obtain the multi-scale pyramid features; The multi-scale pyramid features are used as input to the anatomical prior enhancement module.

4. The method for analyzing ovarian lesions using ultrasound video by fusing anatomical priors and spatiotemporal features according to claim 1, characterized in that, The steps of obtaining the anatomical prior enhancement features for each scale based on the predicted segmentation map and the scale enhancement features through a gating mechanism include: For each scale, gating weights are generated using a gating mechanism and the predicted segmentation map for that scale; The anatomical prior enhancement features for that scale are generated based on the scale enhancement features and the gating weights.

5. The method for analyzing ovarian lesions using ultrasound video by fusing anatomical priors and spatiotemporal features according to claim 4, characterized in that, After obtaining the anatomical prior enhancement features at each scale, the process also includes: By summing up the anatomical prior enhancement features at all scales, we obtain the multi-scale features of anatomical prior enhancement. The multi-scale features enhanced by the anatomical prior are used as input to the spatiotemporal feature fusion module.

6. The method for analyzing ovarian lesions using ultrasound video by fusing anatomical priors and spatiotemporal features according to claim 1, characterized in that, The steps for obtaining the fused features at each scale by spatiotemporally fusing the anatomical prior enhancement features at each scale using a pixel-level temporal attention mechanism include: Perform spatial convolution on each frame to obtain spatial features; perform temporal convolution along the time dimension based on the spatial features to obtain temporal features; Pixel-level temporal attention weights are generated based on the temporal features of each frame; Based on the temporal features of each frame and the pixel-level temporal attention weights, fused features are generated; all fused features are then aggregated to obtain a multi-scale feature pyramid sequence.

7. The method for analyzing ovarian lesions using ultrasound video based on anatomical priors and spatiotemporal features according to claim 6, characterized in that, After obtaining the fusion features at each scale, the process also includes: The fused features from all scales are combined to obtain a multi-scale feature pyramid sequence; The multi-scale feature pyramid sequence is used as the input to the target detection head.

8. The method for analyzing ovarian lesions using ultrasound video by fusing anatomical priors and spatiotemporal features according to claim 1, characterized in that, The steps for generating the set of lesion bounding boxes and the set of confidence scores for each frame of image based on the fusion features at each scale include: For each scale of the fused features, channels are compressed and features are fused through a convolutional layer, and then a prediction result is generated through a convolutional head; the prediction result includes: the bounding box regression value and confidence probability of each potential ovarian lesion; Based on the prediction results at the three scales, the set of lesion bounding boxes and the set of confidence scores corresponding to each frame of the image are output.

9. A system for analyzing ovarian lesions using ultrasound video, integrating anatomical priors and spatiotemporal features, characterized in that: The system includes: The backbone network is used to extract features at three scales from each frame of an ultrasound video sequence. Features at the same scale from each frame are stacked along the time dimension to obtain multi-scale frame-level features. The three scales correspond to different resolutions. The feature pyramid module is used to optimize and fuse features of different scales in the multi-scale frame-level features to obtain scale-enhanced features for each scale. The anatomical prior enhancement module is used to generate a predicted segmentation map of the ovary and lesions based on the scale enhancement features at each scale using an auxiliary segmentation head; and to obtain the anatomical prior enhancement features at each scale based on the predicted segmentation map and the scale enhancement features through a gating mechanism. The spatiotemporal feature fusion module is used to perform spatiotemporal fusion of the anatomical prior enhancement features at each scale through a pixel-level temporal attention mechanism to obtain the fused features at each scale. The target detection head is used to generate a set of lesion bounding boxes and a set of confidence scores for each frame of image based on the fusion features at each scale.

10. A computer-readable storage medium storing a plurality of instructions thereon, characterized in that, The instructions are applicable to be loaded and executed by a processor to implement the steps of the ultrasound video ovarian lesion analysis method that integrates anatomical priors and spatiotemporal features as described in any one of claims 1 to 8.

Images