Anti-noise space-time self-supervised ultrasonic image processing model pre-training method

Abstract

The invention discloses an anti-noise space-time self-supervised ultrasonic image processing model pre-training method. The method comprises the following steps: segmenting an ultrasonic video sequence into a plurality of three-dimensional space-time cubes, and generating a feature vector sequence with 3D position information according to the three-dimensional space-time cubes; performing feature extraction on an unshielded region of the feature vector sequence with the 3D position information through a feature extraction module with a space-time pipeline mask mechanism to obtain coding features; through a restoration module, the shielded area is restored according to the coding features, and structural consistency loss and motion consistency loss are calculated; and performing parameter optimization on the ultrasonic image processing model according to the structural consistency loss and the motion consistency loss. According to the method, the problem of difficulty in self-supervision pre-training caused by high noise of the ultrasonic image can be solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a noise-resistant spatiotemporally self-supervised method for pre-training an ultrasound image processing model. Background Technology

[0002] Ultrasound imaging is one of the most widely used imaging modalities in clinical practice, playing an irreplaceable role in key diagnostic and treatment scenarios such as cardiac function assessment, vascular dynamics monitoring, and fetal development assessment. The essence of ultrasound diagnosis is a dynamic analysis process relying on spatiotemporal continuity. Physicians' diagnostic decisions heavily depend on observing tissue motion patterns, deformation laws, and continuous temporal characteristics in the ultrasound video stream, rather than extracting information from a single static section. This makes feature learning from dynamic ultrasound images a core direction for related technology development.

[0003] In recent years, deep learning technology has made significant progress in the field of ultrasound image analysis. However, the success of existing technologies has mainly focused on the fully supervised learning paradigm, which requires expert physicians to meticulously delineate or classify lesions in each frame of ultrasound images. However, ultrasound videos typically have high frame rates, and the cost of manual annotation increases exponentially with the amount of data. To overcome the bottleneck of labeled data, self-supervised learning technology has emerged. Its core idea is to design auxiliary tasks that allow the model to complete pre-training using unlabeled data. Among these methods, masked video modeling (such as Video MAE) has become the mainstream technology in the field of natural scene video processing.

[0004] However, directly transferring the aforementioned video self-supervised techniques from the general computer vision field to ultrasound image analysis faces three fundamental obstacles and shortcomings, all of which require targeted technical solutions to address: First, there is the problem of noise overfitting. Existing mask reconstruction algorithms mostly use pixel-level mean square error as the loss function, which causes a large amount of model computing power to be consumed in fitting high-frequency noise distribution, rather than learning potential pure anatomical structural features. Ultimately, this makes the pre-trained model extremely sensitive to noise and difficult to extract pathological features with diagnostic value. Secondly, there is the phenomenon of shortcut learning. The high frame rate of ultrasound scanning and the smooth movement of the probe result in extremely high visual redundancy between adjacent frames. The time axis discrete random masking strategy of existing technology allows the model to easily complete the reconstruction task through inter-frame copying or pixel interpolation without truly understanding the long-term anatomical motion laws such as heart pulsation and vasoconstriction, and thus cannot learn the core motion features required for dynamic ultrasound diagnosis. Finally, there is the bottleneck of motion source confusion. The visual changes in ultrasound videos actually mix two types of motion: the human movement of the probe and the physiological movement of human tissues. General video models lack anatomical prior constraints and cannot effectively distinguish between these two types of motion. During pre-training, they tend to learn the larger-amplitude background motion of the probe while ignoring the small but critically diagnostic physiological movements of lesions. This results in insufficient robustness of the model in downstream diagnostic tasks and susceptibility to interference from doctors' manipulation techniques.

[0005] Therefore, existing technologies still need improvement and development. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a noise-resistant spatiotemporally self-supervised ultrasound image processing model pre-training method to address the above-mentioned deficiencies of the prior art, aiming to solve the problem that the existing video models cannot effectively extract anatomical features in high-noise and high-redundancy ultrasound scenarios.

[0007] The technical solution adopted by this invention to solve the problem is as follows: In a first aspect, embodiments of the present invention provide a method for pre-training a noise-resistant spatiotemporally self-supervised ultrasound image processing model, the method comprising: The ultrasound video sequence is divided into several three-dimensional spatiotemporal cubes, and a feature vector sequence with 3D position information is generated based on the three-dimensional spatiotemporal cubes. The feature extraction module with a spatiotemporal pipeline masking mechanism is used to extract features from the unmasked region of the feature vector sequence with 3D position information to obtain coded features. The repair module repairs the occluded area based on the encoded features and calculates the structural consistency loss and motion consistency loss. The ultrasonic image processing model is optimized based on the structural consistency loss and the motion consistency loss; the structural consistency loss is calculated based on spatial domain filtering or frequency domain filtering; the ultrasonic image processing model includes the feature extraction module and the repair module.

[0008] In one embodiment, the step of segmenting the ultrasound video sequence into several three-dimensional spatiotemporal cubes includes the following prior to: Obtain the raw ultrasound video sequence; The ultrasound images in the original ultrasound video sequence are preprocessed to remove background and text artifacts outside the fan-shaped and / or cone-shaped scanning areas, and to extract the clean anatomical regions, thus obtaining the ultrasound video sequence composed of clean ultrasound images.

[0009] In one implementation, the three-dimensional spatiotemporal cubes are non-overlapping.

[0010] In one embodiment, the step of generating a feature vector sequence with 3D position information based on the three-dimensional spatiotemporal cube includes: The three-dimensional spatiotemporal cube is flattened and mapped to a feature vector sequence through a linear projection layer, and then a learnable 3D position code is superimposed to obtain the feature vector sequence with 3D position information.

[0011] In one embodiment, the spatiotemporal pipeline masking mechanism includes: randomly sampling spatial coordinates in the first frame plane at a preset masking rate, and continuously occluding the selected spatial coordinates on the time axis to form a spatiotemporal pipeline mask.

[0012] In one embodiment, the spatiotemporal pipeline mask takes the form of a continuous mask whose position changes dynamically over time.

[0013] In one embodiment, the feature extraction module is an encoder; the repair module is a dual-stream decoder, including a first decoder and a second decoder; the steps of repairing the occluded region based on the encoded features and calculating structural consistency loss and motion consistency loss through the repair module include: The first decoder generates a prediction patch based on the encoded features. The prediction patch and the corresponding three-dimensional spatiotemporal cube are transformed into their respective domains. The difference between the transformation domain results of the two after high-frequency component suppression is calculated by combining a preset filter, so as to obtain the structural consistency loss. The second decoder generates predicted dynamic change information based on the encoded features, and calculates the motion consistency loss based on the predicted dynamic change information and the corresponding reference dynamic change information; wherein, when the predicted dynamic change information is a predicted optical flow field, the reference dynamic change information is a dense optical flow field; when the predicted dynamic change information is a predicted frame difference map, the reference dynamic change information is a true frame difference map; when the predicted dynamic change information is a predicted motion vector, the reference dynamic change information is a true motion vector.

[0014] In one embodiment, the step of optimizing the parameters of the ultrasound image processing model based on the structural consistency loss and the motion consistency loss includes: The total loss is obtained by weighted summation of the structural consistency loss and the motion consistency loss. The parameters of the ultrasound image processing model are optimized based on the total loss.

[0015] Secondly, embodiments of the present invention also provide a noise-resistant spatiotemporally self-supervised ultrasound image feature extraction method, the method comprising: Acquire ultrasound video sequences; The ultrasound video sequence is input into a trained ultrasound image processing model to obtain the corresponding encoded feature information; the trained ultrasound image processing model includes a feature extraction module and is trained using any of the noise-resistant spatiotemporally self-supervised ultrasound image processing model pre-training methods described above.

[0016] Thirdly, embodiments of the present invention also provide a computer-readable storage medium storing a plurality of instructions, the instructions being adapted to be loaded and executed by a processor to implement the steps of the noise-resistant spatiotemporally self-supervised ultrasound image processing model pre-training method as described above, or the steps of the noise-resistant spatiotemporally self-supervised ultrasound image feature extraction method as described above.

[0017] The beneficial effects of this invention are as follows: In this embodiment, an ultrasound video sequence is divided into several three-dimensional spatiotemporal cubes, and a feature vector sequence with 3D positional information is generated based on these cubes. A feature extraction module with a spatiotemporal pipeline masking mechanism extracts features from the unoccluded areas of the feature vector sequence with 3D positional information to obtain coded features. A repair module repairs the occluded areas based on the coded features and calculates structural consistency loss and motion consistency loss. The parameters of the ultrasound image processing model are optimized based on the structural consistency loss and motion consistency loss. This invention can solve the problem of difficult self-supervised pre-training of ultrasound images due to high noise levels. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram illustrating the steps of the noise-resistant spatiotemporal self-supervised ultrasound image processing model pre-training method provided in this embodiment of the invention.

[0020] Figure 2 This is a flowchart of the noise-resistant spatiotemporal self-supervised ultrasound image processing model pre-training method provided in this embodiment of the invention.

[0021] Figure 3 This is a block diagram illustrating the principle of the noise-resistant spatiotemporal self-supervised ultrasound image processing model pre-training method provided in this embodiment of the invention.

[0022] Figure 4 This is a schematic diagram of the steps of the noise-resistant spatiotemporally self-supervised ultrasound image feature extraction method provided in the embodiments of the present invention.

[0023] Figure 5 This is a schematic diagram of the terminal provided in an embodiment of the present invention. Detailed Implementation

[0024] This invention discloses a noise-resistant spatiotemporally self-supervised pre-training method for ultrasound image processing models. To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention.

[0025] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0026] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0027] To address the aforementioned shortcomings of existing technologies, this invention provides a noise-resistant spatiotemporally self-supervised pre-training method for ultrasound image processing models, such as... Figure 1 As shown, the method specifically includes the following steps: Step S100: Divide the ultrasound video sequence into several three-dimensional spatiotemporal cubes, and generate a feature vector sequence with 3D position information based on the three-dimensional spatiotemporal cubes.

[0028] Furthermore, before the step of segmenting the ultrasound video sequence into several three-dimensional spatiotemporal cubes, the following steps are also included: Obtain the raw ultrasound video sequence; The ultrasound images in the original ultrasound video sequence are preprocessed to remove background and text artifacts outside the fan-shaped and / or cone-shaped scanning areas, and to extract the clean anatomical regions, thus obtaining the ultrasound video sequence composed of clean ultrasound images.

[0029] Specifically, such as Figure 2 As shown, ultrasound video sequences need to be preprocessed before being input into the model to remove invalid interference information and provide high-quality data for subsequent feature learning. First, the original ultrasound video sequence is obtained through the scanning output of the ultrasound equipment. Due to the scanning characteristics of ultrasound imaging, the original ultrasound video sequence (also known as the original video stream) The original ultrasound video sequence contains black backgrounds outside the fan-shaped or cone-shaped scanning areas, and may also contain text artifacts such as patient information. These not only lack anatomical diagnostic value but also introduce noise interference into subsequent model training, affecting the accuracy of feature extraction. Therefore, the key to preprocessing lies in ROI region cleaning: using morphological masking techniques, each frame of the original ultrasound video sequence is cleaned, removing the black backgrounds outside the fan-shaped and / or cone-shaped scanning areas, as well as scattered text artifacts, retaining only the effective areas containing human anatomical structures within the scanning area—the pure anatomical regions. After preprocessing, the original ultrasound video sequence is cleaned into an ultrasound video sequence containing only pure anatomical regions.

[0030] Ultrasound video sequences are temporal streams composed of consecutive frames, which can be viewed as a three-dimensional data volume formed by the width and height of each frame. Directly inputting these into a model makes it difficult to efficiently capture local spatiotemporal relationships. Therefore, this embodiment requires splitting the complete video sequence into several non-overlapping three-dimensional spatiotemporal cubes. Each three-dimensional spatiotemporal cube contains both continuous frame information within a certain time period and spatial pixel information of a specific region within the corresponding frame, allowing the model to learn spatiotemporal relationships within a small range. Secondly, a feature vector sequence with 3D positional information is needed for each three-dimensional spatiotemporal cube. Each feature vector retains the features of the corresponding three-dimensional spatiotemporal cube and carries its spatiotemporal position information in the original video, ensuring that the model can accurately learn the dynamic evolution of anatomical structures at different locations over time.

[0031] For example, the preprocessed video sequence Cut into Three non-overlapping 3D spatiotemporal cubes (3DPatches), each cube having a size of [missing information]. .

[0032] In one implementation, the step of generating a feature vector sequence with 3D positional information based on the three-dimensional spatiotemporal cube includes: The three-dimensional spatiotemporal cube is flattened and mapped to a feature vector sequence through a linear projection layer, and then a learnable 3D position code is superimposed to obtain the feature vector sequence with 3D position information.

[0033] Specifically, the method for obtaining the feature vector sequence with 3D positional information is as follows: flatten the three-dimensional spatiotemporal cube and map it into a feature vector sequence through a linear projection layer. Superimposed learnable 3D positional encoding This serves as the input for the subsequent feature extraction module.

[0034] Step S200: Using a feature extraction module with a spatiotemporal pipeline masking mechanism, feature extraction is performed on the unmasked region of the feature vector sequence with 3D position information to obtain coded features.

[0035] Furthermore, the spatiotemporal pipeline masking mechanism includes: randomly sampling spatial coordinates in the first frame plane according to a preset masking rate, and continuously occluding the selected spatial coordinates on the time axis to form a spatiotemporal pipeline mask.

[0036] Specifically, at the input end, this embodiment does not employ the traditional random discrete masking strategy. Instead, it designs a spatiotemporal pipeline masking mechanism, which continuously occludes the same spatial location in the time dimension, forming an occlusion pipeline that spans spatiotemporal space. This blocks the shortcut of the model using the redundancy of pixels in adjacent frames for simple copying, enabling the model to perform deep inference based on the spatiotemporal evolution logic of the anatomical structure. The input to this step is the previously processed feature vector sequence with 3D positional information. The feature extraction module responsible for performing this step has the ability to capture spatiotemporally correlated features and can mine deep information in the data. The feature extraction module can flexibly select feature extractors adapted to spatiotemporal data, such as encoders, 3D-ResNet (convolutional neural network), and ConvLSTM (recurrent neural network). Through the spatiotemporal pipeline masking mechanism, the feature extraction module performs targeted feature extraction on the unoccluded areas in the feature vector sequence with 3D positional information, and finally outputs the encoded features (i.e., visible token features).

[0037] For example, to prevent inter-frame information leakage, a spatiotemporal pipeline mask matrix is ​​constructed. First, spatial sampling: in the first frame plane, at a high mask rate ( ρ ≈75%–90%) Random sampling spatial coordinates Secondly, there's the time extension: for the selected coordinates... Force it on the timeline The upper part maintains a continuous occlusion state, that is This forms a "hollow pipe" that spans time, namely the spacetime pipe mask matrix.

[0038] It should be noted that the spatiotemporal pipe masking mechanism in this embodiment is essentially continuous occlusion in the temporal dimension, and the spatial occlusion shape can be flexibly adapted. The term "pipe" does not restrict the spatial shape to be a regular shape. In other words, the spatial form of the occlusion is not limited to a fixed tubular shape and can be flexibly adjusted according to the anatomical characteristics of the ultrasound image: it can use regular geometric shapes such as cylinders and ellipses, or it can use superpixel segmentation technology to generate irregular shapes that fit the contours of the anatomical structure.

[0039] In one implementation, the spatiotemporal pipeline mask takes the form of a continuous mask whose position changes dynamically over time.

[0040] Specifically, the key to spatiotemporal pipeline masking lies in continuous occlusion along the time dimension, while continuous masks with dynamically changing mask positions over time represent a flexible optimization based on this. Unlike static pipeline occlusion, the occlusion area of ​​this dynamic mask moves along the trajectory of the moving object, forming a "curved pipeline." This spatiotemporal pipeline masking can be implemented using optical flow-guided dynamic masks. For example, in cardiac ultrasound videos, the optical flow field of the cardiac tissue can be pre-calculated, allowing the mask area to move synchronously with the heart's beating trajectory, always accurately occluding the same anatomical tissue. This conforms to the dynamic motion characteristics of anatomical structures in ultrasound images, avoiding the occlusion misalignment problems that may occur with static masks.

[0041] In one implementation, the spatiotemporal pipeline mask can employ a progressive mechanism, gradually increasing the mask rate as the training rounds increase. This mask rate adjustment strategy during training allows the masking mechanism to better adapt to the characteristics of dynamic ultrasound images.

[0042] Furthermore, the spatiotemporal pipeline masking strategy in this embodiment can also employ a progressive mechanism. During model training, the masking rate gradually increases with each training epoch (e.g., from 50% to 90%): A lower masking rate in the early stages of training allows the model to access more effective features, reducing learning difficulty and helping the model converge quickly; as training progresses, gradually increasing the masking rate continuously increases the task difficulty, enabling the model to delve deeper into the core spatiotemporal features of anatomical structures, rather than relying on surface-visible information, ultimately improving the robustness and generalization ability of the model's feature extraction. This masking rate adjustment strategy during training allows the masking mechanism to better adapt to the characteristics of dynamic ultrasound images.

[0043] Step S300: The occluded area is repaired by the repair module according to the encoded features, and the structural consistency loss and motion consistency loss are calculated.

[0044] The repair module completes the occluded region based on encoded features. Its processing logic utilizes contextual information to infer and recover missing content, encompassing both reconstruction and generation / inpainting paradigms. The specific implementation of the repair module is flexible and diverse, employing either a decoder or models with strong generative capabilities, such as Generative Adversarial Networks (GANs) or Diffusion Models. In practice, the repair module receives the encoded features output from the previous feature extraction module. If a decoder is used, it directly performs structured reconstruction based on the contextual associations of the encoded features, completing the content of the occluded region. If a GAN or Diffusion Model is used, the masked ultrasound video block is input into the model, and the generative network mines the structure and motion patterns in the encoded features to generate occluded region content that closely matches the real scene. Simultaneously, a discriminator judges whether the repaired video conforms to the actual ultrasound anatomical structure and motion patterns, further improving the realism and reasonableness of the repair result. Regardless of the implementation method, the goal of the restoration module is to accurately complete the areas obscured by the spatiotemporal pipeline mask by utilizing effective information from the encoded features, ensuring that the restored ultrasound image is visually coherent and structurally complete. After restoration, the module further calculates two types of key losses: structural consistency loss, used to constrain the anatomical structure of the restored area to be consistent with the original unobscured area, avoiding structural distortion; and motion consistency loss, used to ensure that the motion trend of the restored area matches the dynamic evolution of the overall video, preventing motion misalignment. These two types of losses together constitute the evaluation criteria for restoration effectiveness, ensuring the accuracy of the restored content and allowing the model to learn more robust spatiotemporal features of ultrasound images.

[0045] In one implementation, the feature extraction module is an encoder; the repair module is a dual-stream decoder, including a first decoder and a second decoder; the steps of repairing the occluded region based on the encoded features and calculating structural consistency loss and motion consistency loss through the repair module include: The first decoder generates a prediction patch based on the encoded features. The prediction patch and the corresponding three-dimensional spatiotemporal cube are transformed into their respective domains. The difference between the transformation domain results of the two after high-frequency component suppression is calculated by combining a preset filter, so as to obtain the structural consistency loss. The second decoder generates predicted dynamic change information based on the encoded features, and calculates the motion consistency loss based on the predicted dynamic change information and the corresponding reference dynamic change information; wherein, when the predicted dynamic change information is a predicted optical flow field, the reference dynamic change information is a dense optical flow field; when the predicted dynamic change information is a predicted frame difference map, the reference dynamic change information is a true frame difference map; when the predicted dynamic change information is a predicted motion vector, the reference dynamic change information is a true motion vector.

[0046] Specifically, this embodiment employs a self-supervised learning framework built using spatiotemporal tube masking and two-stream decoupled reconstruction, enabling noise-resistant feature learning from massive amounts of unlabeled ultrasound video data. The two-stream decoder, while repairing occluded areas of the ultrasound video, achieves the dual goals of noise resistance and motion understanding through dual loss constraints (noise-resistant structure reconstruction and motion consistency prediction). First, the feature extraction module, acting as an encoder, performs deep feature extraction on the feature vector sequence with 3D positional information; the output encoded features become the foundational data for the two-stream decoder's repair and prediction. The repair module, acting as a two-stream decoder (first decoder and second decoder), follows the two-stream decoupled reconstruction strategy: one stream focuses on texture structure reconstruction to achieve noise resistance; the other stream focuses on motion consistency prediction to capture dynamic features.

[0047] The first decoder's task is to generate predicted patches for the occluded region and calculate structural consistency loss, such as frequency domain noise-resistant reconstruction loss, to address the problem of ultrasonic speckle noise interference. Specifically, after receiving the encoded features output by the encoder, the first decoder generates predicted patches based on the contextual structure information within those features. Subsequently, it performs transform-domain transformations on both the predicted patch and the corresponding original 3D spatiotemporal cube. These transform-domain transformations can employ various methods, such as Fast Fourier Transform (FFT), Wavelet Transform (DWT), Discrete Cosine Transform (DCT), or Laplace Pyramid Decomposition, all of which separate high- and low-frequency information in the transform domain. For example, wavelet decomposition can be used to divide the image into a low-frequency sub-band (LL, corresponding to clean anatomical structures) and a high-frequency sub-band (LH / HL / HH, corresponding to speckle noise). After the transform-domain transformation, a pre-defined low-pass filter is used to suppress the high-frequency components in both transform-domain results. Only the difference in the low-frequency portion after high-frequency suppression is calculated, thus obtaining the structural consistency loss. This design, by shifting the loss calculation to the frequency domain, forces the model to automatically ignore high-frequency speckle noise and focus solely on learning clean anatomical structural features, achieving the goal of denoising and reconstruction.

[0048] The second decoder's task is to generate predicted dynamic change information and calculate motion consistency loss, forcing the model to capture the dynamic functional characteristics of tissues and organs (such as contraction and relaxation). The second decoder also predicts based on the encoded features output by the encoder, and the predicted dynamic change information can take various forms. These can be computationally intensive dense optical flow fields, frame difference maps (i.e., predicting the difference between I_{t+1}-I_t) as a simplified form of optical flow, feature difference maps in feature space, or even the movement trajectories of significant feature points on the time axis in ultrasound images (i.e., predicting only the displacement of sparse points, not the dense optical flow of the entire image; essentially, it's still a motion consistency constraint, without being limited to dense or sparse forms). The corresponding reference dynamic change information is adapted according to the prediction type: if predicting an optical flow field (dense or sparse), the reference dynamic change information is the dense optical flow field calculated offline (as a pseudo-label); if predicting a frame difference map, the reference dynamic change information is the true frame difference map; if predicting a motion vector, the reference dynamic change information is the true motion vector. The motion consistency loss is obtained by calculating the difference between the predicted dynamic change information and the corresponding reference dynamic change information. Simultaneously, the introduction of global affine transformation constraints effectively decouples the probe's manual movement from the organ's physiological movement, enhancing the model's ability to understand tissue movement patterns while improving noise resistance. Ultimately, this improves the accuracy of downstream dynamic ultrasound diagnostic tasks.

[0049] For example, after the coded features are extracted by the feature extraction module, such as... Figure 3 As shown, the following loss function is optimized using two parallel decoders: Stream 1: Frequency Domain Noise Reconstruction Loss For prediction patches and original tiles Perform a Fast Fourier Transform (FFT) and apply a low-pass filter. : ; in, Represents 2D-FFT transformation. This represents the Hadamard product. This formula forces the model to ignore high-frequency speckle noise from a physical perspective.

[0050] Flow 2: Loss of Motion Consistency Generating dense streamer fields using traditional algorithms (such as TV-L1) As a pseudo-label, the optical flow predicted by the supervised model : ; This formula forces the model to learn the dynamic evolution characteristics of anatomical structures.

[0051] Step S400: Optimize the parameters of the ultrasound image processing model based on the structural consistency loss and the motion consistency loss; the structural consistency loss is calculated based on spatial domain filtering or frequency domain filtering; the ultrasound image processing model includes: the feature extraction module and the repair module.

[0052] Furthermore, the step of optimizing the parameters of the ultrasound image processing model based on the structural consistency loss and the motion consistency loss includes: The total loss is obtained by weighted summation of the structural consistency loss and the motion consistency loss. The parameters of the ultrasound image processing model are optimized based on the total loss.

[0053] Specifically, model parameter optimization employs a joint constraint of multiple losses, including structural consistency loss and motion consistency loss. The calculation of structural consistency loss can flexibly adapt to spatial or frequency domain filtering methods, aiming to achieve noise-resistant reconstruction. Whether calculating structural differences by directly suppressing noise in the spatial domain through filtering, or calculating loss for low-frequency anatomical structures by separating high and low-frequency information in the frequency domain, the model is forced to ignore ultrasound speckle noise and focus on learning pure tissue texture and morphological features. Motion consistency loss, on the other hand, constrains the model's accuracy in capturing dynamic changes. By comparing the model's predicted dynamic changes with corresponding reference dynamic changes, it ensures the model accurately grasps the physiological movement patterns of tissues and organs, such as contraction and relaxation. Finally, based on the total loss, the model iteratively adjusts all learnable parameters of the feature extraction and repair modules using a backpropagation algorithm. This allows the model to gradually develop a stable ability to extract pure anatomical structures and a deep understanding of physiological movement patterns through self-supervised learning on massive amounts of unlabeled ultrasound video data, ultimately improving the accuracy of downstream dynamic diagnostic tasks, such as cardiac function assessment and vascular dynamics monitoring.

[0054] For example, the key steps in the model training and optimization process are as follows: 1. Joint Optimization Objective: To jointly optimize the total loss using a weighted summation method: ; in, , Balance coefficient (default) =1.0, =0.5), or dynamically adjust using an uncertainty-weighted strategy.

[0055] 2. Optimizer configuration: AdamW optimizer is used ( =0.9, =0.95), weight decay coefficient =0.05.

[0056] 3. Learning rate scheduling: Implement a Warm-up + Cosine Decay strategy. Current step number learning rate The calculation is as follows: ; in, Warm-up is 40 epochs.

[0057] 4. Training parameters: 500 Epochs, Batch Size=256.

[0058] In one approach, the total loss function can also be a joint loss function constructed from structural consistency loss, motion consistency loss, and a global affine transformation constraint term. Introducing a global affine transformation constraint term into the loss function can separate global background motion from local tissue deformation, allowing the model to simultaneously possess both noise-resistant structural reconstruction capabilities and accurate motion understanding capabilities, providing robust feature support for downstream dynamic diagnostic tasks.

[0059] The advantages of this invention are: 1. The frequency-domain noise-resistant reconstruction mechanism overcomes the bottleneck of high-noise pre-training. Compared with the existing mask autoencoder method of calculating mean square error (MSE) in pixel space, this invention innovatively transfers the reconstruction loss calculation to the frequency domain space, combined with low-pass filtering, to avoid the model's fitting to high-frequency speckle noise from a physical perspective. This allows the model to learn clean anatomical features directly from unwashed noisy videos without relying on any noise reduction preprocessing or using noise reduction algorithms to destroy image details, significantly improving the robustness of features and successfully overcoming the limitation of difficult self-supervised pre-training of ultrasound images due to high noise.

[0060] 2. The spatiotemporal pipeline masking strategy enhances the capture of long-term spatiotemporal features. Unlike the random scattered point masks commonly used in existing video self-supervised methods, this invention designs a time-axis-locked pipeline mask that continuously occludes tiles at the same spatial location along the time dimension. This strategy effectively blocks the shortcut learning of pixel copying based on the high similarity between adjacent frames, forcing the model to infer missing content based on the anatomical structure of surrounding tissues and long-term motion trends (such as the cardiac cycle), thereby learning high-order spatiotemporal semantic features. This solves the problem of models only looking at single frames and not the process, significantly improving the recognition accuracy of dynamic lesions (such as abnormal ventricular wall motion).

[0061] 3. The texture-motion dual-stream decoupled reconstruction architecture enhances dynamic feature capture capabilities. Unlike existing basic models that typically only perform a single image appearance reconstruction task, this invention constructs a parallel dual-stream decoder that simultaneously performs frequency domain texture reconstruction and optical flow motion prediction tasks. This architecture solves the problem that a single visual task is insufficient to capture dynamic functional features of ultrasound. Even when the image is blurred due to acoustic shadowing, the model can still understand the movement patterns of tissues (such as contraction / diffuse) through the optical flow prediction branch, significantly improving the sensitivity to dynamic lesions (such as abnormal wall motion). This allows the model to capture anatomical structural features while possessing a deep understanding of tissue movement patterns.

[0062] 4. Significantly reduces development costs and exhibits outstanding generalization ability. This invention can fully utilize millions of hours of unlabeled video recordings accumulated in hospital PACS systems for self-evolution, eliminating the need for extensive manual annotation data and greatly reducing model development costs. The trained base model possesses strong cross-device and cross-department generalization capabilities, providing universal underlying technical support for the intelligentization and standardization of ultrasound diagnosis.

[0063] Based on the above embodiments, the present invention also provides a noise-resistant spatiotemporally self-supervised ultrasound image feature extraction method, such as... Figure 4 As shown, the method includes: Step S500: Obtain the ultrasound video sequence; Step S600: Input the ultrasound video sequence into the trained ultrasound image processing model to obtain the corresponding encoded feature information; the trained ultrasound image processing model includes a feature extraction module and is trained using any of the noise-resistant spatiotemporally self-supervised ultrasound image processing model pre-training methods described above.

[0064] Specifically, during inference and deployment, the repair module needs to be removed during model export, retaining only the feature extraction module as a general feature extractor. Taking an autoencoder model as an example, after training, the dual-stream decoder is removed, and only the trained encoder is exported as a general feature extractor. In practical applications, to adapt to downstream tasks, the encoder parameters can be frozen, a lightweight task head (Linear Head or UpperNet Head) can be connected, and fine-tuning can be performed using a small amount of labeled data to adapt to tasks such as section classification and lesion segmentation.

[0065] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 5As shown, the terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a noise-resistant spatiotemporally self-supervised ultrasound image processing model pre-training method, or a noise-resistant spatiotemporally self-supervised ultrasound image feature extraction method. The display screen can be a liquid crystal display (LCD) or an e-ink display.

[0066] Those skilled in the art will understand that Figure 5 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0067] In one implementation, the terminal's memory stores one or more programs, and these programs are configured to be executed by one or more processors. The programs include instructions for a noise-resistant spatiotemporally self-supervised ultrasound image processing model pre-training method or a noise-resistant spatiotemporally self-supervised ultrasound image feature extraction method.

[0068] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0069] In summary, this invention discloses a noise-resistant spatiotemporally self-supervised pre-training method for ultrasound image processing models. The method includes: dividing an ultrasound video sequence into several three-dimensional spatiotemporal cubes; generating a feature vector sequence with 3D positional information based on the three-dimensional spatiotemporal cubes; extracting features from the unoccluded regions of the feature vector sequence with 3D positional information using a feature extraction module with a spatiotemporal pipeline masking mechanism to obtain coded features; repairing the occluded regions using a repair module based on the coded features, and calculating structural consistency loss and motion consistency loss; and optimizing the parameters of the ultrasound image processing model based on the structural consistency loss and motion consistency loss. This invention can solve the problem of difficult self-supervised pre-training of ultrasound images due to high noise.

[0070] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for pre-training a noise-resistant spatiotemporally self-supervised ultrasound image processing model, characterized in that, The method includes: The ultrasound video sequence is divided into several three-dimensional spatiotemporal cubes, and a feature vector sequence with 3D position information is generated based on the three-dimensional spatiotemporal cubes. The feature extraction module with a spatiotemporal pipeline masking mechanism is used to extract features from the unmasked region of the feature vector sequence with 3D position information to obtain coded features. The repair module repairs the occluded area based on the encoded features and calculates the structural consistency loss and motion consistency loss. The ultrasonic image processing model is optimized based on the structural consistency loss and the motion consistency loss; the structural consistency loss is calculated based on spatial domain filtering or frequency domain filtering; the ultrasonic image processing model includes the feature extraction module and the repair module.

2. The method for pre-training a noise-resistant spatiotemporally self-supervised ultrasound image processing model according to claim 1, characterized in that, Before the step of dividing the ultrasound video sequence into several three-dimensional spatiotemporal cubes, the following steps are also included: Obtain the raw ultrasound video sequence; The ultrasound images in the original ultrasound video sequence are preprocessed to remove background and text artifacts outside the fan-shaped and / or cone-shaped scanning areas, and to extract the clean anatomical regions, thus obtaining the ultrasound video sequence composed of clean ultrasound images.

3. The method for pre-training a noise-resistant spatiotemporally self-supervised ultrasound image processing model according to claim 1, characterized in that, The three-dimensional spacetime cubes are non-overlapping.

4. The method for pre-training a noise-resistant spatiotemporally self-supervised ultrasound image processing model according to claim 1, characterized in that, The steps for generating a feature vector sequence with 3D position information based on the three-dimensional spatiotemporal cube include: The three-dimensional spatiotemporal cube is flattened and mapped to a feature vector sequence through a linear projection layer, and then a learnable 3D position code is superimposed to obtain the feature vector sequence with 3D position information.

5. The method for pre-training a noise-resistant spatiotemporally self-supervised ultrasound image processing model according to claim 1, characterized in that, The spatiotemporal pipeline masking mechanism includes: randomly sampling spatial coordinates in the first frame plane according to a preset masking rate, and continuously occluding the selected spatial coordinates on the time axis to form a spatiotemporal pipeline mask.

6. The method for pre-training a noise-resistant spatiotemporally self-supervised ultrasound image processing model according to claim 5, characterized in that, The spatiotemporal pipeline mask can take the form of a continuous mask whose position changes dynamically over time.

7. The method for pre-training a noise-resistant spatiotemporally self-supervised ultrasound image processing model according to claim 1, characterized in that, The feature extraction module is an encoder; The repair module is a dual-stream decoder, including a first decoder and a second decoder. The steps of repairing the occluded region based on the encoded features using the repair module, and calculating structural consistency loss and motion consistency loss, include: The first decoder generates a prediction patch based on the encoded features. The prediction patch and the corresponding three-dimensional spatiotemporal cube are transformed into their respective domains. The difference between the transformation domain results of the two after high-frequency component suppression is calculated by combining a preset filter, so as to obtain the structural consistency loss. The second decoder generates predicted dynamic change information based on the encoded features, and calculates the motion consistency loss based on the predicted dynamic change information and the corresponding reference dynamic change information; wherein, when the predicted dynamic change information is a predicted optical flow field, the reference dynamic change information is a dense optical flow field; when the predicted dynamic change information is a predicted frame difference map, the reference dynamic change information is a true frame difference map; when the predicted dynamic change information is a predicted motion vector, the reference dynamic change information is a true motion vector.

8. The method for pre-training a noise-resistant spatiotemporally self-supervised ultrasound image processing model according to claim 1, characterized in that, The steps for optimizing the parameters of the ultrasound image processing model based on the structural consistency loss and the motion consistency loss include: The total loss is obtained by weighted summation of the structural consistency loss and the motion consistency loss. The parameters of the ultrasound image processing model are optimized based on the total loss.

9. A noise-resistant spatiotemporally self-supervised method for extracting ultrasonic image features, characterized in that, The method includes: Acquire ultrasound video sequences; The ultrasound video sequence is input into a trained ultrasound image processing model to obtain the corresponding encoded feature information; the trained ultrasound image processing model includes a feature extraction module and is trained using the noise-resistant spatiotemporal self-supervised ultrasound image processing model pre-training method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a plurality of instructions thereon, characterized in that, The instructions are applicable to be loaded and executed by a processor to implement the steps of the noise-resistant spatiotemporal self-supervised ultrasound image processing model pre-training method as described in any one of claims 1 to 8, or the steps of the noise-resistant spatiotemporal self-supervised ultrasound image feature extraction method as described in claim 9.

Images