Multi-modal ultrasonic data fusion method and system

Abstract

The invention relates to the technical field of data processing, in particular to a multi-modal ultrasonic data fusion method and system, and the method comprises the following steps: dividing time-space synchronization partitions based on a multi-modal ultrasonic image, recognizing the signal aggregation and interference characteristics of each region, dynamically adjusting the weight of each modal according to an index, and calculating the gray scale change trend of the fused region. And adjusting a fusion sequence, analyzing inter-frame signal-noise fluctuation, correcting partition weights, and outputting an inter-frame fusion consistency index. According to the method, the signal distribution state is decomposed region by region, the stability and interference characteristics of signals in each spatial partition are independently judged, the fusion proportion of modal information is dynamically distributed according to partition expression, and in the image splicing and continuous fusion process, the signal transmission path and the region expression sequence are automatically optimized; the continuity of the structure boundary and the tissue texture in the fusion output is enhanced, and the regional signal change in the time sequence is smoothed.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for multimodal ultrasound data fusion. Background Technology

[0002] Data processing encompasses the entire process of collecting, storing, organizing, analyzing, and applying various types of raw data. This includes digital representation of data, multi-source data fusion, algorithm modeling and analysis, and visualization and intelligent output. It is widely applied in various scenarios such as medicine, biology, transportation, and industry. Traditional multimodal ultrasound data fusion methods involve spatially registering and temporally synchronizing data from different ultrasound modalities, such as B-mode ultrasound, color Doppler ultrasound, and elastography, through preprocessing techniques. Then, rule-based algorithms or weighted methods are used to achieve information fusion. Strategies such as gray-scale averaging, maximum value selection, and linear weighted fusion are often used to overlay images from different modalities to enhance the comprehensive recognition of tissue structures.

[0003] Existing technologies employ a unified fusion rule after spatial and temporal registration, but lack detailed analysis of signal performance differences in local areas. This can easily lead to the failure to effectively identify noise and interference information in local areas of different modal signals. As a result, problems such as loss of structural details, discontinuity of boundary area information, and local noise diffusion often occur in the fusion results. Furthermore, the failure to effectively track and adjust the signal change trend between consecutive frames results in abrupt changes and insufficient consistency in the fused image over time, affecting the stable interpretation of medical image data. Summary of the Invention

[0004] To address the technical problems existing in the prior art, embodiments of the present invention provide a multimodal ultrasound data fusion method and system. The technical solution is as follows: On the one hand, a multimodal ultrasound data fusion method is provided, including the following steps: S1: Based on multimodal ultrasound image data, analyze the data stream temporal sequence and modal attributes, compare spatial distribution, determine synchronization response and adjust data arrangement temporal sequence, parse pixel spatial coordinates, filter spatial partitions, and obtain a set of spatiotemporal synchronization partitions; S2: Based on the spatiotemporal synchronization partition set, compare the signal aggregation characteristics with the changes in spurious signals, determine the signal distribution status, calculate the signal and interference ratios, identify feature data, and obtain the partition signal stability index. S3: Based on the partition signal stability index, compare signal performance, determine interference propagation characteristics, adjust mode fusion weights, screen key modes of signal aggregation and adjust their participation ratios, and then correct the overall fusion ratio to obtain mode partition weight parameters. S4: Based on the modal partitioning weight parameters, analyze the weight distribution, calculate the grayscale signal contribution, compare the grayscale trend, determine the signal stability, and obtain the spatial continuous signal-to-noise characteristics; S5: Based on the spatial continuous signal-to-noise characteristics, analyze the signal-to-noise ratio changes, compare the stability characteristics of adjacent frame partition signals, determine abnormal fluctuations, adjust the current frame partition fusion weights, splice the region fusion results, and obtain the inter-frame fusion consistency index.

[0005] On the other hand, the spatiotemporal synchronization partition set includes spatial partition labels, partition corresponding time identifiers, and partition pixel positions. The partition signal stability index includes regional structural clarity, signal uniformity, and noise ratio. The modal partition weight parameters include modal partition weights, signal saliency coefficients, and interference sensitivity coefficients. The spatial continuous signal-to-noise characteristics include fused signal-to-noise distribution, regional coherence indexes, and boundary continuity parameters. The inter-frame fusion consistency index includes inter-frame structural consistency, signal-to-noise trend changes, and continuous frame fusion level.

[0006] On the other hand, the specific steps for obtaining the spatiotemporal synchronization partition set are as follows: S101: Based on multimodal ultrasound image data, analyze the intra-frame time stamps and modal information, determine the time synchronization between image frames of each modality, compare the correspondence between modalities at the same time point, filter the combination of image frames with time consistency, and obtain the time-series correlation sequence. S102: Based on the time-series correlation sequence, compare the pixel layout of the image frame combination in the unified coordinate domain, determine the consistency of pixel distribution under each mode, adjust the spatial position correspondence of the image frames, analyze the acquisition order of the image frames with offset, correct the data arrangement method, and obtain the spatial alignment sequence. S103: Based on the spatial alignment sequence, the region is divided according to the pixel layout, the temporal and spatial matching relationship of the multimodal image corresponding to each region is calculated, the synchronization processing conditions of the region are determined, and a set of spatiotemporal synchronization partitions is obtained.

[0007] On the other hand, the specific steps for obtaining the partition signal stability index are as follows: S201: Based on the spatiotemporal synchronization partition set, analyze the grayscale matrix of each region, judge the difference in grayscale distribution by statistically analyzing the concentration of pixel grayscale within the partition, compare the regularity and degree of change of pixel arrangement, screen the signal features of regions with local clustering phenomena, aggregate the grayscale distribution state, and obtain the clustering characteristic parameter group. S202: Based on the clustering characteristic parameter set, compare the gray-scale structure of each region, analyze the continuity and edge changes of the gray-scale distribution of the partition, determine whether the signal of each region shows continuous diffusion or structural ambiguity, optimize the clustering and summarization of the gray-scale characteristics of the partition, and obtain the distribution pattern label set. S203: Based on the distribution pattern label set, calculate the pixel gray intensity of the partition and its neighborhood difference, analyze the proportion of signal and interference in the gray distribution, determine the persistence of regional signal aggregation and distribution state, and obtain the partition signal stability index.

[0008] On the other hand, the steps for obtaining the modal partitioning weight parameters are as follows: S301: Based on the partitioned signal stability index, analyze the signal aggregation and interference diffusion performance of each mode in the same area, compare the distribution pattern and consistency of pixel grayscale in the area, determine whether the signal distribution of each mode in the same area is discrete or interference diffusion occurs, identify the mode types with different spatial signal performance, and obtain the interference feature type set. S302: Based on the set of interference feature types, analyze the differences in spatial features and grayscale coherence of each mode, determine the mode labels with signal aggregation, optimize the mode labels with signal aggregation, and obtain the dominant response mode group; S303: Based on the dominant response mode group, adjust the weight distribution of the corresponding mode in the partition, calculate the contribution ratio of each mode signal, optimize the participation ratio of interference mode and dominant mode, uniformly correct the regional fusion parameter configuration, and obtain the mode partition weight parameters.

[0009] On the other hand, the specific steps for obtaining the spatial continuous signal-to-noise features are as follows: S401: Based on the modal partition weight parameters, analyze the multimodal weight distribution of each image partition, compare the fusion participation ratio of each modality in the region, determine the difference in modal weight configuration within the partition, calculate the partition distribution change of each modal weight in the whole image, and obtain the region weight mapping information; S402: Based on the region weight mapping information, calculate the weighted gray signal contribution of each modality in the corresponding partition, compare the continuous changes in gray distribution in each region of the fused image, determine the integrity of gray transition between partitions, identify the partition fusion order where gray distribution changes, and obtain the partition fusion time sequence. S403: Based on the partition fusion time sequence, adjust the fusion output order of the partitions, perform grayscale consistency analysis on the fusion boundary of each partition according to the modal weight configuration, determine the signal-to-noise connection status between adjacent partitions, and obtain spatial continuous signal-to-noise features.

[0010] On the other hand, the steps for obtaining the inter-frame fusion consistency index are as follows: S501: Based on the spatial continuous signal-to-noise characteristics, analyze the changing trend of the signal-to-noise ratio of each frame partition during continuous multi-frame imaging, determine the fluctuation of the signal distribution of the same spatial partition in continuous frames on the time axis, compare the continuity and consistency of the partition signals of adjacent frames, filter the partition positions with signal-to-noise changes, and obtain the frame order fluctuation interval set. S502: Based on the frame order fluctuation interval set, perform fusion weight adjustment on the fluctuation partition for the current frame, analyze the signal change direction of the current frame and the signal structure characteristics of the previous frame, determine whether the partition fusion configuration is consistent with the signal trend, optimize the fusion ratio of each mode in the partition, and obtain frame-level fusion ratio data. S503: Based on the frame-level fusion matching data, spatially stitch the fusion image results of all current frame partitions, analyze the boundary grayscale continuity between adjacent partitions, compare the structural coherence of the same partition between consecutive frames, and obtain the inter-frame fusion consistency index.

[0011] On the other hand, the data stream timing refers to the order in which the original signals are acquired over time during the multimodal ultrasound image acquisition process, and the spatial distribution refers to the physical correspondence of each pixel in two-dimensional or three-dimensional space for different modal images acquired at the same time.

[0012] On the other hand, the signal aggregation characteristic refers to whether the pixel grayscale value or signal intensity is concentrated within a certain partition, and the stray signal change refers to the phenomenon that the grayscale or signal intensity is randomly distributed, uneven, and has a lot of noise components within the partition.

[0013] On the other hand, a multimodal ultrasound data fusion system is provided, which is applied to a multimodal ultrasound data fusion method, including: The partition extraction module is based on multimodal ultrasound image data. It analyzes the temporal and modal attributes of the data stream, compares the spatial distribution, judges the synchronization response and adjusts the acquisition order, calculates the pixel arrangement, filters spatial partitions, and obtains a set of spatiotemporal synchronization partitions. The signal feature discrimination module, based on the spatiotemporal synchronization partition set, compares the signal aggregation characteristics with the changes in spurious signals, determines the signal distribution status, calculates the proportion of signal and interference, identifies feature data, and obtains the partition signal stability index. The weight adjustment module compares signal performance based on the partition signal stability index, judges interference propagation characteristics, adjusts mode fusion weights, filters key modes of signal aggregation and adjusts their participation ratios, and then corrects the overall fusion ratio to obtain mode partition weight parameters. Based on the modal partitioning weight parameters, the fusion characteristic analysis module analyzes the weight distribution, calculates the grayscale signal contribution, compares the grayscale trend, judges the signal stability, adjusts the region fusion order, and obtains spatial continuous signal-to-noise characteristics. Based on the spatial continuous signal-to-noise characteristics, the consistency evaluation module analyzes the signal-to-noise ratio changes, compares the stability characteristics of adjacent frame partition signals, judges abnormal fluctuations, adjusts the current frame partition fusion weights, splices the region fusion results, and obtains the inter-frame fusion consistency index.

[0014] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following: By decomposing the signal distribution state region by region, the stability and interference characteristics of the signal in each spatial partition are independently determined. The fusion weight of each modality information is dynamically allocated according to the partition performance. During image stitching and continuous fusion, the signal transmission path and regional expression order are automatically optimized. The continuity of structural boundaries and tissue texture in the fusion output is enhanced, and the regional signal changes in the time series are smoothed. Overall, the fusion results are improved in terms of structural integrity, spatial consistency and inter-frame stability. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of the main steps of the present invention; Figure 2 This is a flowchart of steps S1 of the present invention; Figure 3 This is a flowchart of steps S2 of the present invention; Figure 4 This is a flowchart of steps S3 of the present invention; Figure 5 This is a flowchart of step S4 of the present invention; Figure 6 This is a flowchart of steps S5 of the present invention; Figure 7 This is a system block diagram of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0019] This invention provides a method for multimodal ultrasound data fusion, such as... Figure 1 As shown, it includes the following steps: S1: Based on multimodal ultrasound image data, analyze the temporal and modal attributes of continuous imaging data stream, compare the spatial distribution of each modal image at the same time point, determine whether the signal source responds synchronously, adjust the acquisition order to reduce signal offset, calculate the arrangement of pixels in each frame of image, filter spatial partitions that meet the conditions for synchronous processing, and obtain a set of spatiotemporal synchronous partitions. S2: Based on the spatiotemporal synchronization partition set, compare the signal aggregation characteristics and spurious signal changes in the grayscale distribution of each region, determine whether the signal distribution shows local aggregation or uniform diffusion, calculate the signal and interference ratios corresponding to each pixel in the region, identify the feature data reflecting the region's focusing stability, organize the region's feature descriptions, and obtain the partition signal stability index. S3: Based on the partition signal stability index, compare the aggregation and interference performance of different modes of signals in the same area, determine whether there are characteristics of non-aggregation and interference spread in the partition, adjust the weight allocation of the mode participating in the fusion, at the same time screen the key modes of signal aggregation, adjust the participation ratio, and simultaneously correct the overall fusion ratio of the area to obtain the modal partition weight parameters. S4: Based on the modal partition weight parameters, analyze the weight distribution of each partition, calculate the gray signal contribution of different modes in each region, compare the gray change trend of each region after fusion, judge the signal stability in the partition, adjust the region fusion order, ensure that the whole image maintains spatial continuity, and obtain spatial continuous signal-noise characteristics. S5: Based on the spatial continuous signal-to-noise characteristics, analyze the changes in signal-to-noise ratio in each region during the continuous imaging process, compare the signal stability characteristics of adjacent frame partitions, determine whether there are abnormal signal-to-noise fluctuations between regions, adjust the fusion weight of the current frame partition to make the signal trend conform to the previous state, and re-stitch all region fusion results into a whole frame image to obtain the inter-frame fusion consistency index.

[0020] The spatiotemporal synchronization partition set includes spatial partition labels, partition corresponding time identifiers, and partition pixel positions. The partition signal stability indicators include regional structural clarity, signal uniformity, and noise ratio. The modal partition weight parameters include modal partition weights, signal saliency coefficients, and interference sensitivity coefficients. The spatial continuous signal-to-noise characteristics include fused signal-to-noise distribution, regional coherence indicators, and boundary continuity parameters. The inter-frame fusion consistency indicators include inter-frame structural consistency, signal-to-noise trend changes, and continuous frame fusion levels.

[0021] In S1, data stream timing refers to the sequential order in which the original signals are acquired over time during multimodal ultrasound image acquisition, such as the time stamp of each frame acquired by probes like B-mode ultrasound, color Doppler ultrasound, and elastography in the same operation; modal attributes refer to the data types and characteristics corresponding to different ultrasound imaging methods (e.g., B-mode ultrasound uses single-channel grayscale images, color Doppler ultrasound uses multi-channel color coding, and elastography has elastic distribution information); spatial distribution refers to the physical correspondence of pixels in two-dimensional or three-dimensional space for different modal images acquired at the same time; and synchronous response refers to the timing of signal acquisition by different modal probes at the same anatomical location and at the same time. Whether the data acquisition process achieves spatial and temporal alignment to avoid misalignment; signal offset refers to the spatial or temporal misalignment or drift of multimodal signals caused by factors such as probe physical placement and asynchronous electronic clocks; arrangement refers to the layout order of an ultrasound image frame in the pixel matrix, involving the actual arrangement method of horizontal and vertical pixel coordinates; synchronous processing conditions refer to the requirement that only multimodal images that meet the conditions of being at the same time and location can participate in fusion, and that spatial and temporal correspondence must be satisfied; spatial partitioning refers to dividing the entire image into multiple relatively independent small regions or blocks (such as sliding windows or grid blocks) to facilitate local feature analysis and subsequent fusion.

[0022] In S2, each region refers to the spatial partitions / windows already defined in S1, i.e., several small blocks divided from the entire ultrasound image. Each small block is analyzed separately. Signal aggregation characteristics refer to whether the pixel gray values ​​or signal intensity within a certain partition show a clear concentrated distribution (such as areas with clear boundaries, structures, and textures). Stray signal variation refers to the phenomenon of disordered, uneven distribution of gray values ​​or signal intensity and high noise components within a partition. Local aggregation refers to the discovery of a core concentrated distribution of signals within a partition, reflecting structural information (such as organ outlines and tissue interfaces). Uniform diffusion refers to the signal distribution within a partition without primary or secondary characteristics, showing a state of stray or noise dominance. Signal-to-interference ratio refers to the relative proportion between the true structural signal components and noise or interference components within each partition. Feature data refers to the quantifiable descriptions extracted from the analysis, used to reflect the signal quality, structural stability, or noise dominance of the region.

[0023] In S3, aggregation and interference performance refer to the comparison of signal concentration and noise interference intensity in different modes of each partition (i.e., which modes have a large amount of information and which have a lot of noise in this area). The characteristic of non-aggregation and interference diffusion refers to the situation where a certain mode has no clear signal structure in a certain area and the noise disturbance component is significantly enhanced. The key mode of signal aggregation refers to the mode with the highest signal concentration in the partition that can highlight the structural information (e.g., B-ultrasound performs better at the edge of certain areas). The participation ratio refers to the weight coefficient assigned to each mode when it is fused in the region, which determines the degree of contribution of the mode to the fusion result. The fusion ratio refers to the actual proportion of each mode's weighted participation in a region, which is dynamically adjusted according to the signal performance during the fusion process.

[0024] In S4, grayscale signal contribution refers to the actual influence of different modalities on the final grayscale value of the fused image within a region (the contribution after weighting). The regional grayscale change trend refers to the direction of change of the grayscale distribution of the region after fusion compared to the original modal image, such as contrast enhancement, texture detail enhancement, etc. The signal stability refers to whether the signal of each region after fusion exhibits continuity and coherence, reflecting the uniformity of the data after fusion and the level of noise control. The regional fusion order refers to the order or stitching method of the data of each region when synthesizing into the overall image in the regional stitching and fusion output stages, which is used to control the spatial continuity of the image.

[0025] In S5, the signal-to-noise ratio (SNR) change in each region refers to the characteristics of the SNR change of each spatial partition over time or frame order during the continuous acquisition of multiple frames of images. The signal stability characteristics refer to the degree of signal retention in the corresponding partition of adjacent frames, such as the amplitude of SNR change and structural coherence. Abnormal SNR fluctuations refer to a large fluctuation in the SNR of a certain region between frames, or a sudden abnormal change, reflecting the discontinuity and instability in the fusion process. The previous state refers to the fusion weight and signal distribution characteristics of the same partition in the previous frame or several previous frames, which is used as a dynamic reference for the fusion parameters of the current frame.

[0026] like Figure 2 As shown, the specific steps for obtaining the spatiotemporal synchronization partition set are as follows: S101: Based on multimodal ultrasound image data, analyze the intra-frame time stamps and modal information, determine the time synchronization between image frames of each modality, compare the correspondence between modalities at the same time point, filter the combination of image frames with time consistency, and obtain the time-series correlation sequence. Extract the time stamps and modality types from the image frames. For example, the time stamp for an ultrasound image is 10000ms, for a color Doppler ultrasound image it's 10040ms, and for an elastography image it's 10090ms. Assign modality types M1, M2, and M3 respectively. Sort all image frames along the time axis and generate a modality-time index table. Set the time synchronization threshold to 50 milliseconds. Compare the time difference between each pair of modal image frames. If the time difference is less than or equal to 50 milliseconds, the frames are considered synchronized. For example, if an ultrasound frame has a time of 10000ms and a color Doppler ultrasound frame has a time of 10040ms, and the time difference is 40ms (less than 50ms), it is included in the synchronized frame list. The time window is 10090ms, and the time difference is 90ms. If the time difference exceeds the threshold, the frame is excluded. Then, the same method is used to construct image frame combinations by sliding the image sequence. The step size is set to 40 milliseconds. The time window is slid forward to construct new image frame combination groups. For example, the first image combination is B001-C001, the second image combination is B002-C002, and so on. Each combination contains at least two modal image frames. When any modal image in a combination is missing, has an incorrect label, or exceeds the synchronization threshold, the combination is invalidated and not processed. The set of all image combination sequences is obtained and used as the time synchronization basis data for subsequent processing.

[0027] S102: Based on the temporal correlation sequence, compare the pixel layout of the image frame combination in the unified coordinate domain, determine the consistency of pixel distribution under each mode, adjust the spatial position correspondence of the image frames, analyze the acquisition order of the image frames with offset, correct the data arrangement method, and obtain the spatial alignment sequence. For the obtained time-synchronized image set, the next step is to process its spatial consistency by extracting image pixel size and resolution parameters. For example, if a set of image frames are all 512×512 pixels with a pixel pitch of 0.3 mm, all images are uniformly mapped to a 1024×1024 pixel reference coordinate domain. For smaller images, edge padding is used, for example, adding 256 rows of pixels at the top and bottom and 256 columns of pixels on the left and right. For larger images, a center region cropping method is used, retaining the middle 512×512 pixels for uniform matching. Then, the structural center position of each image in the unified coordinate system is calculated, and the centroid coordinates of the image edge regions or gray-level distribution center regions are extracted. Finally, the structures of each modal image are compared. The centroid offset value is calculated. If the horizontal or vertical offset between the centroids of two images is greater than or equal to 15 pixels, it is recorded as an offset image frame. For example, if the center of the B-ultrasound image is at (270, 280) and the center of the color Doppler ultrasound image is at (285, 275), the offsets are 15 pixels and 5 pixels respectively, which are considered to exceed the offset tolerance. The offset direction is recorded as right offset and vertical offset. At the same time, the position of the frame image is adjusted according to the offset. The image is subjected to overall matrix offset processing. The translation method is to move the image pixel by pixel according to the coordinate difference. After adjustment, the centroid is recalculated. If the positional error between the two images is confirmed to be less than 10 pixels after adjustment, the alignment is confirmed to be successful. This process is performed on all combined images in sequence until all combined image frames have completed spatial position correction and are included in the spatially aligned image sequence.

[0028] S103: Based on the spatial alignment sequence, the region is divided according to the pixel layout, the temporal and spatial matching relationship of the multimodal image corresponding to each region is calculated, the synchronization processing conditions of the region are determined, and the spatiotemporal synchronization partition set is obtained. Each spatially aligned image is divided into multiple local regions, each with a size of 32×32 pixels, resulting in a total image size of 1024×1024 pixels, which can be divided into 1024 regions, labeled R_1 to R_1024. Pixel block data from different modal images are extracted from each region, and it is determined whether they meet the synchronization processing conditions. First, the pixel distribution of image blocks in the region is analyzed, such as comparing edge sharpness, grayscale contrast, and texture intensity. Regions with a grayscale variance greater than 30 are identified as regions rich in structural information. Simultaneously, the acquisition time tag difference and spatial offset residual of each modal image frame within this region are calculated. If a certain region... If the time tag difference between any two modal images does not exceed 50 milliseconds and the spatial centroid offset is less than 15 pixels, then the region is marked as a region that can be synchronized. The maximum allowable value for the time tag difference is set to 50, and the maximum allowable value for the spatial offset residual is set to 15. Under this condition, all regions are traversed, and it is determined whether each one meets the time and space synchronization requirements. For example, in region 32, the time difference between the B-mode ultrasound frame and the color ultrasound frame is 35 milliseconds, and the centroid offset is 10 pixels. Therefore, it is considered to meet the synchronization conditions, and the region is retained. All regions that meet the requirements, along with their corresponding modal image frame indexes, time tags, and region pixel position numbers, are formed into a unified record set, and the output is a set of spatiotemporal synchronization partitions.

[0029] like Figure 3 As shown, the specific steps for obtaining the partition signal stability index are as follows: S201: Based on the spatiotemporal synchronous partition set, analyze the gray-level matrix of each region, judge the difference in gray-level distribution by statistically analyzing the concentration of pixel gray-level within the partition, compare the regularity and degree of change of pixel arrangement, screen the signal features of regions with local clustering phenomena, aggregate the gray-level distribution state, and obtain the clustering characteristic parameter group. Analyzing the grayscale matrix within each region, the grayscale distribution characteristics of each region are obtained by extracting the pixel values ​​and calculating the grayscale mean and standard deviation. For example, in a certain region, the mean grayscale value is 120 and the standard deviation is 10, indicating that the signal in that region is relatively concentrated. Next, by statistically analyzing the changes in grayscale values ​​within each region and comparing the fluctuation range of grayscale values ​​in that region, the concentration is determined. If the grayscale value changes little in a certain region, it indicates that the signal in that region is concentrated; conversely, if the changes are large, it indicates that the signal is relatively dispersed. Based on the changes, regions with obvious local clustering phenomena are selected. For example, in a certain region, the grayscale value fluctuates between 50 and 130, and the grayscale value of most pixels is concentrated around 100. This allows us to determine if the region exhibits local clustering characteristics. Next, we calculate the regularity of pixel arrangement within the region. By analyzing the uniformity and regularity of pixel value changes, if pixel values ​​in a certain region exhibit a regular arrangement, such as obvious edge or structural features, it indicates that the region has stronger clustering characteristics. If the grayscale value in a certain region changes relatively smoothly between pixel positions x=10 and x=50, while the variation is larger between y=100 and y=130, it indicates that local clustering has occurred in the region. After aggregating the statistical characteristics of the grayscale values, we obtain a clustering characteristic parameter set, which includes multiple data such as the mean grayscale value, standard deviation, variation range, and regularity within the region, providing basic data for subsequent regional signal characteristic analysis.

[0030] S202: Based on the clustering characteristic parameter set, compare the gray-scale structure of each region, analyze the continuity and edge changes of the gray-scale distribution of the partition, determine whether the signal of each region shows continuous diffusion or structural ambiguity, optimize the clustering and summarization of the gray-scale characteristics of the partition, and obtain the distribution pattern label set. By comparing the grayscale structure of each region and calculating the continuity of grayscale value distribution within the region, the trend of grayscale value changes within the region is observed. For example, in a certain region, the trend of grayscale value change is relatively gentle, and the grayscale value distribution is continuous without obvious jumps, indicating that the grayscale structure of this region is relatively stable. However, in some regions, the grayscale value changes drastically, and the edges are relatively blurred, indicating a discontinuous or blurred grayscale distribution. In this case, by analyzing the speed and direction of grayscale value changes, it can be determined whether the signal in this region has experienced continuous diffusion or structural blurring. Furthermore, the clustering and summarization of grayscale features in each region are optimized by using the pixel grayscale values ​​within the region... Features such as difference and mean are used to cluster similar gray-scale structural regions. These features are then used to classify regions, grouping those with continuous gray-scale distribution and clear structure into one category, and those with irregular and ambiguous distribution into another. For example, suppose in a group of regions, some regions have a mean gray-scale value of 130 and a variance of 5, showing a gentle trend, and are classified as "stable"; while other regions exhibit larger gray-scale fluctuations, with gray-scale values ​​ranging from 100 to 150 and higher variance, and are classified as "unstable". Labeling each region with a corresponding distribution pattern set helps to assess the regularity and trend of the region's gray-scale distribution as a whole.

[0031] S203: Based on the distribution pattern label set, calculate the pixel gray intensity of the partition and its neighborhood difference, analyze the proportion of signal and interference in the gray distribution, judge the persistence of regional signal aggregation and distribution state, and obtain the partition signal stability index. Calculate the pixel grayscale intensity within each region and statistically analyze the signal-to-noise ratio within each region. For example, for a region with grayscale values ​​ranging from 50 to 130, the signal portion's grayscale values ​​are mainly concentrated around 100, while the noise portion is mainly distributed between 50 and 70. The signal quality of the region can be determined by calculating the pixel ratio of the signal and noise portions. Assuming the region has a total of 1024 pixels, with 800 pixels in the signal region and 224 pixels in the noise region, the signal-to-noise ratio is 800:224 = 3.57:1, indicating that the signal in this region is relatively clear. Next, the persistence of signal aggregation and distribution in a region is determined by analyzing the changes in grayscale intensity within that region. If the grayscale value changes little, the signal is relatively stable; if the changes are large, signal fluctuations occur. For example, if the grayscale value of a region does not change by more than 5 in three consecutive frames, the signal is stable; while in some regions, the grayscale value fluctuation is greater than 10, indicating that the signal in that region is unstable. Based on calculation and judgment, a regional signal stability index is derived, which includes information such as signal stability, the ratio of signal to interference, and signal change trends. This index provides a reference for subsequent image fusion and quality assessment.

[0032] like Figure 4 As shown, the specific steps for obtaining the modal partitioning weight parameters are as follows: S301: Based on the partitioned signal stability index, analyze the signal aggregation and interference diffusion performance of each mode in the same area, compare the distribution pattern and consistency of pixel gray level in the area, determine whether the signal distribution of each mode in the same area is discrete or interference diffusion occurs, identify the mode types with different spatial signal performance, and obtain the interference feature type set. The signal aggregation and interference diffusion characteristics of each modality image in the same region are analyzed, and pixel grayscale information within the region is extracted. For example, in a certain region, the mean grayscale value of the B-mode ultrasound image is 110, and the standard deviation is 12, while the mean grayscale value of the color Doppler ultrasound image is 115, and the standard deviation is 25. The grayscale standard deviation is used to determine the degree of signal aggregation; a smaller standard deviation indicates signal aggregation, while a larger one may indicate interference diffusion. The degree of grayscale variation is calculated for further analysis. If the grayscale value variation of a certain modality is greater than 20, interference signals may be present. The interference and signal distribution in the region are compared by feature analysis. If a certain region B-mode ultrasound image has a mean grayscale value of 110, and the standard deviation is 12, while the standard deviation is 115, and the standard deviation is 25, the standard deviation is 25. The ultrasound signal is stable and concentrated, while the color Doppler ultrasound signal shows large fluctuations and a more scattered grayscale value distribution, indicating that the color Doppler ultrasound mode has obvious interference diffusion. Further identification of interference signal characteristics is possible. Specifically, for a set of regions, assuming that the grayscale variation range of the B-mode ultrasound image is 100 to 120, while that of the color Doppler ultrasound image is 90 to 140, the regions where the B-mode ultrasound signal is more concentrated show smaller grayscale variations, while the color Doppler ultrasound shows larger fluctuations. By comparing the differences between the modes, it is determined which mode has stronger interference characteristics, and the interference feature type set is obtained, including which modes have obvious interference or diffusion phenomena.

[0033] S302: Based on the interference feature type set, analyze the differences in spatial features and grayscale coherence of each mode, determine the mode labels with signal aggregation, optimize the mode labels with signal aggregation, and obtain the dominant response mode group; To analyze the differences in spatial features and grayscale continuity among different modalities, the spatial distribution of each modality image is first compared. The Euclidean distance and structural similarity between pixels are used to evaluate the continuity of grayscale. For example, within a certain region, the grayscale variation of the B-mode ultrasound image is relatively uniform and the structure within the region is relatively clear, while the grayscale variation of the color Doppler ultrasound image in the same region is larger and the edges of some regions are blurred. By comparing the differences in grayscale and structure, it is determined which modalities exhibit stronger signal aggregation in that region. For example, if the grayscale value variation of the B-mode ultrasound image in that region is less than 5, and the grayscale value variation of the color Doppler ultrasound image is greater than 10, and there is obvious [signal aggregation], then [the analysis is more accurate]. In areas with ambiguous structures, ultrasound (B-mode) will be identified as a mode with strong signal aggregation, while color Doppler ultrasound will exhibit unstable and interfering modes. This process identifies modes with signal aggregation and optimizes their identification. By comprehensively analyzing the grayscale and spatial characteristics of different modes within a region, a dominant response mode group is determined, including those modes exhibiting significant signal aggregation characteristics. Specifically, assuming that in a certain region, the grayscale variation of the ultrasound mode is small in two frames and the signal aggregation area is relatively clear, while the color Doppler ultrasound mode exhibits large grayscale variation and dissimilar signal, ultrasound will be identified as the dominant response mode, thus obtaining the dominant response mode group.

[0034] S303: Based on the dominant response mode group, adjust the weight distribution of the corresponding mode in the partition, calculate the contribution ratio of each mode signal, optimize the participation ratio of interference mode and dominant mode, uniformly correct the regional fusion parameter configuration, and obtain the mode partition weight parameters. The weighting of corresponding modes within a partition is adjusted. First, the signal contribution ratio of each mode within the partition is calculated. For example, assuming that in a certain region, the signal proportion of B-mode ultrasound is 70%, the signal proportion of color Doppler ultrasound is 20%, and the signal proportion of elastography is 10%, then B-mode ultrasound will be the dominant mode, contributing the most to the signal in that region. Next, the participation ratio of interfering modes and the dominant mode is optimized. By evaluating the signal strength and interference level of each mode, the fusion weight within the region is adjusted to ensure that the signal contribution of the dominant mode is maximized while reducing the influence of interfering modes. For example, in a region, if the color Doppler ultrasound mode contributes 70%, the signal contribution of the dominant mode is 20%, and the signal contribution of elastography is 10%, then B-mode ultrasound will be the dominant mode, contributing the most to the signal in that region. If a modal signal has a large amount of interference and its signal contribution drops to 5%, its weight is reduced during the fusion process. Conversely, if the ultrasound modal signal is stable and contributes significantly, its weight is increased. This results in a clearer image with less noise in the fused region. By adjusting the weights, the fusion parameter configuration of the region is corrected, resulting in modal partition weight parameters. These parameters reflect the weight distribution of different modalities within the region. For example, the weight coefficient of the dominant modality is 0.7, and the weight coefficient of the interference modality is 0.2. The optimized parameters provide a more reasonable modal weight allocation scheme, ensuring that the signal fusion effect reaches its optimal level.

[0035] like Figure 5 As shown, the specific steps for obtaining spatial continuous signal-to-noise features are as follows: S401: Based on the modal partition weight parameters, analyze the multimodal weight distribution of each image partition, compare the fusion participation ratio of each modality in the region, determine the difference in modal weight configuration within the partition, calculate the partition distribution change of each modality weight in the whole image, and obtain the region weight mapping information; The multimodal weight distribution within each image partition is analyzed, and the weight values ​​of different modalities within each partition are extracted. For example, for a certain region, the weight of ultrasound is 0.6, color Doppler ultrasound is 0.3, and elastography is 0.1. By comparing the weight values, the fusion participation ratio of the modalities in that region is determined. If the weight of ultrasound is greater than that of other modalities, it indicates that the signal contribution of that modality in that region is greater; conversely, it indicates that the influence of color Doppler ultrasound or elastography on that region is more significant. Next, the changes in the partition distribution of each modal weight in the entire image are calculated. Assuming that in different partitions, the weight of ultrasound is 0.7 in one region and 0.4 in another, color Doppler ultrasound is 0.4, and elastography is 0.2, the weight distribution of each modality in different regions of the image can be evaluated through calculation, and the differences in modal weight configuration in certain regions can be identified. For example, the weight of ultrasound in the edge region is greater than that in the central region, indicating that the signal in the edge region is more dependent on the ultrasound modality. The regional weight mapping information is obtained, which records the weight distribution and change pattern of each modality in each image partition.

[0036] S402: Based on the regional weight mapping information, calculate the weighted gray signal contribution of each modality in the corresponding partition, compare the continuous changes in gray distribution in each region of the fused image, determine the integrity of gray transition between partitions, identify the partition fusion order where gray distribution changes, and obtain the partition fusion time sequence. To calculate the weighted grayscale signal contribution of each modality in its corresponding region, firstly, calculate the weighted grayscale value of each modality image in its respective region. Assuming the average grayscale value of ultrasound is 120, color Doppler ultrasound is 130, and elastography is 140, multiply these values ​​by the corresponding modality's weight value (e.g., 0.7 for ultrasound, 0.2 for color Doppler ultrasound, and 0.1 for elastography). The calculated weighted grayscale values ​​are 120 × 0.7 = 84, 130 × 0.2 = 26, and 140 × 0.1 = 14, respectively. The resulting weighted signal contribution is 84 + 26 + 14 = 124. Next, compare the continuous changes in grayscale distribution across different regions in the fused image. By observing the transition effect of grayscale values ​​in different regions, if the grayscale value changes smoothly and without abrupt changes in adjacent regions, it indicates that the grayscale transition is good. Otherwise, if there is a large difference in grayscale values ​​between a certain region and its adjacent regions, it indicates that there is a discontinuity in the grayscale transition, which may lead to poor fusion effect. For example, if the grayscale value of a certain region jumps rapidly from 100 to 150 and then drops sharply to 80 in adjacent regions, it indicates that there is a problem with the fusion. Through comparison and calculation, a partition fusion time sequence is obtained, which records the order and change pattern of grayscale transition in each region, and determines the partition fusion order of discontinuous grayscale transition, which helps in subsequent fusion optimization.

[0037] S403: Based on the partition fusion time sequence, adjust the fusion output order of the partitions, perform grayscale consistency analysis on the fusion boundary of each partition according to the modal weight configuration, determine the signal-to-noise connection status between adjacent partitions, and obtain spatial continuous signal-to-noise characteristics; Adjust the fusion output order of the partitions, and perform grayscale consistency analysis on the fusion boundary of each partition based on the modal weight configuration, using the formula: ; Determine the signal-to-noise transition status between adjacent segments to obtain spatially continuous signal-to-noise characteristics, where... The signal-to-noise ratio (SNR) characteristic value of the fused image represents the comprehensive characteristics of the SNR and grayscale consistency in the fused image. The image after fusion is the first The grayscale value of each pixel represents the specific grayscale intensity of that pixel after image fusion. The image at the boundary is the first The grayscale value of each pixel refers to the original grayscale value of the boundary region, used for comparison with the merged grayscale value. The total number of partitions indicates the number of regions the image is divided into, and is used for normalization processing. Refers to the first The signal-to-noise ratio value of the frame region indicates the value in the first frame. The signal-to-noise ratio in a frame image reflects the clarity of the signal and the degree of noise interference. This refers to the adjustment factor, used to balance the impact of signal-to-noise ratio changes and grayscale consistency, ensuring an appropriate weight between the two. The region frame index indicates the index of the image frame in which the current partition is located, and is used to indicate the specific image frame. The pixel position indicates the location of the currently calculated pixel in the image, and is used to perform calculations on each pixel individually.

[0038] The signal-to-noise (SNR) consistency feature value refers to a quantized value obtained during image fusion by calculating the changes in signal-to-noise ratio (SNR) and grayscale consistency between adjacent partitions or frames. It reflects the degree of signal and noise consistency between different partitions or frames in the fused image, i.e., whether the SNR is smoothly connected between different regions or frames, and whether the grayscale values ​​remain consistent. This feature value is used to measure the smoothness and continuity of the fused image, especially in boundary regions and between different frames, to avoid abrupt changes in signal or noise. Grayscale consistency analysis: The first part of the formula calculates the grayscale difference between the fused image and the boundary regions. If this difference is large, it indicates a significant grayscale discontinuity during the fusion process, i.e., an inconsistent SNR, which may lead to… Obvious segmentation lines or noise issues appear at image boundaries or transition areas; Signal-to-noise ratio (SNR) coherence analysis: The second part of the formula calculates the magnitude of SNR variation by comparing the SNR values ​​of adjacent frames or partitions; if the SNR variation is large, it indicates that there are large fluctuations in the signal-to-noise ratio between different regions or frames, which will lead to unstable or discontinuous image fusion effects, affecting the overall image quality; therefore, the SNR coherence feature value is a comprehensive index used to measure the degree of coherence between signal and noise during image fusion, especially in the transition parts between partitions or frames; the smaller this value, the smoother the signal transition and the better the fusion effect; conversely, it indicates that there are obvious fusion discontinuities or inconsistent SNR in the image.

[0039] The input image is partitioned based on modality weights. Assuming the input image resolution is 1920×1080 pixels, it is divided into 6 equal-spaced partitions in the horizontal direction, resulting in a total of 6 partitions. Each partition is 320 pixels wide, and the partitions are numbered sequentially from Region 1 to Region 6. After partition calibration, the fusion order is determined based on the weight values ​​of each modality in the current fusion task. The weights of modality A, modality B, and modality C are set to 0.6, 0.3, and 0.1, respectively, and the corresponding fusion execution order is Modality A-Modality B-Modality C. During the specific partition fusion process, the fusion operation is performed point-by-point on each pixel within each partition, selecting a pixel within Region 1. As an example, the original grayscale values ​​of this pixel in the three modal images are respectively , , The three grayscale values ​​are multiplied and summed according to predetermined weights to obtain the fused grayscale value of the pixel: ; After completing pixel fusion in region 1, the partition boundary between region 1 and region 2 is located. Continuous pixel samples are extracted from the boundary location for grayscale consistency analysis. Five pixels are selected at the boundary location as calculation samples, corresponding to the grayscale values ​​of the fused image. The values ​​are 110, 118, 122, 115, and 117, corresponding to the original grayscale values ​​of the boundary image. The values ​​are 110, 120, 121, 113, and 118. For each pair of pixel grayscale values, an absolute difference operation is performed, resulting in a difference sequence of 0, 2, 1, 2, and 1. These differences are accumulated and divided by the number of pixels (5) to obtain the average grayscale difference for this boundary region. ; The above grayscale difference mean is the original dimensionless value. To ensure it participates in the same feature calculation as the subsequent signal-to-noise ratio difference, the grayscale difference result is processed using a minimum-maximum normalization method. The maximum reference value for the grayscale difference is set to 255, and the corresponding normalization result is: ; After the grayscale consistency analysis is completed, the signal-to-noise ratio (SNR) values ​​of region 1 in adjacent image frames are further obtained, and the first [frame name] is set. The signal-to-noise ratio of this region in the frame is , No. The signal-to-noise ratio of the corresponding region in the frame is The signal-to-noise ratio (SNR) values ​​of the two frames are interpolated and their absolute values ​​are taken to obtain the SNR difference value: ; Since the unit of signal-to-noise ratio (SNR) difference is dB, the minimum-maximum normalization method is used for unit unification. The maximum reference range for the SNR difference is set to 6 dB. Therefore, the normalized SNR difference is: ; After completing the two normalization processes, a weighting adjustment factor is introduced. Substituting the normalized grayscale difference term and signal-to-noise ratio difference term into the fusion output feature calculation formula, the fusion output feature value of the partition boundary is obtained as follows: ; ; ; Based on the image fusion continuity evaluation criteria, the interval is divided into the following three sub-intervals: when When the boundary fusion continuity is excellent, the grayscale transition and signal-to-noise ratio change are minimal, and the fusion transition can be judged to be smooth. when When the boundary fusion continuity is generally poor, there is a certain degree of difference between the grayscale and signal-to-noise changes, and it is recommended to evaluate whether to optimize the fusion path. when When this occurs, it indicates poor continuity of boundary fusion, with abrupt changes or discontinuities in the boundary region. The current fusion order or weight configuration should be adjusted.

[0040] The current calculation result is The result falls within the first interval, which satisfies the condition. The results show that the current fusion boundary of the region is in a stable state under the normalized grayscale consistency and signal-to-noise ratio connection index, indicating that the current partition fusion effect has good continuity in the spatial dimension.

[0041] like Figure 6 As shown, the specific steps for obtaining the inter-frame fusion consistency index are as follows: S501: Based on the spatial continuous signal-to-noise characteristics, analyze the changing trend of the signal-to-noise ratio of each frame partition during continuous multi-frame imaging, determine the fluctuation of the signal distribution of the same spatial partition in continuous frames on the time axis, compare the continuity and consistency of the partition signals of adjacent frames, filter the partition positions of signal-to-noise changes, and obtain the frame order fluctuation interval set. This study analyzes the signal-to-noise ratio (SNR) trend of each frame partition during continuous multi-frame imaging, extracting SNR data for each image partition. Assuming the ultrasound SNR of a certain region is 15dB in the first frame, 13dB in the second, 16dB in the third, and so on, the fluctuation of the SNR of each partition along the time axis is calculated. By comparing the SNR variation amplitude between different frames, it is determined whether the signal of that partition remains stable. If the variation amplitude exceeds a set threshold (e.g., ±3dB), the signal of that partition is considered to have experienced significant fluctuation. Further comparisons are made between adjacent frame partitions. Signal continuity and consistency: Assuming that in a certain region, the signal-to-noise ratio (SNR) of B-mode ultrasound and color Doppler ultrasound changes relatively steadily, while the SNR of elastography fluctuates significantly, this indicates that the elastography signal is unstable in that region and needs adjustment. By screening out the partitions with large SNR fluctuations, a set of frame sequence fluctuation intervals is obtained. Between the 4th and 5th frames, the SNR of a certain partition drops sharply from 12dB to 8dB and then rises back to 14dB, significantly exceeding the ±3dB fluctuation range. This region is marked as the fluctuation interval, and a set of frame sequence fluctuation intervals is constructed for subsequent fusion optimization processing.

[0042] S502: Based on the frame order fluctuation interval set, perform fusion weight adjustment on the fluctuation partition for the current frame, analyze the signal change direction of the current frame and the signal structure characteristics of the previous frame, determine whether the partition fusion configuration is consistent with the signal trend, optimize the fusion ratio of each mode in the partition, and obtain frame-level fusion ratio data. For fluctuating partitions, the fusion weight adjustment for the current frame is performed. First, the signal change direction of each modality in the current frame is obtained. For example, if in a certain region, the ultrasound signal shows a gradually increasing trend, while the color Doppler ultrasound signal shows a gradually decreasing trend, then the modal fusion weight of the current frame needs to be adjusted according to this signal change direction. For example, the weight of the ultrasound modality is increased, and the weight of the color Doppler ultrasound is decreased. Next, the signal change direction of the current frame and the signal structure characteristics of the previous frame are analyzed. Assuming that in the previous frame, the ultrasound signal in this region is a relatively concentrated signal, while the color Doppler ultrasound signal is relatively dispersed, by comparing the signal structure of the current frame and the previous frame, it is determined whether the partition fusion configuration of the current frame is consistent with the signal trend. If the signal structure of the previous frame has stabilized, but the signal of the current frame shows large fluctuations, then the weight of the current frame is adjusted to optimize the fusion ratio of each modality in the partition. For example, if the contribution of the ultrasound signal continues to increase, then the weight of the ultrasound is increased, and the weight of the modal with greater interference is reduced. Frame-level fusion ratio data is obtained, which includes the fusion weight adjustment results for each partition in the current frame, so that the signal fusion of each partition is more consistent with its signal change trend.

[0043] S503: Based on frame-level fusion matching data, spatial stitching is performed on the fusion image results of all current frame partitions, the boundary gray level continuity between adjacent partitions is analyzed, the structural coherence of the same partition between consecutive frames is compared, and the inter-frame fusion consistency index is obtained. Spatial stitching is performed on the fused image results of all current frame partitions. First, the images of each partition are weighted and synthesized. Assuming the signal weight for the B-mode ultrasound is 0.7, the signal weight for the color Doppler ultrasound is 0.2, and the signal weight for the elastography mode is 0.1, the images of each partition are weighted and synthesized according to their weight values ​​to obtain the fused images of each partition. Next, the continuity of grayscale at the boundaries between adjacent partitions is analyzed. By comparing the differences in grayscale values ​​of boundary pixels, if the grayscale changes between adjacent regions are large, for example, if the grayscale value of a certain region jumps from 100 to 200, it indicates a problem with the boundary connection, which may be due to... To improve the fusion effect, it is necessary to adjust the fusion parameters of the boundary transition region. Further comparison of the structural coherence of the same partition between consecutive frames is required. For example, if the structural details do not change significantly and the grayscale transition is smooth in the same region of adjacent frames, it indicates good structural coherence. If obvious cracks or breaks are found in the structure of a certain region, it is necessary to improve the structural consistency by adjusting the weight ratio or adding smoothing processing. This yields the inter-frame fusion consistency index, which includes key indicators such as the boundary grayscale continuity and structural coherence of the fusion results of each partition of the image, ensuring the overall consistency and smoothness of the image fusion process.

[0044] like Figure 7 As shown, a multimodal ultrasound data fusion system includes: The partition extraction module is based on multimodal ultrasound image data. It analyzes the temporal and modal attributes of the data stream, compares the spatial distribution, judges the synchronization response and adjusts the acquisition order, calculates the pixel arrangement, filters spatial partitions, and obtains a set of spatiotemporal synchronization partitions. The signal feature discrimination module is based on a set of spatiotemporal synchronous partitions. It compares the signal aggregation characteristics with the changes in spurious signals, determines the signal distribution status, calculates the proportion of signals and interference, identifies feature data, and obtains the partition signal stability index. The weight adjustment module compares signal performance based on the regional signal stability index, judges interference propagation characteristics, adjusts the mode fusion weights, screens key modes of signal aggregation and adjusts their participation ratios, and then corrects the overall fusion ratio to obtain the mode partition weight parameters. The fusion characteristic analysis module analyzes the weight distribution, calculates the contribution of grayscale signal, compares grayscale trends, judges the stability of signal, adjusts the regional fusion order, and obtains spatial continuous signal-noise characteristics based on modal partitioning weight parameters. The consistency evaluation module analyzes the signal-to-noise ratio changes based on the spatial continuous signal-to-noise characteristics, compares the stability characteristics of adjacent frame partition signals, judges abnormal fluctuations, adjusts the current frame partition fusion weights, splices the region fusion results, and obtains the inter-frame fusion consistency index.

[0045] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of protection of the described technical solutions.

Claims

1. A method for multimodal ultrasound data fusion, characterized in that, The method includes: S1: Based on multimodal ultrasound image data, analyze the data stream temporal sequence and modal attributes, compare spatial distribution, determine synchronization response and adjust data arrangement temporal sequence, parse pixel spatial coordinates, filter spatial partitions, and obtain a set of spatiotemporal synchronization partitions; S2: Based on the spatiotemporal synchronization partition set, compare the signal aggregation characteristics with the changes in spurious signals, determine the signal distribution status, calculate the signal and interference ratios, identify feature data, and obtain the partition signal stability index. S3: Based on the partition signal stability index, compare signal performance, determine interference propagation characteristics, adjust mode fusion weights, screen key modes of signal aggregation and adjust their participation ratios, and then correct the overall fusion ratio to obtain mode partition weight parameters. S4: Based on the modal partitioning weight parameters, analyze the weight distribution, calculate the grayscale signal contribution, compare the grayscale trend, determine the signal stability, and obtain the spatial continuous signal-to-noise characteristics; S5: Based on the spatial continuous signal-to-noise characteristics, analyze the signal-to-noise ratio changes, compare the stability characteristics of adjacent frame partition signals, determine abnormal fluctuations, adjust the current frame partition fusion weights, splice the region fusion results, and obtain the inter-frame fusion consistency index.

2. The multimodal ultrasound data fusion method according to claim 1, characterized in that, The spatiotemporal synchronization partition set includes spatial partition labels, partition corresponding time identifiers, and partition pixel positions. The partition signal stability indicators include regional structural clarity, signal uniformity, and noise ratio. The modal partition weight parameters include modal partition weights, signal saliency coefficients, and interference sensitivity coefficients. The spatial continuous signal-to-noise characteristics include fused signal-to-noise distribution, regional coherence indicators, and boundary continuity parameters. The inter-frame fusion consistency indicators include inter-frame structural consistency, signal-to-noise trend changes, and continuous frame fusion levels.

3. The multimodal ultrasound data fusion method according to claim 1, characterized in that, The specific steps for obtaining the spatiotemporal synchronization partition set are as follows: S101: Based on multimodal ultrasound image data, analyze the intra-frame time stamps and modal information, determine the time synchronization between image frames of each modality, compare the correspondence between modalities at the same time point, filter the combination of image frames with time consistency, and obtain the time-series correlation sequence. S102: Based on the time-series correlation sequence, compare the pixel layout of the image frame combination in the unified coordinate domain, determine the consistency of pixel distribution under each mode, adjust the spatial position correspondence of the image frames, analyze the acquisition order of the image frames with offset, correct the data arrangement method, and obtain the spatial alignment sequence. S103: Based on the spatial alignment sequence, the region is divided according to the pixel layout, the temporal and spatial matching relationship of the multimodal image corresponding to each region is calculated, the synchronization processing conditions of the region are determined, and a set of spatiotemporal synchronization partitions is obtained.

4. The multimodal ultrasound data fusion method according to claim 1, characterized in that, The specific steps for obtaining the partition signal stability index are as follows: S201: Based on the spatiotemporal synchronization partition set, analyze the grayscale matrix of each region, judge the difference in grayscale distribution by statistically analyzing the concentration of pixel grayscale within the partition, compare the regularity and degree of change of pixel arrangement, screen the signal features of regions with local clustering phenomena, aggregate the grayscale distribution state, and obtain the clustering characteristic parameter group. S202: Based on the clustering characteristic parameter set, compare the gray-scale structure of each region, analyze the continuity and edge changes of the gray-scale distribution of the partition, determine whether the signal of each region shows continuous diffusion or structural ambiguity, optimize the clustering and summarization of the gray-scale characteristics of the partition, and obtain the distribution pattern label set. S203: Based on the distribution pattern label set, calculate the pixel gray intensity of the partition and its neighborhood difference, analyze the proportion of signal and interference in the gray distribution, determine the persistence of regional signal aggregation and distribution state, and obtain the partition signal stability index.

5. The multimodal ultrasound data fusion method according to claim 1, characterized in that, The specific steps for obtaining the modal partitioning weight parameters are as follows: S301: Based on the partitioned signal stability index, analyze the signal aggregation and interference diffusion performance of each mode in the same area, compare the distribution pattern and consistency of pixel grayscale in the area, determine whether the signal distribution of each mode in the same area is discrete or interference diffusion occurs, identify the mode types with different spatial signal performance, and obtain the interference feature type set. S302: Based on the set of interference feature types, analyze the differences in spatial features and grayscale coherence of each mode, determine the mode labels with signal aggregation, optimize the mode labels with signal aggregation, and obtain the dominant response mode group; S303: Based on the dominant response mode group, adjust the weight distribution of the corresponding mode in the partition, calculate the contribution ratio of each mode signal, optimize the participation ratio of interference mode and dominant mode, uniformly correct the regional fusion parameter configuration, and obtain the mode partition weight parameters.

6. The multimodal ultrasound data fusion method according to claim 1, characterized in that, The specific steps for obtaining the spatial continuous signal-to-noise features are as follows: S401: Based on the modal partition weight parameters, analyze the multimodal weight distribution of each image partition, compare the fusion participation ratio of each modality in the region, determine the difference in modal weight configuration within the partition, calculate the partition distribution change of each modal weight in the whole image, and obtain the region weight mapping information; S402: Based on the region weight mapping information, calculate the weighted gray signal contribution of each modality in the corresponding partition, compare the continuous changes in gray distribution in each region of the fused image, determine the integrity of gray transition between partitions, identify the partition fusion order where gray distribution changes, and obtain the partition fusion time sequence. S403: Based on the partition fusion time sequence, adjust the fusion output order of the partitions, perform grayscale consistency analysis on the fusion boundary of each partition according to the modal weight configuration, determine the signal-to-noise connection status between adjacent partitions, and obtain spatial continuous signal-to-noise features.

7. The multimodal ultrasound data fusion method according to claim 1, characterized in that, The specific steps for obtaining the inter-frame fusion consistency index are as follows: S501: Based on the spatial continuous signal-to-noise characteristics, analyze the changing trend of the signal-to-noise ratio of each frame partition during continuous multi-frame imaging, determine the fluctuation of the signal distribution of the same spatial partition in continuous frames on the time axis, compare the continuity and consistency of the partition signals of adjacent frames, filter the partition positions with signal-to-noise changes, and obtain the frame order fluctuation interval set. S502: Based on the frame order fluctuation interval set, perform fusion weight adjustment on the fluctuation partition for the current frame, analyze the signal change direction of the current frame and the signal structure characteristics of the previous frame, determine whether the partition fusion configuration is consistent with the signal trend, optimize the fusion ratio of each mode in the partition, and obtain frame-level fusion ratio data. S503: Based on the frame-level fusion matching data, spatially stitch the fusion image results of all current frame partitions, analyze the boundary grayscale continuity between adjacent partitions, compare the structural coherence of the same partition between consecutive frames, and obtain the inter-frame fusion consistency index.

8. The multimodal ultrasound data fusion method according to claim 1, characterized in that, The data stream timing refers to the order in which the original signals are acquired over time during the multimodal ultrasound image acquisition process, and the spatial distribution refers to the physical correspondence of each pixel in two-dimensional or three-dimensional space for different modal images acquired at the same time.

9. The multimodal ultrasound data fusion method according to claim 1, characterized in that, The signal aggregation characteristic refers to whether the pixel grayscale value or signal intensity is concentrated within a certain partition, and the stray signal change refers to the phenomenon that the grayscale or signal intensity is randomly distributed, uneven, and has a lot of noise components within the partition.

10. A multimodal ultrasound data fusion system, said system being used to implement the multimodal ultrasound data fusion method as described in any one of claims 1-9, characterized in that, The system includes: The partition extraction module is based on multimodal ultrasound image data. It analyzes the temporal and modal attributes of the data stream, compares the spatial distribution, judges the synchronization response and adjusts the acquisition order, calculates the pixel arrangement, filters spatial partitions, and obtains a set of spatiotemporal synchronization partitions. The signal feature discrimination module, based on the spatiotemporal synchronization partition set, compares the signal aggregation characteristics with the changes in spurious signals, determines the signal distribution status, calculates the proportion of signal and interference, identifies feature data, and obtains the partition signal stability index. The weight adjustment module compares signal performance based on the partition signal stability index, judges interference propagation characteristics, adjusts mode fusion weights, filters key modes of signal aggregation and adjusts their participation ratios, and then corrects the overall fusion ratio to obtain mode partition weight parameters. Based on the modal partitioning weight parameters, the fusion characteristic analysis module analyzes the weight distribution, calculates the grayscale signal contribution, compares the grayscale trend, judges the signal stability, adjusts the region fusion order, and obtains spatial continuous signal-to-noise characteristics. Based on the spatial continuous signal-to-noise characteristics, the consistency evaluation module analyzes the signal-to-noise ratio changes, compares the stability characteristics of adjacent frame partition signals, judges abnormal fluctuations, adjusts the current frame partition fusion weights, splices the region fusion results, and obtains the inter-frame fusion consistency index.

Images