A multi-source data fusion method based on double models
By using dual-model collaborative processing, the problem of accurately locating and aligning morphological transition features and dynamic response parameters in multi-source data fusion was solved, achieving precise fusion of multi-source data streams and improving data consistency and robustness.
Patent Information
- Application Number
- CN202611416316.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-14
- Publication Date
- 2026-11-10
AI Technical Summary
Existing multi-source data fusion technologies struggle to accurately pinpoint the morphological transition features and dynamic response parameters of heterogeneous data streams, leading to deviations in cross-source data segment extraction from actual boundaries and insufficient accuracy in dynamic phase alignment, thus affecting the consistency and continuity of the fused data.
A dual-model approach is adopted to differentiate the data streams of the first and second sensor groups. Data alignment and fusion are performed by utilizing data morphological features and signal dynamic response characteristics. Morphological feature sequences and dynamic response parameters are extracted through signal separation and dynamic response operations to generate phase matching coefficients for time-domain alignment.
It achieves precise fusion of multi-source data streams in terms of morphological semantics and temporal phase, improves fusion accuracy and data continuity, adapts to complex dynamic environments, and outputs multi-source data fusion results with complete structure and accurate time alignment.
Smart Images

Figure CN123069283A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of multi-source data fusion technology, specifically a multi-source data fusion method based on a dual-model approach. Background Technology
[0002] Multi-source data fusion integrates data collected from different sensors to generate more consistent and information-rich results. In multi-sensor systems, due to differences in sensor physical characteristics, sampling mechanisms, and signal transmission paths, data streams often exhibit drastically different morphological characteristics and dynamic response behaviors. Effectively fusing such heterogeneous data requires simultaneously addressing the precise temporal and phase correspondence of cross-source data segments. Existing multi-source data fusion techniques largely rely on a single analytical model. A common approach is to divide various data streams into uniform time windows and then perform feature matching or mutual alignment within these windows. This paradigm implicitly assumes that each data stream possesses a similar or linearly transformable morphological structure, and that its phase variation remains stable throughout the observation interval. However, in practical applications, data collected by the first sensor group often contains rich waveform fluctuations and boundary abrupt changes, forming complex data morphological profiles; the signals output by the second sensor group may exhibit dynamic oscillation characteristics such as energy attenuation and frequency drift. The unified time window segmentation method struggles to capture the natural segmentation boundaries defined by morphological transitions in the first data stream. This results in data segments extracted from the second data stream not semantically corresponding to these morphological boundaries, with cross-boundary information mixed into the segments, causing misalignment of the fused objects. On the other hand, the correlation alignment algorithm with fixed parameters cannot track the time-varying characteristics of the phase relationship during the dynamic response of the second data stream. The alignment deviation introduced during the signal energy attenuation stage gradually accumulates, significantly deteriorating the temporal consistency of the fused data segments.
[0003] In the scenario of collaborative fusion of multi-source heterogeneous data streams, two issues urgently need to be addressed: how to accurately locate and extract data segments with morphological semantic consistency from the second data stream based on the morphological transition features contained in the first data stream; and how to perform adaptive phase matching and temporal alignment on the extracted segments under the constraints of the dynamic response parameters of the second data stream itself. Summary of the Invention
[0004] This invention provides a multi-source data fusion method based on a dual-model approach to address the problems of cross-source data segment extraction deviating from actual morphological boundaries and insufficient dynamic phase alignment accuracy in existing technologies, thereby achieving accurate fusion of multi-source heterogeneous data streams in terms of morphological semantics and temporal phase.
[0005] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a multi-source data fusion method based on a dual-model approach, which differentiates the multi-source data streams collected by the first sensor group and the second sensor group respectively, and utilizes the data morphology characteristics and signal dynamic response characteristics to collaboratively complete data alignment and fusion, thereby improving fusion accuracy and data continuity.
[0006] In the technical solution of this invention, a first multi-source data stream collected by a first sensor group and a second multi-source data stream collected by a second sensor group are acquired. Using the first multi-source data stream as input, a signal separation operation based on data morphology features is performed through a preset first signal processing model to obtain a data morphology feature sequence corresponding to the first multi-source data stream. Simultaneously, using the second multi-source data stream as input, a dynamic response operation based on signal oscillation characteristics is performed through a preset second signal processing model to obtain dynamic response parameters corresponding to the second multi-source data stream. Based on the data morphology feature sequence, data segments corresponding to the data morphology feature sequence are extracted from the second multi-source data stream and marked as data segments to be fused. Subsequently, based on the dynamic response parameters, phase matching coefficients for the data segments to be fused within a fusion time window are generated, and these phase matching coefficients are used to perform time-domain alignment operations on the data segments to be fused, resulting in aligned fused data segments. The aligned fused data segments are then sequentially written into a preset fusion data buffer according to the order of the data morphology feature sequence, ultimately obtaining the multi-source data fusion result.
[0007] In a preferred embodiment of the present invention, a first signal processing model performs a sliding window sampling operation on a first multi-source data stream to obtain multiple first data windows. Extreme point detection is performed on each first data window to obtain a set of window extreme points, and the data oscillation period of each first data window is determined based on this set of extreme points. The first multi-source data stream is segmented according to the data oscillation period of each first data window to form multiple data shape segments. The start and end markers of each data shape segment are extracted in chronological order, and the start and end markers are combined into a data shape feature sequence. This scheme adaptively segments based on the signal's own shape changes, without relying on external synchronization signals, and can accurately capture the temporal structural features of the data stream.
[0008] Furthermore, when determining the data oscillation period, target extreme points with amplitudes greater than a preset amplitude threshold are selected from the set of extreme points in the window, resulting in a sequence of target extreme points. The sampling interval between two adjacent target extreme points in the sequence is calculated to obtain multiple candidate interval values. The candidate interval value that appears most frequently among these multiple candidate interval values is determined as the initial oscillation period of the first data window. The initial oscillation periods are arranged according to their time positions within the first data window to obtain an oscillation period sequence. Median filtering is then performed on the oscillation period sequence, and the filtered result is used as the data oscillation period. This effectively suppresses period misjudgment caused by noise interference, making the extracted oscillation period more stable and reliable.
[0009] In the above scheme, when generating the data morphology feature sequence, the sampling time of the first sampling point of each data morphology segment is recorded as the segment start timestamp and converted into a segment start flag, and the sampling time of the last sampling point is recorded as the segment end timestamp and converted into a segment end flag. These are arranged in chronological order to obtain the initial morphology feature sequence. The time interval between adjacent segment end flags and segment start flags in the initial morphology feature sequence is checked to see if it is less than a preset time threshold. If it is less, the adjacent segment end flags and segment start flags are deleted, and the remaining flag sequence after deletion is used as the data morphology feature sequence. Through redundant flag removal, fragmented segments caused by excessive subdivision can be eliminated, ensuring that the feature sequence provides a more concise description of the macroscopic morphology of the data.
[0010] As another preferred embodiment of the present invention, the second signal processing model performs a spectral transformation operation on the second multi-source data stream to obtain spectral distribution information; extracts the dominant frequency component from the spectral distribution information, and records the dominant frequency amplitude and dominant frequency phase corresponding to the dominant frequency component. An envelope detection operation is performed on the second multi-source data stream to obtain the signal envelope; the envelope slope of the signal envelope within a preset time interval is calculated, and the energy attenuation coefficient of the second multi-source data stream is determined based on the envelope slope. The dominant frequency amplitude, dominant frequency phase, and energy attenuation coefficient are combined into a dynamic response parameter. This method can comprehensively characterize the dynamic characteristics of the signal in the frequency and time domains, providing an accurate basis for subsequent phase matching.
[0011] Furthermore, in the extraction of the dominant frequency component, the frequency component with the largest amplitude is selected from the spectral distribution information as the initial dominant frequency component; the initial frequency value corresponding to the initial dominant frequency component is obtained, and the initial frequency value is merged with other frequency components within a preset frequency tolerance range to obtain a merged frequency band; the total energy value within the merged frequency band is calculated, and the center frequency corresponding to the total energy value is determined as the final dominant frequency; then, the amplitude corresponding to the final dominant frequency is read from the spectral distribution information as the dominant frequency amplitude, and the phase value corresponding to the final dominant frequency is read as the dominant frequency phase. Through frequency band merging, frequency point dispersion caused by spectral leakage can be avoided, improving the accuracy of dominant frequency characterization.
[0012] When extracting data segments to be fused, the segment start and end markers in the data morphology feature sequence are parsed to obtain the start time position corresponding to each segment start marker and the end time position corresponding to each segment end marker. Candidate data segments are extracted from the second multi-source data stream, using the start time position as the extraction starting point and the end time position as the extraction endpoint. It is then determined whether the data length of the candidate data segment is less than a preset length threshold; if it is, the candidate data segment is concatenated with the next adjacent candidate data segment to obtain a concatenated data segment. This concatenated data segment or candidate data segment is used as the data segment to be fused. Concatenating excessively short segments prevents damage to the continuity of the fused data due to overly fine segmentation.
[0013] When generating phase matching coefficients, the dominant frequency phase is extracted from the dynamic response parameters and mapped to the start time of the fusion time window to obtain the initial phase value. The energy attenuation coefficient is extracted from the dynamic response parameters, and the phase compensation factor for each sampling moment within the fusion time window is calculated based on the energy attenuation coefficient. The instantaneous phase prediction value for each sampling moment is calculated based on the initial phase value and the phase compensation factor for each sampling moment. The difference between the actual phase value and the instantaneous phase prediction value for each sampling moment in the data segment to be fused is calculated to obtain the phase deviation. The reciprocal of the phase deviation for each sampling moment is used as the phase matching coefficient for that sampling moment. This coefficient reflects the degree of deviation between the data segment to be fused and the dynamic response trend, ensuring that subsequent time-domain alignment can take into account the phase drift caused by signal attenuation.
[0014] During temporal alignment, the temporal stretching factor for each sampling moment is calculated based on the phase matching coefficient at each sampling moment, and the new time coordinates of each sampling point in the data segment to be fused are determined according to the temporal stretching factor. A resampling operation is then performed on the data segment to be fused according to the new time coordinates, resulting in a resampled data segment. It is then checked whether the number of sampling points in the resampled data segment is equal to the number of standard sampling points within the fusion time window; if not, interpolation or decimation operations are performed on the resampled data segment to obtain the aligned fused data segment. This method ensures that the data segment to be fused accurately corresponds to the standard time grid in the time dimension, eliminating temporal deviations between multi-source data.
[0015] When generating multi-source data fusion results, the order of the segment start markers in the data morphology feature sequence corresponding to the aligned fused data segments is determined, and the write start address of the fused data segments in the fused data buffer is determined according to this order. The write end address is determined based on the data length of the aligned fused data segments, and the fused data segments are written to the fused data buffer according to the write start address and write end address. After all aligned fused data segments have been written, the data in the fused data buffer is output as the multi-source data fusion result. Through ordered writing and buffered output, the temporal order of the fusion result is ensured to be consistent with the overall temporal order of the original multi-source data stream.
[0016] This invention employs a dual-model collaborative processing approach. On one hand, it extracts the data morphology feature sequence of the first multi-source data stream to delineate the time-domain boundary. On the other hand, it extracts the dynamic response parameters of the second multi-source data stream to describe the signal oscillation characteristics and attenuation behavior. Then, it combines the two to generate phase matching coefficients and achieve time-domain alignment, ultimately obtaining a multi-source data fusion result with high consistency and low distortion, which still exhibits good adaptability and robustness in complex dynamic environments.
[0017] The technical effects and advantages provided by the present invention in the above technical solution are as follows: A pre-defined first signal processing model is used to perform signal separation based on data morphology features on the first multi-source data stream. This model employs sliding window sampling and extreme point detection, determining the data oscillation period by using the candidate interval value that appears most frequently within the statistical window, and using median filtering to suppress transient disturbances in period estimation, thereby obtaining a morphological segmentation basis consistent with the inherent rhythm of the data. Based on this, the start and end time markers of each data morphology segment are extracted, and adjacent markers with excessively short intervals are merged to generate a data morphology feature sequence. This feature sequence reflects the naturally formed data morphology boundaries in the first data stream, rather than the mechanical segmentation positions of a fixed window. Guided by this morphology feature sequence, time-series-stricken data segments are extracted from the second multi-source data stream, and segments with insufficient length are spliced with adjacent segments. This ensures that the final data segments to be fused maintain morphological semantic consistency with the data structure captured by the first sensor group, avoiding segment overlap and content disorder caused by time window offset, and achieving high-fidelity correspondence between cross-source data segments. A pre-defined second signal processing model is then used to perform dynamic response operations based on signal oscillation characteristics on the second multi-source data stream. The model extracts the dominant frequency amplitude and phase from the spectral distribution information and calculates the energy attenuation coefficient based on the envelope slope, forming dynamic response parameters. When generating phase matching coefficients, the dominant frequency phase is mapped to the starting phase value of the fusion time window. Simultaneously, the phase compensation factor is calculated for each sampling moment based on the energy attenuation coefficient, synthesizing the instantaneous phase prediction value. The reciprocal of the deviation between this prediction value and the actual phase is used as the phase matching coefficient, thereby deriving the time-domain stretching factor and resampling and mapping the data segment to the time coordinates. This process ensures that the time position of each sampling point is compensated according to the dynamic oscillation characteristics of the second data stream itself, maintaining precise phase tracking even in intervals of significant signal energy changes, and significantly improving the phase consistency of the aligned fused data segment. The resampling results are written into the fused data buffer according to the original sorting of the morphological feature sequence, outputting a structurally complete and time-aligned multi-source data fusion result, effectively enhancing the usability of the fused data in subsequent joint analysis. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.
[0019] Figure 1 This is a flowchart of a multi-source data fusion method based on a dual-model approach. Figure 2 This is a flowchart of the candidate data fragment extraction and splicing process; Figure 3This is a flowchart of phase alignment processing based on dynamic response parameters; Figure 4 This is a schematic diagram of the extraction and splicing of data segments to be merged based on data morphology characteristics in a multi-source data stream. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0021] See Figure 1 This invention provides a multi-source data fusion method based on a dual-model approach. The method includes: acquiring a first multi-source data stream collected by a first sensor group and a second multi-source data stream collected by a second sensor group; using the first multi-source data stream as input, performing a signal separation operation based on data morphology features through a preset first signal processing model to obtain a data morphology feature sequence corresponding to the first multi-source data stream; using the second multi-source data stream as input, performing a dynamic response operation based on signal oscillation characteristics through a preset second signal processing model to obtain dynamic response parameters corresponding to the second multi-source data stream; extracting data segments corresponding to the data morphology feature sequence from the second multi-source data stream according to the data morphology feature sequence, and marking the data segments as data segments to be fused; generating phase matching coefficients for the data segments to be fused within a fusion time window based on the dynamic response parameters; performing time-domain alignment on the data segments to be fused based on the phase matching coefficients to obtain aligned fused data segments; and sequentially writing the aligned fused data segments into a preset fusion data buffer according to the order of the data morphology feature sequence to obtain the multi-source data fusion result.
[0022] In specific implementation, a first multi-source data stream is acquired from the first sensor group. This first multi-source data stream consists of multiple sampling points arranged in chronological order, with each sampling point containing a sampling time and a sampling amplitude. A sliding window sampling operation is performed on the first multi-source data stream. The window length of the sliding window is set to the product of the sampling frequency of the first sensor group and a preset basic time unit, and the sliding step size is set to half the window length. Starting from the beginning of the first multi-source data stream, the sliding window sequentially extracts consecutive sampling points to form a first data window until it covers the end of the first multi-source data stream, thus obtaining multiple first data windows.
[0023] For each first data window, an extreme point detection operation is performed. This operation is achieved by calculating the amplitude difference between adjacent sampling points within the first data window. When the sign of the amplitude difference changes from positive to negative, the sampling point corresponding to the sign change is recorded as a local maximum; when the sign of the amplitude difference changes from negative to positive, the sampling point corresponding to the sign change is recorded as a local minimum. All detected local maxima and local minima within the first data window are then added to the window extreme point set corresponding to that first data window.
[0024] Based on the set of extreme points corresponding to each first data window, the data oscillation period of each first data window is determined. The determination process is as follows: Extreme points with absolute amplitude values greater than a preset amplitude threshold are selected from the set of extreme points. These selected extreme points are taken as target extreme points and arranged in chronological order to obtain a sequence of target extreme points. The preset amplitude threshold is set at three times the root mean square value of the amplitude of the stationary, oscillation-free interval in the first multi-source data stream to filter out extreme points with minor fluctuations caused by background noise. The sampling interval between two adjacent target extreme points in the target extreme point sequence is calculated. The sampling interval is the difference between the sampling time number of the later target extreme point and the sampling time number of the earlier target extreme point. Each sampling interval is taken as a candidate interval value, resulting in multiple candidate interval values. The number of repetitions of each different value among the multiple candidate interval values is counted, and the candidate interval value with the highest repetition frequency is selected. This selected candidate interval value is determined as the initial oscillation period of the first data window. The initial oscillation periods of each first data window are arranged according to their chronological position in the first multi-source data stream to form an oscillation period sequence. Median filtering is performed on the oscillation period sequence. The window width of the median filtering operation is selected according to the number of elements contained in the oscillation period sequence. When the number of elements in the median filtering window is odd, the median value is output as the filtering result; when the number of elements in the median filtering window is even, the average of the two median values is output as the filtering result. The sequence value processed by the median filtering operation is used as the data oscillation period of the corresponding first data window.
[0025] The first multi-source data stream is segmented based on the data oscillation period of each first data window. The relative rate of change of the data oscillation periods of adjacent first data windows is calculated. This relative rate of change is the absolute value of the difference between the data oscillation periods of the subsequent and preceding first data windows, divided by the data oscillation period of the preceding first data window. When the relative rate of change exceeds a preset rate of change threshold, segmentation points are set at the time boundary between the two first data windows. The preset rate of change threshold is set to 0.2. All segmentation points are then used to cut the first multi-source data stream along the time axis, forming multiple data form segments. The data oscillation periods of the sampling points within each data form segment are consistent.
[0026] Extract the start and end flags of each data form segment in chronological order. Record the sampling time of the first sampling point of each data form segment as the segment start timestamp. Apply a preset flag mapping encoding to convert the segment start timestamp into a segment start flag, which is a binary code containing the segment start timestamp value. Record the sampling time of the last sampling point of each data form segment as the segment end timestamp. Apply a flag mapping encoding to convert the segment end timestamp into a segment end flag, which is a binary code containing the segment end timestamp value. Arrange the segment start and end flags corresponding to all data form segments in the order in which the data form segments appear to form an initial morphological feature sequence. In the initial morphological feature sequence, detect the time interval between an adjacent segment end flag and a subsequent segment start flag. The time interval is the difference between the timestamp in the segment end flag and the timestamp in the segment start flag. If the time interval is less than a preset time threshold, delete the adjacent segment end flag and segment start flag. The preset time threshold is set to the time length corresponding to the sliding step size of the first data window. After removing all pairs of flags that meet the deletion conditions in the initial morphological feature sequence, the remaining flag sequence is used as the data form feature sequence.
[0027] In the specific implementation, a second multi-source data stream is acquired from the second sensor group. This second multi-source data stream consists of continuous sampling points that vary over time. A spectrum transformation operation is performed on the second multi-source data stream to obtain its spectral distribution information. The spectrum transformation operation uses a Fast Fourier Transform (FFT). The transform length of the FFT is selected as an integer power of the number of sampling points in the second multi-source data stream. When the number of sampling points in the second multi-source data stream is not an integer power of 2, zeros are padded to the end of the second multi-source data stream until the nearest integer power of 2 is reached. After the FFT, spectral distribution information containing frequency coordinates and corresponding complex amplitudes is obtained. Each frequency point in the spectral distribution information corresponds to an amplitude and a phase.
[0028] Extract the dominant frequency component from the spectral distribution information and record its amplitude and phase. Among all frequency components in the spectral distribution information, compare the amplitudes of each component and select the component with the largest amplitude as the initial dominant frequency component. Obtain the frequency value corresponding to the initial dominant frequency component and record it as the initial frequency value. Using the initial frequency value as the center, determine a preset frequency tolerance range. The upper limit of the preset frequency tolerance range is the sum of the initial frequency value and the frequency tolerance value, and the lower limit is the difference between the initial frequency value and the frequency tolerance value. The frequency tolerance value is set to twice the frequency resolution in the spectral distribution information, which is the interval between adjacent frequency points after the Fast Fourier Transform. Mark the frequency components in the spectral distribution information that fall within the preset frequency tolerance range (excluding the initial dominant frequency component) as adjacent frequency components. Merge the initial dominant frequency component with all adjacent frequency components to form a merged frequency band. Calculate the total energy value within the merged frequency band; the total energy value is the sum of the squares of the amplitudes of each frequency component within the merged frequency band. The frequency value at the peak position corresponding to the total energy value within the merged frequency band is determined as the final dominant frequency. This is achieved by calculating a weighted average frequency value using the square of the amplitude of each frequency component within the merged frequency band as the weight. The formula for calculating the weighted average frequency value is as follows:
[0029] in, Indicates the final clock frequency. This represents the total number of frequency components within the combined frequency band. Indicates the first frequency band within the merged frequency band The amplitude of each frequency component, Indicates the first frequency band within the merged frequency band The frequency values of each frequency component are calculated. The complex amplitude value corresponding to the final dominant frequency is read from the spectral distribution information. The magnitude of the complex amplitude value is taken as the dominant frequency amplitude, and the argument of the complex amplitude value is taken as the dominant frequency phase.
[0030] An envelope detection operation is performed on the second multi-source data stream to obtain the signal envelope of the second multi-source data stream. The envelope detection operation is implemented using Hilbert transform. The amplitude of each sampling point in the second multi-source data stream is subjected to Hilbert transform to obtain the corresponding analytic signal. The modulus value of the analytic signal is taken to obtain the envelope value of the corresponding sampling time on the signal envelope line. The envelope values of each sampling time are connected to form the signal envelope line.
[0031] The envelope slope of the signal envelope is calculated within a preset time interval, which is the time interval corresponding to the decrease in signal energy from its peak value to one-third of the peak energy in the second multi-source data stream. Within the preset time interval, the natural logarithm of the envelope value of the signal envelope is taken to obtain a logarithmic envelope sequence. The logarithmic envelope sequence is then fitted using the least squares method to obtain a fitted straight line, and the slope of the fitted straight line is the envelope slope. The energy attenuation coefficient of the second multi-source data stream is determined based on the envelope slope, and the energy attenuation coefficient is taken as the absolute value of the envelope slope.
[0032] The main frequency amplitude, main frequency phase, and energy attenuation coefficient are combined into dynamic response parameters. The combination method is to store the main frequency amplitude, main frequency phase, and energy attenuation coefficient sequentially into a preset data structure. The data structure contains three consecutive storage units, which respectively store the floating-point representation of the main frequency amplitude, the floating-point representation of the main frequency phase, and the floating-point representation of the energy attenuation coefficient, thus forming the dynamic response parameters.
[0033] In specific implementation, please refer to Figure 2 The process involves obtaining the generated data morphology feature sequence, which consists of alternating segment start and end markers arranged chronologically. The segment start and end markers are parsed. The segment start timestamp value is extracted from the binary encoding of the segment start marker and converted into a time offset relative to the start time of the second multi-source data stream. This time offset is the difference between the segment start timestamp value and the start timestamp value of the second multi-source data stream, and is used as the starting time position. Similarly, the segment end timestamp value is extracted from the binary encoding of the segment end marker and converted into a time offset relative to the start time of the second multi-source data stream, and is used as the ending time position.
[0034] Using each starting time position as the extraction start point and the next segment ending time position adjacent to the segment start mark as the extraction end point, candidate data segments are extracted from the second multi-source data stream. The extraction operation starts from the sampling point in the second multi-source data stream where the time offset is equal to the starting time position corresponding to the extraction start point, including that sampling point, and ends at the sampling point in the second multi-source data stream where the time offset is equal to the ending time position corresponding to the extraction end point, excluding that sampling point. All consecutive sampling points between the starting and ending sampling points are extracted as candidate data segments.
[0035] Determine if the length of the candidate data segment is less than a preset length threshold. The data length is the number of sampling points contained in the candidate data segment. The preset length threshold is set as the product of the sampling frequency of the second sensor group and the duration of the minimum effective signal. The duration of the minimum effective signal is the minimum time segment length in the second multi-source data stream that can reflect the complete signal pattern. This minimum time segment length is determined based on the response characteristics of the second sensor group. When the second sensor group is an acceleration sensor group, the duration of the minimum effective signal is determined by the minimum resonance period of the acceleration sensor group. The minimum resonance period is the reciprocal of the resonant frequency given in the factory parameters of the acceleration sensor group. The formula for calculating the preset length threshold is:
[0036] in, This indicates the preset length threshold, expressed in units of the number of sampling points. This indicates the sampling frequency of the second sensor group, measured in Hertz. Indicates the minimum effective signal duration, in seconds. The value is the reciprocal of the resonant frequency of the accelerometer sensor array, which is obtained from the technical specifications of the accelerometer sensor array. Read from the configuration parameters of the second sensor group.
[0037] If the length of a candidate data segment is less than a preset length threshold, the candidate data segment is concatenated with the next adjacent candidate data segment to obtain a concatenated data segment. The next adjacent candidate data segment refers to the candidate data segment that immediately follows the current candidate data segment on the time axis of the second multi-source data stream, and is truncated by the next segment start flag and the subsequent segment end flag. The concatenation operation places the last sample point of the previous candidate data segment and the first sample point of the next adjacent candidate data segment consecutively, merging them into a continuous data sequence. The number of sample points in the concatenated data segment is the sum of the number of sample points in the two candidate data segments. If the length of the concatenated data segment is still less than the preset length threshold, the concatenated data segment is continued to be concatenated with the next adjacent candidate data segment until the length of the concatenated data segment reaches the preset length threshold or there are no unused adjacent candidate data segments.
[0038] If the length of the candidate data segment is not less than the preset length threshold, the splicing operation will not be performed.
[0039] All spliced data segments, whether spliced or candidate data segments without splicing and with a length not less than a preset threshold, are uniformly marked as data segments to be fused. The marking method involves appending a segment identifier to the beginning of each data segment to be fused. This segment identifier contains the number of sampling points in the data segment and the original corresponding segment start flag.
[0040] See Figure 4 In the graph, the horizontal axis represents time in seconds (s), and the vertical axis represents amplitude in volts (V). The gray solid line represents the amplitude variation curve of the second multi-source data stream over time. Blue and red dashed lines mark the start and end points of segments, respectively, with these markers alternating to form several candidate data segments. Green-filled areas represent data segments to be merged, all with lengths greater than or equal to a preset length threshold; orange-filled areas represent spliced data segments, with lengths less than the preset length threshold but meeting the length requirement after splicing with adjacent data segments.
[0041] The first green area in the diagram begins at approximately 0.2 seconds (the blue dashed line, the segment start marker) and ends at approximately 0.8 seconds (the red dashed line, the segment end marker), corresponding to a complete data segment to be fused. The amplitude exhibits a trend of initial high-amplitude oscillations gradually decaying to noise levels. The second green area begins at approximately 1.2 seconds (the segment start marker) and ends at approximately 2 seconds (the segment end marker). Its amplitude is relatively large and decays slowly, indicating the signal energy attenuation process. The third orange-filled area is formed by splicing two adjacent candidate data segments, starting at approximately 2.4 seconds (the segment start marker) and ending at approximately 2.5 seconds (the segment end marker). After splicing, the length threshold requirement is met, and the amplitude exhibits a rapid oscillating decay trend.
[0042] In specific implementation, please refer to Figure 3 The generated dynamic response parameters are obtained, including the dominant frequency amplitude, dominant frequency phase, and energy attenuation coefficient. The dominant frequency phase is extracted from the dynamic response parameters and mapped to the start time of the fusion time window to obtain the starting phase value. The mapping process is as follows: the time value corresponding to the start time of the fusion time window is obtained. The fusion time window is a preset fixed-length time interval, the length of which is determined by the target time resolution of the multi-source data fusion result. The start time of the fusion time window is aligned with the start sampling time of the data segment to be fused in the second multi-source data stream. According to the phase propagation relationship of the dominant frequency component to which the dominant frequency phase belongs in the second multi-source data stream, the phase propagation relationship is described by the product of the dominant frequency and time. The product of the dominant frequency phase and the dominant frequency multiplied by the time offset of the start time of the fusion time window relative to the initial zero time of the second multi-source data stream is added together, and the result is modulo 2π to obtain the starting phase value.
[0043] The energy attenuation coefficient is extracted from the dynamic response parameters, and the phase compensation factor is calculated for each sampling moment within the fusion time window based on the energy attenuation coefficient. The sampling moments within the fusion time window are set to zero at the start of the fusion time window and increment according to a preset time step. The time step is the length of the fusion time window divided by the total number of sampling points within the fusion time window minus one. The total number of sampling points within the fusion time window is set based on the target sampling rate of the multi-source data fusion result. The phase compensation factor has a linear relationship with the energy attenuation coefficient and the temporal position of the sampling moment within the fusion time window. The formula for calculating the phase compensation factor is:
[0044] in, Indicates the first fusion time window Phase compensation factor at each sampling time. This refers to the index number of the sampling time within the fusion time window. The value range is an integer from 0 to the total number of sampling points within the fusion time window minus one. This represents the energy decay coefficient, which is obtained by taking the absolute value of the envelope slope. This represents the time step between adjacent sampling moments within the fusion time window. The value of the time step is the length of the fusion time window divided by the total number of sampling points within the fusion time window minus one.
[0045] Based on the initial phase value and the phase compensation factor at each sampling time, the instantaneous phase prediction value at each sampling time is calculated. The initial phase value is then added to the phase compensation factor to obtain the instantaneous phase prediction value.
[0046] The actual phase value is obtained at each sampling moment in the data segment to be fused. The method for obtaining the actual phase value is as follows: perform Hilbert transform on each sampling point in the data segment to be fused to obtain the corresponding analytical signal, extract the argument from the analytical signal as the actual phase value, and perform phase unwrapping processing on the actual phase value to eliminate phase jump.
[0047] Calculate the difference between the actual phase value and the instantaneous phase prediction value at each sampling moment in the data segment to be fused, and take the absolute value of the difference to obtain the phase deviation at each sampling moment.
[0048] The reciprocal of the phase deviation at each sampling moment is used as the phase matching coefficient for that sampling moment. When the phase deviation is zero, the phase matching coefficient is set to a preset value, which is determined based on the upper limit of the precision of the floating-point representation and is 10 to the power of 6.
[0049] The time-domain stretching factor is calculated for each sampling time based on the phase matching coefficient. The relationship between the time-domain stretching factor and the phase matching coefficient is set to inversely proportional; the value of the time-domain stretching factor is obtained by dividing a constant 1 by the phase matching coefficient.
[0050] Based on the temporal stretching factor at each sampling time, the new time coordinates of each sampling point in the data segment to be fused are determined. The original time coordinates of each sampling point in the data segment to be fused are represented with the starting time of the fusion time window as zero. The new time coordinates are obtained by multiplying the original time coordinates of each sampling point by the corresponding temporal stretching factor.
[0051] According to the new time coordinates, a resampling operation is performed on the data segment to be fused. The resampling operation is implemented using linear interpolation. On a preset uniform target time grid, the target time grid consists of equally spaced sampling moments within the fusion time window. The interval of the target time grid is consistent with the time step between adjacent sampling moments within the fusion time window. For each target time point on the target time grid, the two nearest sampling points before and after that target time point in the new time coordinates are found. Based on the new time coordinates and sampling amplitudes of the two sampling points before and after that target time point, the amplitude of the target time point is calculated using a linear interpolation formula to obtain the resampled data segment.
[0052] The test checks whether the number of sampling points in the resampled data segment is equal to the number of standard sampling points within the fusion time window. The number of standard sampling points is the total number of sampling points within the fusion time window.
[0053] If the number of sampling points in the resampled data segment is not equal to the number of standard sampling points, then interpolation or decimation operations are performed on the resampled data segment. When the number of sampling points in the resampled data segment is less than the number of standard sampling points, new sampling points are inserted at equal intervals between adjacent sampling points in the resampled data segment. The amplitude of the inserted sampling points is calculated by linear interpolation of the amplitudes of adjacent sampling points. The number of inserted sampling points is equal to the number of standard sampling points minus the number of sampling points in the resampled data segment. When the number of sampling points in the resampled data segment is greater than the number of standard sampling points, redundant sampling points are uniformly deleted from the resampled data segment. The number of deleted sampling points is equal to the number of sampling points in the resampled data segment minus the number of standard sampling points. The remaining sequence of sampling points after deletion is used as the aligned fused data segment.
[0054] In the specific implementation, the generated aligned and fused data segments are obtained. Each aligned and fused data segment retains its correspondence with the segment start marker in the data morphology feature sequence during generation. The order of the segment start markers in the data morphology feature sequence corresponding to the aligned and fused data segments is determined. The order is the position of the segment start markers in the data morphology feature sequence when arranged in ascending order of timestamp values. All segment start markers in the data morphology feature sequence are extracted and sorted in ascending order according to the size of the segment start timestamp value. The position number corresponding to each segment start marker after sorting is the arrangement order of that segment start marker. The aligned and fused data segments are associated with the arrangement order of the segment start markers. The association method is to match the original segment start markers corresponding to the aligned and fused data segments with the sorted segment start markers. After a successful match, the aligned and fused data segments are assigned the corresponding arrangement order number.
[0055] Based on the arrangement order, the starting address for writing the aligned fused data segments in the fused data buffer is determined. The fused data buffer is a pre-defined contiguous address storage space. The total capacity of the fused data buffer is set according to the sum of the data lengths of all aligned fused data segments. The fused data buffer is divided into multiple storage blocks, each corresponding to a specific arrangement order position. The size of each storage block is set according to the largest data length among all aligned fused data segments. The size of each storage block is a pre-defined fixed value, which is 1.2 times the maximum data length, rounded up to the nearest power of 2. After determining the storage block size, the starting address for writing is calculated. The starting address for writing is calculated as: the base address of the fused data buffer plus the arrangement order number minus one multiplied by the offset of the storage block size.
[0056] The formula for calculating the starting address to be written is:
[0057] in, Indicates the order number. The write start address corresponding to the aligned and merged data segment. This represents the sequence number, which ranges from 1 to an integer equal to the total number of starting markers in the data morphology feature sequence. This represents the base address of the fused data cache, which is the physical address of the starting location of the fused data cache in system memory. The size of the storage block is determined by: counting the length of all aligned and merged data segments, taking the maximum value of the length of all aligned and merged data segments, multiplying the maximum value by 1.2, rounding the product up to the nearest power of 2, and using the power of 2 as the value that satisfies the power of 2 condition.
[0058] Based on the data length of the aligned fused data segment, the write termination address of the aligned fused data segment in the fused data buffer is determined. The data length of the aligned fused data segment is the number of sampling points contained in the aligned fused data segment. Each sampling point occupies one storage unit in the fused data buffer, and the size of the storage unit is determined according to the data type of the sampling point amplitude. The write termination address is the address value obtained by adding the data length of the aligned fused data segment to the write start address and subtracting one. The write termination address is calculated by adding the data length of the aligned fused data segment to the write start address and multiplying it by the size of a single storage unit.
[0059] The aligned and fused data segments are written to the fused data buffer according to the start and end addresses. The write operation begins at the memory location in the fused data buffer where the start address is equal to the write address. The amplitude of the first sample point in the aligned and fused data segment is stored in the memory location pointed to by the start address. The amplitudes of subsequent sample points in the aligned and fused data segment are then stored in memory locations with progressively increasing addresses, until the amplitude of the last sample point in the aligned and fused data segment is stored in the memory location where the end address is equal to the write address. If the length of the aligned and fused data segment is less than the size of the memory block, the memory locations from the end address to the end of the current memory block retain their initial state, which is filled with zero values.
[0060] After all aligned fused data segments have been written, the data in the fused data buffer is output as the multi-source data fusion result. The condition for detecting completion of the write operation is that all aligned fused data segments corresponding to the permutation sequence number have been written. The output operation reads the sample point amplitude values stored in each memory cell in the fused data buffer, starting from the base address and proceeding in ascending address order. The read sample point amplitude values are then sequentially combined into a continuous data sequence, which is the multi-source data fusion result.
[0061] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. A multi-source data fusion method based on a dual-model approach, characterized in that, The method includes: Acquire the first multi-source data stream collected by the first sensor group, and acquire the second multi-source data stream collected by the second sensor group; Using the first multi-source data stream as input, a signal separation operation based on data morphology features is performed through a preset first signal processing model to obtain the data morphology feature sequence corresponding to the first multi-source data stream. Using the second multi-source data stream as input, a dynamic response operation based on signal oscillation characteristics is performed through a preset second signal processing model to obtain the dynamic response parameters corresponding to the second multi-source data stream; Based on the data morphology feature sequence, extract data segments corresponding to the data morphology feature sequence from the second multi-source data stream, and mark the data segments as data segments to be fused. Based on the dynamic response parameters, the phase matching coefficients of the data segment to be fused within the fusion time window are generated; Based on the phase matching coefficient, the data segment to be fused is time-domain aligned to obtain the aligned fused data segment; The aligned fused data segments are sorted according to the data morphology feature sequence and sequentially written into a preset fused data buffer to obtain the multi-source data fusion result.
2. The multi-source data fusion method based on a dual-model according to claim 1, characterized in that, Using the first multi-source data stream as input, a signal separation operation based on data morphology features is performed through a preset first signal processing model to obtain a data morphology feature sequence corresponding to the first multi-source data stream, including: Perform a sliding window sampling operation on the first multi-source data stream to obtain multiple first data windows; Perform extreme point detection on each of the first data windows to obtain the set of window extreme points corresponding to each of the first data windows; The data oscillation period of each first data window is determined based on the set of window extreme points corresponding to each first data window; Based on the data oscillation period of each first data window, the first multi-source data stream is segmented to obtain multiple data form segments; Extract the start and end markers of each data form segment in chronological order, and combine the start and end markers into the data form feature sequence.
3. The multi-source data fusion method based on a dual-model according to claim 2, characterized in that, The step of determining the data oscillation period of each first data window based on the set of window extreme points corresponding to each first data window includes: Target extreme points with amplitudes greater than a preset amplitude threshold are selected from the set of extreme points in the window to obtain a sequence of target extreme points; Calculate the sampling interval between two adjacent target extreme points in the target extreme point sequence to obtain multiple candidate interval values; The candidate interval value that appears most frequently among the multiple candidate interval values is determined as the initial oscillation period of the first data window; The initial oscillation periods are arranged according to their time positions within the first data window to obtain an oscillation period sequence; A median filtering operation is performed on the oscillation period sequence, and the filtering result is used as the data oscillation period.
4. The multi-source data fusion method based on a dual-model according to claim 2, characterized in that, The step of extracting the segment start flag and segment end flag of each data form segment in chronological order, and combining the segment start flag and segment end flag into the data form feature sequence includes: Record the sampling time of the first sampling point of each data form segment as the segment start timestamp, and convert the segment start timestamp into a segment start flag; The sampling time of the last sampling point of each data form segment is recorded as the segment termination timestamp, and the segment termination timestamp is converted into a segment termination flag. Arrange the segment start flag and the segment end flag sequentially according to the time order of the corresponding data form segments to obtain the initial morphological feature sequence; Detect whether the time interval between adjacent segment termination markers and segment start markers in the initial morphological feature sequence is less than a preset time threshold; If the time is less than the preset time threshold, then the adjacent segment termination flag and segment start flag are deleted, and the remaining flag sequence after deletion is used as the data morphology feature sequence.
5. The multi-source data fusion method based on a dual-model according to claim 2, characterized in that, Using the second multi-source data stream as input, a dynamic response operation based on signal oscillation characteristics is performed through a preset second signal processing model to obtain the dynamic response parameters corresponding to the second multi-source data stream, including: Perform a spectrum transformation operation on the second multi-source data stream to obtain the spectrum distribution information of the second multi-source data stream; Extract the dominant frequency component from the spectrum distribution information, and record the dominant frequency amplitude and dominant frequency phase corresponding to the dominant frequency component; Perform an envelope detection operation on the second multi-source data stream to obtain the signal envelope of the second multi-source data stream; Calculate the envelope slope of the signal envelope within a preset time interval, and determine the energy attenuation coefficient of the second multi-source data stream based on the envelope slope; The main frequency amplitude, the main frequency phase, and the energy attenuation coefficient are combined to form the dynamic response parameters.
6. The multi-source data fusion method based on a dual-model according to claim 5, characterized in that, Extracting the dominant frequency component from the spectral distribution information and recording the dominant frequency amplitude and phase corresponding to the dominant frequency component, including: Select the frequency component with the largest amplitude from the spectrum distribution information as the initial dominant frequency component; Obtain the initial frequency value corresponding to the initial main frequency component, and merge the initial frequency value with other frequency components within a preset frequency tolerance range to obtain the merged frequency band; Calculate the total energy value within the combined frequency band, and determine the center frequency corresponding to the total energy value as the final main frequency. The amplitude corresponding to the final main frequency is read from the spectrum distribution information and used as the main frequency amplitude; The phase value corresponding to the final main frequency is read from the spectrum distribution information and used as the main frequency phase.
7. The multi-source data fusion method based on a dual-model according to claim 1, characterized in that, Based on the data morphology feature sequence, data segments corresponding to the data morphology feature sequence are extracted from the second multi-source data stream, and the data segments are marked as data segments to be fused, including: Analyze the segment start flag and segment end flag in the data morphology feature sequence to obtain the start time position corresponding to each segment start flag and the end time position corresponding to each segment end flag. Using the start time position as the extraction starting point and the end time position as the extraction ending point, candidate data segments are extracted from the second multi-source data stream; Determine whether the length of the candidate data segment is less than a preset length threshold; If the length is less than the preset length threshold, the candidate data segment is spliced with the next adjacent candidate data segment to obtain a spliced data segment. The spliced data segment or the candidate data segment is used as the data segment to be fused.
8. The multi-source data fusion method based on a dual-model according to claim 1, characterized in that, Based on the dynamic response parameters, the phase matching coefficients of the data segment to be fused within the fusion time window are generated, including: The dominant frequency phase is extracted from the dynamic response parameters, and the dominant frequency phase is mapped to the start time of the fusion time window to obtain the starting phase value; The energy attenuation coefficient is extracted from the dynamic response parameters, and the phase compensation factor at each sampling moment within the fusion time window is calculated based on the energy attenuation coefficient. Based on the initial phase value and the phase compensation factor at each sampling time, calculate the instantaneous phase prediction value at each sampling time; The difference between the actual phase value and the instantaneous phase prediction value at each sampling moment in the data segment to be fused is calculated to obtain the phase deviation at each sampling moment; The reciprocal of the phase deviation at each sampling moment is used as the phase matching coefficient at that sampling moment.
9. The multi-source data fusion method based on a dual-model according to claim 8, characterized in that, Based on the phase matching coefficient, a time-domain alignment operation is performed on the data segment to be fused to obtain the aligned fused data segment, including: Calculate the time-domain stretching factor for each sampling time based on the phase matching coefficient at each sampling time. Based on the time-domain stretching factor, determine the new time coordinates of each sampling point in the data segment to be fused; According to the new time coordinates, a resampling operation is performed on the data segment to be fused to obtain a resampled data segment; Detect whether the number of sampling points in the resampled data segment is equal to the number of standard sampling points within the fusion time window; If the number of samples is not equal to the number of standard sampling points, then interpolation or extraction is performed on the resampled data segment to obtain the aligned fused data segment.
10. The multi-source data fusion method based on a dual-model according to claim 1, characterized in that, The aligned fused data segments are sorted according to the data morphological feature sequence and sequentially written into a preset fused data buffer to obtain multi-source data fusion results, including: Determine the order of the segment start markers in the data morphology feature sequence corresponding to the aligned fused data segment; According to the aforementioned arrangement order, determine the starting address for writing the aligned fused data segment in the fused data buffer. Based on the data length of the aligned fused data segment, determine the write termination address of the aligned fused data segment in the fused data buffer; The aligned fused data segment is written into the fused data buffer according to the write start address and the write end address; After all the aligned and fused data segments have been written, the data in the fused data buffer is output as the multi-source data fusion result.
