A method and computer device for transient analysis of very large amounts of data

By using segmentation and splicing methods to perform transient analysis on ultra-large data volumes, the problem of insufficient computing resources in existing technologies is solved, phase-continuous data processing is achieved, and it is suitable for transient analysis of ultra-large data volumes.

CN118734056BActive Publication Date: 2024-11-05成都玖锦科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411181016.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-05
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

Existing technologies cannot effectively handle transient analysis of massive amounts of data, leading to insufficient computing resources, potential program crashes, and an inability to perform phase continuity analysis under high sampling rates and long measurement times.

Method used

The total number of sampling points is divided into segments for transient analysis, and the data is spliced ​​together after each segment analysis. Phase continuity is ensured through downsampling, and frequency domain filters and downsampling techniques are used to address the problem of inconsistent data lengths.

Benefits of technology

It enables transient analysis of massive amounts of data, reduces memory usage, ensures phase continuity, does not affect subsequent demodulation, and can handle signals with large sampling rates and long measurement times.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118734056B_ABST
    Figure CN118734056B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of data transient analysis, and specifically to a method and computer device for transient analysis of very large data volumes. It is capable of performing transient analysis on very large data volumes, i.e., large sampling rate and long measurement time conditions, to obtain all output parameters, and can ensure phase continuity at the same time. The method of transient analysis of very large data volumes of the present invention comprises: segmenting the total number of sampling points, calculating segmentation information, performing transient analysis on the current segment data to obtain the current segment output parameters, adding 1 to the last signal end point detected in the current segment to store the segment tail data, and splicing it with the next segment signal data, performing transient analysis on the next segment using the spliced ​​data to obtain the next segment output parameters, splicing the current segment output parameters with the next segment output parameters together to obtain two segments of total output parameters, until splicing to the last segment data, and finally obtaining the total output parameters of the total number of sampling points transient analysis. The present invention is suitable for transient analysis of very large data volumes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data transient analysis, and in particular to a method and a computer device for transient analysis of very large amounts of data. Background Art

[0002] Transient analysis is usually used to analyze signals that change briefly or rapidly in time or frequency, such as linear frequency modulation signals, frequency hopping signals, etc. While ensuring that each measurement result is gapless, it can analyze time-varying and frequency-varying signals with large bandwidths, measure frequency hopping in signals, and analyze characteristic parameters.

[0003] However, due to the limitation of computing resources and computer memory, transient analysis can only process data of a certain length and cannot perform transient analysis on extremely large amounts of data. Forcing transient analysis on extremely large amounts of data may cause serious consequences such as program crashes. In other words, current transient analysis technology cannot handle situations with high sampling rates and long measurement times. Currently, there is little research on transient analysis of extremely large amounts of data, and how to analyze them while ensuring phase continuity has become a difficult problem. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a very large data volume transient analysis method and a computer device, which can perform transient analysis on very large data volumes, i.e., large sampling rate and long measurement time, to obtain all output parameters while ensuring phase continuity.

[0005] The present invention adopts the following technical solutions to achieve the above-mentioned purpose. In a first aspect, the present invention provides a method for transient analysis of very large data volumes, comprising:

[0006] The total number of sampling points is segmented, the segmentation information is calculated, and the transient analysis is performed on the current segment data to obtain the current segment output parameters. The last signal end point detected in the current segment is added by 1 and stored in the segment tail data, and spliced ​​with the next segment signal data. The next segment uses the spliced ​​data to perform transient analysis to obtain the next segment output parameters. The current segment output parameters and the next segment output parameters are spliced ​​together to obtain the total output parameters of the two segments, until the last segment data is spliced, and finally the total output parameters of the transient analysis of the total number of sampling points are obtained.

[0007] Furthermore, the total number of sampling points is segmented, and the segmentation information is calculated specifically including:

[0008] S21. Determine the total number of sampling points required TotalDataLen :

[0009]

[0010] In the formula, is the total number of sampling points; is the sampling rate; To measure time; is the filter coefficient length; To analyze bandwidth; To measure bandwidth;

[0011] S22, determine the maximum data length of each segment MaxSegLen ;

[0012] S23, determine the length of data that needs to be repeatedly taken for each segment ExtSegLen , data length ExtSegLen Frequency domain filter coefficient length by regional analysis The number of points lost when calculating the signal frequency modulation slope or frequency difference SigDiffProcLen The composition is calculated as follows: , is the filter coefficient length;

[0013] S24. Calculate the number of segments SegNum , according to the maximum data length of each segment MaxSegLen , total number of sampling points TotalDataLen And the length of data that needs to be repeatedly retrieved for each segment ExtSegLen Calculate the number of segments SegNum , calculated as:

[0014] ;

[0015] S25, calculate the starting point array of each segment NSegSrtIdx And the length array of each data segment NSegDataLen ,like , then the starting point and data length of the first segment of data are:

[0016] ;

[0017] like , then the starting point and data length of the nth segment of data are:

[0018]

[0019] In the formula, is the starting point of the total sampling data, , is the starting point of the time gate, is the sampling rate.

[0020] Furthermore, performing transient analysis on the current segment data to obtain the current segment output parameter, and adding 1 to the last signal end point detected in the current segment to store the segment end data specifically includes:

[0021] After transient analysis of each segment of data, retain the last signal end point detected in the segment plus 1 to the segment end data LastSegLastDataPri , and splice it with the next segment of data, and then perform the next segment of transient analysis on the spliced ​​data;

[0022] If the length of the current segment plus the next segment is greater than the set length, the current segment and the next segment are downsampled, and the data after downsampling of the current segment is stored as LastSegLastData ;

[0023] If the current segment is not the last segment, and the calculated length of the spliced ​​data exceeds the set length, downsampling is performed and the flag bit indicating whether the next segment is sampled is set. IsCurrSegDownSampling Set to 1, the sampling times are increased by 1, the sampling rate of the current segment data is halved, and the half-band filter is used to retain the data in the current segment. LastSegLastDataPri Filter, then extract the filtered data and save it as LastSegLastData For splicing with the next section of data;

[0024] If the current segment is not the last segment, and the calculated length of the spliced ​​data does not exceed the set length, but the current segment has been downsampled, the flag bit indicating whether the next segment is sampled is set. IsCurrSegDownSampling Set to 1 to make the sampling rates of the two segments of data consistent;

[0025] If the current segment data is the last segment, or the current segment data is not the last segment, but the calculated spliced ​​data length does not exceed the set length, and the current segment has not been downsampled, no processing is performed.

[0026] Furthermore, data splicing specifically includes:

[0027] will store LastSegLastData The data is spliced ​​with the current segment data;

[0028] If the current segment data is not the first segment, and the flag is IsCurrSegDownSampling If it is 1, the current segment data is first downsampled until the sampling rate of this segment is the same as that of the previous segment, and then the stored LastSegLastData It is concatenated with the downsampled data of the current segment and the index at this time is recorded;

[0029] If the current segment data is not the first segment, and the flag is IsCurrSegDownSampling If it is 0, the stored LastSegLastData It is concatenated with the downsampled data of the current segment and the index at this time is recorded;

[0030] If the current segment of data is the first segment, record the index.

[0031] In a second aspect, the present invention provides a computer device, including a memory, wherein the memory stores program instructions, and when the program instructions are run, the above-mentioned method for transient analysis of extremely large amounts of data is executed.

[0032] The beneficial effects of the present invention are:

[0033] The present invention can reduce memory space occupation, can perform transient analysis on extremely large data volumes, that is, large sampling rates and long measurement times, to obtain all output parameters, and can simultaneously ensure phase continuity without affecting subsequent demodulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of a method for transient analysis of very large amounts of data provided by an embodiment of the present invention;

[0035] Figure 2 is a schematic diagram of effective data provided by an embodiment of the present invention;

[0036] Figure 3 It is a segmented schematic diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0038] The present invention provides a method for analyzing transient state of extremely large data volume, such as Figure 1 As shown, specifically including:

[0039] S1, segment the total number of sampling points and calculate segment information;

[0040] Determine the maximum length of each segment MaxSegLen This value is determined based on system memory, algorithm operation rate and other factors, and can be set to 2 million.

[0041] Determine the total number of sampling points required TotalDataLen At this time, the relationship between the analysis bandwidth and the measurement bandwidth during regional analysis should be considered:

[0042]

[0043] In the formula, is the total number of sampling points; is the sampling rate; To measure time; is the filter coefficient length; To analyze bandwidth; To measure bandwidth.

[0044] The present invention uses the filter function in Matlab to perform frequency domain filtering, so the effective data The number of invalid data points before and after is ,like Figure 2 shown.

[0045] Determine the length of data that needs to be repeatedly retrieved for each segment ExtSegLen This value is mainly determined by the length of the frequency domain filter coefficients in the regional analysis The number of points lost by differential calculation when calculating the signal frequency modulation slope (linear frequency modulation signal) or frequency (frequency hopping signal) SigDiffProcLen composition.

[0046]

[0047] In the formula, if it is a linear frequency modulation signal, is 2, if it is a frequency hopping signal, is 1, is the filter coefficient length, such as Figure 3 This processing makes the phase continuous and does not affect the transient analysis.

[0048] Calculate the number of segments SegNum . According to the maximum data length of each segment MaxSegLen , total number of sampling points TotalDataLen And the length of data that needs to be repeatedly retrieved for each segment ExtSegLen Calculated.

[0049] , where Indicates a round-up operation.

[0050] Calculate the starting point array for each segment NSegSrtIdx And the length array of each data segment NSegDataLen .

[0051] like , that is, there is only one segment, the starting point and data length of the first segment are:

[0052]

[0053] like , the starting point and data length of the nth segment of data are:

[0054]

[0055] In the formula, is the starting point of the total sampling data, ,

[0056] is the starting point of the time gate, is the sampling rate.

[0057] S2, current segment data storage;

[0058] After transient analysis of each segment of data, the last signal end point detected in the segment plus 1 will be retained to the end of the segment data. LastSegLastDataPri It is then spliced ​​with the next segment of data, and then the spliced ​​data is processed for the next segment of transient analysis. TotalLen Greater than the set length (which can be set to ), the current segment and the next segment need to be downsampled (half-band filtering + 2x decimation). The data of the current segment after downsampling is stored as LastSegLastData .

[0059] If the current segment of data is not the last segment, and the length of the spliced ​​data is calculated TotalLen If the length exceeds the set length, downsampling is required. IsCurrSegDownSampling Set to 1, the number of samples increases by 1, the sampling rate of the current segment is halved, and the half-band filter is used to retain data for the current segment. LastSegLastDataPri Filter, then decimate the filtered data by 2 times and save it as LastSegLastData For splicing with the next piece of data.

[0060] If the current segment is not the last segment, and the calculated concatenated data length TotalLen does not exceed the set length, but the current segment has been downsampled, the flag bit indicating whether the next segment is sampled needs to be set. IsCurrSegDownSampling Set to 1, indicating that the next segment needs to be downsampled to make the sampling rates of the two segments consistent.

[0061] If the current segment of data is the last segment, or the current segment of data is not the last segment, but the length of the spliced ​​data is calculated TotalLen If the length does not exceed the set length and the current segment has not been downsampled, no processing will be performed.

[0062] S3, segmented data splicing;

[0063] The previous segment stored in step S2 LastSegLastData The data is concatenated with the current segment data for subsequent signal processing.

[0064] If the current segment data is not the first segment, and the flag is IsCurrSegDownSampling If it is 1, it means that the current segment needs to be downsampled, and the downsampling process (half-band filtering + 2 times decimation) is performed until the sampling rate of this segment is the same as that of the previous segment, and then the previous segment stored in step S2 is LastSegLastData It is concatenated with the downsampled data of the current segment and the index at this time is recorded.

[0065] If the current segment data is not the first segment, and the flag is IsCurrSegDownSamplingIf it is 0, it means that the current segment does not need to be downsampled, and the previous segment stored in step S2 is directly LastSegLastData Join it with the current segment data and record the index at this time.

[0066] If the current segment of data is the first segment, just record the index.

[0067] S4, output parameter splicing.

[0068] Perform transient analysis on the current segment to obtain the current segment output parameters, and concatenate the previous segment output parameters with it to obtain all data output parameters.

[0069] The last segment of data is subjected to transient analysis, and the output parameters of the previous segment are concatenated with the output parameters of this segment to complete the transient analysis processing of extremely large amounts of data.

[0070] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A method for transient analysis of very large amounts of data, characterized in that: include: The total number of sampling points is segmented, the segmentation information is calculated, the current segment data is transiently analyzed to obtain the current segment output parameters, the last signal end point detected in the current segment is added by 1 and stored in the segment tail data, and spliced ​​with the next segment signal data, the next segment uses the spliced ​​data to perform transient analysis to obtain the next segment output parameters, the current segment output parameters and the next segment output parameters are spliced ​​together to obtain the two-segment total output parameters, until the last segment data is spliced, and finally the total output parameters of the transient analysis of the total number of sampling points are obtained; The total number of sampling points is divided into segments, and the segment information is calculated including: S21. Determine the total number of sampling points required: In the formula, is the total number of sampling points; is the sampling rate; To measure time; is the filter coefficient length; To analyze bandwidth; To measure bandwidth; S22, determine the maximum data length of each segment MaxSegLen ; S23, determine the length of data that needs to be repeatedly taken for each segment. The data length is composed of the length of the regional analysis frequency domain filter coefficient and the number of points for obtaining the signal frequency modulation slope or the frequency time difference loss. The calculation method is as follows: , is the filter coefficient length, SigDiffProcLen To obtain the signal frequency modulation slope or the number of points of frequency time difference loss, ExtSegLen is the data length; S24. Calculate the number of segments SegNum , according to the maximum data length of each segment MaxSegLen , total number of sampling points TotalDataLen And the length of data that needs to be repeatedly retrieved for each segment ExtSegLen Calculate the number of segments SegNum , calculated as: ; S25, calculate the starting point array of each segment NSegSrtIdx And the length array of each data segment NSegDataLen ,like , then the starting point and data length of the first segment of data are: ; like , then the starting point and data length of the nth segment of data are: In the formula, is the starting point of the total sampling data, , is the starting point of the time gate, is the sampling rate; Performing transient analysis on the current segment data to obtain the current segment output parameters, adding 1 to the last signal end point detected in the current segment to store the segment end data specifically includes: After transient analysis of each segment of data, retain the last signal end point detected in the segment plus 1 to the segment end data LastSegLastDataPri , and splice it with the next segment of data, and then perform the next segment of transient analysis on the spliced ​​data; If the length of the current segment plus the next segment is greater than the set length, the current segment and the next segment are downsampled, and the current segment after downsampling is stored as LastSegLastData ; If the current segment is not the last segment and the calculated length of the spliced ​​data exceeds the set length, downsampling is performed and the flag indicating whether the next segment is sampled is set to IsCurrSegDownSampling Set to 1, the sampling times are increased by 1, the sampling rate of the current segment data is halved, and the half-band filter is used to retain the data in the current segment. LastSegLastDataPri Filter, then extract the filtered data and save it as LastSegLastData For splicing with the next section of data; If the current segment is not the last segment, and the calculated length of the spliced ​​data does not exceed the set length, but the current segment has been downsampled, the flag bit indicating whether the next segment is sampled is set. IsCurrSegDownSampling Set to 1 to make the sampling rates of the two segments of data consistent; If the current segment data is the last segment, or the current segment data is not the last segment, but the calculated spliced ​​data length does not exceed the set length, and the current segment data has not been downsampled, no processing is performed.

2. The method for transient analysis of very large amounts of data according to claim 1, characterized in that: Data splicing specifically includes: will store LastSegLastData The data is spliced ​​with the current segment data; If the current segment data is not the first segment, and the flag is IsCurrSegDownSampling If the sampling rate is 1, downsampling is performed until the sampling rate of this segment of data is the same as that of the previous segment, and then the stored LastSegLastData It is concatenated with the downsampled data of the current segment and the index at this time is recorded; If the current segment data is not the first segment, and the flag is IsCurrSegDownSampling If it is 0, the stored LastSegLastData It is concatenated with the downsampled data of the current segment and the index at this time is recorded; If the current segment of data is the first segment, record the index.

3. A computer device, comprising a memory, wherein the memory stores program instructions, characterized in that: When the program instructions are executed, the very large data volume transient analysis method as described in any one of claims 1-2 is executed.