Time sequence data adaptive dynamic sampling method, device, equipment, medium and product

Through real-time sampling frequency adjustment based on the fitting model, the balance between accuracy and efficiency of traditional sampling methods is solved, and accurate and efficient sampling of high-frequency timing data is achieved, and it is suitable for industrial monitoring and environmental monitoring.

CN119087805APending Publication Date: 2024-12-06润电能源科学技术有限公司
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411189716.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The traditional fixed frequency sampling method cannot dynamically adapt to the characteristics of time sequence data, resulting in oversampling or insufficient sampling, affecting the accuracy of data analysis and decision-making. The existing adaptive sampling method has high computational complexity, making it difficult to calculate the sampling accuracy and efficiency. to achieve a balance between the two.

Method used

By training the fitting model based on historical time series data, generating fitting functions, adjusting the sampling frequency in real time, and dynamically adjusting the sampling strategy to achieve accurate and efficient sampling of high-frequency time series data.

Benefits of technology

With limited computing and storage resources, redundant sampling is significantly reduced, data sampling accuracy and representativeness are improved, resource consumption is reduced, and system performance and resource utilization efficiency are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119087805A_ABST
    Figure CN119087805A_ABST
Patent Text Reader

Abstract

The invention discloses a time sequence data adaptive dynamic sampling method, device and equipment, a medium and a product. The method comprises the following steps: training a selected target fitting model according to historical time sequence data to generate a fitting function; and adjusting the current sampling frequency in real time according to the fitting function, and sampling according to the current sampling frequency in real time to obtain sampling data. According to the technical scheme provided by the invention, the sampling strategy is dynamically adjusted based on data fitting, high-frequency and large-scale time sequence data can be accurately and efficiently sampled under the condition of limited calculation and storage resources, redundant sampling is reduced on the premise of ensuring the accuracy and representativeness of data sampling, and the sampling efficiency is improved. Therefore, the number of sampling points is remarkably reduced, the resource consumption of data calculation, storage and transmission is reduced, and the overall performance of the system and the resource utilization efficiency are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of data acquisition and processing, and in particular to a method, device, equipment, medium and product for adaptive dynamic sampling of time series data. Background Art

[0002] In modern industry, environmental monitoring, the Internet of Things and other fields, time series data collection is the basis for system monitoring, data analysis and intelligent decision-making. However, due to limitations in sensor sampling capabilities, data transmission bandwidth and storage space, it is impossible to perform high-frequency, continuous and full-volume sampling of time series data. At the same time, with the rapid growth of data volume and the increase in real-time processing requirements, traditional sampling methods face many challenges.

[0003] For example, the traditional fixed-frequency sampling method is widely used in time series data collection. This method samples at a predetermined fixed frequency regardless of the state of the data change. This one-size-fits-all sampling strategy cannot dynamically adapt to the characteristics of the data and may lead to oversampling or undersampling problems. Oversampling not only increases the burden of data storage and transmission, but also wastes computing resources, while undersampling may miss key data changes, thereby affecting the accuracy of data analysis and decision-making.

[0004] In the face of the problems faced by fixed-frequency sampling methods, research on adaptive sampling methods has gradually begun, in an attempt to cope with data changes by adjusting the sampling frequency. However, the current adaptive sampling methods need to adjust the sampling strategy by analyzing data characteristics in real time, which usually requires a lot of computing resources. Although the defects of the fixed-frequency sampling method have been solved to a certain extent, the computational complexity is high and the implementation is difficult. The sampling accuracy may still be insufficient when the data changes dramatically. Therefore, it is difficult for existing sampling methods to strike a balance between sampling accuracy and sampling efficiency. Summary of the invention

[0005] Embodiments of the present invention provide a method, device, equipment, medium and product for adaptive dynamic sampling of time series data to solve the trade-off problem between accuracy and efficiency in traditional sampling methods and improve system performance and resource utilization.

[0006] In a first aspect, an embodiment of the present invention provides a method for adaptive dynamic sampling of time series data, the method comprising:

[0007] The selected target fitting model is trained according to the historical time series data to generate a fitting function;

[0008] The current sampling frequency is adjusted in real time according to the fitting function, and sampling is performed in real time according to the current sampling frequency to obtain sampling data.

[0009] Optionally, adjusting the current sampling frequency in real time according to the fitting function includes:

[0010] Determining the current data change rate in real time according to the fitting function;

[0011] Update the current sampling frequency in real time according to the current data change rate.

[0012] Optionally, the real-time adjustment of the current sampling frequency according to the fitting function further includes:

[0013] The current sampling frequency is updated in real time according to the current fitting error of the fitting function.

[0014] Optionally, before training the selected target fitting model according to the historical time series data, the method further includes:

[0015] Based on the data characteristics of the historical time series data, the fitting accuracy of the fitting model and / or the available computing resources of the system, the target fitting model is determined according to the selected fitting models; the selected fitting models include at least one of a linear model, a polynomial model, an exponential model, a spline interpolation model and a machine learning model.

[0016] Optionally, the training of the selected target fitting model according to the historical time series data to generate a fitting function includes:

[0017] Segmenting the historical time series data based on time windows or system driven events;

[0018] According to each segment of data, the target fitting model corresponding to each segment is fit-trained separately to obtain the fitting function of each segment of data.

[0019] Optionally, after adjusting the current sampling frequency in real time according to the fitting function and sampling in real time according to the current sampling frequency to obtain the sampled data, the method further includes:

[0020] Compressing the sampled data using a lossy compression algorithm or a lossless compression algorithm according to the sampling frequency and / or importance level of the sampled data;

[0021] The compressed data is stored according to the preset data storage format.

[0022] In a second aspect, an embodiment of the present invention further provides a time series data adaptive dynamic sampling device, the device comprising:

[0023] A fitting function generation module is used to train the selected target fitting model according to the historical time series data to generate a fitting function;

[0024] The sampling frequency adjustment module is used to adjust the current sampling frequency in real time according to the fitting function, and to perform sampling in real time according to the current sampling frequency to obtain sampling data.

[0025] In a third aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

[0026] one or more processors;

[0027] A memory for storing one or more programs;

[0028] When the one or more programs are executed by the one or more processors, the one or more processors implement the time series data adaptive dynamic sampling method provided by any embodiment of the present invention.

[0029] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for adaptive dynamic sampling of time series data provided by any embodiment of the present invention.

[0030] In a fifth aspect, an embodiment of the present invention further provides a computer program product, which includes a computer program, and when the program is executed by a processor, the method for adaptive dynamic sampling of time series data provided by any embodiment of the present invention is implemented.

[0031] The embodiment of the present invention provides a method for adaptive dynamic sampling of time series data. First, the selected target fitting model is trained according to the historical time series data to generate a fitting function. Then, when sampling is required, the current sampling frequency is adjusted in real time according to the obtained fitting function, and sampling is performed in real time according to the current sampling frequency, thereby obtaining sampled data. The method for adaptive dynamic sampling of time series data provided by the embodiment of the present invention can achieve accurate and efficient sampling of high-frequency and large-scale time series data under limited computing and storage resources by dynamically adjusting the sampling strategy based on data fitting. Under the premise of ensuring the accuracy and representativeness of data sampling, redundant sampling is reduced, thereby significantly reducing the number of sampling points, reducing the resource consumption of data calculation, storage and transmission, and improving the overall performance and resource utilization efficiency of the system. Moreover, it has a wide range of applications and can be applied to a variety of application scenarios, especially in the fields of industrial monitoring and environmental monitoring that require high sampling accuracy and resource utilization efficiency, and the effect is particularly significant, so it has broad application prospects and important practical significance. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flowchart of a method for adaptive dynamic sampling of time series data provided in Embodiment 1 of the present invention;

[0033] Figure 2A schematic diagram of exemplary historical time series data and a fitting curve provided in the first embodiment of the present invention;

[0034] Figure 3 A schematic diagram of the structure of a time series data adaptive dynamic sampling device provided in Embodiment 2 of the present invention;

[0035] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0037] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the steps as sequential processes, many of the steps therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the steps can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0038] Embodiment 1

[0039] Figure 1 This is a flowchart of the method for adaptive dynamic sampling of time series data provided in the first embodiment of the present invention. This embodiment can be applied to data sampling in scenarios such as industrial monitoring, environmental monitoring, and the Internet of Things, where real-time tracking and analysis of data changes are required. The method can be executed by the adaptive dynamic sampling device for time series data provided in the embodiment of the present invention, which can be implemented by hardware and / or software and can generally be integrated into a computer device. Figure 1 As shown, the specific steps include:

[0040] S11. Train the selected target fitting model according to the historical time series data to generate a fitting function.

[0041] S12. Adjust the current sampling frequency in real time according to the fitting function, and perform sampling in real time according to the current sampling frequency to obtain sampling data.

[0042] Specifically, the data collected in various scenarios generally have a certain change pattern, so the fitting model can be used to analyze the change trend of the data. By constructing a mathematical model to capture the regularity in the data, the future trend of the time series data can be predicted in advance through the fitting model, and the sampling frequency can be adjusted adaptively in real time to reduce redundant sampling and improve the representativeness and accuracy of the sampling. Specifically, the pre-selected target fitting model can be fitted and trained according to the existing historical time series data to generate a fitting function for describing the data change trend, so as to guide the adjustment of the sampling frequency through the fitting function. When starting sampling, an initial sampling frequency can be preset based on experience, and then the sampling operation can be started according to the initial sampling frequency. During the sampling process, the sampling frequency can be adjusted at any time according to the fitting function, and the sampling can be performed in real time according to the latest current sampling frequency. The sampling points obtained in succession are the required sampling data, so as to ensure that the sampling data can accurately reflect the actual change trend of the time series data.

[0043] Optionally, the real-time adjustment of the current sampling frequency according to the fitting function includes: determining the current data change rate according to the fitting function in real time; updating the current sampling frequency according to the current data change rate in real time. Specifically, the current data change rate of the time series data can be obtained by calculating the absolute value of the derivative of the fitting function in real time. In areas where the data changes dramatically, that is, where the current data change rate is large, the sampling points can be increased to ensure that the details of the data changes are captured, which can be achieved by increasing the sampling frequency. In areas where the data changes slowly, that is, where the current data change rate is small, the sampling points can be reduced to avoid redundant sampling and save computing and storage resources, which can be achieved by reducing the sampling frequency. This is equivalent to adjusting the overall sampling point position to be more concentrated in areas where the data changes dramatically, while only a small number of samples are required as representatives in areas where the data changes slowly, thereby ensuring high-precision sampling in key areas and ensuring that every key change point can be collected.

[0044] Further optionally, the real-time adjustment of the current sampling frequency according to the fitting function also includes: updating the current sampling frequency in real time according to the current fitting error of the fitting function. Specifically, since there may be a certain deviation between the data change trend predicted by the fitting model and the actual situation of the time series data, the current sampling frequency can be further updated based on the current data change rate, combined with the current fitting error of the fitting function. Specifically, the fitting error between the predicted value and the actual value of the sampling point can be calculated after each sampling, and then the obtained fitting error can be compared with the preset error threshold. When the preset error threshold range is exceeded, the sampling points can be increased to improve the accuracy of the data, which can be achieved by increasing the sampling frequency. When it is within the preset error threshold range, the current sampling frequency can be kept unchanged, or the sampling frequency can be appropriately reduced to keep the sampling frequency in a suitable range and reduce the calculation and storage burden. In the entire sampling process, the sampling frequency used in the subsequent use can be adjusted in real time and dynamically by continuously monitoring the data change rate and fitting error, so as to ensure that the data changes can be responded to in time, the accuracy and representativeness of the sampled data can be guaranteed, the sampling accuracy and efficiency can be improved, and unnecessary sampling can be minimized.

[0045] On the basis of the above technical solution, optionally, before the selected target fitting model is trained according to the historical time series data, it also includes: determining the target fitting model according to the selected fitting model based on the data characteristics of the historical time series data, the fitting accuracy of the fitting model and / or the available computing resources of the system; the selected fitting model includes at least one of a linear model, a polynomial model, an exponential model, a spline interpolation model and a machine learning model.

[0046] Specifically, in the above-mentioned sampling process of time series data, the choice of fitting model directly determines the accuracy of data fitting, thereby affecting the effectiveness of subsequent sampling strategies. In terms of data characteristics, the complexity of time series data is the first factor to be considered when selecting a fitting model. For example, simple data changes can usually be fitted using a linear model, while complex, nonlinear changes require a more complex model to describe. The characteristics and application scenarios of various fitting models are as follows: Linear models are suitable for scenarios where data presents a linear trend, such as linear temperature changes, linearly growing sales data, etc. Linear models are simple and fast to calculate, and are suitable for data acquisition systems with high real-time requirements, but their limitation is that they can only capture linear relationships in the data and cannot describe more complex nonlinear changes; polynomial models can capture nonlinear trends in data by increasing the order of the polynomial, and can be applied to scenarios where there is curvature in data changes, such as fluctuating data in natural phenomena, seasonal sales data, etc. Increasing the order can improve the fitting ability, but it should be noted that too high an order may lead to overfitting, that is, the fitting model fits the training data too well and performs poorly in actual applications; exponential models are suitable for scenarios where data presents an exponential growth or decay trend, such as describing economic growth. , population growth, etc., the exponential model can capture the rapid changes of data very well, but the effect is average for slowly changing data; the spline interpolation model uses piecewise polynomials to fit the data, which is suitable for situations where the data has local characteristics, complex change trends and is difficult to fit with a single polynomial, such as mechanical vibration data, sensor data in complex environments, etc. The spline interpolation model can smoothly connect each segment of data to provide a more accurate fit; the machine learning model can process high-dimensional and complex data, and can capture deep-level patterns through training with a large amount of historical data. It is suitable for scenarios with high-precision requirements, such as financial market forecasts, fault prediction of complex equipment, etc. For high-dimensional or highly complex data, traditional statistical models may be difficult to process effectively. At this time, regression models, neural networks and other machine learning models can be selected for fitting. Then, the appropriate target fitting model can be selected based on the data characteristics of historical time series data.

[0047] Fitting accuracy is a core indicator for measuring the quality of fitting model selection, which can usually be measured by mean square error (MSE), mean absolute error (MAE), etc. Among them, mean square error refers to the average value of the sum of squares of the difference between the predicted value and the true value. It is suitable for scenarios that are sensitive to large errors. The smaller the MSE, the higher the accuracy of the fitting model. Mean absolute error refers to the average value of the sum of absolute values ​​of the difference between the predicted value and the true value. It can directly reflect the degree of deviation between the predicted value and the actual value. MAE is more commonly used in scenarios that are not sensitive to large errors. Fitting accuracy will directly affect the effect of subsequent sampling strategies. If the fitting accuracy is high, the main change trend of the data can be captured through fewer sampling points. On the contrary, if the fitting accuracy is low, more sampling points may be required to ensure the accuracy of the sampled data. You can select a suitable target fitting model based on the fitting accuracy of various fitting models. Specifically, you can evaluate the fitting accuracy of different models through cross-validation methods and select the model with the best performance as the target fitting model.

[0048] In terms of available computing resources in the system, different fitting models have different requirements for computing resources. Simple models such as linear models and low-order polynomial models have fast calculation speeds and occupy fewer resources, and are suitable for systems with high real-time requirements. Complex models such as high-order polynomial models, spline interpolation models, and machine learning models have high computational complexity and are suitable for scenarios with high accuracy requirements and sufficient computing resources. When selecting a fitting model, you need to make a trade-off based on the actual available computing resources of the system. If computing resources are limited, you can choose a simpler model to ensure real-time performance. If computing resources are sufficient, you can choose a complex model to improve fitting accuracy.

[0049] In practical applications, model selection is often a dynamic adjustment process. As data characteristics and system requirements change, the fitting model may need to be adjusted to adapt to the new environment. For example, in some complex scenarios, the above single model may not meet the needs. At this time, a combination of multiple models can be used, such as a collection of linear models and nonlinear models, and further, the weighted average of different models can be used. For example, in some systems with high real-time requirements, machine learning models can use online learning to continuously update the model as new data arrives to maintain fitting accuracy. Through scientific and reasonable model selection and adjustment strategies, the efficiency and accuracy of data sampling can be significantly improved, laying the foundation for the efficient operation of the system.

[0050] Specifically, we can first select a fitting model based on the overall trend and complexity of the historical time series data and the available computing resources of the system. For example, a linear model can be selected for a simple linear trend, a polynomial model or an exponential model can be selected for a nonlinear trend, and a spline interpolation model or a machine learning model can be selected for complex and changeable data. Then, we use the historical time series data to perform fitting training on the preliminarily selected fitting model and evaluate the fitting effect of the fitting model. If the accuracy is poor, we can replace the fitting model or adjust the model parameters until the expected fitting accuracy is achieved, thereby determining the final target fitting model to generate the required fitting function.

[0051] On the basis of the above technical solution, optionally, the selected target fitting model is trained according to the historical time series data to generate a fitting function, including: segmenting the historical time series data based on time windows or system driven events; and fitting training is performed separately on the target fitting model corresponding to each segment of the data to obtain the fitting function of each segment of the data.

[0052] Specifically, after the target fitting model is selected, the target fitting model can be trained using historical time series data. Since historical time series data may show different data characteristics in different time periods or different system event states, in order to improve the fitting accuracy, the historical time series data can be segmented based on the time window or system drive event (such as a switch), and the appropriate target fitting model can be selected for the corresponding segment, and then the corresponding target fitting model can be separately fitted and trained using each segment of data to obtain the fitting function of each segment of data. Each fitting function can describe the change trend of the segment of data and can also predict the possible changes of the same type of regional data in the future. After completing the fitting training, the fitting function and fitting accuracy of each segment of data can be saved to provide a basis for the subsequent sampling strategy adjustment. Correspondingly, in the actual sampling process, the current sampling frequency can be adjusted in real time according to the corresponding fitting function according to the current time period or the current system event state. After each application period, the fitting accuracy of the target fitting model used can be evaluated again based on the newly collected data, such as comparing it with the previously stored fitting accuracy. If the accuracy is poor, the target fitting model can be reselected and retrained using the newly collected data.

[0053] Before fitting training, specifically before selecting the target fitting model, the historical time series data may be preprocessed, such as data cleaning and standardization. Data cleaning can remove noise and outliers in the historical time series data to ensure data quality, and standardization can reduce the impact between different dimensions and improve fitting accuracy.

[0054] On the basis of the above technical solution, optionally, after adjusting the current sampling frequency in real time according to the fitting function and sampling in real time according to the current sampling frequency to obtain the sampling data, it also includes: compressing the sampling data according to the sampling frequency and / or importance level of the sampling data using a lossy compression algorithm or a lossless compression algorithm; and storing the compressed data according to a preset data storage format.

[0055] Specifically, on the basis of optimizing the sampling strategy, the overall performance of the system can be further improved by optimizing the storage strategy. After obtaining the sampled data, the sampled data can be stored in a standard preset data storage format for subsequent data analysis and processing, and the storage space can be optimized in combination with the compression algorithm. Specifically, a suitable compression algorithm can be selected for data compression in combination with the sampling frequency and importance level of the data. For example, a lossy compression algorithm can be used for redundant data segments (i.e., frequently sampled data segments) to maximize storage space savings while maintaining data accuracy, while a lossless compression algorithm can be used for key data segments to ensure data integrity.

[0056] On the basis of the above technical solutions, the effectiveness of this method can be verified through a series of experiments and performance comparisons. Specifically, the effectiveness of the sampling strategy and the fitting model can be verified through experiments in different application scenarios. For example, in industrial monitoring scenarios, the accuracy of the fitting model and the efficiency of the sampling strategy can be verified by collecting equipment operation status data. This method can also be compared with traditional fixed-frequency schemes and adaptive sampling schemes, such as comparing the sampling accuracy in the same data scenario to verify whether this method can better capture data changes, and evaluating the utilization efficiency of computing resources and storage resources to verify whether this method can reduce resource consumption while improving sampling accuracy. This embodiment uses an example of bearing temperature monitoring to verify the advantages of this method in improving data sampling accuracy, reducing storage requirements and optimizing resource utilization. The curves of historical time series data and the fitting function obtained using a polynomial model (the red curve in the figure, i.e., the curve in the middle) are shown in Figure 2. Figure 2 As shown. Based on the fitting function, the sampling process of subsequent time series data is optimized. Through simulation analysis, when processing 1 million data, the data compression rate is about 89.99% after dynamic sampling using this method. That is, the data volume can be reduced to about 10% of the original using this method. It can be seen that redundant data is significantly reduced, and efficient data compression is achieved, thereby greatly reducing the storage and transmission requirements. Such a high compression rate has a very obvious advantage when processing large-scale time series data. At the same time, the experimental results also show that this method also has a good performance in data sampling accuracy.

[0057] The technical solution provided by the embodiment of the present invention first trains the selected target fitting model according to the historical time series data to generate a fitting function, and then when sampling is required, the current sampling frequency is adjusted in real time according to the obtained fitting function, and sampling is performed in real time according to the current sampling frequency, thereby obtaining sampled data. By dynamically adjusting the sampling strategy based on data fitting, it is possible to accurately and efficiently sample high-frequency and large-scale time series data under limited computing and storage resources, and reduce redundant sampling while ensuring the accuracy and representativeness of data sampling, thereby significantly reducing the number of sampling points, reducing the resource consumption of data calculation, storage and transmission, and improving the overall performance of the system and resource utilization efficiency. Moreover, it has a wide range of applications and can be applied to a variety of application scenarios, especially in the fields of industrial monitoring and environmental monitoring that require high sampling accuracy and resource utilization efficiency, and the effect is particularly significant, so it has broad application prospects and important practical significance.

[0058] Embodiment 2

[0059] Figure 3 This is a schematic diagram of the structure of the time series data adaptive dynamic sampling device provided in the second embodiment of the present invention. The device can be implemented by hardware and / or software, and can generally be integrated into a computer device to execute the time series data adaptive dynamic sampling method provided in any embodiment of the present invention. Figure 3 As shown, the device comprises:

[0060] A fitting function generating module 31 is used to train the selected target fitting model according to the historical time series data to generate a fitting function;

[0061] The sampling frequency adjustment module 32 is used to adjust the current sampling frequency in real time according to the fitting function, and perform sampling in real time according to the current sampling frequency to obtain sampling data.

[0062] The technical solution provided by the embodiment of the present invention first trains the selected target fitting model according to the historical time series data to generate a fitting function, and then when sampling is required, the current sampling frequency is adjusted in real time according to the obtained fitting function, and sampling is performed in real time according to the current sampling frequency, thereby obtaining sampled data. By dynamically adjusting the sampling strategy based on data fitting, it is possible to accurately and efficiently sample high-frequency and large-scale time series data under limited computing and storage resources, and reduce redundant sampling while ensuring the accuracy and representativeness of data sampling, thereby significantly reducing the number of sampling points, reducing the resource consumption of data calculation, storage and transmission, and improving the overall performance of the system and resource utilization efficiency. Moreover, it has a wide range of applications and can be applied to a variety of application scenarios, especially in the fields of industrial monitoring and environmental monitoring that require high sampling accuracy and resource utilization efficiency, and the effect is particularly significant, so it has broad application prospects and important practical significance.

[0063] On the basis of the above technical solution, optionally, the sampling frequency adjustment module 32 includes:

[0064] A data change rate determination unit, used to determine the current data change rate in real time according to the fitting function;

[0065] The first sampling frequency updating unit is used to update the current sampling frequency in real time according to the current data change rate.

[0066] On the basis of the above technical solution, optionally, the sampling frequency adjustment module 32 further includes:

[0067] The second sampling frequency updating unit is used to update the current sampling frequency in real time according to the current fitting error of the fitting function.

[0068] On the basis of the above technical solution, optionally, the time series data adaptive dynamic sampling device further includes:

[0069] A target fitting model determination module is used to determine the target fitting model according to the selected fitting model based on the data characteristics of the historical time series data, the fitting accuracy of the fitting model and / or the available computing resources of the system before training the selected target fitting model according to the historical time series data; the selected fitting model includes at least one of a linear model, a polynomial model, an exponential model, a spline interpolation model and a machine learning model.

[0070] On the basis of the above technical solution, optionally, the fitting function generating module 31 includes:

[0071] A data segmentation unit, configured to segment the historical time series data based on a time window or a system-driven event;

[0072] The segmented fitting unit is used to perform fitting training on the target fitting model corresponding to each segment according to each segment of data, so as to obtain the fitting function of each segment of data.

[0073] On the basis of the above technical solution, optionally, the time series data adaptive dynamic sampling device further includes:

[0074] A data compression module, configured to compress the sampled data by using a lossy compression algorithm or a lossless compression algorithm according to the sampling frequency and / or importance level of the sampled data after adjusting the current sampling frequency in real time according to the fitting function and sampling according to the current sampling frequency in real time to obtain the sampled data;

[0075] The data storage unit is used to store the compressed data according to a preset data storage format.

[0076] The time series data adaptive dynamic sampling device provided in the embodiment of the present invention can execute the time series data adaptive dynamic sampling method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0077] It is worth noting that in the above-mentioned embodiment of the adaptive dynamic sampling device for timing data, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0078] Embodiment 3

[0079] Figure 4 The schematic diagram of the structure of the computer device provided for the third embodiment of the present invention shows a block diagram of an exemplary computer device suitable for implementing the implementation mode of the present invention. Figure 4 The computer device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Figure 4 As shown, the computer device includes a processor 41, a memory 42, an input device 43 and an output device 44; the number of processors 41 in the computer device can be one or more. Figure 4 Taking a processor 41 as an example, the processor 41, memory 42, input device 43 and output device 44 in the computer device can be connected through a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0080] The memory 42, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the time series data adaptive dynamic sampling method in the embodiment of the present invention (for example, the fitting function generation module 31 and the sampling frequency adjustment module 32 in the time series data adaptive dynamic sampling device). The processor 41 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 42, that is, realizes the above-mentioned time series data adaptive dynamic sampling method.

[0081] The memory 42 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 42 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 42 may further include a memory remotely arranged relative to the processor 41, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0082] The input device 43 may be used to obtain sampling data, and generate key signal inputs related to user settings and function control of the computer device, etc. The output device 44 may include a display screen, which may be used to display sampling conditions, etc. to the user.

[0083] Embodiment 4

[0084] Embodiment 4 of the present invention further provides a storage medium containing computer executable instructions, wherein the computer executable instructions are used to execute a method for adaptive dynamic sampling of time series data when executed by a computer processor, the method comprising:

[0085] The selected target fitting model is trained according to the historical time series data to generate a fitting function;

[0086] The current sampling frequency is adjusted in real time according to the fitting function, and sampling is performed in real time according to the current sampling frequency to obtain sampling data.

[0087] The storage medium may be any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the computer system in which the program is executed, or may be located in a different second computer system, which is connected to the computer system via a network (such as the Internet). The second computer system may provide program instructions to the computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., embodied as a computer program) that may be executed by one or more processors.

[0088] Of course, the computer executable instructions of a storage medium containing computer executable instructions provided by an embodiment of the present invention are not limited to the method operations described above, and can also execute related operations in the time series data adaptive dynamic sampling method provided by any embodiment of the present invention.

[0089] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0090] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0091] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0092] Embodiment 5

[0093] Embodiment 5 of the present invention further provides a computer program product, which includes a computer program (also referred to as code, instruction), which can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it is used to execute the method for adaptive dynamic sampling of time series data provided in any of the above embodiments, and has the corresponding beneficial effects of the execution method.

[0094] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for adaptive dynamic sampling of time series data, characterized in that: include: The selected target fitting model is trained according to the historical time series data to generate a fitting function; The current sampling frequency is adjusted in real time according to the fitting function, and sampling is performed in real time according to the current sampling frequency to obtain sampling data.

2. The method for adaptive dynamic sampling of time series data according to claim 1, characterized in that: The real-time adjustment of the current sampling frequency according to the fitting function comprises: Determining the current data change rate in real time according to the fitting function; Update the current sampling frequency in real time according to the current data change rate.

3. The method for adaptive dynamic sampling of time series data according to claim 2, characterized in that: The real-time adjustment of the current sampling frequency according to the fitting function also includes: The current sampling frequency is updated in real time according to the current fitting error of the fitting function.

4. The method for adaptive dynamic sampling of time series data according to claim 1, characterized in that: Before training the selected target fitting model according to the historical time series data, the method further includes: Based on the data characteristics of the historical time series data, the fitting accuracy of the fitting model and / or the available computing resources of the system, the target fitting model is determined according to the selected fitting models; the selected fitting models include at least one of a linear model, a polynomial model, an exponential model, a spline interpolation model and a machine learning model.

5. The method for adaptive dynamic sampling of time series data according to claim 1, characterized in that: The training of the selected target fitting model according to the historical time series data to generate a fitting function includes: Segmenting the historical time series data based on time windows or system driven events; According to each segment of data, the target fitting model corresponding to each segment is fit-trained separately to obtain the fitting function of each segment of data.

6. The method for adaptive dynamic sampling of time series data according to claim 1, characterized in that: After adjusting the current sampling frequency in real time according to the fitting function and sampling in real time according to the current sampling frequency to obtain the sampling data, the method further includes: Compressing the sampled data using a lossy compression algorithm or a lossless compression algorithm according to the sampling frequency and / or importance level of the sampled data; The compressed data is stored according to the preset data storage format.

7. A time series data adaptive dynamic sampling device, characterized in that: include: A fitting function generation module is used to train the selected target fitting model according to the historical time series data to generate a fitting function; The sampling frequency adjustment module is used to adjust the current sampling frequency in real time according to the fitting function, and to perform sampling in real time according to the current sampling frequency to obtain sampling data.

8. A computer device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the time series data adaptive dynamic sampling method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for adaptive dynamic sampling of time series data as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method for adaptive dynamic sampling of time series data as described in any one of claims 1 to 6.

Citation Information

Cited By

  • Intelligent detection method and system based on aluminum alloy production line

    CN120521538A

  • Cluster data adaptive sampling method and device

    CN122309289A