Periodic time series data analysis and treatment method and product

Abstract

The invention provides a periodic time series data analysis and treatment method and product, and relates to the technical field of data analysis and treatment. In the embodiment of the invention, the time series data are cut according to the periodicity of the data, the data are coded according to the occurrence time of the periodic time series data, the coded time series data are clustered, and the time series data are clustered based on the clustered result obtained by clustering the coded time series data. The abnormal data cannot correspond to the time slots of the data of other periods, so that a class cluster can be independently formed, and the abnormal data can be found and eliminated based on a clustering result. For the remaining data after the abnormal data is eliminated, noise reduction can be performed on the data in the slot position at each moment, the deviation of the data corresponding to the slot position at the same moment is corrected, and the purpose of improving the data quality is achieved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of data analysis and management, and in particular, to a method and product for analyzing and managing periodic time series data. Background Art

[0002] Time series data refers to a sequence of times when an event or action occurs continuously, such as a sequence of times when a subway line departs, or a sequence of times when a device starts at a fixed time. Periodic time series data refers to time series data with a certain periodicity, such as a subway line that departs at 5:30 every morning, or a lawn sprinkler that starts at a fixed time every morning and afternoon. Time series data is acquired by a collection device or a receiving device and transmitted to the data storage via the network. There is often noise in the process, which causes data deviation and makes it difficult to accurately reflect the time when an event or action occurs.

[0003] Therefore, there is an urgent need for a method to govern time series data. Summary of the invention

[0004] An embodiment of the present invention provides a method for analyzing and managing periodic time series data, which cuts and encodes the time series data according to the periodicity of the data, reorganizes and reduces noise on the data through data clustering and Kalman filter, thereby correcting data deviations and achieving the purpose of improving data quality.

[0005] A first aspect of an embodiment of the present invention provides a method for analyzing and managing periodic time series data, the method comprising:

[0006] The input periodic time series data is cut into periods, and the time series data in each period is encoded according to the occurrence time of the periodic time series data to obtain the encoded time series data;

[0007] Cluster the encoded time series data, exclude abnormal data, and locate the time slot where periodic time series data occurs;

[0008] De-noising the encoded time series data in each time slot;

[0009] The denoised data is reassembled and reversely encoded to obtain the output data.

[0010] Optionally, the time series data in each cycle is encoded, including:

[0011] The time series data of each cycle is encoded respectively based on the offset of the time corresponding to the time series data within the cycle relative to the starting point of the cycle time.

[0012] Optionally, cluster the encoded time series data, exclude abnormal data, and locate the time slot where the periodic time series data occurs, including:

[0013] Clustering the encoded time series data of multiple periods to obtain a clustering result; a cluster in the clustering result corresponds to a time slot;

[0014] Based on the clustering result, a cluster in which the amount of data is less than a first preset threshold is determined as an abnormal cluster, and the abnormal data in the corresponding time slot is removed.

[0015] Optionally, the method further comprises:

[0016] When the period with the most encoded time series data among all periods contains N encoded time series data, the encoded time series data are clustered M+2 times, M≥1, and the number of target clusters is specified as {N-1, N, N+1, …, N+M} during the clustering process;

[0017] After each clustering is completed, the sum of the variances of all encoded time series data to the cluster center is calculated to obtain the variance set {S N-1 ,S N ,S N+1 ,…,S N+M};

[0018] Determine a value P between N-1 and N+M such that |S P-1 -S P The value of | is greater than the second preset threshold, and |S P -S P+1 The value of | is less than a second preset threshold value, and the second preset threshold value is between 1.5 and 3.5;

[0019] Let P be the optimal number of slots at the moment;

[0020] Clustering of encoded time series data of multiple periods, including:

[0021] The optimal number of time slots is used as the number of clusters for clustering.

[0022] Optionally, the method further comprises:

[0023] When it is impossible to determine a value P such that |S P-1 -S P The value of | is greater than the second preset threshold, and |S P -S P+1 When the value of | is less than the second preset threshold, the value N is used as the optimal number of time slots.

[0024] Optionally, denoising is performed on the encoded time series data in each time slot, including:

[0025] For the encoded time series data that fall into the same time slot with similar occurrence time, the Kalman filter is used to correct the encoded time series data in the same time slot one by one to obtain the denoised data.

[0026] A second aspect of an embodiment of the present invention provides a device for analyzing and managing periodic time series data, the device comprising:

[0027] The encoding module is used to cut the input periodic time series data into periods and encode the time series data in each period to obtain encoded time series data;

[0028] The clustering module is used to cluster the encoded time series data, exclude abnormal data, and locate the time slot where the periodic time series data occurs;

[0029] A noise reduction module, used to reduce noise on the encoded time series data in each slot at each moment;

[0030] The reverse encoding module is used to reassemble and reverse encode the denoised data to obtain output data.

[0031] Optionally, the encoding module is specifically used to encode the time series data of each cycle respectively based on the offset of the moment corresponding to the time series data in the cycle relative to the starting point of the cycle moment.

[0032] Optionally, the clustering module is specifically used to:

[0033] Clustering the encoded time series data of multiple periods to obtain a clustering result; a cluster in the clustering result corresponds to a time slot;

[0034] Based on the clustering result, a cluster in which the amount of data is less than a first preset threshold is determined as an abnormal cluster, and the abnormal data in the corresponding time slot is removed.

[0035] Optionally, the device further comprises:

[0036] The first determination module is used to cluster the encoded time series data M+2 times when the period with the most encoded time series data in all periods contains N encoded time series data, M≥1, and the number of target clusters is specified as {N-1, N, N+1, ..., N+M} during the clustering process; after each clustering is completed, the sum of the variances of all encoded time series data to the cluster center is calculated to obtain the variance set {S N-1 ,S N ,S N+1 ,…,S N+M}; Determine a value P between N-1 and N+M such that |S P-1 -S PThe value of | is greater than the second preset threshold, and |S P -S P+1 | is less than a second preset threshold value, and the second preset threshold value is between 1.5 and 3.5; P is used as the optimal number of slots at the moment;

[0037] The clustering module is specifically used to: perform clustering by taking the optimal number of time slots as the number of clusters.

[0038] Optionally, the device further comprises:

[0039] The second determination module is used to determine a value P when it is impossible to determine |S P-1 -S P The value of | is greater than the second preset threshold, and |S P -S P+1 When the value of | is less than the second preset threshold, the value N is used as the optimal number of time slots.

[0040] Optionally, the noise reduction module is specifically used to:

[0041] For the encoded time series data that fall into the same time slot with similar occurrence time, the Kalman filter is used to correct the encoded time series data in the same time slot one by one to obtain the denoised data.

[0042] A third aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when executed, the processor implements the method for analyzing and managing periodic time series data as described in the first aspect of the present invention.

[0043] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for analyzing and managing periodic time series data as described in the first aspect of the present invention.

[0044] A fifth aspect of an embodiment of the present invention provides a computer program product, including a computer program / instruction, which is used by a processor to implement the steps in the method for analyzing and managing periodic time series data as described in the first aspect of the present invention.

[0045] In the embodiment of the present invention, the time series data is cut according to the periodicity of the data, and the data is encoded according to the time when the periodic time series data occurs, and the encoded time series data is clustered. In the clustering results obtained by clustering the encoded time series data, the abnormal data cannot correspond to the time slots of the data of other periods, and will form a cluster separately, so that the abnormal data can be found and excluded based on the clustering results. For the data remaining after excluding the abnormal data, the data in each time slot can be denoised respectively, and the deviation of the data corresponding to the same time slot can be corrected to achieve the purpose of improving the data quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0047] Figure 1 It is a flowchart of the steps of the method for analyzing and managing periodic time series data provided by an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of clustering results obtained in the method for analyzing and managing periodic time series data provided by an embodiment of the present invention;

[0049] Figure 3 It is a schematic diagram of the result of searching for the optimal number of time slots by using elbowmethod in the method for analyzing and managing periodic time series data provided by an embodiment of the present invention;

[0050] Figure 4 It is a schematic diagram of data noise reduction results in the method for analyzing and managing periodic time series data provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] The present invention provides a method for analyzing and managing periodic time series data. Figure 1 As shown, the method may include the following steps:

[0053] S101, cutting the input periodic time series data into periods, encoding the time series data in each period according to the occurrence time of the periodic time series data, and obtaining encoded time series data.

[0054] S102, clustering the encoded time series data, eliminating abnormal data, and locating the time slot where the periodic time series data occurs.

[0055] S103, performing noise reduction on the encoded time series data in each time slot.

[0056] S104, rejoining and reverse encoding the noise-reduced data to obtain output data.

[0057] The method provided by an embodiment of the present invention can be applied to a data processing device, which can be configured on any electronic device to receive input periodic timing data, process the received data and output the processed data. The input periodic timing data can be uploaded through an external interface of the electronic device, stored locally in the electronic device, or obtained through a network device that is communicatively connected to the electronic device.

[0058] In an embodiment of the present invention, in step S101, first, the periodicity of the data can be determined, that is, the length of time each cycle contains. It can be determined based on the knowledge of business logic, or it can be a statistical period obtained by analyzing the data. Secondly, the starting point of each cycle can be determined. It can be a fixed time interval (such as the beginning of each day, week, or month), or it can be determined based on certain feature points in the data. Once the starting point of the cycle is determined, the data set can be cut into multiple subsets according to these points. Each subset contains data for a complete cycle. In the process of cutting data, if the beginning or end of the data set does not completely contain a complete cycle, these data can be discarded or processed separately or used as a complete cycle to participate in subsequent clustering steps.

[0059] In an embodiment of the present invention, the input periodic time series data is cut according to the period, specifically, the input time series data is divided into units of period, and the period length can be hours, days or weeks, etc., which is determined by the periodic characteristics of the time series data. Therefore, the periodic time series data can be cut according to the periodicity of the time series data. For example, if the subway starts at 5:30 every morning and a train is sent every 5 minutes, then one day can be taken as a cycle, and the time series data of the subway departure in one day can be taken as the time series data within a cycle, and the periodic time series data can be cut with one day as a cycle. In the cut data, the time series data generated in one day is a time series data group.

[0060] In an embodiment of the present invention, encoding the time series data may include: encoding the time series data of each cycle separately based on the offset of the moment corresponding to the time series data within the cycle relative to the starting point of the cycle moment, thereby representing each time series data based on the occurrence moment of each time series data.

[0061] Specifically, in an embodiment of the present invention, the time series data within a cycle can be encoded based on the accuracy of the time series data (usually seconds or minutes), and the corresponding code is calculated according to the offset of each time series data (moment) within the cycle relative to the starting point of the cycle. For example, taking a day as a cycle, starting at 0 o'clock and ending at 24 o'clock, with an accuracy of minutes, there are 1440 coding positions in a cycle, and an event or behavior occurring at 12 noon is encoded as 720. Each moment at which an event or behavior occurs within the cycle can be converted into a corresponding code. After encoding, the time series of events or behaviors occurring within each cycle is represented by a set of integer codes.

[0062] In the embodiment of the present invention, the k-means algorithm can be used to cluster the encoded time series data to determine the time slot where the event or behavior occurs, and each cluster corresponds to a time slot.

[0063] The k-means algorithm can divide the samples in the encoded time series data into K clusters, so that the samples within the clusters are as similar as possible, and the samples between clusters are as different as possible. Specifically, the k-means algorithm can include the following steps:

[0064] Step 1: Randomly select K data points as the initial cluster centers.

[0065] Step 2: For each sample in the data set, calculate its distance from the K cluster centers and assign it to the cluster represented by the nearest cluster center.

[0066] Step 3: For each cluster, calculate the mean of all samples in the cluster and use this mean as the new cluster center.

[0067] Repeat steps 2 and 3 until the cluster center no longer changes or the preset number of iterations is reached. At this point, each sample is assigned to a cluster and the clustering results are output.

[0068] In an embodiment of the present invention, the time series data of events or behaviors occurring in each period are encoded according to their occurrence times, and then the encoded time series data corresponding to multiple periods are clustered. It can be understood that the encoded time series data at the same time in multiple periods will form a cluster, and the time when the data corresponding to the cluster occurs is a time slot that will periodically generate time series data.

[0069] For example, if the subway departs at 5:30 every morning and a train departs every 5 minutes, then one day can be regarded as a cycle, and the time series data of subway departures in one day can be regarded as the time series data within a cycle. After clustering, 5:30 a.m. every day can be determined as a time slot, and a time slot can be determined every 5 minutes.

[0070] In the embodiment of the present invention, in the clustering process, the encoded time series data at the same time in multiple cycles will form a cluster, and the time when the data corresponding to the cluster occurs is a time slot that periodically generates time series data. For abnormal data generated at a certain time in a certain cycle, since there is no data corresponding to the time when it occurs in other cycles, the abnormal data will form a cluster alone, based on which the abnormal data can be discovered and excluded.

[0071] Specifically, in the embodiment of the present invention, step S102 may include the following sub-steps:

[0072] S1021, clustering the encoded time series data of multiple periods to obtain a clustering result; a cluster in the clustering result corresponds to a time slot.

[0073] S1022: Based on the clustering result, a cluster in which the amount of data is less than a first preset threshold is determined as an abnormal cluster, and abnormal data in the corresponding time slot is removed.

[0074] Due to equipment failure or program errors, there may be outliers in the time series data. Considering that outliers often deviate from the normal data range, outliers will form their own category after clustering. In the embodiment of the present invention, these outliers can be found by clustering the encoded time series data, and then outliers can be excluded. Specifically, all clusters obtained after clustering can be checked one by one, and clusters with less than a preset threshold in the clusters are determined as abnormal clusters, and the values ​​in the abnormal clusters are determined as abnormal values ​​and excluded.

[0075] In an embodiment of the present invention, the first preset threshold can be set according to actual conditions, and can specifically be Max(total number of cycles×0.05, 1). If the number of cycles×0.05 is greater than 1, the first preset threshold is the number of cycles×0.05; if the result is less than 1, the first preset threshold is 1.

[0076] like Figure 2 As shown, it shows a schematic diagram of the clustering results, where the data of period 1 to period n are obtained based on the cutting of periodic time series data. Figure 2 It can be seen that events that occur periodically at normal times will be clustered into a cluster, while outliers will form a separate cluster, so outliers can be found intuitively and accurately.

[0077] In the embodiment of the present invention, after excluding abnormal data, the number of remaining clusters is the number of time slots at which periodic time series data occurs, and each cluster corresponds to a time slot.

[0078] In the embodiment of the present invention, the incomplete cycle with missing data can be filled based on the located time slot to further improve the data quality. Specifically, after determining the time slot, the data of each cycle after cutting can be searched according to the time slot. In the case of missing data in a certain time slot, the corresponding data can be filled in the corresponding time slot.

[0079] In the embodiment of the present invention, during the clustering process, it is also necessary to determine the optimal number of time slots. Specifically, the optimal number of time slots may be determined based on the cycle with the largest number of time slots among all cycles.

[0080] In the embodiment of the present invention, the elbow method can be used to find the optimal number of time slots. The specific method includes: when the period with the most encoded time series data in all periods contains N encoded time series data, the encoded time series data is clustered M+2 times, M≥1, and the number of target clusters is specified as {N-1, N, N+1, ..., N+M} during the clustering process; after each clustering is completed, the sum of the variances of all encoded time series data to the cluster center is calculated to obtain the variance set {S N-1 ,S N ,S N+1 ,…,S N+M}; Determine a value P between N-1 and N+M such that |S P-1 -S P The value of | is greater than the second preset threshold, and |S P -S P+1 The value of | is less than the second preset threshold value, and the second preset threshold value is between 1.5 and 3.5; P is used as the optimal number of time slots. The optimal number of time slots is used as the number of clusters for clustering.

[0081] Specifically, Figure 3 As shown, it shows a schematic diagram of the result of using the elbow method to find the optimal number of slots at the moment. Figure 3 In the figure, the horizontal axis represents the number of target clusters {N-1, N, N+1, …, N+M}, and the vertical axis represents the variance set {S N-1 ,S N ,S N+1 ,…,S N+M}, where the number of target clusters is 3, and the sum of the variances of all encoded time series data to the cluster centers obtained based on the clustering results meets the requirements for the optimal number of time slots, then the number of target clusters 3 is taken as the optimal number of time slots.

[0082] In the embodiment of the present invention, when a value P cannot be determined such that |S P-1 -S PThe value of | is greater than the second preset threshold, and |S P -S P+1 When the value of | is less than the second preset threshold, the value N is used as the optimal number of time slots.

[0083] In the embodiment of the present invention, it is also considered that the input data has periodicity, that is, events or behaviors will occur at similar times across periods. After the aforementioned clustering process, similar times will fall into the same cluster. Due to measurement errors, the time when events or behaviors occur in the same cluster will fluctuate.

[0084] Based on this, in the embodiment of the present invention, in step S103, the coded time series data in each time slot is denoised. Specifically, for the coded time series data falling into the same time slot with similar occurrence time, the Kalman filter is used to correct the coded time series data in the same time slot one by one to obtain the denoised data. Specifically, the Kalman filter is a linear filter, which requires that the state at the current moment has a linear relationship with the state at the previous moment. The occurrence time of events or behaviors across periods can be described as X t =X t-1 , that is, in different cycles, the time when events or behaviors occur is theoretically consistent. This linear relationship just meets the prerequisite for using the Kalman filter. Therefore, the Kalman filter can be used to correct the values ​​in the same cluster one by one. Figure 4 As shown, it shows a schematic diagram of the data denoising result, where the red dotted line is the reference true value, the green solid line is the noisy measurement value (i.e., the occurrence time of the input data in the embodiment of the present invention), and the blue dashed line is the corrected value (i.e., the occurrence time of the output data in the embodiment of the present invention). It can be seen that the corrected value is closer to the true value as a whole. The Kalman filter can process noisy signals through system input and output observation data to obtain the optimal estimate of the system state or the true signal.

[0085] In the embodiment of the present invention, in step S104, the time series data after the above steps S101 to S103 are re-joined in order of periods, and the data are reversely encoded, the codes are converted into time values, and the final data processing results are output.

[0086] In the embodiment of the present invention, the data processing device first cuts the input time series data into periods and encodes the time series data within the period. Secondly, the encoded time series data is clustered to locate the time slots where periodic events or actions occur. Afterwards, the abnormal time series data is checked and eliminated based on the clustering results (such as Figure 24). After judging the normal clusters, the Kalman filter is used to reduce the noise of the data in the clusters. Finally, the data is reassembled and reversely encoded to obtain the output data. This can achieve the purpose of improving data quality.

[0087] Based on the same inventive concept, an embodiment of the present invention further provides a device for analyzing and managing periodic time series data, the device comprising:

[0088] The encoding module is used to cut the input periodic time series data into periods and encode the time series data in each period to obtain encoded time series data;

[0089] The clustering module is used to cluster the encoded time series data, exclude abnormal data, and locate the time slot where the periodic time series data occurs;

[0090] A noise reduction module, used to reduce noise on the encoded time series data in each slot at each moment;

[0091] The reverse encoding module is used to reassemble and reverse encode the denoised data to obtain output data.

[0092] Optionally, the encoding module is specifically used to encode the time series data of each cycle respectively based on the offset of the moment corresponding to the time series data in the cycle relative to the starting point of the cycle moment.

[0093] Optionally, the clustering module is specifically used to:

[0094] Clustering the encoded time series data of multiple periods to obtain a clustering result; a cluster in the clustering result corresponds to a time slot;

[0095] Based on the clustering result, a cluster in which the amount of data is less than a first preset threshold is determined as an abnormal cluster, and the abnormal data in the corresponding time slot is removed.

[0096] Optionally, the device further comprises:

[0097] The first determination module is used to cluster the encoded time series data M+2 times when the period with the most encoded time series data in all periods contains N encoded time series data, M≥1, and the number of target clusters is specified as {N-1, N, N+1, ..., N+M} during the clustering process; after each clustering is completed, the sum of the variances of all encoded time series data to the cluster center is calculated to obtain the variance set {S N-1 ,S N ,S N+1 ,…,S N+M}; Determine a value P between N-1 and N+M such that |S P-1 -S PThe value of | is greater than the second preset threshold, and |S P -S P+1 | is less than a second preset threshold value, and the second preset threshold value is between 1.5 and 3.5; P is used as the optimal number of slots at the moment;

[0098] The clustering module is specifically used to: perform clustering by taking the optimal number of time slots as the number of clusters.

[0099] Optionally, the device further comprises:

[0100] The second determination module is used to determine a value P when it is impossible to determine |S P-1 -S P The value of | is greater than the second preset threshold, and |S P -S P+1 When the value of | is less than the second preset threshold, the value N is used as the optimal number of time slots.

[0101] Optionally, the noise reduction module is specifically used to:

[0102] For the encoded time series data that fall into the same time slot with similar occurrence time, the Kalman filter is used to correct the encoded time series data in the same time slot one by one to obtain the denoised data.

[0103] Based on the same inventive concept, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes, the steps in the method for analyzing and managing periodic time series data as described in any of the above embodiments are implemented.

[0104] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the method for analyzing and managing periodic time series data described in any of the above embodiments are implemented.

[0105] Based on the same inventive concept, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in the method for analyzing and managing periodic time series data described in any of the above embodiments.

[0106] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0107] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0108] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (apparatus), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0112] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0113] The above is a detailed introduction to the analysis and management method and product of periodic time series data provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for analyzing and managing periodic time series data, characterized in that: The method comprises: The input periodic time series data is cut into periods, and the time series data in each period is encoded according to the occurrence time of the periodic time series data to obtain the encoded time series data; Cluster the encoded time series data, exclude abnormal data, and locate the time slot where the periodic time series data occurs; De-noise the encoded time series data in each time slot; The denoised data is reassembled and reversely encoded to obtain the output data.

2. The method for analyzing and managing periodic time series data according to claim 1, characterized in that: According to the occurrence time of periodic time series data, the time series data in each period is encoded, including: The time series data of each cycle is encoded respectively based on the offset of the time corresponding to the time series data within the cycle relative to the starting point of the cycle time.

3. The method for analyzing and managing periodic time series data according to claim 1, characterized in that: Cluster the encoded time series data, exclude abnormal data, and locate the time slots where periodic time series data occurs, including: Clustering the encoded time series data of multiple periods to obtain a clustering result; a cluster in the clustering result corresponds to a time slot; Based on the clustering result, a cluster in which the amount of data is less than a first preset threshold is determined as an abnormal cluster, and the abnormal data in the corresponding time slot is removed.

4. The method for analyzing and managing periodic time series data according to claim 3, characterized in that: The method further comprises: When the period with the most encoded time series data among all periods contains N encoded time series data, the encoded time series data are clustered M+2 times, M≥1, and the number of target clusters is specified as {N-1, N, N+1, …, N+M} during the clustering process; After each clustering is completed, the sum of the variances of all encoded time series data to the cluster center is calculated to obtain the variance set {S N-1 ,S N ,S N+1 ,…,S N+M }; Determine a value P between N-1 and N+M such that |S P-1 -S P The value of | is greater than the second preset threshold, and |S P -S P+1 The value of | is less than a second preset threshold value, and the second preset threshold value is between 1.5 and 3.5; Let P be the optimal number of slots at the moment; Clustering of encoded time series data of multiple periods, including: The optimal number of time slots is used as the number of clusters for clustering.

5. The method for analyzing and managing periodic time series data according to claim 4, characterized in that: The method further comprises: When it is impossible to determine a value P such that |S P-1 -S P The value of | is greater than the second preset threshold, and |S P -S P+1 When the value of | is less than the second preset threshold, the value N is used as the optimal number of time slots.

6. The method for analyzing and managing periodic time series data according to claim 1, characterized in that: De-noise the encoded time series data in each slot at each moment, including: For the encoded time series data that fall into the same time slot with similar occurrence time, the Kalman filter is used to correct the encoded time series data in the same time slot one by one to obtain the denoised data.

7. A device for analyzing and managing periodic time series data, characterized in that: The device comprises: The encoding module is used to cut the input periodic time series data into periods, encode the time series data in each period according to the occurrence time of the periodic time series data, and obtain the encoded time series data; The clustering module is used to cluster the encoded time series data, exclude abnormal data, and locate the time slot where the periodic time series data occurs; A noise reduction module, used to reduce noise on the encoded time series data in each slot at each moment; The reverse encoding module is used to re-join and reverse encode the denoised data to obtain output data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for analyzing and managing periodic time series data described in any one of claims 1-6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method for analyzing and managing periodic time series data described in any one of claims 1-6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps in the method for analyzing and managing periodic time series data described in any one of claims 1-6 are implemented.

Images