A data processing method and related apparatus

Abstract

Embodiments of the present application disclose a data processing method and related device, which are used for detecting faults of a business system through business data, sequentially dividing the business data into n segment data according to a to-be-determined time period, determining a correlation parameter of the jth segment data with the ith segment data after aligning the jth segment data to the ith segment data according to a time offset of a variation trend of the jth segment data with respect to the ith segment data in a time length, determining a trend alignment object for actual alignment according to the correlation parameters corresponding to the n segment data respectively, and performing alignment processing on other segment data based on the trend alignment object, so that the distortion and distortion degree of the obtained alignment data can be effectively controlled under the premise of eliminating the time offset. Therefore, if the business data is identified as periodic data based on the alignment data, a target period determined according to the to-be-determined time period is more accurate, and data anomaly identification of the business data based on the target period is also more accurate.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method and related apparatus. Background Technology

[0002] In the process of providing services to users through business systems, these systems generate various types of time series data. Time series data is data collected at different times and is used to reflect how the phenomena described by the data change over time, such as basic indicator data in the business system. By monitoring the time series data of the business system for anomalies, alerts can be issued when abnormal fluctuations occur, enabling business operators to promptly investigate, identify, and locate problems in the business system.

[0003] Although time series data appear to change continuously over time, some time series data exhibit a repetitive trend within a certain period (day, week, month, etc.), and are called periodic data. Other time series data do not possess this characteristic and are called aperiodic data.

[0004] Related technologies use the Discrete Fourier Transform (DFT) to determine whether time series data is periodic and identify the period of the identified periodic data. By identifying the period, the current data can be compared with historical data to determine whether the current data is abnormal.

[0005] However, the aforementioned technologies are not very accurate in determining the period, and may even mistakenly identify periodic data as non-periodic data. As a result, subsequent data anomaly monitoring based on this will result in missed alarms and false alarms, and the accuracy of anomaly monitoring is low. Summary of the Invention

[0006] To address the aforementioned technical problems, this application provides a data processing method and related apparatus to improve the accuracy of cycle determination and subsequent anomaly monitoring.

[0007] The embodiments of this application disclose the following technical solutions:

[0008] On the one hand, this application provides a data processing method, the method comprising:

[0009] The business data to be identified is divided into n segments according to the undetermined time period.

[0010] Based on the time offset of the j-th data segment relative to the i-th data segment at the time length indicated by the undetermined time period, determine the correlation parameter between the j-th data segment and the i-th data segment after aligning the j-th data segment to the i-th data segment. The i-th data segment is one of the n data segments, and the j-th data segment is any one of the n data segments except the i-th data segment.

[0011] Based on the correlation parameters corresponding to the n data segments, one data segment is determined from the n data segments as the trend alignment object for actual alignment;

[0012] The business data is adjusted into aligned data by aligning the other fragments of data (excluding the trend alignment object) based on the trend alignment object.

[0013] In response to identifying the business data as periodic data based on the alignment data, a target period for the business data is determined according to the pending time period, and data anomaly identification is performed on the business data according to the target period.

[0014] On the other hand, this application provides a data processing apparatus, which includes: a division unit, a determination unit, an adjustment unit, and a data anomaly identification unit;

[0015] The partitioning unit is used to sequentially divide the business data to be identified into n data segments according to a predetermined time period.

[0016] The determining unit is configured to determine, based on the time offset of the j-th segment data relative to the i-th segment data within the time length identified by the undetermined time period, the correlation parameter between the j-th segment data and the i-th segment data after alignment, wherein the i-th segment data is one of the n segment data, and the j-th segment data is any one of the n segment data except the i-th segment data; and to determine, based on the correlation parameter corresponding to each of the n segment data, a segment data as the trend alignment object for actual alignment.

[0017] The adjustment unit is used to adjust the business data into aligned data by aligning the other fragment data in the n fragment data except for the trend alignment object based on the trend alignment object;

[0018] The data anomaly identification unit is configured to respond to identifying the business data as periodic data based on the alignment data, determine the target period of the business data according to the pending time period, and perform data anomaly identification on the business data according to the target period.

[0019] On the other hand, this application provides a computer device, the device including a processor and a memory:

[0020] The memory is used to store program code and transmit the program code to the processor;

[0021] The processor is configured to execute the methods described above according to instructions in the program code.

[0022] On the other hand, embodiments of this application provide a computer-readable storage medium for storing a computer program for performing the methods described above.

[0023] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods described above.

[0024] As can be seen from the above technical solution, for the business data to be identified, the business data is sequentially divided into n segments according to the undetermined time period. Since the change trend between each segment may have a relative time offset under the time length indicated by the undetermined time period, such as the change trend being lagging or leading, in order to eliminate the impact of the relative time offset on the accuracy of period identification, the correlation parameter between the j-th segment and the i-th segment is determined based on the time offset of the change trend of the j-th segment relative to the i-th segment under the time length. Since the correlation parameter can identify the similarity between the change trend of the j-th segment and the i-th segment after the above alignment, the trend alignment object for actual alignment is determined from the n segments based on the correlation parameters corresponding to the n segments. After aligning the other segments except the trend alignment object with the trend alignment object, the distortion and aberration of the obtained aligned data can be effectively controlled under the premise of eliminating the time offset. Therefore, if the business data is identified as periodic based on the aligned data, the target period determined according to the pending time period is more consistent with the actual period of the business data. This allows for more accurate identification of data anomalies based on the target period. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram illustrating an application scenario of a data processing method provided in an embodiment of this application;

[0027] Figure 2 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0028] Figure 3 This is a schematic diagram illustrating the determination of correlation parameters according to an embodiment of this application.

[0029] Figure 4 This application provides a schematic diagram of processing business data as an embodiment of the present application.

[0030] Figure 5 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0031] Figure 6 This application provides a schematic diagram of processing business data as an embodiment of the present application.

[0032] Figure 7 This application provides a schematic diagram of processing business data as an embodiment of the present application.

[0033] Figure 8 A schematic diagram of a data processing apparatus provided in an embodiment of this application;

[0034] Figure 9 This is a schematic diagram of the server structure provided in an embodiment of this application;

[0035] Figure 10 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0036] The embodiments of this application will now be described with reference to the accompanying drawings.

[0037] In related technologies, the Directional Time-of-Flight (DFT) based method can transform time series data from the time domain to the frequency domain, and then analyze the spectral data to see if there are large spikes or discrete signals. The amplitude of these spikes or discrete signals represents the period of the time series data. This allows determination of whether the time series data is periodic or aperiodic. However, some time series data exhibit a repetitive fluctuation trend within each period, but show varying degrees of lag or advance in different periods. Using only the DFT method in such cases can lead to inaccurate period determination, or even misjudgment.

[0038] Based on this, embodiments of this application provide a data processing method and related apparatus for eliminating the impact of relative time offset on the accuracy of period identification.

[0039] The data processing method provided in this application can be applied to data processing devices with data processing capabilities, such as terminal devices and servers. Specifically, terminal devices can be smartphones, desktop computers, laptops, tablets, smart speakers, smartwatches, etc., but are not limited to these. Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this connection.

[0040] The data processing method provided in this application is based on cloud computing technology. Cloud computing refers to the delivery and usage model of IT infrastructure, which means obtaining the required resources in an on-demand and easily scalable manner through the network. In a broader sense, cloud computing refers to the delivery and usage model of services, which means obtaining the required services in an on-demand and easily scalable manner through the network. Such services can be IT and software, Internet-related, or other services. Cloud computing is a product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0041] With the development of the internet, real-time data streams, and the diversification of connected devices, as well as the demands for search services, social networks, mobile commerce, and open collaboration, cloud computing has rapidly developed. Unlike previous parallel distributed computing, cloud computing will fundamentally revolutionize the entire internet model and enterprise management model.

[0042] In this application's embodiments, the main cloud computing technologies involved include big data. Big data refers to data sets that cannot be captured, managed, and processed within a certain timeframe using conventional software tools. It represents massive, rapidly growing, and diverse information assets that require new processing models to achieve stronger decision-making, insight discovery, and process optimization capabilities. With the advent of the cloud era, big data has attracted increasing attention. Big data requires specialized technologies to effectively process large amounts of data within a tolerable timeframe. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

[0043] In the data processing method provided in this application embodiment, the collected business data is periodically identified through big data so that data anomaly monitoring can be performed based on the periodic identification results.

[0044] To facilitate understanding of the technical solution of this application, the data processing method provided in the embodiments of this application will be described below in conjunction with a real-world application scenario, using a server as a data processing device.

[0045] See Figure 1 This figure is a schematic diagram illustrating an application scenario of a data processing method provided in an embodiment of this application. Figure 1 The application scenario shown includes server 100, which can process the business data to be identified.

[0046] like Figure 1 As shown in (a), server 100 acquired business data to be identified over a period of 5 days. To determine whether this business data is periodic, server 100 sequentially divided the business data into n segments according to the undetermined time period. For example, if the undetermined time period is 1 day, then... Figure 1 (b) divides the business data into 5 segments by dashed lines. The data between each pair of dashed lines represents business data with a time length of 1 day. From left to right, they are the first segment, the second segment, ..., the fifth segment.

[0047] exist Figure 1 In the application scenario shown, each data segment exhibits a repetitive fluctuation trend within its corresponding time length, but there are varying degrees of lag and advance in relative time offset between them. In order to... Figure 1 The relative time offset is shown in the figure, with the first change in each data segment marked by a solid line, such as... Figure 1As shown in (b), the time lengths marked between the dashed and solid lines corresponding to each data segment are not equal, thus there is a relative time offset between each data segment.

[0048] Under the time length indicated by the undetermined time period, there is a relative time offset between the changing trends of each data segment. The relative time offset will affect the accuracy of period identification. Therefore, in order to improve the accuracy of period identification, the relative time offset between each data segment can be eliminated. The following uses the first data segment and the second data segment as an example, i.e., j=2, i=1.

[0049] Based on the time offset of the change trend of the second data segment relative to the change trend of the first data segment, a correlation parameter is determined between the second data segment and the first data segment after alignment. This correlation parameter identifies the degree of similarity between the change trend of the j-th data segment and the i-th data segment after alignment. In other words, the correlation parameter clarifies the similarity between the second and first data segments after alignment, thereby controlling the degree of distortion and aberration in the resulting aligned data.

[0050] Therefore, based on the correlation parameters corresponding to the five data segments, the trend alignment object used for actual alignment can be determined from the five data segments. Figure 1 In the application scenario shown, the fourth data segment is used as the trend alignment object for actual alignment. Based on the fourth data segment, the other four data segments are aligned, as shown in 1(c), resulting in aligned data. Compared to... Figure 1 The business data shown in (a) Figure 1 (c) shows that the alignment data eliminates the relative time offset between different data segments, and effectively controls the degree of distortion and aberration of the obtained alignment data based on the correlation parameter.

[0051] exist Figure 1 In the application scenario shown, since the alignment data eliminates the impact of relative time offset, the business data can be accurately identified as periodic data based on the alignment data. Furthermore, the accuracy of the target period determined based on the pending time period is higher, meaning that the target period is more consistent with the actual period of the business data. This allows for more accurate identification of data anomalies based on the target period, reducing the number of missed alarms and false alarms.

[0052] The data processing method provided in the embodiments of this application will be described below with reference to the accompanying drawings and using a server as a data processing device.

[0053] See Figure 2This figure is a flowchart of a data processing method provided in an embodiment of this application. Figure 2 As shown, the data processing method includes the following steps:

[0054] S201: Divide the business data to be identified into n segments according to the pending time period.

[0055] Currently, the presence of abnormal fluctuations in the business data generated by the business system is used to determine whether the system is experiencing a malfunction. Business data is generally divided into periodic and non-periodic data, and these different types of business data have different characteristics. To facilitate subsequent anomaly identification, different strategies can be employed for different types of business data.

[0056] However, some business data exhibit similar fluctuation trends, but the start time of the cycle is relatively random. For example, business data generated by triggered request tasks. These tasks start daily and generate requests for a period of time, exhibiting a repetitive fluctuation trend that can be considered periodic data. However, because the start time of triggered request tasks is relatively random—meaning the trends between different triggered request tasks may have relative time shifts, such as lag or lead—the accuracy of determining the cycle of the business data is not high. It may even be incorrectly identified as non-periodic data, which is detrimental to subsequent anomaly identification.

[0057] Based on this, in order to eliminate the impact of relative time offset on the accuracy of period recognition, after acquiring the business data to be identified, the business data to be identified is divided into n segments according to the undetermined time period, thereby finding the relative time offset between the n segments and eliminating it.

[0058] The pending time period can be a preliminary estimate of the actual period of the business data. It can be determined based on experience or by methods such as DFT. This application does not impose specific limitations on this. It should be noted that the pending time period is an estimated value, which can be the same as the actual period of the business data or may have errors and differ from the actual period of the business data. This application does not impose specific limitations on this.

[0059] When dividing business data into segments, the division should be done sequentially according to the pending time period. For example, if the business data consists of three consecutive days of data, and the pending time period is one day, then the first segment (0:00 to 23:59) can be divided into the first segment, the second segment (0:00 to 23:59) into the second segment, and the third segment (0:00 to 23:59) into the third segment. Alternatively, the first segment (8:00 to 7:59) can be divided into the first segment, and the second segment (8:00 to 7:59) into the third segment, and so on.

[0060] S202: Based on the time offset of the j-th segment data relative to the i-th segment data under the time length indicated by the undetermined time period, determine the correlation parameter between the j-th segment data and the i-th segment data after aligning the j-th segment data to the i-th segment data.

[0061] After dividing the business data according to a unified standard, different data segments are all within the time length indicated by the undetermined time period. Different data segments can be compared to determine the relative time offset between them.

[0062] To eliminate potential relative time offsets between data segments, a single data segment is used as the trend alignment object for actual alignment. Other data segments (excluding this trend alignment object) are then aligned using the trend alignment object as a reference. However, alignment operations can lead to data distortion and aberration. Therefore, to effectively control the degree of distortion and aberration in the subsequently obtained aligned data, this application uses a correlation parameter to determine the trend alignment object. The correlation parameter can identify the similarity between the trend of change of the other data segments after alignment and the trend alignment object. This effectively controls the degree of distortion and aberration in the subsequently obtained aligned data while eliminating time offsets, ensuring the accuracy of business data periodicity identification.

[0063] This application does not specifically limit the method of determining the correlation parameters. The following explanation uses two data segments from n data segments, the i-th segment and the j-th segment, as an example, in conjunction with steps S301 and S302. Where i j.

[0064] S301: By determining the temporal similarity between the m data feature points in the j-th data segment and the m data feature points in the i-th data segment, a temporal alignment path for identifying time offsets is obtained.

[0065] The business data includes m data feature points within a time length specified by the undetermined time marker. For example, the i-th data segment includes m data feature points, and the j-th data segment also includes m data feature points. These data feature points characterize the waveform trend of the business data within that time length; for instance, a data feature point can be the business data value corresponding to each moment within the time length.

[0066] As mentioned above, there may be a relative time offset between the i-th data segment and the j-th data segment. Therefore, there may also be a relative time offset between the m data feature points included in the i-th data segment and the m data feature points included in the j-th data segment. For example, the first data feature point of the j-th data segment may not correspond to the first data feature point of the i-th data segment, but may correspond to the third data feature point of the i-th data segment.

[0067] Therefore, in order to eliminate relative time offset, the temporal similarity between the m data feature points included in the i-th data segment and the m data feature points included in the j-th data segment can be determined. The temporal similarity characterizes the degree of temporal similarity between the two data feature points. Through the temporal similarity, a temporal alignment path for identifying time offset can be obtained so that the j-th data segment can be aligned to the i-th data segment according to the temporal alignment path.

[0068] For example, the third data feature point in the i-th data segment has a high temporal similarity to the sixth data feature point in the j-th data segment, and the fourth data feature point in the i-th data segment has a high temporal similarity to the seventh data feature point in the j-th data segment. Thus, the temporal alignment path used to identify the time offset between the i-th and j-th data segments can be determined through the correspondence of the data feature points.

[0069] As one possible implementation, the timing alignment path can be determined using the Dynamic Time Warping (DTW) algorithm, which will be explained below.

[0070] Calculating the i-th data segment and the j-th fragment data When determining temporal similarity, it can be done by calculating and The traditional way to obtain the distance between them is by calculation. and The Euclidean distance, but requires and They must have the same length, that is, they must be... and If the number of data feature points is the same, then the distances between corresponding positions are compared point by point, such as... Figure 3 As shown in (b).

[0071] The DTW algorithm is an algorithm used to calculate the similarity between two time series data. It is generally used to identify the similarity between time series data that identify speech, and it does not require... and Having the same length, through twisting and This process aims to ensure that the data feature points of two data segments are as temporally consistent as possible, i.e., have high temporal similarity, to achieve better alignment. Then, the distance between corresponding positions of the aligned data feature points is calculated, such as... Figure 3 As shown in (c).

[0072] The following describes the implementation in detail using S3011-S3013, as shown in the figure. Figure 3 The alignment shown in (c) is as follows.

[0073] S3011: Based on the position information of the p-th data feature point in the i-th data segment and the position information of the q-th data feature point in the j-th data segment, determine the distance parameter between the p-th and q-th data feature points, such as... ,in, .

[0074] As one possible implementation, the distance parameter can also be determined by using the position information of the p-th data feature point in the i-th data segment, the position information of the q-th data feature point in the j-th data segment, and the difference between p and q, to ​​determine the distance parameter between the p-th and q-th data feature points. ,in, , The corresponding penalty intensity is used to make the data feature points in the j-th data segment as close as possible to the corresponding data feature points in the i-th data segment, thereby reducing distortion and aberration caused by the alignment operation. On the other hand, it can also reduce the misalignment effect caused by the random extreme values ​​(noisy data) of the fluctuation trend of the two data segments.

[0075] S3012: Determine the set of distance parameters that reflect temporal similarity based on the distance parameters between any data feature point in the i-th data segment and any data feature point in the j-th data segment.

[0076] In order to achieve such Figure 3 The alignment shown in (c) generally requires the construction of a The cumulative distance matrix, where Let be the lengths of the two data segments, such that the i-th data segment contains n data feature points and the j-th data segment contains m data feature points. They can be equal or unequal; what is understandable is that when... In this case, the method of determining the similarity parameter through the DTW algorithm is the same as the method described in S301 above.

[0077] The following is based on For example, Figure 3 As shown in (a), the elements in the constructed cumulative distance matrix This indicates that the j-th data segment is the first... The data feature points are directed to the i-th data segment. The minimum cumulative distance (a type of distance parameter) required to align data feature points is calculated by adding the distances of all previously reached data feature points as each data feature point is reached, until the endpoint is reached. The cumulative distance matrix that fills the distance parameters is the distance parameter set.

[0078] S3013: Obtain the timing alignment path for identifying time offsets based on the distance parameter set.

[0079] The path that minimizes the cumulative distance is the timing alignment path used to identify the time offset, such as... Figure 3 As shown in (a), the path indicated by the arrow in the figure represents and The timing alignment path. For example, element 0, number 1 in the sequence will be aligned with... The first element, -9, and the second element, -5, are aligned.

[0080] Calculated using traditional methods Figure 3 (b) and The Pearson correlation coefficient between them is 0.80. After processing using the DTW algorithm, i.e., calculating... Figure 3 (c) and The Pearson correlation coefficient between them is 0.99. Therefore, for business data that have similar fluctuation trends but relatively random start times (not aligned on the time axis), the correlation parameters determined by the DTW algorithm are more accurate, as it eliminates the impact of relative time offset.

[0081] S302: Based on the timing alignment path and the j-th segment data, determine the correlation parameters between the j-th segment data and the i-th segment data after aligning the j-th segment data to the i-th segment data.

[0082] After obtaining the temporal alignment path, the j-th data segment can adjust its m data feature points according to this path, making the j-th data segment more accurately aligned with the i-th data segment. Therefore, if the i-th data segment is used as the trend alignment object, the distortion and aberration of the j-th data segment after alignment can be effectively controlled. It is understandable that the more similar the obtained correlation parameters indicate that the j-th data segment is to the i-th data segment after alignment, the less distortion and aberration the j-th data segment will have.

[0083] Therefore, by finely adjusting the alignment method of the j-th segment data, the distortion and aberration effects of the alignment operation on the j-th segment data can be further reduced, making the determined correlation parameters more accurate. The trend alignment object determined based on the correlation parameters can reduce the impact of the alignment operation in subsequent actual alignment operations, thereby effectively controlling the degree of distortion and aberration of the obtained aligned data while eliminating time offset.

[0084] S203: Based on the correlation parameters corresponding to the n fragment data, determine one fragment data as the trend alignment object for actual alignment.

[0085] After obtaining the correlation parameters between the j-th data segment and the i-th data segment after alignment, the correlation parameters corresponding to the n data segments can be obtained respectively. The correlation parameters are used as the basis to determine the trend alignment object. The correlation parameters can identify the similarity between the change trend of the other data segments after alignment and the data segments used as the benchmark. This can effectively control the distortion and aberration of the subsequently obtained aligned data while eliminating time offset, and ensure the accuracy of business data period identification.

[0086] The embodiments of this application do not specifically limit the method of determining the trend alignment object. The following is an example of one method, see S2031-S2032.

[0087] S2031: For the i-th data segment, determine the average correlation parameter corresponding to the i-th data segment based on the correlation parameters corresponding to the i-th data segment after the n-1 data segments are aligned to the i-th data segment.

[0088] For the i-th data segment First, obtain the j-th segment data. To the i-th data segment The correlation parameter between the aligned data and the i-th segment can be expressed as: ,in, This represents the time-series alignment path. Then, iterate through all values ​​of j to obtain the correlation parameters between each of the n-1 data segments aligned to the i-th data segment and the i-th data segment. Based on these n-1 correlation parameters, obtain the average correlation parameter, which can be expressed as: Among them, n-1 fragment data are the fragment data other than the i-th fragment data among the n fragment data.

[0089] S2032: Based on the average correlation parameters corresponding to the n data segments, the data segments whose average correlation parameters meet the conditions are identified as the trend alignment objects for actual alignment.

[0090] Traversal Given all values, obtain the average correlation parameter corresponding to each of the n data segments, and select the data segments that meet the conditions from the n average correlation parameters to determine the trend alignment objects used for actual alignment.

[0091] For example, a data segment that meets the condition of being the data segment corresponding to the largest average correlation parameter among n average correlation parameters can be represented as follows: of : .Will corresponding This serves as the trend alignment object for actual alignment. Therefore, when aligning n-1 data segments using this trend alignment object as a reference, the translation amount of the n-1 data segments is minimized, further reducing distortion and aberration caused by the alignment operation, and ensuring the authenticity of the waveform curve as much as possible.

[0092] S204: Adjust the business data into aligned data by aligning the other fragment data (excluding the trend alignment object) based on the trend alignment object among the n fragment data.

[0093] After determining the trend alignment object, the trend alignment object is used as the benchmark. The other data segments in the n data segments, excluding the trend alignment object, are actually aligned, thereby adjusting the business data to the alignment object.

[0094] This application does not specifically limit the process of adjusting business data to aligned data. The following description takes the splicing method as an example. After S302, it also includes S303 and S304.

[0095] S303: Align other fragment data with the trend alignment object based on the time alignment path between other fragment data and the trend alignment object to obtain aligned fragment data.

[0096] After determining the trend alignment object, each other data segment, except for the trend alignment object, is aligned based on the trend alignment object according to its own temporal alignment path with the trend alignment object, thus obtaining n-1 aligned data segments, which can be represented as follows: .

[0097] S304: The aligned fragment data and the trend alignment object are spliced ​​together in chronological order to obtain aligned data.

[0098] Iterate through all values ​​of j and extract n-1 data fragments. Align objects with trends The data is stitched together in the order of the undetermined time period to obtain aligned data. .

[0099] S205: In response to identifying business data as periodic data based on aligned data, determine the target period of the business data according to the pending time period, and identify data anomalies in the business data according to the target period.

[0100] After obtaining the aligned data, compared to the business data, the aligned data eliminates the relative time offset between the fragment data, thus making the identification of whether the business data is periodic based on the aligned data more accurate.

[0101] Different types of business data have different characteristics. To facilitate subsequent anomaly identification, different strategies can be adopted for different types of business data. The processing methods for two types of business data are explained below.

[0102] If the business data is periodic, the target period of the business data can be determined based on the pending time period, and data anomaly identification can be performed on the business data based on the target period. For example, data anomaly identification can be performed on the business data based on the target period and the first strategy.

[0103] If the business data is non-periodic, the second strategy can be used to identify data anomalies, thereby identifying different types of business data and using different strategies for data anomaly identification.

[0104] As one possible approach, since the periods of periodic data may vary, relying on only one strategy for data anomaly handling may result in many missed alarms and false alarms. Therefore, different strategies can be adopted for different target periods to identify data anomalies in business data.

[0105] The embodiments of this application do not specifically limit the first strategy and the second strategy. For example, the first strategy can be a year-on-year strategy, such as a time-series decomposition algorithm, the three-sigma rule (3-sigma), etc., and the second strategy can be a month-on-month strategy.

[0106] Therefore, even if the volume of business data is large and changes frequently, the data processing method provided in this application embodiment can automatically determine the type of business data, or even the target period of the business data, and select corresponding strategies to identify data anomalies based on the characteristics of the business data, thereby improving the accuracy of data anomaly identification.

[0107] As can be seen from the above technical solution, for the business data to be identified, the business data is sequentially divided into n segments according to the undetermined time period. Since the change trend between each segment may have a relative time offset under the time length indicated by the undetermined time period, such as the change trend being lagging or leading, in order to eliminate the impact of the relative time offset on the accuracy of period identification, the correlation parameter between the j-th segment and the i-th segment is determined based on the time offset of the change trend of the j-th segment relative to the i-th segment under the time length. Since the correlation parameter can identify the similarity between the change trend of the j-th segment and the i-th segment after the above alignment, the trend alignment object for actual alignment is determined from the n segments based on the correlation parameters corresponding to the n segments. After aligning the other segments except the trend alignment object with the trend alignment object, the distortion and aberration of the obtained aligned data can be effectively controlled under the premise of eliminating the time offset. Therefore, if the business data is identified as periodic based on the aligned data, the target period determined according to the pending time period is more consistent with the actual period of the business data. This allows for more accurate identification of data anomalies based on the target period.

[0108] Currently, the fluctuation trends of business data generated by the business systems are very similar, but the data volume (trend) differs to some extent, such as... Figure 4 As shown in (a), the fluctuation trend of the segment data under the time length identified by each undetermined time period has a certain degree of repetition, but the corresponding business data values ​​(amplitudes) are different. In order to eliminate the existing differences and improve the accuracy of subsequent data anomaly identification, the trend part of the n segment data is removed before S202.

[0109] The trend component refers to the difference in the total amount of data at different times among the n data segments, reflecting the changing trend. For example, in... Figure 4In (a), the first and last data segments are basically similar in terms of fluctuation trend, but the total amount of data at the time corresponding to the time length indicated by the undetermined time period is different.

[0110] The embodiments of this application do not specifically limit the method of removing the trend part from n data segments. The following description uses the classic time series decomposition algorithm as an example.

[0111] Using the classic time-series decomposition algorithm, business data can be divided into three distinct parts: a trend component, a seasonal component, and a residual. The trend component represents the changes in the volume of business data, while the seasonal component demonstrates the periodicity of the business data.

[0112] Classical time series decomposition algorithms include two types: additive and multiplicative models. The additive model can be represented as follows: The multiplication model can be represented as .in, This is represented as the trend portion. Indicates the seasonal component. This represents the residual. The two models can be converted to each other; taking the logarithm of both sides of the multiplicative model yields the additive model.

[0113] Analysis of business data revealed that data classified as periodic generally exhibits a characteristic of changing periodically over time, or in other words, the seasonal or cyclical changes are proportional to the level of the business data. Therefore, a multiplicative model was chosen to decompose the business data and remove the trend component (see S401-S404).

[0114] S401: Obtain Business Data .

[0115] For example, to obtain 15 days of business data, see [link / reference]. Figure 4 (a). In Figure 4 In (a), the business data to be identified has been sequentially divided into 15 segments by a pending time period of 1 day.

[0116] S402: Yes Take the logarithm.

[0117] The fluctuation trend of business data after taking the logarithm is as follows Figure 4 As shown in (b), it can be represented as Among them, for Adding 1 is to avoid The singularity is generated by the occurrence of 0.

[0118] S403: Determine by moving average The trending part.

[0119] If the moving average uses a window of 240, the business data is collected in 6-minute increments, and 240 business data values ​​can be collected per day, which is the number of points within a certain time period.

[0120] The calculated trend can be represented as Among them, the moving average is obtained The length of the data is .

[0121] S404: Calculate the seasonal component and residuals.

[0122] It can be represented as .

[0123] S405: Yes Smoothing is performed to obtain business data with trend components removed. .

[0124] Exponential moving averages can be used Smoothing can be represented as: The processed data is as follows: Figure 4 As shown in (c), since the data length is reduced by moving average in S403, therefore here... compared to The length of one cycle has been reduced.

[0125] Therefore, after removing the trend component, the relative time offset can be eliminated using methods S201-S204, such as... Figure 4 As shown in (d).

[0126] It should be noted that if there is no relative time offset between the data segments, and the determined time offset is zero, then even alignment operations will not affect the fluctuation trend of the j-th data segment. Figure 4 (c) and Figure 4 As shown in (d), if there is a relative time offset between the fragment data, and the determined time offset is not zero, the relative time offset between different fragment data can be adaptively adjusted through the alignment operation, so that the j-th fragment data is more similar to the i-th fragment data after being aligned with the i-th fragment data, and the determined similarity parameter is more accurate.

[0127] After obtaining the alignment data, it is possible to identify whether the business data is periodic based on the alignment data. The following explanation uses S501-S502 as examples.

[0128] S501: Determine the spectrum data based on the alignment data.

[0129] For example, the DFT method can be used to transform the aligned data from the time domain to the frequency domain to obtain the corresponding spectral data. See [link to relevant documentation]. Figure 4 (e).

[0130] The basic idea of ​​DFT is that it meets certain conditions. Point Discrete Function It can be decomposed into the sum of a series of simple complex exponential functions, as shown in the following formula:

[0131]

[0132] in, It is the imaginary unit. For the corresponding complex exponential function angular frequency, is a complex number representing the coefficient of the corresponding complex exponential function. Its magnitude reflects the strength of the influence of the complex exponential function on the discrete function. This formula is the inverse Fourier transform. Correspondingly, Yes The Fourier transform is used to calculate the result, as shown in the following formula:

[0133]

[0134] in, Indicates frequency, and They differ by only one coefficient. This constitutes a discrete function The spectrum.

[0135] It should be noted that even It is a discrete function, after transformation It is also a continuous function, but if Having a periodicity At a specific frequency Only then can there be a non-zero value, which can be represented as Therefore, the aligned data can be calculated using the DFT method. The spectral intensity can be expressed as By analyzing the intensity of various frequencies, it can be determined whether the business data is periodic, or even the corresponding period.

[0136] S502: Determine the frequency intensity ratio between the sum of the spectral intensities of the peaks in the spectrum data and the spectral intensity of the spectrum data. If the frequency intensity ratio is greater than or equal to the threshold, the service data is periodic data; if the frequency intensity ratio is less than the threshold, the service data is non-periodic data.

[0137] like Figure 4 As shown in (e), the peaks in the spectral data are 7, 14, 21, etc. The frequency intensity ratio between the sum of the spectral intensities of the peaks and the spectral intensities of the spectral data is obtained. For example, in... Figure 4In (e), there are 240 data points within one undetermined time period, and the total data length for d days is [data length missing]. Therefore, if the business data has a target period of days, it can be represented as follows: Then the spectral intensity of the spectrum data Non-zero values ​​only exist when the frequency is an integer multiple of d. Therefore, the spectral intensity at frequencies that are integer multiples of d can be statistically analyzed. occupy The ratio of frequency intensity to the periodicity of business data is used to measure its quality. The formula for the frequency intensity ratio is as follows:

[0138]

[0139] The first-order norm is used here for aggregation. and Alternatively, a second-order norm can be used, but this application does not specifically limit this. It should be noted that the calculated... The larger the value, the better the periodicity of the business data. Figure 4 In (e), .

[0140] As one possible implementation, the threshold can be set to 0.6. If the business data is periodic, then the business data is periodic data. If so, the business data is non-periodic.

[0141] After determining that the business data is periodic data, Frequency reflects its periodic information. It can be recovered using an inverse Fourier transform. The corresponding discrete function in the time domain, i.e. the periodic part of the index data, can be expressed as: This determines the target cycle for business data. Figure 4 After filtering the frequencies that are not multiples of 14 in (e), the periodic portion of the business data can be obtained by performing an inverse discrete Fourier transform, such as... Figure 4 As shown in (f).

[0142] As one possible implementation, when determining the target period, the period of the peaks in the spectrum data can be obtained. Based on the ratio of the number of data segments included in the spectrum data to the period of the peaks in the spectrum data, the target period of the service data can be determined. It is understood that the data segments are determined based on a predetermined time period. For example, Figure 4 (e) includes 14 data segments, and the period of the peaks in the spectral data is 14. Therefore, Figure 4 The business data shown has a period of 1 day.

[0143] As one possible implementation, the high-frequency part of the DFT spectrum can be filtered out before performing an inverse Fourier transform to achieve the purpose of noise reduction.

[0144] To better understand the data processing method provided in the embodiments of this application, the following is combined with... Figures 4-7 Taking three types of business data as examples, the data processing method provided in this application embodiment will be described. The three examples of business data are: (1) periodic data without relative time offset; the processing procedure is described in [reference needed]. Figure 4 (2) There are periodic data with relative time offsets. For the processing procedure, please refer to [link / reference]. Figure 6 (3) For non-periodic data, please refer to the processing procedure. Figure 7 .

[0145] Large internet companies provide a large number of applications and network services to the outside world through servers that host their business systems. From a monitoring perspective, these business systems generate business data during operation. Once a service failure occurs, the corresponding business data will show abnormal fluctuations.

[0146] To improve the accuracy of data anomaly identification, different strategies are used for different types of business data. If the business data is periodic, a year-on-year comparison strategy can be used to determine if there is an anomaly at the current moment. If the business data is non-periodic, a month-on-month comparison strategy can be used. If the current business data is outside the reasonable fluctuation range of historical data at the same time, it can be considered that the business system has an anomaly at that moment, requiring maintenance personnel to handle it and push an alarm message.

[0147] If the type of business data is incorrectly determined, leading to the adoption of an incorrect strategy, or if the period of periodic data is incorrectly determined and a strategy corresponding to a different period is used, the fluctuation range of business data will be too large. Even if an anomaly occurs in the business system, it will be difficult to detect, resulting in missed alarms or false alarms.

[0148] However, there exists a type of business data with a relative time offset. This offset can affect the judgment of the business data type, leading to misjudgment or incorrect calculation of the target period. Therefore, this application provides a data processing method that can accurately determine the type of business data, especially the aforementioned types, and automatically configure a switching strategy based on the periodic changes of the business data to achieve the purpose of algorithmic routing.

[0149] See Figure 5 The figure is a flowchart of a data processing method provided in an embodiment of this application.

[0150] S1: Decompose the business data and remove the trending parts.

[0151] Business data is processed using the methods described in S401-S405 above. The data is broken down and trend-following components are removed to obtain business data with the trend components removed. For details, please refer to Figure 4 (a)- Figure 4 (c) Figure 6 (a)- Figure 6 (b) and Figure 7 (a)- Figure 7 (c).

[0152] in, Figure 4 The fluctuation trends of each segment of the business data shown are very similar, but the numerical values ​​differ significantly. By removing the trending parts, the remaining parts can well reflect the periodicity of the business data.

[0153] Business data The data can be offline or online; this application does not specifically limit it. As one possible implementation, business data can also be... Noise reduction.

[0154] S2: Remove relative time offsets from business data.

[0155] Obtain business data (this business data can be business data with trend components removed). Afterwards, it can also be used for business data. After that, remove it through the aforementioned steps S201-S204. The relative time offset is used to obtain aligned data. For details, please refer to [link / reference]. Figure 4 (d) Figure 6 (c) and Figure 7 (d).

[0156] in, Figure 4 and Figure 7 There is no relative time offset between the data segments shown, and S2 will not affect their fluctuation trend. Figure 6 The data segments shown exhibit similar fluctuation trends, but they are not consistent in time, exhibiting either lag or lead, meaning there is a relative time offset between the data segments. S2 can adaptively remove this relative time offset. Without removing the relative time offset using S2, the subsequent calculations... In this case, the business data will be judged as non-periodic data, which contradicts the fact that the business data should be periodic data.

[0157] S3: Perform a discrete Fourier transform on the aligned data.

[0158] For aligned data Perform a discrete Fourier transform to obtain the spectral intensity of the spectral data. The sum of the spectral intensities of the peaks in the spectrum data For details, please refer to [link / reference]. Figure 4 (e) Figure 6 (d) and Figure 7 (e).

[0159] S4: Calculate the frequency intensity proportion of the spectrum data .

[0160] Among them, Figure 4 (e) only shows a portion of the spectral data, while the calculated... It applies to all spectral intensities. The sum of the non-peak spectral intensities accounts for a small proportion, and their effect can be ignored. Figure 6 In (d), The sum of the non-peak spectral intensities accounts for a small proportion, and their effect can be ignored. Figure 7 In (e), The sum of the non-peak spectral intensities accounts for a large proportion, and their effect cannot be ignored; otherwise, the periodic data generated by the inverse Fourier transform will produce significant distortion, such as... Figure 7 As shown in (f).

[0161] S5: Judgment If the value is greater than or equal to the threshold, then execute S6; otherwise, execute S8.

[0162] The S3-S5 process is based on aligned data. To determine whether business data is periodic, refer to the aforementioned S501-S502, where the threshold can be set to 0.6. If the business data is periodic, then execute S6. If the business data is non-periodic, then execute S8.

[0163] S6: Perform inverse discrete Fourier transform on the spectral data.

[0164] Figure 4 and Figure 6 The business data shown are all periodic data. Their period can be determined by performing an inverse discrete Fourier transform. For details, please refer to [link to relevant documentation]. Figure 4 (f) and Figure 6 (e).

[0165] S7: Use a year-on-year comparison strategy to identify data anomalies.

[0166] S8: Use a month-on-month comparison strategy to identify data anomalies.

[0167] Therefore, by combining classical time series decomposition, DTW, and DFT algorithms, the ability to determine and detect the periodicity of business data is improved. The determined target period is more consistent with the actual period of business data, which can more accurately identify data anomalies based on the target period and configure different strategies for business data with different characteristics to achieve the purpose of algorithm routing.

[0168] In addition to the data processing method provided in the above embodiments, this application also provides a data processing apparatus.

[0169] See Figure 8 This figure is a schematic diagram of a data processing device provided in an embodiment of this application. Figure 8 As shown, the data processing device 800 includes: a division unit 801, a determination unit 802, an adjustment unit 803, and a data anomaly identification unit 804;

[0170] The partitioning unit 801 is used to sequentially divide the business data to be identified into n fragments of data according to a predetermined time period.

[0171] The determining unit 802 is configured to determine, based on the changing trend of the j-th segment data relative to the time offset of the i-th segment data within the time length identified by the undetermined time period, the correlation parameter between the j-th segment data and the i-th segment data after alignment, wherein the i-th segment data is one of the n segment data, and the j-th segment data is any one of the n segment data except the i-th segment data; and based on the correlation parameter corresponding to each of the n segment data, determine one segment data as the trend alignment object for actual alignment from the n segment data.

[0172] The adjustment unit 803 is used to adjust the business data into aligned data by aligning the other fragment data in the n fragment data except for the trend alignment object based on the trend alignment object;

[0173] The data anomaly identification unit 804 is configured to respond to identifying the business data as periodic data based on the alignment data, determine the target period of the business data according to the pending time period, and perform data anomaly identification on the business data according to the target period.

[0174] As one possible implementation, the business data includes m data feature points over the time length, and the determining unit is used to:

[0175] By determining the temporal similarity between m data feature points in the j-th data segment and m data feature points in the i-th data segment, a temporal alignment path for identifying the time offset is obtained;

[0176] Based on the time alignment path and the j-th segment data, determine the correlation parameters between the j-th segment data and the i-th segment data after aligning the j-th segment data to the i-th segment data.

[0177] As one possible implementation, the determining unit 802 is used to:

[0178] Based on the position information of the p-th data feature point in the i-th data segment, the position information of the q-th data feature point in the j-th data segment, and the difference between p and q, the distance parameter between the p-th data feature point and the q-th data feature point is determined, where p and q are both less than or equal to m;

[0179] Based on the distance parameter between any data feature point in the i-th data segment and any data feature point in the j-th data segment, determine the set of distance parameters that reflect the temporal similarity;

[0180] A timing alignment path for identifying the time offset is obtained based on the set of distance parameters.

[0181] As one possible implementation, the determining unit 802 is used to:

[0182] By using the temporal alignment path between the other fragment data and the trend alignment object, the other fragment data is aligned based on the trend alignment object to obtain the aligned fragment data;

[0183] The aligned fragment data and the trend alignment object are concatenated in chronological order to obtain the aligned data.

[0184] As one possible implementation, the determining unit 802 is used to:

[0185] For the i-th data segment, the average correlation parameter corresponding to the i-th data segment is determined based on the correlation parameters corresponding to the i-th data segment after the n-1 data segments are aligned to the i-th data segment. The n-1 data segments are the data segments other than the i-th data segment among the n data segments.

[0186] Based on the average correlation parameters corresponding to the n data segments, the data segments whose average correlation parameters meet the conditions are determined as trend alignment objects for actual alignment.

[0187] As one possible implementation, the device further includes a removal unit for:

[0188] Before determining the correlation parameter between the j-th segment data and the i-th segment data after aligning the j-th segment data with the i-th segment data based on the time offset of the time length identified by the undetermined time period according to the changing trend of the j-th segment data, the trend part in the n segments data is removed. The trend part is caused by the difference in the total amount of data at different times in the changing trend of the n segments data.

[0189] As one possible implementation, the device further includes a periodic determination unit, used for:

[0190] The spectral data is determined based on the alignment data;

[0191] The frequency intensity ratio between the sum of the spectral intensities of the peaks in the spectrum data and the spectral intensity of the spectrum data is determined. If the frequency intensity ratio is greater than or equal to a threshold, the service data is periodic data; if the frequency intensity ratio is less than the threshold, the service data is non-periodic data.

[0192] As one possible implementation, the periodicity determination unit is used for:

[0193] Obtain the period of the peaks in the spectrum data;

[0194] The target period of the service data is determined based on the number of data segments included in the spectrum data and the ratio of the periods of the peaks in the spectrum data, wherein the data segments are determined based on the undetermined time period.

[0195] As one possible implementation, in response to identifying the business data as periodic data based on the alignment data, the data anomaly identification unit 804 is configured to:

[0196] The business data is identified as having anomalies based on the target period and the first strategy.

[0197] In response to identifying the business data as non-periodic based on the alignment data, the data anomaly identification unit 804 is configured to:

[0198] The business data is used to identify data anomalies according to the second strategy.

[0199] The data processing apparatus provided in this application divides the business data to be identified into n segments according to a predetermined time period. Since the change trend between each segment may have a relative time offset under the time length indicated by the predetermined time period, such as the change trend being lagging or leading, in order to eliminate the impact of the relative time offset on the accuracy of period identification, the correlation parameter between the j-th segment and the i-th segment is determined based on the time offset of the change trend of the j-th segment relative to the i-th segment under the time length. Since the correlation parameter can identify the similarity between the change trend of the j-th segment and the i-th segment after alignment, the trend alignment object for actual alignment is determined from the n segments based on the correlation parameters corresponding to the n segments. After aligning the other segments except the trend alignment object with the trend alignment object, the distortion and aberration of the obtained aligned data can be effectively controlled while eliminating the time offset. Therefore, if the business data is identified as periodic based on the aligned data, the target period determined according to the pending time period is more consistent with the actual period of the business data. This allows for more accurate identification of data anomalies based on the target period.

[0200] The aforementioned data processing device can be a computer device, which can be a server or a terminal device. The computer device provided in this application embodiment will be described below from a hardware implementation perspective. Figure 9 The diagram shown is a structural schematic of the server. Figure 10 The diagram shown is a structural schematic of the terminal device.

[0201] See Figure 9 , Figure 9 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1400 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1422 (e.g., one or more processors) and memory 1432, and one or more storage media 1430 (e.g., one or more mass storage devices) for storing application programs 1442 or data 1444. The memory 1432 and storage media 1430 can be temporary or persistent storage. The program stored in the storage media 1430 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 1422 may be configured to communicate with the storage media 1430 and execute the series of instruction operations in the storage media 1430 on the server 1400.

[0202] Server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input / output interfaces 1458, and / or one or more operating systems 1441, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0203] The steps performed by the server in the above embodiments can be based on this Figure 9 The server structure shown.

[0204] CPU1422 is used to perform the following steps:

[0205] The business data to be identified is divided into n segments according to the undetermined time period.

[0206] Based on the time offset of the j-th data segment relative to the i-th data segment at the time length indicated by the undetermined time period, determine the correlation parameter between the j-th data segment and the i-th data segment after aligning the j-th data segment to the i-th data segment. The i-th data segment is one of the n data segments, and the j-th data segment is any one of the n data segments except the i-th data segment.

[0207] Based on the correlation parameters corresponding to the n data segments, one data segment is determined from the n data segments as the trend alignment object for actual alignment;

[0208] The business data is adjusted into aligned data by aligning the other fragments of data (excluding the trend alignment object) based on the trend alignment object.

[0209] In response to identifying the business data as periodic data based on the alignment data, a target period for the business data is determined according to the pending time period, and data anomaly identification is performed on the business data according to the target period.

[0210] Optionally, CPU1422 may also execute method steps of any specific implementation of the data processing method in the embodiments of this application.

[0211] See Figure 10 , Figure 10 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 10The diagram shown is a block diagram of a portion of the structure of a smartphone related to the terminal device provided in the embodiments of this application. The smartphone includes components such as: a radio frequency (RF) circuit 1510, a memory 1520, an input unit 1530, a display unit 1540, a sensor 1550, an audio circuit 1560, a wireless fidelity (WiFi) module 1570, a processor 1580, and a power supply 1590. Those skilled in the art will understand that... Figure 10 The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0212] The following is combined with Figure 10 A detailed introduction to the various components of a smartphone:

[0213] RF circuit 1510 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 1580; additionally, it transmits uplink data to the base station. Typically, RF circuit 1510 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), and a duplexer. Furthermore, RF circuit 1510 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0214] The memory 1520 can be used to store software programs and modules. The processor 1580 executes the software programs and modules stored in the memory 1520 to realize various functions and data processing of the smartphone. The memory 1520 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the smartphone (such as audio data, phonebook, etc.). In addition, the memory 1520 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0215] The input unit 1530 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the smartphone. Specifically, the input unit 1530 may include a touch panel 1531 and other input devices 1532. The touch panel 1531, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 1531), and drive corresponding connected devices according to a pre-set program. Optionally, the touch panel 1531 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 1580, and can receive and execute commands sent by the processor 1580. In addition, the touch panel 1531 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1531, the input unit 1530 may also include other input devices 1532. Specifically, other input devices 1532 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0216] Display unit 1540 can be used to display information input by the user or information provided to the user, as well as various menus of the smartphone. Display unit 1540 may include display panel 1541, optionally configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel 1541. Further, touch panel 1531 may cover display panel 1541. When touch panel 1531 detects a touch operation on or near it, it transmits the information to processor 1580 to determine the type of touch event. Subsequently, processor 1580 provides corresponding visual output on display panel 1541 based on the type of touch event. Although in Figure 10 In this embodiment, the touch panel 1531 and the display panel 1541 are two separate components to realize the input and output functions of the smartphone. However, in some embodiments, the touch panel 1531 and the display panel 1541 can be integrated to realize the input and output functions of the smartphone.

[0217] Smartphones may also include at least one sensor 1550, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 1541 according to the ambient light level, and the proximity sensor can turn off the display panel 1541 and / or the backlight when the smartphone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the smartphone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometers, taps), etc. Other sensors that smartphones may also be equipped with, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0218] Audio circuit 1560, speaker 1561, and microphone 1562 provide an audio interface between the user and the smartphone. Audio circuit 1560 converts received audio data into electrical signals and transmits them to speaker 1561, where speaker 1561 converts them into sound signals for output. On the other hand, microphone 1562 converts collected sound signals into electrical signals, which are received by audio circuit 1560, converted into audio data, and then processed by processor 1580 before being transmitted via RF circuit 1510 to, for example, another smartphone, or the audio data can be output to memory 1520 for further processing.

[0219] WiFi is a short-range wireless transmission technology. Smartphones using the WiFi module 1570 can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 10 WiFi module 1570 is shown, but it is understood that it is not an essential component of a smartphone and can be omitted as needed without changing the nature of the invention.

[0220] The processor 1580 is the control center of the smartphone, connecting various parts of the smartphone through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 1520 and calling data stored in the memory 1520, thereby providing overall monitoring of the smartphone. Optionally, the processor 1580 may include one or more processing units; preferably, the processor 1580 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 1580.

[0221] The smartphone also includes a power supply 1590 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 1580 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0222] Although not shown, smartphones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0223] In this embodiment of the application, the memory 1520 included in the smartphone can store program code and transmit the program code to the processor.

[0224] The processor 1580 included in the smartphone can execute the data processing method provided in the above embodiments according to the instructions in the program code.

[0225] This application also provides a computer-readable storage medium for storing a computer program that executes the data processing method provided in the above embodiments.

[0226] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data processing methods provided in the various optional implementations of the above aspects.

[0227] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium can be at least one of the following media: read-only memory (ROM), RAM, magnetic disk or optical disk, and other media capable of storing program code.

[0228] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0229] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, The method includes: The business data to be identified is divided into n segments according to the undetermined time period. Based on the time offset of the j-th data segment relative to the i-th data segment at the time length indicated by the undetermined time period, determine the correlation parameter between the j-th data segment and the i-th data segment after aligning the j-th data segment to the i-th data segment. The i-th data segment is one of the n data segments, and the j-th data segment is any one of the n data segments except the i-th data segment. Based on the correlation parameters corresponding to the n data segments, a data segment is selected from the n data segments as the trend alignment object for actual alignment. This includes: for the i-th data segment, determining the average correlation parameter corresponding to the i-th data segment based on the correlation parameters corresponding to the i-th data segment after alignment with the n-1 data segments, where the n-1 data segments are all data segments other than the i-th data segment; and based on the average correlation parameters corresponding to the n data segments, determining the data segment whose average correlation parameter satisfies the condition as the trend alignment object for actual alignment. The business data is adjusted into aligned data by aligning the other fragments of data (excluding the trend alignment object) based on the trend alignment object. In response to identifying the business data as periodic data based on the alignment data, a target period for the business data is determined according to the pending time period, and data anomaly identification is performed on the business data according to the target period; Wherein, the business data includes m data feature points within the time length, and the step of determining the correlation parameter between the j-th segment data and the i-th segment data after alignment based on the time offset of the j-th segment data relative to the i-th segment data within the time length identified by the undetermined time period, according to the changing trend of the j-th segment data, includes: By determining the temporal similarity between m data feature points in the j-th data segment and m data feature points in the i-th data segment, a temporal alignment path for identifying the time offset is obtained. The temporal alignment path for identifying the time offset is the path that minimizes the cumulative distance. Based on the time alignment path and the j-th segment data, determine the correlation parameters between the j-th segment data and the i-th segment data after aligning the j-th segment data to the i-th segment data.

2. The method according to claim 1, characterized in that, The step of determining the temporal similarity between m data feature points in the j-th data segment and m data feature points in the i-th data segment to obtain a temporal alignment path for identifying the time offset includes: Based on the position information of the p-th data feature point in the i-th data segment, the position information of the q-th data feature point in the j-th data segment, and the difference between p and q, the distance parameter between the p-th data feature point and the q-th data feature point is determined, where p and q are both less than or equal to m; Based on the distance parameter between any data feature point in the i-th data segment and any data feature point in the j-th data segment, determine the set of distance parameters that reflect the temporal similarity; A timing alignment path for identifying the time offset is obtained based on the set of distance parameters.

3. The method according to claim 1, characterized in that, The step of aligning the business data into aligned data by aligning the other data segments (excluding the trend alignment object) among the n data segments based on the trend alignment object includes: By using the temporal alignment path between the other fragment data and the trend alignment object, the other fragment data is aligned based on the trend alignment object to obtain the aligned fragment data; The aligned fragment data and the trend alignment object are concatenated in chronological order to obtain the aligned data.

4. The method according to claim 1, characterized in that, Before determining the correlation parameter between the j-th segment data and the i-th segment data after alignment based on the time offset of the j-th segment data relative to the i-th segment data within the time length identified by the undetermined time period according to the changing trend of the j-th segment data, the method further includes: The trend portion is removed from the n data segments. The trend portion is caused by the difference in the total amount of data at different times in the trend of the n data segments.

5. The method according to claim 1, characterized in that, The method further includes: The spectral data is determined based on the alignment data; The frequency intensity ratio between the sum of the spectral intensities of the peaks in the spectrum data and the spectral intensity of the spectrum data is determined. If the frequency intensity ratio is greater than or equal to a threshold, the service data is periodic data; if the frequency intensity ratio is less than the threshold, the service data is non-periodic data.

6. The method according to claim 5, characterized in that, Determining the target period for the business data based on the pending time period includes: Obtain the period of the peaks in the spectrum data; The target period of the service data is determined based on the number of data segments included in the spectrum data and the ratio of the periods of the peaks in the spectrum data, wherein the data segments are determined based on the undetermined time period.

7. The method according to any one of claims 1-6, characterized in that, In response to identifying the business data as periodic data based on the alignment data, the method for identifying data anomalies in the business data according to the target period includes: The business data is identified as having anomalies based on the target period and the first strategy. In response to identifying the business data as aperiodic data based on the alignment data, the method further includes: The business data is used to identify data anomalies according to the second strategy.

8. A data processing apparatus, characterized in that, The device includes: a division unit, a determination unit, an adjustment unit, and a data anomaly identification unit; The partitioning unit is used to sequentially divide the business data to be identified into n data segments according to a predetermined time period. The determining unit is configured to determine, based on the time offset of the j-th segment data relative to the i-th segment data within the time length identified by the undetermined time period, a correlation parameter between the j-th segment data and the i-th segment data after alignment, wherein the i-th segment data is one of the n segment data, and the j-th segment data is any one of the n segment data except the i-th segment data; and to determine a segment data as a trend alignment object for actual alignment from the n segment data according to the correlation parameters corresponding to the n segment data, including: for the i-th segment data, determining the average correlation parameter corresponding to the i-th segment data based on the correlation parameters corresponding to the n-1 segment data after alignment, wherein the n-1 segment data are segment data other than the i-th segment data; and to determine the segment data whose average correlation parameter satisfies the condition among the n segment data as the trend alignment object for actual alignment based on the average correlation parameter corresponding to the n segment data. The adjustment unit is used to adjust the business data into aligned data by aligning the other fragment data in the n fragment data except for the trend alignment object based on the trend alignment object; The data anomaly identification unit is configured to respond to identifying the business data as periodic data based on the alignment data, determine the target period of the business data according to the pending time period, and perform data anomaly identification on the business data according to the target period. Wherein, the business data includes m data feature points within the time length, and the determining unit is used for: By determining the temporal similarity between m data feature points in the j-th data segment and m data feature points in the i-th data segment, a temporal alignment path for identifying the time offset is obtained. The temporal alignment path for identifying the time offset is the path that minimizes the cumulative distance. Based on the time alignment path and the j-th segment data, determine the correlation parameters between the j-th segment data and the i-th segment data after aligning the j-th segment data to the i-th segment data.

9. The apparatus according to claim 8, characterized in that, The determining unit is used for: Based on the position information of the p-th data feature point in the i-th data segment, the position information of the q-th data feature point in the j-th data segment, and the difference between p and q, the distance parameter between the p-th data feature point and the q-th data feature point is determined, where p and q are both less than or equal to m; Based on the distance parameter between any data feature point in the i-th data segment and any data feature point in the j-th data segment, determine the set of distance parameters that reflect the temporal similarity; A timing alignment path for identifying the time offset is obtained based on the set of distance parameters.

10. The apparatus according to claim 8, characterized in that, The determining unit is used for: By using the temporal alignment path between the other fragment data and the trend alignment object, the other fragment data is aligned based on the trend alignment object to obtain the aligned fragment data; The aligned fragment data and the trend alignment object are concatenated in chronological order to obtain the aligned data.

11. The apparatus according to claim 8, characterized in that, The device further includes a removal unit for: Before determining the correlation parameter between the j-th segment data and the i-th segment data after aligning the j-th segment data with the i-th segment data based on the time offset of the time length identified by the undetermined time period according to the changing trend of the j-th segment data, the trend part in the n segments data is removed. The trend part is caused by the difference in the total amount of data at different times in the changing trend of the n segments data.

12. The apparatus according to claim 8, characterized in that, The device further includes a periodic determination unit, used for: The spectral data is determined based on the alignment data; The frequency intensity ratio between the sum of the spectral intensities of the peaks in the spectrum data and the spectral intensity of the spectrum data is determined. If the frequency intensity ratio is greater than or equal to a threshold, the service data is periodic data; if the frequency intensity ratio is less than the threshold, the service data is non-periodic data.

13. The apparatus according to claim 12, characterized in that, The periodicity determination unit is used for: Obtain the period of the peaks in the spectrum data; The target period of the service data is determined based on the number of data segments included in the spectrum data and the ratio of the periods of the peaks in the spectrum data, wherein the data segments are determined based on the undetermined time period.

14. A computer device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1-7 according to the instructions in the program code.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the method according to any one of claims 1-7.

16. A computer program product, characterized in that, The computer program product includes computer instructions, which are executed by the processor of the computer device to cause the computer device to perform the method according to any one of claims 1-7.

Images