A method, apparatus and device for processing delay data

Abstract

An embodiment of this specification discloses a method, device, and equipment for processing delay data. The method includes: obtaining delay data of each subtask of a stream computing task, associatively storing the delay data of the subtask with a preset label of the subtask, dividing the delay data associatively stored with any preset label into a group to obtain each first delay data sequence. According to a preset rule, the delay data included in at least part of the first delay data sequences is processed to obtain target delay data required for the target device to perform analysis and processing on the stream computing task. This solution records each delay data in the form of a label, and the delay data provided to the target device is the delay data obtained after relevant processing on a certain number of real-time delay data, so as to improve the simplicity of recording the delay data and improve the efficiency of the target device in analyzing the delay data.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular, to a method, apparatus, and device for processing delay data. Background Art

[0002] In the prior art, the delay data of each subtask in the process of processing streaming data is recorded in the form of logs. Since the logs record the corresponding delay data in the form of strings, the delay data of each subtask corresponds to a string template. When it is necessary to record the delay data of other subtasks, it is necessary to modify the string template in the logs. Since it is difficult to modify the strings in the logs, the method of recording the delay data of each subtask in the process of processing streaming data in the prior art is relatively cumbersome.

[0003] In addition, after recording all the delay data in the form of logs in the prior art, all the delay data in the form of logs will be saved in the memory. When the target device needs to analyze the delay data, all the delay data in the form of logs will be sent to the target device. The target device will perform a large amount of analysis and processing on all the delay data in the form of logs, and then obtain the required target delay data. Therefore, this will require a large amount of computing resources of the target device and affect the efficiency of the target device for analyzing the delay data.

[0004] Based on this, how to improve the simplicity of recording the delay data of each subtask and the real-time performance and efficiency of the target device for analyzing the delay data has become an urgent technical problem to be solved. Summary of the Invention

[0005] A method, apparatus, and device for processing delay data provided in the embodiments of this specification use tags to record the delay data of each subtask, and the target delay data sent to the target device for analysis is the delay data obtained by corresponding processing of the real-time received delay data within a preset time, thereby improving the simplicity of recording the delay data of each subtask and the real-time performance and efficiency of the target device for analyzing the delay data.

[0006] To solve the above technical problems, the embodiments of this specification are implemented as follows:

[0007] A method for processing delay data includes:

[0008] Obtain the delay data of each subtask of the stream computing task to obtain a set of delay data.

[0009] Associate and store the delay data of the subtask with the preset tag of the subtask.

[0010] Divide the latency data associated with any one of the preset tags into a group to obtain each first latency data sequence.

[0011] According to a preset rule, process the latency data included in at least part of the first latency data sequences to obtain target latency data required for the target device to perform analysis and processing on the stream computing task.

[0012] Send the target latency data to the target device.

[0013] A service request processing device includes

[0014] An acquisition module, configured to acquire the latency data of each subtask of a stream computing task to obtain a latency data set.

[0015] A storage module, configured to associatively store the latency data of the subtask and the preset tag of the subtask.

[0016] A division module, configured to divide the latency data associated with any one of the preset tags into a group to obtain each first latency data sequence.

[0017] A processing module, configured to process the latency data included in at least part of the first latency data sequences according to a preset rule to obtain target latency data required for the target device to perform analysis and processing on the stream computing task.

[0018] A sending module, configured to send the target latency data to the target device.

[0019] A service request processing device includes

[0020] At least one processor; and

[0021] A memory communicatively connected to the at least one processor; wherein

[0022] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to:

[0023] Acquire the latency data of each subtask of a stream computing task to obtain a latency data set.

[0024] Associatively store the latency data of the subtask and the preset tag of the subtask.

[0025] Divide the latency data associated with any one of the preset tags into a group to obtain each first latency data sequence.

[0026] Process the delay data included in at least part of the first delay data sequence according to preset rules to obtain target delay data required for the target device to analyze and process the stream computing task.

[0027] Send the target delay data to the target device.

[0028] At least one embodiment provided in this specification can achieve the following beneficial effects:

[0029] The delay data of each subtask in the process of processing stream data is recorded in the form of tags, and the delay data sent to the target device for analysis is the data obtained by correspondingly processing each delay data received in real time within a preset time, so as to improve the simplicity of recording each delay data involved in the process of processing the stream computing task, and improve the real-time performance and efficiency of the target device in analyzing the obtained delay data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0031] Figure 1 It is a schematic flowchart of a method for processing delay data provided in an embodiment of this specification;

[0032] Figure 2 It is a schematic diagram of each subtask included in the stream computing task provided in an embodiment of this specification;

[0033] Figure 3 Corresponding to Figure 1 It is a schematic structural diagram of a device for processing delay data provided in an embodiment of this specification;

[0034] Figure 4 Corresponding to Figure 1 It is a schematic structural diagram of a device for processing delay data provided in an embodiment of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] To make the objectives, technical solutions, and advantages of one or more embodiments of this specification clearer, the following will clearly and completely describe the technical solutions of one or more embodiments of this specification in conjunction with specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all of them. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by one or more embodiments of this specification.

[0036] The following will, with reference to the drawings, detail the technical solutions provided by each embodiment of this application.

[0037] Figure 1 It is a schematic flowchart of a method for processing delay data provided by an embodiment of this specification. From a program perspective, the execution subject of this process can be a device for processing delay data, or an application program running on the device for processing delay data. As Figure 1 shown, this process may include the following steps:

[0038] Step 102: Obtain the delay data of each subtask of the stream computing task to obtain a set of delay data.

[0039] In the embodiments of this specification, as Figure 2 shown, each frame of stream data is input from the beginning, processed through each subtask, and finally the processed stream data is output. The processing of each subtask takes a certain amount of time, and the time consumed by each subtask is the corresponding delay data of each subtask. By statistically calculating the delay data of multiple subtasks, a set of delay data is obtained. A subtask can be any step that is the smallest unit in the stream computing task, such as: the step of calling a function, and so on.

[0040] Step 104: Associatively store the delay data of the subtask with the preset label of the subtask.

[0041] In an embodiment of the present specification, a preset label is set for each subtask in a stream computing task. When the corresponding subtask is executed and generates corresponding delay data, the delay data generated by the subtask is associated with the preset label of the subtask and stored, so that multiple delay data generated after the same subtask is executed multiple times are respectively set with the same preset label. The storage location of the delay data of the subtask has a certain correlation with the storage location of the preset label of the subtask. The delay data of the subtask and the preset label of the subtask can be stored in one field, or the delay data of the subtask and the preset label of the subtask can be stored in two different fields respectively, and the two different fields have a certain corresponding correlation. For example: assuming that a function in subtask 1 is called, the generated delay data takes N milliseconds. When the delay data of the function being called in subtask 1 and the preset label of the subtask are stored in the same field, they can be stored as: function·subtask 1 function 1·N milliseconds, where function·subtask 1 function 1 is the preset label of the subtask in which the function in subtask 1 is called; when the delay data of the function being called in subtask 1 and the preset label of the subtask are stored in two different fields, they can be stored as: preset label stored in field 1: function·subtask 1 function 1, delay data stored in field 2: N milliseconds, and a corresponding correlation is set between field 1 and field 2. This solution does not impose any specific restrictions on the storage location of the delay data of the subtask and the preset tag of the subtask. It only needs to store the delay data of the subtask and the preset tag of the subtask in an associated storage manner.

[0042] Step 106: Divide the time delay data stored in association with any one of the preset tags into a group to obtain first time delay data sequences.

[0043] In an embodiment of the present specification, each delay data associated with the same preset tag is divided into a group to obtain multiple first delay data sequences. Each delay data in each delay data sequence is the delay data generated after any same subtask is executed multiple times. Each delay data in each delay data sequence can be sorted according to the order of their respective generation times. The earlier the generation time of the delay data, the later its position in the first delay data sequence to which it belongs. The later the generation time of the delay data, the earlier its position in the first delay data sequence to which it belongs. The most forward position in the first delay data sequence represents that the delay data at this position is the latest generated delay data, so as to ensure that the delay data at the most forward position in the first delay data sequence is the delay data generated in real time.

[0044] Step 108: Process the delay data included in at least part of the first delay data sequence according to a preset rule to obtain target delay data required for the target device to analyze and process the stream computing task.

[0045] In the embodiments of the present specification, according to the requirements of the delay data required by the target device, a corresponding first delay data sequence is selected, and the delay data in the first delay data sequence is processed to obtain the target delay data required by the target device. When the target delay data required by the target device is data such as the maximum delay data, minimum delay data, average delay data, variance of delay data, quantile delay data, etc. of any sub-task, any number of consecutive delay data can be selected from the corresponding first delay data sequence. For the any number of consecutive delay data, the corresponding maximum delay data, minimum delay data, average delay data, variance of delay data, quantile delay data, etc. are calculated. The any number of consecutive delay data can be all the delay data included in the first delay data sequence or part of the delay data included in the first delay data sequence. This solution does not specifically limit the delay data required for calculating the maximum delay data, minimum delay data, average delay data, variance of delay data, quantile delay data, etc.

[0046] Step 110: Send the target delay data to the target device.

[0047] In the embodiments of the present specification, after obtaining the corresponding target delay data according to the requirements of the target device for the delay data, the corresponding target delay data is sent to the target device. The target delay data sent to the target device is not the original delay data, but the delay data obtained by processing a preset number of original delay data according to the requirements of the target device. The target device can directly perform corresponding analysis and processing based on the processed delay data without preprocessing the target delay data.

[0048] In the embodiments of the present specification, the delay data of each sub-task in the process of stream data processing is recorded in the form of labels, and the delay data sent to the target device for analysis is the data obtained by processing the respective delay data received in real time within a preset time, thereby improving the simplicity of recording each delay data involved in the processing of the stream computing task and improving the real-time performance and efficiency of the target device in analyzing the obtained delay data.

[0049] Based on Figure 1 the method in, the embodiments of the present specification also provide some specific implementation schemes of the method, which will be described below.

[0050] The stream computing task is used to process a plurality of consecutive stream data obtained within a preset time.

[0051] The preset tag of the subtask is a tag in the key-value pair format. The key data of the preset tag is used to reflect the type of data processing method used by the subtask; the value data of the preset tag is used to reflect the unique task identifier of the subtask and the technical information required for data processing according to the data processing method.

[0052] In the embodiments of this specification, the stream computing task is a computing task for a plurality of consecutive stream data received within a preset time, and the plurality of stream data are all of the same type of stream data. Stream data is a sequence of data that arrives sequentially, in large quantities, quickly, and continuously. Generally, stream data can be regarded as a dynamically growing data set that grows infinitely over time. It is applied in fields such as network monitoring, sensor networks, aerospace, meteorological measurement and control, and financial services. For example, stream data can be the vehicle operation data collected in real time by a data acquisition device for an autonomous vehicle.

[0053] In the embodiments of this specification, the preset tag of each delay data can be a tag in the key-value pair format. A key-value pair means that a corresponding value data can be obtained according to a key data. The key data of the preset tag is used to reflect the type of data processing method used by the subtask, and the value data of the preset tag is used to reflect the unique task identifier of the subtask and the technical information required for data processing according to the data processing method. For example: if a certain function needs to be called in subtask 1, the preset tag set when the function in this subtask 1 is called can be set as follows: key data: function, value data: subtask 1 function 1. When this function is also called in subtask 2, the preset tag set when the function in this subtask 2 is called can be set as follows: key data: function, value data: subtask 2 function 1. When the same function is called in different subtasks, the preset tags set are different. For calling the same function in the same subtask for different frame data, the preset tags set are the same.

[0054] In the embodiments of this specification, tags in the key-value pair format are used to record each delay data generated in the stream computing task, thereby improving the simplicity of recording each delay data generated in the stream computing task.

[0055] The stream computing task contains various subtasks. Therefore, the delay data set obtained according to the stream computing task contains delay data generated during the processing of multiple subtasks. For the convenience of calculation and processing, it is necessary to classify different types of delay data. Based on this, step 106: The step of dividing the delay data associated with any one of the preset tags and storing them into a group to obtain each first delay data sequence may specifically include:

[0056] For any one of the preset tags, obtain the delay data of the key-value pair carrying the any one of the preset tags from the delay data set, so as to obtain a delay data subset.

[0057] According to the preset duration and the first preset quantity for the first time window, determine a first delay data sequence from the delay data subset, where the first delay data sequence includes the delay data generated within the first time window with the preset quantity of consecutive ones.

[0058] In the embodiments of this specification, the delay data corresponding to the preset tags with exactly the same tag key-value pairs are divided into one group to obtain each delay data subset. Each delay data in each delay data subset is the delay data generated for the same subtask. Obtain the generation time of each delay data in each delay data subset, and put the delay data that meets the preset time requirement into a time window. Each time window corresponds to a certain preset time period. For example, each time window corresponds to a time period of 5 seconds. Then, all the delay data generated within the first 5 seconds are all put into the 1# time window, and all the delay data generated within the second 5 seconds are all put into the 2# time window, and so on, to obtain multiple time windows. If according to the above classification, all the delay data in a certain delay data subset obtained are the delay data generated within the first 5 seconds, then all the delay data in the delay data subset are all put into the 1# time window. Because the stream computing task is a continuously occurring task, as the stream data task continues, new delay data will be generated. Then, all the new delay data generated within the second 5 seconds are all put into the 2# time window, and so on, and the delay data generated within each 5 seconds are put into the corresponding time window. For the delay data of the same type of subtask, as the stream computing task progresses, multiple consecutive time windows will be obtained, and each time window includes the delay data generated within the corresponding preset time period. All the delay data included in multiple consecutive time windows will form a first delay data sequence. According to the actual requirement situation, each first delay data sequence can include a preset quantity of time windows, and the specific quantity of time windows is not limited. If the time windows included in the delay data sequence are more, the delay data included in the delay data sequence are also more, and when corresponding analysis is performed according to the delay data sequence, the accuracy of the obtained analysis result is higher. If the time windows included in the delay data sequence are less, the delay data included in the delay data sequence are also less, and when corresponding analysis is performed according to the delay data sequence, the real-time performance of the obtained analysis result is higher.

[0059] In the embodiments of the present specification, each delay data generated by the stream computing task is classified accordingly, and the delay data with exactly the same preset tags is divided into a group for corresponding analysis and processing, thereby improving the convenience and accuracy of analyzing and calculating the delay data.

[0060] Each first delay data sequence contains a preset number of time windows. When processing the delay data within the corresponding first delay data sequence, it can be processed according to the delay data within some or all of the time windows. Based on this, step 108: Processing the delay data included in at least part of the first delay data sequences according to a preset rule to obtain target delay data required for the target device to perform analysis and processing on the stream computing task may specifically include:

[0061] For any one of the first delay data sequences, select any one of the first time windows from the respective first time windows corresponding to the first delay data sequence to obtain a second time window.

[0062] Calculate the target delay data according to the respective delay data within the second time window, where the target delay data includes at least one of maximum delay data, minimum delay data, average delay data, delay data variance, and quantile delay data.

[0063] In the embodiments of the present specification, it is assumed that the first delay data sequence contains 6 consecutive time windows, namely time windows 1# to 6#. The time period corresponding to each time window is assumed to be 5 seconds. The 1# time window is the delay data generated within the first 5 seconds, and so on. The 6# time window is the delay data generated within the sixth 5 seconds. According to the requirements of the target device, the delay data within any one of the time windows from 1# to 6# can be selected for corresponding calculation to obtain the target delay data required by the target device. For example: If the target device needs the real-time maximum delay data within the latest time period, the maximum delay data is selected from the 6# time window according to the priority queue of each delay data within the time window and sent to the target device; when the target device needs the average value data of the delay data within the third 5 seconds, all the delay data included in the 3# time window is averaged, and the obtained average delay data is sent to the target device. In this solution, the types of target delay data required by the target device may include at least one of maximum delay data, minimum delay data, average delay data, delay data variance, quantile delay data, etc.

[0064] Step 108: Processing the delay data included in at least part of the first delay data sequences according to a preset rule to obtain target delay data required for the target device to perform analysis and processing on the stream computing task may further include:

[0065] For any one of the first delay data sequences, select a second preset number of the first time windows from each of the first time windows corresponding to the first delay data sequence to obtain the second preset number of third time windows.

[0066] Perform priority sorting on each of the delay data included in the second preset number of the third time windows to obtain a priority sequence for each of the delay data.

[0067] Select the required delay data from the priority sequence to obtain the target delay data, where the target delay data includes at least one of maximum delay data, minimum delay data, average delay data, delay data variance, and quantile delay data.

[0068] In the embodiments of this specification, it is assumed that the preset number of time windows included in the first delay data sequence is the same as the scenario of the 6 consecutive time windows included in the above first delay data sequence. According to the requirements of the target device, various delay data included in multiple consecutive time windows within the 1# time window to the 6# time window can be selected for corresponding calculation and processing. For example, if the delay data at the 99th percentile within the third 5 seconds to the fifth 5 seconds is required. Then all the delay data included in the three time windows from the 3# time window to the 5# time window need to be sorted in ascending order according to the delay data, and then the delay data at the 99% position is selected from the sorted priority sequence and sent to the target device. It should be noted that according to the requirements of the target device, other data such as the maximum delay data, minimum delay data, average delay data, and delay data variance can also be calculated from the sorted priority sequence.

[0069] For priority sorting of all the delay data within multiple consecutive time windows, sorting can be performed according to certain rules. Based on this, the performing priority sorting on each of the delay data included in the second preset number of the third time windows to obtain a priority sequence for each of the delay data may specifically include:

[0070] Determine the delay data with the smallest value in each of the third time windows to obtain each first delay data.

[0071] Sort each of the first delay data in ascending order of value to obtain a first queue.

[0072] Select the first delay data with the smallest value from the first queue to obtain the second delay data belonging to the first priority level.

[0073] Select a third delay data from within the third time window to which the second delay data belongs. The third delay data is the delay data with the smallest value within the third time window to which the second delay data belongs, excluding the second delay data.

[0074] Put the third delay data into the first queue after deleting the second delay data to obtain a second queue.

[0075] Select the delay data with the smallest value from the second queue to obtain the fourth delay data belonging to the second priority level. The second priority level is lower than the first priority level and higher than other priority levels other than the first priority level.

[0076] And so on until the priority levels to which all the processing delays within each of the third time windows belong are determined, obtaining a priority sequence for each of the delay data.

[0077] In the embodiments of this specification, assuming that the 90th percentile delay data of all delay data within the 3# time window to the 5# time window is required, the method for sorting the priority levels of each delay data within the 3# time window to the 5# time window is as follows:

[0078] Since the delay data within each time window has been automatically sorted by size, select the smallest delay data from within the 3# time window to the 5# time window respectively. For example: the smallest delay data within the 3# time window is the a delay data, the smallest delay data within the 4# time window is the b delay data, and the smallest delay data within the 5# time window is the c delay data. Put the a delay data, b delay data, and c delay data into a first queue respectively. This first queue can automatically sort the data put into the queue. Assuming that the smallest delay data among the a delay data, b delay data, and c delay data is the a delay data, select the a delay data from the first queue and use the a delay data as the first data for priority sorting. Then select the second smallest d delay data from within the 3# time window to which the a delay data belongs. The d delay data is the delay data within the 3# time window that is only greater than the a delay data. Put the d delay data into the first queue that currently only has the b delay data and c delay data. Assuming that the smallest delay data among the b delay data, c delay data, and d delay data is the c delay data, select the c delay data as the second data for priority sorting. According to the above method, select all the delay data within the 3# time window to the 5# time window in sequence, and finally obtain a priority sequence.

[0079] In the embodiments of this specification, the target delay data sent to the target device is the delay data obtained by performing corresponding calculation and processing on each delay data within the corresponding time window according to the requirements of the target device. The target device can directly perform analysis and processing based on the received target delay data without preprocessing the delay data, thereby saving the computing resources of the target device and improving the real-time performance and rate of the target device for analyzing delay data.

[0080] Step 108: Before processing the delay data included in at least part of the first delay data sequence according to the preset rule, it may further include:

[0081] For any one of the first delay data sequences, when the duration from the creation time of the first delay data sequence to the last creation of a time window reaches a preset duration, a fourth time window is created for the first delay data sequence.

[0082] Judge whether the target number of the time windows created at the position of the first delay data sequence is greater than a third preset number to obtain a first judgment result.

[0083] If the first judgment result indicates that the target number of the created time windows is greater than the third preset number, then in the order of the creation time of the created time windows from early to late, delete a fourth preset number of the created time windows in sequence, where the fourth preset number is the difference between the target number and the third preset number.

[0084] In the embodiments of this specification, the target delay data is sent to the target device at a certain preset frequency. Before each corresponding calculation and processing of the target delay data required by the target device, a new time window needs to be created for each delay data sequence. If the target delay data is sent to the target device once every 5 seconds, then a new time window is created for the corresponding first delay data sequence every 5 seconds. The new time window is used to receive the delay data generated within the next 5 seconds. After calculating the corresponding target delay data based on the time window included in the first delay data sequence, it should be noted that since the new time window has just been created, the time window required for calculation at this time does not include the newly created time window. Determine the total number of time windows created in the first delay data sequence. The total number at this time includes the newly created time window. When the total number of time windows created in the first delay data sequence is greater than the third preset number of time windows set by the system for the first delay data sequence, calculate the difference e between the total number of the above time windows and the third preset number, and then delete the corresponding e time windows in order of the creation time of the time windows included in the first delay data sequence from the earliest to the latest. The number of time windows included in the first delay data sequence after deleting the e time windows is equal to the third preset number. It should be noted that the comparison between the total number of time windows created for the first delay data sequence and the third preset number of time windows set for the first delay data sequence can be made once for each time window created for the first delay data sequence, or can be made once when a time window is randomly created for the first delay data sequence.

[0085] In the embodiments of this specification, only the corresponding delay data within the preset number of time windows created at the most recent moment is retained in each first delay data sequence, so as to improve the real-time performance of the target delay data obtained after corresponding calculation and processing based on each delay data in the first delay data sequence.

[0086] In order to ensure the real-time performance of each delay data in the delay data set, before grouping the delay data in the delay data set, it is also necessary to determine whether the total number in the delay data set meets the preset requirements. Based on this, step 106: Before dividing the delay data associated with any one of the preset tags and stored into a group to obtain each first delay data sequence, it may further include:

[0087] Count the total number of the delay data included in the delay data set to obtain a first value.

[0088] Determine whether the first value is greater than the fifth preset number to obtain a second judgment result.

[0089] If the second judgment result indicates that the first value is greater than the fifth preset quantity, then, in the order of the generation time of the delay data from early to late, delete the sixth preset quantity of the delay data in sequence, where the sixth preset quantity is the difference between the first value and the fifth preset quantity.

[0090] In the embodiments of this specification, the total quantity of each piece of delay data received from the stream computing task is counted, and it is judged whether the total quantity of each piece of delay data is greater than the fifth preset quantity. When the total quantity of each piece of delay data is greater than the fifth preset quantity, calculate the difference t between the total quantity of each piece of delay data and the fifth preset quantity, and then delete t corresponding pieces of delay data in sequence according to the generation time of each piece of delay data included in the delay data set from early to late. The quantity of delay data included in the delay data set after deleting t pieces of delay data is equal to the fifth preset quantity. Thus, the delay data with a longer generation time in the delay data set is deleted accordingly to ensure that the delay data in the delay data set is the delay data that is newly generated in the most recent time period, so as to improve the timeliness of the target delay data required by the target device.

[0091] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 3 For the embodiments of this specification to provide the corresponding Figure 1 structural schematic diagram of a device for processing delay data. As Figure 3 shown, the device may include:

[0092] An obtaining module 302, configured to obtain the delay data of each subtask of the stream computing task to obtain a delay data set.

[0093] A storage module 304, configured to store the delay data of the subtask and the preset label of the subtask in an associated manner.

[0094] A dividing module 306, configured to divide the delay data stored in association with any one of the preset labels into a group to obtain each first delay data sequence.

[0095] A processing module 308, configured to process the delay data included in at least part of the first delay data sequences according to a preset rule to obtain the target delay data required for the target device to perform analysis and processing on the stream computing task.

[0096] A sending module 310, configured to send the target delay data to the target device.

[0097] Optionally, the stream computing task is used to process a continuous plurality of stream data obtained within a preset time.

[0098] The preset tag of the subtask is a tag in the key-value pair format. The key data of the preset tag is used to reflect the type of data processing method used by the subtask; the value data of the preset tag is used to reflect the unique task identifier of the subtask and the technical information required for data processing according to the data processing method.

[0099] Optionally, the partitioning module 306 may specifically include:

[0100] An acquisition unit, configured to obtain, for any one of the preset tags, the delay data carrying the key-value pair of any one of the preset tags from the delay data set, so as to obtain a delay data subset.

[0101] A determination unit, configured to determine, according to a preset duration and a first preset quantity for a first time window, a first delay data sequence from the delay data subset, where the first delay data sequence includes the delay data generated within the first time window that is continuous and has the first preset quantity.

[0102] Optionally, the processing module 308 may specifically include:

[0103] A first selection unit, configured to select, for any one of the first delay data sequences, any one of the first time windows from each of the first time windows corresponding to the first delay data sequence, so as to obtain a second time window.

[0104] A calculation unit, configured to calculate the target delay data according to each of the delay data within the second time window, where the target delay data includes at least one of maximum delay data, minimum delay data, average delay data, delay data variance, and quantile delay data.

[0105] Optionally, the processing module 308 may further include:

[0106] A second selection unit, configured to select, for any one of the first delay data sequences, a second preset quantity of the first time windows from each of the first time windows corresponding to the first delay data sequence, so as to obtain the second preset quantity of third time windows.

[0107] A sorting unit, configured to perform priority sorting on each of the delay data included in the second preset quantity of the third time windows, so as to obtain a priority sequence for each of the delay data.

[0108] A third selection unit, configured to select the required delay data from the priority sequence to obtain the target delay data, where the target delay data includes at least one of maximum delay data, minimum delay data, average delay data, delay data variance, and quantile delay data.

[0109] Optionally, the sorting unit may specifically include:

[0110] A first determination subunit, configured to determine the delay data with the smallest value within each of the third time windows to obtain respective first delay data.

[0111] A sorting subunit, configured to sort the respective first delay data in ascending order of value to obtain a first queue.

[0112] A first selection subunit, configured to select the first delay data with the smallest value from the first queue to obtain second delay data belonging to the first priority.

[0113] A second selection subunit, configured to select a third delay data from within the third time window to which the second delay data belongs, where the third delay data is the delay data with the smallest value other than the second delay data within the third time window to which the second delay data belongs.

[0114] A putting subunit, configured to put the third delay data into the first queue after deleting the second delay data to obtain a second queue.

[0115] A third selection subunit, configured to select the delay data with the smallest value from the second queue to obtain fourth delay data belonging to the second priority, where the second priority is lower than the first priority and higher than other priorities other than the first priority.

[0116] A second determination subunit, configured to determine the priorities to which all the processing delays within each of the third time windows belong, for generating a priority sequence for each of the delay data.

[0117] Optionally, before the processing module 308, it may further include:

[0118] A creation module, configured to, for any one of the first delay data sequences, when the duration of the first delay data sequence from the last creation time window reaches a preset duration, create a fourth time window for the first delay data sequence.

[0119] A first judgment module, configured to judge whether the target number of time windows created at the first delay data sequence is greater than a third preset number to obtain a first judgment result.

[0120] The first deletion module is configured to, if the first judgment result indicates that the target number of the created time windows is greater than the third preset number, sequentially delete the fourth preset number of the created time windows in the order from the earliest creation time to the latest creation time of the created time windows, where the fourth preset number is the difference between the target number and the third preset number.

[0121] Optionally, before the partitioning module 306, it may further include:

[0122] A statistics module is configured to count the total number of the delay data included in the delay data set to obtain a first value.

[0123] A second judgment module is configured to judge whether the first value is greater than a fifth preset number to obtain a second judgment result.

[0124] A second deletion module is configured to, if the second judgment result indicates that the first value is greater than the fifth preset number, sequentially delete the sixth preset number of the delay data in the order from the earliest generation time to the latest generation time of the delay data, where the sixth preset number is the difference between the first value and the fifth preset number.

[0125] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method.

[0126] Figure 4 For the Figure 1 corresponding to the embodiments of this specification Figure 4 is a schematic structural diagram of a processing device for delay data. As

[0127] shown, the device 400 may include:

[0128] At least one processor 410; and,

[0129] A memory 430 communicatively connected to the at least one processor; where

[0130] The memory 430 stores instructions 420 executable by the at least one processor 410, and when the instructions are executed by the at least one processor 410, the at least one processor 410 can:

[0131] Obtain the delay data of each subtask of the stream computing task to obtain a delay data set.

[0132] Associatively store the delay data of the subtask with the preset label of the subtask.

[0133] Process the delay data included in at least part of the first delay data sequence according to a preset rule to obtain target delay data required for the target device to perform analysis and processing on the stream computing task.

[0134] Send the target delay data to the target device.

[0135] It should be understood that in the method described in one or more embodiments of this specification, the order of some steps can be adjusted according to actual needs, or some steps can be omitted.

[0136] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for Figure 4 the device shown, since it is basically similar to the method embodiment, the description is relatively simple. For related parts, reference can be made to the partial description of the method embodiment.

[0137] The above are only embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A method for processing delay data, characterized in that, Including: Obtain the latency data of each subtask of the stream computing task to obtain a latency data set; Associate and store the latency data of the subtask with the preset label of the subtask; Divide the latency data associated and stored with any one of the preset labels into a group to obtain each first latency data sequence; According to a preset rule, process the latency data included in at least part of the first latency data sequences to obtain target latency data required for the target device to perform analysis and processing on the stream computing task; Send the target latency data to the target device; The stream computing task is used to process a continuous plurality of stream data obtained within a preset time; The preset label of the subtask is a label in key-value pair format. The key data of the preset label is used to reflect the type of data processing method used by the subtask; the value data of the preset label is used to reflect the unique task identifier of the subtask and the technical information required for data processing according to the data processing method; The dividing the latency data associated and stored with any one of the preset labels into a group to obtain each first latency data sequence specifically includes: For any one of the preset labels, obtain the latency data carrying the key-value pair of the any one of the preset labels from the latency data set to obtain a latency data subset; According to the preset duration of the first time window and the first preset quantity, determine one of the first latency data sequences from the latency data subset. The first latency data sequence includes the latency data generated within the first time window with the preset quantity of consecutive ones; The processing the latency data included in at least part of the first latency data sequences according to a preset rule to obtain target latency data required for the target device to perform analysis and processing on the stream computing task specifically includes: For any one of the first latency data sequences, select any one of the first time windows from each of the first time windows corresponding to the first latency data sequence to obtain a second time window; Calculate the target latency data according to the latency data within the second time window. The target latency data includes at least one of maximum latency data, minimum latency data, average latency data, latency data variance, and quantile latency data.

2. The method according to claim 1, wherein The processing the latency data included in at least part of the first latency data sequences according to a preset rule to obtain target latency data required for the target device to perform analysis and processing on the stream computing task further includes: For any one of the first latency data sequences, select a second preset quantity of the first time windows from each of the first time windows corresponding to the first latency data sequence to obtain the second preset quantity of third time windows; Perform priority sorting on the latency data included in the second preset quantity of the third time windows to obtain a priority sequence for each of the latency data; Select the required delay data from the priority sequence to obtain the target delay data, where the target delay data includes at least one of the maximum delay data, minimum delay data, average delay data, variance of delay data, and quantile delay data.

3. The method according to claim 2, characterized in that, The prioritizing the delay data included in each of the second preset number of the third time windows to obtain a priority sequence for each of the delay data specifically includes: Determine the delay data with the smallest value in each of the third time windows to obtain respective first delay data; Sort the respective first delay data in ascending order of value to obtain a first queue; Select the first delay data with the smallest value from the first queue to obtain second delay data belonging to the first priority; Select a third delay data from the third time window to which the second delay data belongs, where the third delay data is the delay data with the second smallest value in the third time window to which the second delay data belongs except the second delay data; Put the third delay data into the first queue after deleting the second delay data to obtain a second queue; Select the delay data with the smallest value from the second queue to obtain fourth delay data belonging to the second priority, where the second priority is lower than the first priority and higher than other priorities other than the first priority; And so on, until the priorities to which all the processing delays in each of the third time windows belong are determined, to obtain a priority sequence for each of the delay data.

4. The method according to claim 1, wherein Before processing the delay data included in at least part of the first delay data sequences according to a preset rule, it further includes: For any one of the first delay data sequences, when the duration from the time when the first delay data sequence was created to the previous time window creation reaches a preset duration, create a fourth time window for the first delay data sequence; Judge whether the target number of the time windows already created at the first delay data sequence is greater than a third preset number to obtain a first judgment result; If the first judgment result indicates that the target number of the time windows already created is greater than the third preset number, then delete the fourth preset number of the already created time windows in order of the creation time of the already created time windows from earliest to latest, where the fourth preset number is the difference between the target number and the third preset number.

5. The method according to claim 1, characterized in that Before dividing the delay data associated with any one of the preset tags into a group to obtain respective first delay data sequences, it further includes: Count the total number of the delay data included in the delay data set to obtain a first value; Judge whether the first value is greater than a fifth preset number to obtain a second judgment result; If the second judgment result indicates that the first value is greater than the fifth preset quantity, then, in the order from the earliest to the latest generation time of the delay data, delete the sixth preset quantity of the delay data in sequence, where the sixth preset quantity is the difference between the first value and the fifth preset quantity.

6. A processing device for delay data, characterized in that, Including: An acquisition module, configured to acquire the delay data of each subtask of the stream computing task to obtain a delay data set; The stream computing task is used to process a continuous plurality of stream data acquired within a preset time; A storage module, configured to associatively store the delay data of the subtask and the preset label of the subtask; the preset label of the subtask is a label in the key-value pair format, and the key data of the preset label is used to reflect the type of data processing method used by the subtask; the value data of the preset label is used to reflect the unique task identifier of the subtask and the technical information required for data processing according to the data processing method; A division module, configured to divide the delay data associatively stored with any one of the preset labels into a group to obtain each first delay data sequence; specifically including: for any one of the preset labels, acquire the delay data carrying the key-value pair of any one of the preset labels from the delay data set to obtain a delay data subset; according to the preset duration and the first preset quantity of the first time window, determine one of the first delay data sequences from the delay data subset, and the first delay data sequence includes the delay data generated within the first time window of the preset quantity of consecutive ones; A processing module, configured to process the delay data included in at least part of the first delay data sequences according to a preset rule to obtain target delay data required for the target device to perform analysis and processing on the stream computing task; specifically including: for any one of the first delay data sequences, select any one of the first time windows from each of the first time windows corresponding to the first delay data sequence to obtain a second time window; calculate the target delay data according to the delay data within the second time window, and the target delay data includes at least one of maximum value delay data, minimum value delay data, average value delay data, delay data variance, and quantile delay data; A sending module, configured to send the target delay data to the target device.

7. A processing device for delay data, characterized in that Including: At least one processor; And, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor can: Acquire the delay data of each subtask of the stream computing task to obtain a delay data set; Associatively store the delay data of the subtask and the preset label of the subtask; Divide the delay data associatively stored with any one of the preset labels into a group to obtain each first delay data sequence; Process the delay data included in at least part of the first delay data sequence according to preset rules to obtain target delay data required for the target device to analyze and process the stream computing task; Send the target delay data to the target device; The stream computing task is used to process a continuous plurality of stream data obtained within a preset time; The preset tag of the subtask is a tag in key-value pair format. The key data of the preset tag is used to reflect the type of data processing method used by the subtask; the value data of the preset tag is used to reflect the unique task identifier of the subtask and the technical information required for data processing according to the data processing method; The dividing the delay data associated with and stored for any one of the preset tags into a group to obtain each first delay data sequence specifically includes: For any one of the preset tags, obtain the delay data carrying the key-value pair of the any one of the preset tags from the delay data set to obtain a delay data subset; According to the preset duration of the first time window and the first preset quantity, determine one of the first delay data sequences from the delay data subset. The first delay data sequence includes the delay data generated within the first time window with the preset quantity of consecutive ones; The processing the delay data included in at least part of the first delay data sequence according to preset rules to obtain target delay data required for the target device to analyze and process the stream computing task specifically includes: For any one of the first delay data sequences, select any one of the first time windows from each of the first time windows corresponding to the first delay data sequence to obtain a second time window; Calculate the target delay data according to the delay data within the second time window. The target delay data includes at least one of maximum delay data, minimum delay data, average delay data, delay data variance, and quantile delay data.

Images