A method and related device for concurrently scheduling and executing time series data

Abstract

The present application discloses a method and related device for concurrently scheduling and executing time series data, which is applied to a time series data scheduling and execution system. The method includes: obtaining and caching input time series data, the time series data includes a first number of different instance data; obtaining and assembling the first number of different instance data to obtain a second number of instance data and corresponding processing requests; concurrently processing the second number of instance data and the corresponding processing requests through a second number of threads to obtain a second number of data packets; performing time reconstruction and rectification operations on the time series data of the second number of data packets, and outputting the first number of concurrent instance data. By implementing the embodiments of the present invention, multiple threads can simultaneously serve one instance data, and one thread can jointly schedule multiple instance data, thereby improving the processing performance for a single instance data and achieving the best performance of concurrent scheduling and execution of time series data.

Description

Technical Field

[0001] The present invention relates to the field of electronic technology, and in particular to a method and related device for concurrently scheduling and executing time series data. Background Art

[0002] The so-called time series data refers to the data that is input in time order, and the processing output needs to be consistent with the original input time order, such as alarm data, indicator data, performance data, etc. in telecommunications systems. Especially in AI video processing. The detection / feature extraction stage of all video data can be considered as the processing of time series data. In actual systems, time series data processing is very critical. It is the biggest constraint factor of the system's service throughput. In the prior art, for the processing of time series data, multi-threaded concurrent processing is enabled, but each time series processing instance starts a thread; or each thread is bound to the time series processing instance, and a thread can only process the thread bound to it. There may be problems such as the processing capacity of a single time series data instance needs to exceed the processing capacity of a single thread or the number of threads generally has a reasonable upper limit, and exceeding the upper limit will deteriorate system performance. In summary, a technology that can efficiently complete high-concurrency scheduling of time series data is urgently needed. Summary of the invention

[0003] An embodiment of the present invention provides a method and related apparatus for concurrently scheduling and executing time series data. By implementing the embodiment of the present invention, it is hoped that multiple threads can simultaneously serve one instance data, and one thread can jointly schedule multiple instance data, thereby improving the processing performance for a single instance data and achieving concurrent scheduling and optimal execution performance of time series data.

[0004] In a first aspect, an embodiment of the present application provides a method for concurrently scheduling and executing time series data, which is applied to a time series data scheduling and execution system, and the method includes:

[0005] Acquire and cache input time series data, wherein the time series data includes a first number of different instance data;

[0006] Acquire and assemble the first quantity of different instance data to obtain a second quantity of instance data and corresponding processing requests;

[0007] Concurrently processing the second number of instance data and corresponding processing requests through a second number of threads to obtain a second number of data packets;

[0008] Perform timing reconstruction and rectification operations on the timing data of the second number of data packets, and output the first number of concurrent instance data.

[0009] In a second aspect, an embodiment of the present application provides a device for concurrently scheduling and executing time series data, including a processing unit and a communication unit, wherein:

[0010] The processing unit is used to obtain and cache input timing data through the communication unit, where the timing data includes a first number of different instance data; obtain and assemble the first number of different instance data to obtain a second number of instance data and corresponding processing requests; concurrently process the second number of instance data and the corresponding processing requests through a second number of threads to obtain a second number of data packets; perform timing reconstruction and rectification operations on the timing data of the second number of data packets, and output a first number of concurrent instance data.

[0011] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps of any method in the first aspect of the embodiment of the present application.

[0012] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in any method of the first aspect of the embodiment of the present application.

[0013] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiment of the present application. The computer program product may be a software installation package.

[0014] It can be seen that in the embodiment of the present application, the input timing data is obtained and cached, and the timing data includes a first number of different instance data; the first number of different instance data is obtained and assembled to obtain a second number of instance data and corresponding processing requests; the second number of instance data and the corresponding processing requests are processed concurrently by a second number of threads to obtain a second number of data packets; the timing data of the second number of data packets are subjected to timing reconstruction and rectification operations to output the first number of concurrent instance data. It can be seen that the present application processes the second number of instance data and the corresponding processing requests concurrently by a second number of threads to obtain a second number of data packets, that is, multiple threads are implemented to serve a timing data instance at the same time, improving the processing performance for a single instance data; and different input instance data can be combined into the same data packet for execution, that is, one thread can jointly schedule multiple instance data to achieve concurrent scheduling of timing data and optimal execution performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0016] Figure 1 A schematic diagram of a system for concurrently scheduling and executing time series data provided by one embodiment of the present invention;

[0017] Figure 2 A flowchart of a method for concurrently scheduling and executing time series data provided by yet another embodiment of the present invention;

[0018] Figure 3a A schematic diagram of the structure of a time series data input and instance cache unit provided for one embodiment of the present invention;

[0019] Figure 3b A schematic diagram of a multiplexing configuration information structure provided by an embodiment of the present invention;

[0020] Figure 3c A schematic structural diagram of a random-order packaging unit provided by an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of the structure of a timing reconstruction unit provided by an embodiment of the present invention;

[0022] Figure 5 A flowchart of another method for concurrently scheduling and executing time series data provided by an embodiment of the present invention;

[0023] Figure 6 A schematic diagram of an electronic device provided by an embodiment of the present invention;

[0024] Figure 7 A block diagram of the functional units of a device for concurrently scheduling and executing time series data provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The following are detailed descriptions of each.

[0027] The terms "first" and "second" in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0028] like Figure 1 As shown, Figure 1The schematic diagram of a system 100 for concurrently scheduling and executing time series data is shown. The system 100 for concurrently scheduling and executing time series data includes a time series data input and instance cache unit 110, a random package assembly unit 120, a random concurrent execution unit 130, and a time series reconstruction unit 140. The time series data input and instance cache unit 110, the random package assembly unit 120, the random concurrent execution unit 130, and the time series reconstruction unit 140 are connected in sequence. The time series data input and instance cache unit 110 is used to implement the data cache processing of the time series data at the input end and send it to the random package assembly unit 120 to complete the package data acquisition and assembly for different instance data. Then the random concurrent execution unit 130 implements the concurrent processing of all requests through multi-threading. Finally, the time series reconstruction unit 140 completes the reconstruction operation of the time series data and the rectification operation of the data. The output data can be output in time series independently according to the instance. The system 100 for concurrently scheduling and executing timing data may include an integrated single device or multiple devices. For the convenience of description, the system 100 for concurrently scheduling and executing timing data is collectively referred to as an electronic device in this application. Obviously, the electronic device may include various handheld devices with wireless communication functions, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem, as well as various forms of user equipment (UE), mobile station (MS), terminal device, etc.

[0029] At present, the processing capacity of a single time series data instance needs to exceed the processing capacity of a single thread; due to the characteristics of the computing system, the number of threads generally has a reasonable upper limit. Exceeding the upper limit will deteriorate system performance. Due to the characteristics of the computing system, the underlying calculation hopes to further combine data from a larger instance range for concurrent processing, which makes it impossible to achieve the most efficient scheduling and execution of time series data.

[0030] Based on this, an embodiment of the present application proposes a method for concurrent scheduling and execution of timing data to solve the above-mentioned problems. The embodiment of the present application is introduced in detail below.

[0031] First, see Figure 2 , Figure 2 A flowchart of a method for concurrently scheduling and executing time series data provided by an embodiment of the present invention is applied to Figure 1 The time series data scheduling execution system shown in FIG. Figure 2 As shown, a method for concurrently scheduling and executing time series data provided by an embodiment of the present invention may include:

[0032] S201, acquiring and caching input time series data, where the time series data includes a first number of different instance data.

[0033] The time series data can be input in different time series data queues. By caching data with time series data instances as the granularity, a first-in first-out (FIFO)-based operation is implemented.

[0034] S202: Acquire and assemble the first quantity of different instance data to obtain a second quantity of instance data and corresponding processing requests.

[0035] In the specific implementation, actual batch processing and packaging operations are performed on the cached time series data, and necessary processing delay management is performed.

[0036] S203, concurrently processing the second number of instance data and corresponding processing requests through a second number of threads to obtain a second number of data packets;

[0037] S204, performing timing reconstruction and rectification operations on the timing data of the second number of data packets, and outputting the first number of concurrent instance data.

[0038] The timing reconstruction unit stores cache information in the timing data reconstruction process, such as a timing data buffer queue and a Reorder Timing clock, wherein the Reorder Timing clock refers to the most recent time when all previous data processed by the current Reorder has been completed.

[0039] It can be seen that in the embodiment of the present application, the input timing data is obtained and cached, and the timing data includes a first number of different instance data; the first number of different instance data is obtained and assembled to obtain a second number of instance data and corresponding processing requests; the second number of instance data and the corresponding processing requests are processed concurrently by a second number of threads to obtain a second number of data packets; the timing data of the second number of data packets are subjected to timing reconstruction and rectification operations to output the first number of concurrent instance data. It can be seen that the present application processes the second number of instance data and the corresponding processing requests concurrently by a second number of threads to obtain a second number of data packets, that is, multiple threads are implemented to serve a timing data instance at the same time, improving the processing performance for a single instance data; and different input instance data can be combined into the same data packet for execution, that is, one thread can jointly schedule multiple instance data to achieve concurrent scheduling of timing data and optimal execution performance.

[0040] In one possible example, the timing data scheduling execution system includes a timing data queue and a data multiplexing unit, the data multiplexing unit includes a multiplexing configuration unit and a data acquisition and packaging unit, and the acquisition and assembly of the first number of different instance data to obtain a second number of instance data and corresponding processing requests includes: caching the timing data at the granularity of timing data instances through the timing data queue; storing and providing multiplexing configuration information for outputting the timing data through the multiplexing configuration unit; and acquiring and packaging the first number of different instance data according to preset rules through the data acquisition and packaging unit to obtain a second number of instance data and corresponding processing requests.

[0041] The time series data is cached at the time series data instance granularity, and then output according to the preset data rules through the data multiplexing unit, such as Figure 3a As shown, Figure 3a This is a structural diagram of a timing data input and instance cache unit. Different timing data queues (for example: timing data queue 1, timing data queue 2...timing data queue N-1, timing data queue N) are input into the data multiplexing unit for data caching. The multiplexing configuration unit stores and provides multiplexing-related configuration information, and then obtains data and packages the data according to preset rules.

[0042] It can be seen that in this example, by caching the time series data at the granularity of time series data instances and outputting the cached time series data according to preset rules, different input instances are combined into the same package, and multiple time series data instances are jointly scheduled to achieve the best scheduling performance.

[0043] In a possible example, the acquisition and packaging of the first number of different instance data according to preset rules by the data acquisition and packaging unit to obtain the second number of instance data and corresponding processing requests include: obtaining the packaging request by the data acquisition and packaging unit; after obtaining the packaging request, obtaining the size of the level package and initializing the level packaging variable; inputting the level package size and the level packaging variable into the multiplexing configuration unit for processing, and outputting corresponding timing data.

[0044] Among them, Figure 3b , Figure 3b The schematic diagram of the multiplexing configuration information structure is as follows. Data packet acquisition includes waiting for a packet assembly request. When a packet assembly request is obtained and triggered, the size of the level packet is obtained, and the level assembly packet variable is initialized. Then, the size of the packet and the level assembly packet variable are input as follows. Figure 3b The multiplexing configuration information structure shown polls the instance data and then outputs the corresponding timing data.

[0045] In a specific implementation, when initializing the packet group, a timer is initialized and set to time T1. Then, whenever T1 times out, polling of instance data is performed, the quota margin is made equal to the quota configuration, and then polling of instance data is performed again.

[0046] It can be seen that in this example, by obtaining the packet group request; obtaining the size of the level packet and initializing the packet group variables; and then processing through the multiplexing configuration unit to output corresponding timing data, different input instances are combined into the same packet, improving the data processing efficiency.

[0047] In a possible example, the multiplexing configuration unit includes multiple multiplexing configuration information structures. The multiplexing configuration information structure includes a multiplexing configuration management buffer unit, an instance number acquisition unit, a quota configuration unit, and a quota margin unit. Inputting the size of the timing data packet group and the packet group variables into the multiplexing configuration unit for processing and outputting a second quantity of instance data and corresponding processing requests includes: determining whether an instance element of the timing data is successfully obtained; if so, obtaining the instance number of the instance element through the instance number acquisition unit, and obtaining two variables, namely the configuration margin variable, through the quota configuration unit and the quota margin unit; obtaining the queue depth of the corresponding instance queue according to the instance number; and outputting a second quantity of instance data and corresponding processing requests or performing the above operations on the next multiplexing configuration information structure according to the comparison result of the two variables of the queue depth and the configuration margin variable.

[0048] Among them, as Figure 3b shown, the multiplexing configuration information structure includes a multiplexing configuration management buffer unit, an instance number acquisition unit, a quota configuration unit, and a quota margin unit.

[0049] In a specific implementation, when a request is triggered, first obtain the size of the level packet, set it to N, and initialize the packet group variable M = 0; then perform polling of instance data, that is, after obtaining an instance element, obtain two variables, namely the instance ID (x) in the instance element and the configuration margin variable Left(x), and obtain the queue depth Depth(x) of the instance queue according to the instance ID (x). The obtaining principle is: when Depth(x) >= Left(x), obtain Left(x) data and make M = M + Left(x); when Depth(x) < Left(x), obtain Depth(x) elements and make M = M + Depth(x); determine the size of M and N. When M < N, obtain another instance element. When M = N, output the current data packet. Then detect the quota margin of all instance elements. If the margin is 0, send a quota reset signal and re-detect the packet group request.

[0050] In the specific implementation, when it is detected that the quota balance of all instance elements is 0, a quota reset signal is issued, an instance element is obtained for instance polling, and the quota balance is made equal to the quota configuration, and then the instance data is polled.

[0051] It can be seen that in this example, the size of the level package is obtained and the level group package variables are initialized; then the size of the level package and the level group package variables are processed, and then corresponding operations are taken according to the processing results, so that the same input instance is executed concurrently by different execution units, and different input instances are combined into the same package to improve data processing efficiency.

[0052] In one possible example, the time series data scheduling execution system includes an out-of-order packet assembly unit, and the out-of-order packet assembly unit includes a batch packet assembly unit, a batch configuration unit, a batch signal unit and a delay signal unit. The acquisition and assembly of the first number of different instance data to obtain the second number of instance data and corresponding processing requests include: obtaining a data packet pending signal, and querying the queue depth corresponding to the data packet pending signal; when the queue depth is greater than a preset depth, performing a batch packet assembly operation through the batch packet assembly unit; obtaining configuration information of the batch packet assembly and storing it in the batch configuration unit; performing signal management of the batch packet assembly through the batch signal unit to obtain signal information; processing the configuration information and the signal information through the batch packet assembly unit to obtain the second number of instance data and the corresponding processing request; when the queue depth is less than the preset depth, performing processing delay management through the delay signal unit.

[0053] Among them, Figure 3c As shown, Figure 3c It is a structural diagram of the out-of-order packet assembly unit. The batch configuration unit 303, the batch signal unit 304 and the delay signal unit 302 are all connected to the batch packet assembly unit 301. Among them, after obtaining the pending signal of the data packet, query the queue depth corresponding to the pending signal of the data packet; when the queue depth is greater than the preset depth, perform batch packet assembly operation through the batch packet assembly unit 301; obtain the configuration information of the batch packet assembly and store it in the batch configuration unit 303; perform signal management of the batch packet assembly through the batch signal unit 304 to obtain signal information; process the configuration information and signal information through the batch packet assembly unit to obtain the second number of instance data and the corresponding processing request; when the queue depth is less than the preset depth, perform processing delay management through the delay signal unit 302. Among them, the configuration information of the batch packet assembly stored in the batch configuration unit includes the batch processing capacity Capacity (y) of each downstream processing unit (i.e., the maximum batch packet size).

[0054] Among them, when the batch processing packet assembly unit receives the data packet pending signal, it queries the total queue depth TotalDepth of the timing data at this time and performs signal polling, that is, obtains an example element in the data instance, reads the status information. If the status is Ready, it reads the ID(x) number of the processing unit, and obtains the batch processing capacity Capacity(y) in the batch processing configuration information through this ID(x) number. When TotalDepth < Capacity(y), it waits for the data packet Pending signal again; when TotalDepth >= Capacity(y), it obtains the batch processing packet data and sends it to the downstream processing unit.

[0055] In a specific implementation, when the queue depth is less than the preset depth, the processing delay management is performed by the delay signal unit. For example, when the delay signal unit receives the data packet Pending signal and then clears the WatchDog. After the delay signal unit is triggered, it waits for the WatchDog timeout signal. When the timeout signal is received, it queries the total queue depth TotalDepth at this time and performs signal polling, that is, obtains an instance element, reads the status therein. If the status is Ready, it reads the ID(x) number of the processing unit, and obtains the batch processing capacity Capacity(y) in the batch processing configuration information through this ID(x) number. When TotalDepth < Capacity(y), it waits for the data packet Pending signal again; when TotalDepth >= Capacity(y), it obtains the batch processing packet data and sends it to the downstream processing unit.

[0056] It can be seen that in this example, the data packet pending signal is obtained, and the queue depth corresponding to the data packet pending signal is queried; when the queue depth is greater than the preset depth, the batch processing packet assembly operation is performed by the batch processing packet assembly unit; the configuration information of the batch processing packet assembly is obtained and stored in the batch processing configuration unit; the signal management of the batch processing packet assembly is performed by the batch processing signal unit to obtain signal information; the configuration information and signal information are processed by the batch processing packet assembly unit to obtain the second quantity of instance data and processing requests of the timing data, which is beneficial to improving the order of magnitude of the performance upper limit of a single instance, and realizing the joint scheduling of multiple timing data instance data to achieve the best scheduling performance.

[0057] In one possible example, the timing data scheduling execution system includes a reconstruction output packet scheduling unit and a reconstruction buffer queue; the timing reconstruction and rectification operations are performed on the timing data of the second number of data packets to output the first number of concurrent instance data, including: when the reconstruction output packet scheduling unit detects the input of the second number of instance data, the second number of instance data is added to the reconstruction buffer queue; reconstruction timing detection is performed on the data packet after the timing increment; if the data packet after the timing increment is in the reconstruction buffer queue, the first number of concurrent instance data after the timing increment is output; if the data packet after the timing increment is not in the reconstruction buffer queue, waiting for the second number of instance data to be input

[0058] Among them, Figure 4 As shown, Figure 4 The schematic diagram of the structure of the timing reconstruction unit includes a reconstruction output package and scheduling unit and a reconstruction buffer queue of timing data. The reconstruction buffer queue of timing data includes multiple multiplexing configuration management cache units, each of which includes a timing data number, that is, a timing data instance ID. The content stored in each instance element includes a timing data buffer queue and Reorder Timing.

[0059] In the specific implementation, the reconstruction output and scheduling process includes waiting for new data packets to be input. When a data packet arrives, the data packet is first added to the timing data buffer queue, and then a reconstruction timing (ReorderTime) test is performed. For example, when each timing increment is Delta (T), it is determined whether the data packet corresponding to ReorderTime + Delta (T) is in the Reorder cache queue. If so, the data packet corresponding to the current ReorderTime + Delta (T) is output, and ReorderTime = ReorderTime + Delta (T) is updated. Then, the relationship between the data packet corresponding to ReorderTime + Delta (T) and the Reorder cache queue is determined. If not, the current processing is terminated and waiting for new data packets to be input.

[0060] It can be seen that in this example, the timing reconstruction and rectification operations are performed on the timing data of the multiple groups of packets by reconstructing the output group packet scheduling unit and reconstructing the buffer queue, and the target timing data is output, so as to achieve an order of magnitude improvement in the performance upper limit of a single instance and realize the joint scheduling of multiple timing data instance data to achieve the best scheduling performance.

[0061] In one possible example, when the reconstruction output package scheduling unit detects a second number of instance data input, after adding the second number of instance data to the reconstruction buffer queue, it also includes: identifying the instance number of the input timing data; obtaining the corresponding reconstruction buffer queue according to the instance number, and saving the reconstruction buffer queue and the time when the data concurrency is completed before the reconstruction processing.

[0062] The contents stored in each instance element in the reconstruction buffer queue are the following two parts: a timing data buffer queue and a Reorder Timing clock. The Reorder Timing clock refers to the most recent time when all the previous data processed by the current Reorder has been completed.

[0063] It can be seen that in this example, the timing reconstruction and rectification operations are performed on the timing data of the multiple groups of packets by reconstructing the output group packet scheduling unit and reconstructing the buffer queue, and the target timing data is output, so as to achieve an order of magnitude improvement in the performance upper limit of a single instance and realize the joint scheduling of multiple timing data instance data to achieve the best scheduling performance.

[0064] With the above Figure 2 For details on the embodiments shown in the drawings, please refer to Figure 5 , Figure 5 is a flowchart of a method for concurrently scheduling and executing time series data provided by an embodiment of the present application, which is applied to Figure 1 The time series data scheduling and execution system, as shown in the figure, the method of concurrently scheduling and executing time series data includes:

[0065] S501, caching the time series data at a granularity of a time series data instance through the time series data queue;

[0066] S502, storing and providing multiplexing configuration information for outputting the time series data through the multiplexing configuration unit;

[0067] S503, acquiring and packaging the first quantity of different instance data according to a preset rule by the data acquisition and packaging unit to obtain a second quantity of instance data and corresponding processing requests;

[0068] S504, acquiring and assembling the first quantity of different instance data to obtain a second quantity of instance data and corresponding processing requests;

[0069] S505, concurrently processing the second number of instance data and corresponding processing requests through a second number of threads to obtain a second number of data packets;

[0070] S506: Perform timing reconstruction and rectification operations on the timing data of the second number of data packets, and output the first number of concurrent instance data.

[0071] It can be seen that in the embodiment of the present application, the input timing data is obtained and cached, and the timing data includes a first number of different instance data; the first number of different instance data is obtained and assembled to obtain a second number of instance data and corresponding processing requests; the second number of instance data and the corresponding processing requests are processed concurrently by a second number of threads to obtain a second number of data packets; the timing data of the second number of data packets are subjected to timing reconstruction and rectification operations to output the first number of concurrent instance data. It can be seen that the present application processes the second number of instance data and the corresponding processing requests concurrently by a second number of threads to obtain a second number of data packets, that is, multiple threads are implemented to serve a timing data instance at the same time, improving the processing performance for a single instance data; and different input instance data can be combined into the same data packet for execution, that is, one thread can jointly schedule multiple instance data to achieve concurrent scheduling of timing data and optimal execution performance.

[0072] With the above Figure 2 , Figure 5 For details on the embodiments shown in the drawings, please refer to Figure 6 , Figure 6 6 is a structural diagram of an electronic device 600 provided in an embodiment of the present application. As shown in the figure, the electronic device 600 includes an application processor 610, a memory 620, a communication interface 630, and one or more programs 621, wherein the one or more programs 621 are stored in the memory 620 and are configured to be executed by the application processor 610, and the one or more programs 621 include instructions for executing the following steps;

[0073] Acquire and cache input time series data, wherein the time series data includes a first number of different instance data;

[0074] Acquire and assemble the first quantity of different instance data to obtain a second quantity of instance data and corresponding processing requests;

[0075] Concurrently processing the second number of instance data and corresponding processing requests through a second number of threads to obtain a second number of data packets;

[0076] Perform timing reconstruction and rectification operations on the timing data of the second number of data packets, and output the first number of concurrent instance data.

[0077] It can be seen that in the embodiment of the present application, the input timing data is obtained and cached, and the timing data includes a first number of different instance data; the first number of different instance data is obtained and assembled to obtain a second number of instance data and corresponding processing requests; the second number of instance data and the corresponding processing requests are processed concurrently by a second number of threads to obtain a second number of data packets; the timing data of the second number of data packets are subjected to timing reconstruction and rectification operations to output the first number of concurrent instance data. It can be seen that the present application processes the second number of instance data and the corresponding processing requests concurrently by a second number of threads to obtain a second number of data packets, that is, multiple threads are implemented to serve a timing data instance at the same time, improving the processing performance for a single instance data; and different input instance data can be combined into the same data packet for execution, that is, one thread can jointly schedule multiple instance data to achieve concurrent scheduling of timing data and optimal execution performance.

[0078] In one possible example, the timing data scheduling execution system includes a timing data queue and a data multiplexing unit, the data multiplexing unit includes a multiplexing configuration unit and a data acquisition and packaging unit, and in terms of acquiring and assembling the first number of different instance data to obtain a second number of instance data and corresponding processing requests, the instructions in the program are specifically used to perform the following operations: caching the timing data at the granularity of timing data instances through the timing data queue; storing and providing multiplexing configuration information for outputting the timing data through the multiplexing configuration unit; acquiring and packaging the first number of different instance data according to preset rules through the data acquisition and packaging unit to obtain a second number of instance data and corresponding processing requests.

[0079] In one possible example, in terms of acquiring and packaging the first number of different instance data according to preset rules by the data acquisition and packaging unit to obtain the second number of instance data and corresponding processing requests, the instructions in the program are specifically used to perform the following operations: acquiring a packaging request by the data acquisition and packaging unit; after acquiring the packaging request, acquiring the size of the level package and initializing the level packaging variable; inputting the size of the level package and the level packaging variable into the multiplexing configuration unit for processing, and outputting corresponding timing data.

[0080] In one possible example, the multiplexing configuration unit includes multiple multiplexing configuration information structures, and the multiplexing configuration information structure includes a multiplexing configuration management cache unit, an instance number acquisition unit, a quota configuration unit and a quota margin unit. In terms of inputting the size of the timing data level package and the level group package variable into the multiplexing configuration unit for processing, and outputting a second number of instance data and a corresponding processing request, the instructions in the program are specifically used to perform the following operations: determine whether an instance element of the timing data is successfully acquired; if so, acquire the instance number of the instance element through the instance number acquisition unit, and acquire two variables, the configuration margin variable, through the quota configuration unit and the quota margin unit; acquire the queue depth of the corresponding instance queue according to the instance number; output the second number of instance data and the corresponding processing request according to the comparison result of the queue depth and the configuration margin variable, or perform the above operation on the next multiplexing configuration information structure.

[0081] In one possible example, the time series data scheduling execution system includes an out-of-order packet assembly unit, which includes a batch packet assembly unit, a batch configuration unit, a batch signal unit and a delay signal unit. In terms of acquiring and assembling the first number of different instance data to obtain the second number of instance data and corresponding processing requests, the instructions in the program are specifically used to perform the following operations: acquire a data packet pending signal, and query the queue depth corresponding to the data packet pending signal; when the queue depth is greater than a preset depth, perform batch packet assembly operations through the batch packet assembly unit; acquire configuration information of the batch packet assembly and store it in the batch configuration unit; perform signal management of the batch packet assembly through the batch signal unit to obtain signal information; process the configuration information and the signal information through the batch packet assembly unit to obtain the second number of instance data and the corresponding processing request; when the queue depth is less than the preset depth, perform necessary processing delay management through the delay signal unit.

[0082] In one possible example, the timing data scheduling execution system includes a reconstruction output packet scheduling unit and a reconstruction buffer queue; in terms of performing timing reconstruction and rectification operations on the timing data of the second number of data packets and outputting a first number of concurrent instance data, the instructions in the program are specifically used to perform the following operations: when the reconstruction output packet scheduling unit detects the input of the second number of instance data, the second number of instance data is added to the reconstruction buffer queue; reconstruction timing detection is performed on the data packets after timing increment; if the data packets after the timing increment are in the reconstruction buffer queue, the first number of concurrent instance data after the timing increment is output; if the data packets after the timing increment are not in the reconstruction buffer queue, the second number of instance data is waited for input.

[0083] In one possible example, the instructions in the program are also specifically used to perform the following operations: when a second number of instance data input is detected by the reconstruction output package scheduling unit, the second number of instance data is added to the reconstruction buffer queue, and then the instance number of the input timing data is identified; the corresponding reconstruction buffer queue is obtained according to the instance number and the reconstruction buffer queue and the time when the data is completed before the reconstruction processing are saved.

[0084] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0085] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0086] Figure 7 700 is a functional unit composition block diagram of the apparatus 700 for concurrent scheduling and execution of time series data involved in the embodiment of the present application. The apparatus 700 for concurrent scheduling and execution of time series data is applied to an electronic device, and the electronic device includes a processing unit 701 and a communication unit 702, wherein:

[0087] The processing unit 701 is used to obtain and cache input timing data through the communication unit 702, where the timing data includes a first number of different instance data; obtain and assemble the first number of different instance data to obtain a second number of instance data and corresponding processing requests; concurrently process the second number of instance data and the corresponding processing requests through a second number of threads to obtain a second number of data packets; perform timing reconstruction and rectification operations on the timing data of the second number of data packets, and output a first number of concurrent instance data.

[0088] The apparatus 700 for concurrently scheduling and executing timing data may further include a storage unit 703 for storing program codes and data of the electronic device. The processing unit 701 may be a processor, the communication unit 702 may be an internal communication interface, and the storage unit 703 may be a memory.

[0089] It can be seen that in the embodiment of the present application, the input timing data is obtained and cached, and the timing data includes a first number of different instance data; the first number of different instance data is obtained and assembled to obtain a second number of instance data and corresponding processing requests; the second number of instance data and the corresponding processing requests are processed concurrently by a second number of threads to obtain a second number of data packets; the timing data of the second number of data packets are subjected to timing reconstruction and rectification operations to output the first number of concurrent instance data. It can be seen that the present application processes the second number of instance data and the corresponding processing requests concurrently by a second number of threads to obtain a second number of data packets, that is, multiple threads are implemented to serve a timing data instance at the same time, improving the processing performance for a single instance data; and different input instance data can be combined into the same data packet for execution, that is, one thread can jointly schedule multiple instance data to achieve concurrent scheduling of timing data and optimal execution performance.

[0090] In one possible example, the timing data scheduling execution system includes a timing data queue and a data multiplexing unit, and the data multiplexing unit includes a multiplexing configuration unit and a data acquisition and packaging unit. In acquiring and assembling the first number of different instance data to obtain the second number of instance data and corresponding processing requests, the processing unit 701 is specifically used to: cache the timing data at the granularity of timing data instances through the timing data queue; store and provide multiplexing configuration information for outputting the timing data through the multiplexing configuration unit; acquire and package the first number of different instance data according to preset rules through the data acquisition and packaging unit to obtain the second number of instance data and corresponding processing requests.

[0091] In one possible example, in terms of acquiring and packaging the first number of different instance data according to preset rules through the data acquisition and packaging unit to obtain the second number of instance data and corresponding processing requests, the processing unit 701 is specifically used to: acquire the packaging request through the data acquisition and packaging unit; after acquiring the packaging request, acquire the size of the level package and initialize the level packaging variable; input the size of the level package and the level packaging variable into the multiplexing configuration unit for processing, and output corresponding timing data.

[0092] In a possible example, the multiplexing configuration unit includes multiple multiplexing configuration information structures, and the multiplexing configuration information structure includes a multiplexing configuration management cache unit, an instance number acquisition unit, a quota configuration unit and a quota margin unit. In terms of inputting the size of the timing data level package and the level group package variable into the multiplexing configuration unit for processing, and outputting a second number of instance data and a corresponding processing request, the processing unit 701 is specifically used to: determine whether an instance element of the timing data is successfully acquired; if so, acquire the instance number of the instance element through the instance number acquisition unit, and acquire two variables, the configuration margin variable, through the quota configuration unit and the quota margin unit; acquire the queue depth of the corresponding instance queue according to the instance number; output the second number of instance data and the corresponding processing request according to the comparison result of the queue depth and the configuration margin variable, or perform the above operation on the next multiplexing configuration information structure.

[0093] In one possible example, the time series data scheduling execution system includes an out-of-order packet assembly unit, which includes a batch packet assembly unit, a batch configuration unit, a batch signal unit and a delay signal unit. In terms of acquiring and assembling the first number of different instance data to obtain the second number of instance data and corresponding processing requests, the processing unit 701 is specifically used to: acquire a data packet pending signal, and query the queue depth corresponding to the data packet pending signal; when the queue depth is greater than a preset depth, perform batch packet assembly operations through the batch packet assembly unit; acquire configuration information of the batch packet assembly and store it in the batch configuration unit; perform signal management of the batch packet assembly through the batch signal unit to obtain signal information; process the configuration information and the signal information through the batch packet assembly unit to obtain the second number of instance data and the corresponding processing request; when the queue depth is less than the preset depth, perform necessary processing delay management through the delay signal unit.

[0094] In one possible example, the timing data scheduling execution system includes a reconstruction output packet scheduling unit and a reconstruction buffer queue; in terms of performing timing reconstruction and rectification operations on the timing data of the second number of data packets and outputting a first number of concurrent instance data, the processing unit 701 is specifically used for: when the reconstruction output packet scheduling unit detects the input of the second number of instance data, the second number of instance data is added to the reconstruction buffer queue; a reconstruction timing detection is performed on the data packet after the timing increment; if the data packet after the timing increment is in the reconstruction buffer queue, the first number of concurrent instance data after the timing increment is output; if the data packet after the timing increment is not in the reconstruction buffer queue, the second number of instance data is waited for input.

[0095] In a possible example, when the processing unit 701 detects a second number of instance data input through the reconstruction output group packet scheduling unit, it is also used to add the second number of instance data to the reconstruction buffer queue, and then identify the instance number of the input timing data; obtain the corresponding reconstruction buffer queue according to the instance number and save the reconstruction buffer queue and the time when the data is completed before the reconstruction processing.

[0096] In a possible example, the timing data scheduling execution system includes a timing data queue and a data multiplexing unit, the data multiplexing unit includes a multiplexing configuration unit and a data acquisition and packaging unit, and in terms of acquiring and caching the input timing data, the processing unit 701 is specifically used to: cache the timing data at the granularity of timing data instances through the timing data queue; store and provide multiplexing configuration information for outputting the timing data through the multiplexing configuration unit; and acquire and package the timing data according to preset rules through the data acquisition and packaging unit.

[0097] In a possible example, in terms of acquiring and packaging time series data according to preset rules through the data acquisition and packaging unit, the processing unit 701 is specifically used to: acquire a packaging request through the data acquisition and packaging unit; after acquiring the packaging request, acquire the size of the level package and initialize the level packaging variable; input the level package size and the level packaging variable into the multiplexing configuration unit for processing, and output corresponding time series data.

[0098] In a possible example, the multiplexing configuration unit includes multiple multiplexing configuration information structures, and the multiplexing configuration information structure includes a multiplexing configuration management cache unit, an instance number acquisition unit, a quota configuration unit and a quota margin unit. In terms of inputting the size of the level package and the level group package variable into the multiplexing configuration unit for processing and outputting the corresponding timing data, the processing unit 701 is specifically used to: determine whether an instance element of the timing data is successfully acquired; if so, acquire the instance number of the instance element through the instance number acquisition unit, and acquire two variables, the configuration margin variable, through the quota configuration unit and the quota margin unit; acquire the queue depth of the corresponding instance queue according to the instance number; and determine whether to output the corresponding timing data or perform the above operation on the next multiplexing configuration information structure according to the queue depth and the configuration margin variable.

[0099] In one possible example, the time series data scheduling execution system includes an out-of-order packet assembly unit, and the out-of-order packet assembly unit includes a batch packet assembly unit, a batch configuration unit, a batch signal unit and a delay signal unit. In the acquisition and assembling of the second number of instance data and processing requests of the time series data, the processing unit 701 is specifically used to: obtain a data packet pending signal, and query the queue depth corresponding to the data packet pending signal; when the queue depth is greater than a preset depth, perform batch packet assembly operations through the batch packet assembly unit; obtain configuration information of the batch packet assembly and store it in the batch configuration unit; perform signal management of the batch packet assembly through the batch signal unit to obtain signal information; process the configuration information and the signal information through the batch packet assembly unit to obtain the second number of instance data and processing requests of the time series data; when the queue depth is less than the preset depth, perform necessary processing delay management through the delay signal unit.

[0100] In one possible example, the timing data scheduling execution system includes a reconstruction output packet scheduling unit and a reconstruction buffer queue; in terms of performing timing reconstruction and rectification operations on the timing data of the multiple packets and outputting target timing data, the processing unit 701 is specifically used to: when a second number of instance data input is detected by the reconstruction output packet scheduling unit, the second number of instance data is added to the reconstruction buffer queue; a reconstruction timing detection is performed on the data packet after the timing increment; if the data packet after the timing increment is in the reconstruction buffer queue, the data packet after the timing increment is output; if the data packet after the timing increment is not in the reconstruction buffer queue, waiting for the second number of instance data to be input.

[0101] In a possible example, the processing unit 701 is also specifically used for: when a second number of instance data input is detected by the reconstruction output package scheduling unit, the second number of instance data is added to the reconstruction buffer queue, and then the instance number of the input timing data is identified; according to the instance number, the corresponding reconstruction buffer queue is obtained and the reconstruction buffer queue and the time when the data is completed before the reconstruction processing are saved.

[0102] It can be understood that since the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part in this application should be synchronously adapted to the device embodiment part and will not be repeated here.

[0103] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiments, and the above computer includes an electronic device.

[0104] The embodiment of the present application also provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. The computer program product may be a software installation package, and the computer includes an electronic device.

[0105] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0106] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0107] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0108] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0109] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0110] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or CD-ROM and other media that can store program codes.

[0111] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0112] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for concurrently scheduling and executing time series data, characterized in that: Applied to a time series data scheduling execution system, the method includes: Acquire and cache input time series data, wherein the time series data includes a first number of different instance data; Acquire and assemble the first quantity of different instance data to obtain a second quantity of instance data and corresponding processing requests; Concurrently processing the second number of instance data and corresponding processing requests through a second number of threads to obtain a second number of data packets; Performing timing reconstruction and rectification operations on the timing data of the second number of data packets, and outputting the first number of concurrent instance data; Among them, the timing data scheduling execution system includes a timing data queue and a data multiplexing unit, the data multiplexing unit includes a multiplexing configuration unit and a data acquisition and packaging unit, and the acquisition and assembly of the first number of different instance data to obtain the second number of instance data and corresponding processing requests include: caching the timing data with the timing data instance as the granularity through the timing data queue; storing and providing multiplexing configuration information for outputting the timing data through the multiplexing configuration unit; acquiring and packaging the first number of different instance data according to preset rules through the data acquisition and packaging unit to obtain the second number of instance data and corresponding processing requests.

2. The method according to claim 1, characterized in that The acquiring and packaging of the first quantity of different instance data according to a preset rule by the data acquisition and packaging unit to obtain a second quantity of instance data and corresponding processing requests includes: Acquiring a package assembly request through the data acquisition and package assembly unit; After obtaining the packet assembly request, obtain the size of the time series data level packet and initialize the level packet assembly variable; The size of the time series data level package and the level group package variable are input into the multiplexing configuration unit for processing, and a second number of instance data and corresponding processing requests are output.

3. The method according to claim 2, characterized in that The multiplexing configuration unit includes a plurality of multiplexing configuration information structures, the multiplexing configuration information structure includes a multiplexing configuration management cache unit, an instance number acquisition unit, a quota configuration unit and a quota margin unit, the size of the time series data level package and the level group package variable are input into the multiplexing configuration unit for processing, and the second number of instance data and corresponding processing requests are output, including: Determine whether an instance element of time series data is successfully obtained; If yes, the instance number of the instance element is obtained through the instance number obtaining unit, and two variables, namely, the configuration margin variable, are obtained through the quota configuration unit and the quota margin unit; Obtaining a queue depth of a corresponding instance queue according to the instance number; Output a second amount of instance data and a corresponding processing request according to the comparison result of the queue depth and the configuration margin variable, or perform the above operation on the next multiplexing configuration information structure.

4. The method according to claim 3, characterized in that: The time series data scheduling execution system includes a random order packaging unit, the random order packaging unit includes a batch packaging unit, a batch configuration unit, a batch signal unit and a delay signal unit, and the acquiring and assembling of the first number of different instance data to obtain the second number of instance data and corresponding processing requests includes: Obtain a data packet pending signal, and query a queue depth corresponding to the data packet pending signal; When the queue depth is greater than a preset depth, a batch processing package grouping operation is performed by a batch processing package grouping unit; Obtain configuration information of the batch processing group package and store it in the batch processing configuration unit; The batch signal unit is used to perform signal management of batch processing packages to obtain signal information; Processing the configuration information and the signal information by a batch processing package unit to obtain the second number of instance data and corresponding processing requests; When the queue depth is less than a preset depth, processing delay management is performed through the delay signal unit.

5. The method according to any one of claims 1 to 4, characterized in that: The time series data scheduling execution system includes a reconstruction output packet scheduling unit and a reconstruction buffer queue; the time series data of the second number of data packets are subjected to time series reconstruction and rectification operations to output the first number of concurrent instance data, including: When the reconstruction output packet group scheduling unit detects the input of the second amount of instance data, the second amount of instance data is added to the reconstruction buffer queue; Performing a reconstruction timing detection on the data packet after the timing increment; If the data packet after the timing increment is in the reconstruction buffer queue, output the first number of concurrent instance data after the timing increment; If the data packet after the timing increment is not in the reconstruction buffer queue, the second quantity of instance data input is waited for.

6. The method according to claim 5, characterized in that When the reconstruction output packet group scheduling unit detects a second amount of instance data input, after adding the second amount of instance data to the reconstruction buffer queue, the method further includes: Identify the instance number of the input time series data; The corresponding reconstruction buffer queue is obtained according to the instance number, and the reconstruction buffer queue and the time when the data concurrency before the reconstruction process is completed are saved.

7. A device for concurrently scheduling and executing time series data, characterized in that: It includes a processing unit and a communication unit, wherein: The processing unit is configured to obtain and cache input time series data through the communication unit, the time series data including a first number of different instance data; obtain and assemble the first number of different instance data to obtain a second number of instance data and corresponding processing requests; perform concurrent processing on the second number of instance data and the corresponding processing requests through a second number of threads to obtain a second number of data packets; perform time series reconstruction and rectification operations on the time series data of the second number of data packets to output a first number of concurrent instance data; The time series data scheduling execution system includes a time series data queue and a data multiplexing unit, the data multiplexing unit includes a multiplexing configuration unit and a data acquisition and packaging unit, and the processing unit is also used to cache the time series data at the granularity of time series data instances through the time series data queue; store and provide multiplexing configuration information for outputting the time series data through the multiplexing configuration unit; and acquire and package the first number of different instance data according to preset rules through the data acquisition and packaging unit to obtain the second number of instance data and corresponding processing requests.

8. An electronic device, characterized in that: The method comprises a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for executing the steps in the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 6.

Images