Time series data processing method and device, equipment and storage medium

Abstract

The application discloses a time series data processing method and device, equipment and storage medium, relates to the technical field of data processing, and comprises the following steps: mapping an original time series data file to a virtual address space to obtain mapped time series data; performing parallel pre-scanning on the mapped time series data through a plurality of analysis threads, and counting the amount of data to be processed corresponding to each calculation thread; allocating a dedicated memory area to each calculation thread according to the amount of data to be processed, completely analyzing the mapped time series data through each analysis thread, distributing the analyzed mapped time series data to the dedicated memory area of each calculation thread, and independently performing parallel calculation to obtain the statistical characteristics of the mapped time series data; and aggregating the statistical characteristics to a global result pointer array in the original data order, and asynchronously writing the completed statistical characteristics in the global result pointer array into a storage device in the process of parallel calculation of the calculation thread. The application can realize efficient and low-delay processing of massive time series data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a time-series data processing method, apparatus, device, and storage medium. Background Technology

[0002] Currently, the industry mainly uses traditional file reading and writing methods to acquire data for processing massive amounts of time-series data. The raw data in text format is converted into binary format that computers can directly calculate before processing. At the same time, the data is distributed to different processors for parallel processing through simple block partitioning. Conventional computing logic is used in complex feature calculations, and in a multi-processor environment, synchronization mechanisms such as locking are used to ensure the orderliness of data processing. After the calculation is completed, the results are written back in a unified manner, forming a conventional processing flow from data reading, conversion, allocation, calculation to result output.

[0003] However, these existing technologies exhibit numerous significant shortcomings when faced with the demands of processing massive amounts of time-series data. Traditional file reading and writing methods lag far behind CPU computing speeds, creating data read / write bottlenecks. The text-to-binary format conversion process is inefficient and consumes a large amount of time. Simple data block allocation methods easily lead to uneven task distribution and processor load imbalance. Repetitive calculations in complex feature computations significantly reduce computational efficiency. Furthermore, the locking and synchronization mechanisms between multiple processors cause frequent processor waiting, greatly increasing the overhead of multi-core collaboration. These shortcomings combine to make existing technologies unable to achieve high-efficiency, low-latency processing of massive amounts of time-series data, and thus failing to meet the data processing needs of practical applications. Summary of the Invention

[0004] The main objective of this application is to provide a time-series data processing method, apparatus, device, and storage medium, which aims to achieve high-efficiency and low-latency processing of massive time-series data.

[0005] To achieve the above objectives, this application proposes a time-series data processing method, the method comprising: The original timing data file is acquired and mapped to the virtual address space in virtual memory to obtain the mapped timing data; The mapped time series data is pre-scanned in parallel by multiple parsing threads, and the amount of data to be processed corresponding to each computing thread is counted based on the scan results. Based on the amount of data to be processed corresponding to each computing thread, a dedicated memory area is allocated to each computing thread. Each parsing thread performs complete parsing of the mapped timing data in the virtual address space and distributes the parsed mapped timing data to the dedicated memory area of ​​each computing thread. Each computing thread performs independent parallel computation on the mapped timing data in the dedicated memory area to obtain the statistical characteristics of the mapped timing data. The statistical features calculated by each computing thread are aggregated into a global result pointer array in the order of the original data. During the parallel computing process of the computing threads, the completed statistical features in the global result pointer array are asynchronously written to the storage device to complete the time-series data processing.

[0006] In one possible implementation, the step of performing a parallel pre-scan of the mapped time-series data using multiple parsing threads, and calculating the amount of data to be processed for each computation thread based on the scan results, includes: The mapping time-series data is logically divided into blocks, and corresponding mapping time-series data blocks are allocated to each of the parsing threads. Each of the aforementioned parsing threads performs a parallel pre-scan of the allocated mapped time-series data blocks to parse out the key value corresponding to each record in the mapped time-series data. The computing thread corresponding to each record is determined based on the key value, and the number of data records to be processed corresponding to each computing thread is counted to obtain the amount of data to be processed for each computing thread.

[0007] In one possible implementation, based on the amount of data to be processed corresponding to each computing thread, a dedicated memory region is allocated to each computing thread. Each parsing thread performs complete parsing of the mapped timing data in the virtual address space and distributes the parsed mapped timing data to the dedicated memory region of each computing thread. This allows each computing thread to perform independent parallel computation on the mapped timing data in its dedicated memory region, obtaining statistical characteristics of the mapped timing data, including: Based on the amount of data to be processed corresponding to each computing thread, a matching dedicated memory area is allocated to each computing thread; The mapping time series data is completely parsed by each of the parsing threads, various calculation fields in the mapping time series data are extracted and their formats are converted, and the parsed content is encapsulated into work items. Based on the key value corresponding to each record in the mapping time series data, the work item is distributed to the dedicated memory area of ​​the computing thread corresponding to the work item; Each of the aforementioned computing threads performs independent parallel computation on the work items in the dedicated memory region, maintains the sliding window statistical state corresponding to each key value, and updates and calculates the statistical characteristics of the mapped time-series data.

[0008] In one possible implementation, the parsing threads perform complete parsing of the mapped time-series data, extract various calculated fields from the mapped time-series data and perform format conversion, and encapsulate the parsed content into work items, including: The mapping time series data is parsed line by line by each of the parsing threads to extract various calculation fields; The extracted calculated fields are formatted using a single instruction multiple data stream instruction set, converting text-formatted data into binary-formatted data. The various calculated fields after format conversion are encapsulated to form standardized work items.

[0009] In one possible implementation, the step of independently and in parallel computing each of the computing threads on the work items in the dedicated memory region, maintaining the sliding window statistical state corresponding to each key value, and updating and calculating the statistical characteristics of the mapped time-series data includes: Obtain the calculation status of the key values ​​managed by each of the aforementioned calculation threads using a hash table, and maintain at least one sliding window of statistical status for each of the aforementioned key values; An online incremental algorithm is used to update the mean and variance of the sliding window, and a double-ended queue algorithm is used to track the maximum and minimum values ​​of the sliding window. The intermediate statistical values ​​of each sliding window are calculated in parallel using a single instruction multiple data stream instruction set to obtain the statistical indicators corresponding to each sliding window, and then integrated to form the statistical characteristics of the mapped time series data.

[0010] In one possible implementation, the step of aggregating the statistical features calculated by each computing thread into a global result pointer array in the order of the original data, and asynchronously writing the completed statistical features in the global result pointer array to a storage device during the parallel computing process of the computing threads, thereby completing the time-series data processing, includes: The statistical features obtained by each of the computing threads are converted into text-formatted result strings and stored in a dedicated output buffer. The addresses and lengths of the result strings in each dedicated output buffer are filled into the corresponding positions in the global result pointer array according to the record sequence number of the original data, thus completing the aggregation of statistical features; During the parallel computation process of the computing thread, the global result pointer array is checked in blocks according to the record sequence number of the original data. After confirming that the statistical features in the data blocks have been completed, the statistical features of the data blocks are written to the storage device in batches to complete the time-series data processing.

[0011] In one possible implementation, during the parallel computation of the computing thread, the global result pointer array is checked in blocks according to the record sequence number of the original data. After confirming that the statistical features within the data blocks have been completed, the statistical features of the data blocks are written to the storage device in batches to complete the time-series data processing. This includes: While the computing thread performs parallel computing, an independent write-back process is initiated and the global result pointer array is divided into blocks according to the record sequence number of the original data. The statistical features in the data blocks after the blocks are checked one by one to see if they have all been written into the global result pointer array. If not all statistical features within the data block are completed, the data block will be looped and waited until all statistical features are written. If all statistical features within the data block are completed, the statistical features corresponding to the data block are written to the storage device in batches. The process of checking and writing all data blocks is completed sequentially until all statistical features in the global result pointer array have been written to the storage device, thus completing the time-series data processing.

[0012] Furthermore, to achieve the above objectives, this application also proposes a time-series data processing apparatus, which includes: The acquisition module is used to acquire and map the raw timing data file to the virtual address space in virtual memory to obtain the mapped timing data. The scanning module is used to perform parallel pre-scanning of the mapped time series data through multiple parsing threads, and to count the amount of data to be processed corresponding to each computing thread based on the scanning results; The calculation module is used to allocate a dedicated memory area to each calculation thread according to the amount of data to be processed corresponding to each calculation thread, and to completely parse the mapped timing data in the virtual address space through each parsing thread, and distribute the parsed mapped timing data to the dedicated memory area of ​​each calculation thread, so that each calculation thread can independently and in parallel calculate the mapped timing data in the dedicated memory area to obtain the statistical characteristics of the mapped timing data. The writing module is used to aggregate the statistical features calculated by each computing thread into a global result pointer array in the order of the original data, and asynchronously write the completed statistical features in the global result pointer array into the storage device during the parallel computing process of the computing threads, thereby completing the time-series data processing.

[0013] In addition, to achieve the above objectives, this application also proposes a timing data processing apparatus, the apparatus comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the timing data processing method described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the timing data processing method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the timing data processing method described above.

[0016] This application provides a time-series data processing method, apparatus, device, and storage medium. The time-series data processing method obtains and maps the original time-series data file to a virtual address space in virtual memory to obtain mapped time-series data. Then, multiple parsing threads perform parallel pre-scanning of the mapped time-series data. Based on the scan results, the amount of data to be processed for each computing thread is statistically analyzed. Based on the amount of data to be processed for each computing thread, a dedicated memory region is allocated to each computing thread. Each parsing thread performs complete parsing of the mapped time-series data in the virtual address space and distributes the parsed mapped time-series data to the dedicated memory region of each computing thread. Each computing thread performs independent parallel computation on the mapped time-series data in its dedicated memory region to obtain the statistical characteristics of the mapped time-series data. Finally, the statistical characteristics calculated by each computing thread are processed... Statistical features are aggregated into a global result pointer array in the order of the original data. During the parallel computation of the computing threads, the completed statistical features in the global result pointer array are asynchronously written to the storage device to complete the time-series data processing. This achieves load balancing of the computing threads by pre-scanning the statistical task volume in parallel, avoiding single-thread overload that slows down the overall processing progress. By allocating dedicated memory areas to the computing threads, lock-free computation with data isolation is achieved, eliminating the synchronization overhead of multi-core collaboration. Data parsing and distribution are completed in stages. At the same time, combined with accurate task volume statistics, accurate memory allocation is achieved, reducing memory peaks and data copy pressure. Furthermore, by writing back asynchronously, the disk write operation time is hidden in the computation process, improving the overall system throughput. This forms a multi-threaded parallel participation in the entire process of parsing, computation, and distribution, making full use of hardware computing power and maximizing data processing efficiency. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the time-series data processing method of this application. Figure 2 This is one of the flowcharts provided for the time-series data processing method of this application; Figure 3 The second flowchart provided for the time-series data processing method of this application; Figure 4 The third flowchart provided for the time-series data processing method of this application; Figure 5 This is a schematic diagram of the module structure of the timing data processing device according to an embodiment of this application; Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the timing data processing method in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device, big data service platform, or time-series data processing system capable of performing the above functions. The following description uses a time-series data processing system as an example to illustrate this embodiment and the subsequent embodiments.

[0024] Based on this, embodiments of this application provide a time-series data processing method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the timing data processing method of this application.

[0025] In this embodiment, the time-series data processing method includes steps S11 to S14: Step S11: Obtain and map the original timing data file to the virtual address space in virtual memory to obtain the mapped timing data; It should be noted that raw time-series data files refer to files formed by massive amounts of data generated and stored in chronological order. These files are usually in text format and contain various raw record information of the time-series data. Virtual memory refers to the address space allocated to programs in a computer system, which combines physical memory and disk space. It is the range of memory addresses that programs can access. Virtual address space refers to an independent address region that virtual memory divides for a single program, in which the program can access and manipulate data. Mapped time-series data refers to the time-series data that can be directly accessed by programs after the raw time-series data file is mapped to the virtual address space.

[0026] The purpose of this system operation is to abandon the traditional file read / write method, achieve zero-copy reading of raw time-series data files, fundamentally eliminate the bottleneck of data read / write, and allow programs to access time-series data in files as quickly as accessing memory, thereby improving data access efficiency. At the same time, it lays an efficient data access foundation for subsequent parallel parsing, computation and other operations. The implementation idea is to use memory mapping technology to establish an association between the raw time-series data file on the disk and the program's virtual address space, so that the file data is directly mapped to this address space without the need for operating system-level input / output copying.

[0027] In one possible implementation, the system can dynamically allocate a matching virtual address space region to the original time-series data file based on its size, ensuring that the mapped time-series data can be accessed completely and continuously.

[0028] Specifically, after the system starts up, it does not use the traditional method of reading files, but uses the "memory mapping" (mmap) technology to directly map the entire input data file (such as a large CSV file) into the program's virtual address space. This operation allows subsequent programs to access file data as quickly as accessing memory, avoiding the slow I / O (input / output) copying process of the operating system.

[0029] In one specific implementation, the system obtains a raw time-series data file in CSV format that stores merchant transaction time-series data. This file contains raw records such as merchant identifier, transaction time, and transaction amount. The system calls memory mapping technology to map the entire CSV file to the virtual address space of virtual memory allocated to the program that processes the data. This allows the program to directly access the merchant transaction time-series data in this address space and form corresponding mapped time-series data. The entire process involves no input / output copying by the operating system, and the speed at which the program accesses data is consistent with the speed at which it accesses memory data.

[0030] Step S12: Perform parallel pre-scanning of the mapped time series data through multiple parsing threads, and calculate the amount of data to be processed for each computing thread based on the scan results; It should be noted that parsing threads refer to execution threads in the system specifically used for scanning and parsing time-series data, and multiple threads can be started simultaneously to achieve parallel processing; parallel pre-scanning refers to the rapid scanning operation of the mapped time-series data by multiple parsing threads at the same time. This scan only targets the key information of the data and does not perform complete parsing; computation threads refer to execution threads in the system specifically used for calculating statistical features of time-series data, and are the core execution entities for subsequent data computation; the amount of data to be processed refers to the number of records of mapped time-series data that each computation thread needs to process, and is an important indicator for measuring the workload of computation threads.

[0031] The purpose of this system operation is to accurately calculate the workload of each computing thread in advance, so as to avoid the situation where some computing threads are overloaded and others are idle due to uneven task allocation. The implementation idea is to logically divide the mapped time-series data into blocks and distribute them to multiple parsing threads. Each parsing thread performs fast scanning and key-value parsing in parallel, and then determines the computing thread to which the data belongs based on the key-value. Finally, the number of data records to be processed corresponding to each computing thread is calculated.

[0032] In one possible implementation, the number of parsing threads launched by the system is comparable to the number of CPU cores in the computer system, making full use of the hardware's parallel processing capabilities.

[0033] For details, please refer to steps S21-S23, which will not be repeated here.

[0034] Step S13: Based on the amount of data to be processed corresponding to each computing thread, allocate a dedicated memory area to each computing thread, and use each parsing thread to completely parse the mapped timing data in the virtual address space, and distribute the parsed mapped timing data to the dedicated memory area of ​​each computing thread, so that each computing thread can independently and in parallel compute the mapped timing data in the dedicated memory area to obtain the statistical characteristics of the mapped timing data. It should be noted that a dedicated memory area refers to a memory area allocated separately by the system for each computing thread, accessible and used only by that thread, and the dedicated memory areas of each computing thread are isolated from each other; complete parsing refers to the parsing thread performing a comprehensive analysis of each record of the mapped time-series data, extracting all field information used for subsequent calculations; a work item refers to the standardized data structure formed after format conversion and encapsulation of the calculated fields after complete parsing, which is the basic processing unit of the computing thread; a sliding window refers to a time window set for statistical analysis of the characteristics of time-series data within a specific time range, which can be updated by sliding according to preset rules; the sliding window statistical status refers to the real-time status of various statistical indicators of the time-series data within the sliding window, which is the basis for calculating statistical features; statistical features refer to indicators that reflect the characteristics of the data obtained by calculating the mapped time-series data, including mean, variance, maximum value, minimum value, standard deviation, etc.

[0035] The purpose of this system operation is to achieve efficient parsing, format conversion, and balanced distribution of mapped time-series data. At the same time, it completes the extraction of statistical features through independent parallel computing by computing threads, eliminating the synchronization overhead of multi-core collaboration and improving the efficiency of data processing and computing. The implementation idea is to first allocate a matching dedicated memory area to each computing thread according to the amount of data to be processed, then the parsing thread performs complete parsing and format conversion of the mapped time-series data, encapsulates it into work items, and distributes them to the dedicated memory area of ​​the corresponding computing thread. Finally, each computing thread completes the sliding window state maintenance and statistical feature calculation in its independent memory area.

[0036] Specifically, please refer to steps S31-S53 and Figure 2 , Figure 3 This will not be elaborated upon here.

[0037] Step S14: The statistical features calculated by each computing thread are aggregated into a global result pointer array in the order of the original data. During the parallel computing process of the computing threads, the completed statistical features in the global result pointer array are asynchronously written to the storage device to complete the time-series data processing.

[0038] It should be noted that the global result pointer array refers to a global array created in the system and indexed by the record number of the original time-series data. This array is used to store the address and length of the statistical feature result strings calculated by each computing thread. The dedicated output buffer refers to a memory buffer allocated separately by the system for each computing thread to store the text-format statistical feature result strings. The result string refers to the string formed after converting the binary format statistical features into text format, which is convenient for output and storage. Asynchronous writing refers to the operation of writing statistical features to the storage device simultaneously with the parallel computing operation of the computing thread, without waiting for all calculations to be completed before performing the write operation. The storage device refers to the storage medium such as disks and solid-state drives used to store the final time-series data processing results.

[0039] The purpose of this operation is to ensure the consistency of the statistical feature results with the original data in order, while hiding the time spent on disk write operations in the calculation process, thereby improving the overall throughput of the system and completing the entire time series data processing flow. The implementation idea is to first convert the statistical features of each calculation thread into text format and store them in a dedicated output buffer, then fill the address and length of the result string into the global result pointer array according to the original record sequence number to achieve aggregation, and finally start an independent write-back process to perform block checks and batch write operations in parallel with the calculation.

[0040] Specifically, please refer to steps S61-S74 and Figure 4 This will not be elaborated upon here.

[0041] This embodiment achieves efficient data access by mapping the original time-series data file to a virtual address space. It utilizes multiple parsing threads to complete parallel pre-scanning and full parsing, achieving load balancing of computing threads and efficient data distribution. Statistical feature extraction is completed through independent parallel computation by computing threads in dedicated memory areas. The statistical features are then aggregated in their original order and asynchronously written to the storage device. This systematically solves the technical bottlenecks in traditional time-series data processing, such as slow data read / write, inefficient data conversion, uneven task allocation, low computational efficiency, and high multi-core collaboration overhead. It fully utilizes the parallel processing capabilities of hardware to achieve high-efficiency, low-latency processing of massive amounts of time-series data, while ensuring the consistency of the processing results with the original data order, significantly improving the overall throughput and data processing performance of the system.

[0042] In one feasible implementation, the step of performing parallel pre-scanning of the mapped time-series data through multiple parsing threads, and calculating the amount of data to be processed for each computing thread based on the scan results, includes: Step S21: Logically divide the mapped time-series data into blocks and allocate corresponding mapped time-series data blocks to each of the parsing threads; It should be noted that logical partitioning refers to the operation of dividing the mapped time-series data into multiple data blocks at the program level according to preset rules without changing the physical storage structure of the data; a mapped time-series data block refers to an independent data unit formed after logical partitioning that contains a portion of the mapped time-series data. The purpose of this system operation is to distribute a balanced initial workload among multiple parsing threads, enabling each parsing thread to perform parallel pre-scanning work simultaneously, making full use of the hardware's multi-core parallel processing capabilities, and improving the overall pre-scanning efficiency. The implementation idea is to divide the continuous mapping time-series data into an equal number of logical data blocks according to the number of parsing threads started, and then allocate each data block to each parsing thread one by one to ensure that the amount of data processed by each thread is basically the same.

[0043] In one possible implementation, the system calculates the basic data volume of each mapped time-series data block based on the total data volume of the mapped time-series data and the number of parsing threads, and adaptively adjusts the last data block to ensure the uniformity of the block division.

[0044] Specifically, the system first determines the number of parsing threads to be started, then obtains the total number of records of the mapped time-series data and the continuous address range of the data storage. According to the rule of continuous addresses and basically equal data volume, the mapped time-series data is divided into mapped time-series data blocks with the same number of parsing threads. Then, the address information and data range of each mapped time-series data block are sent to the corresponding parsing thread, so that each parsing thread can only access and process the allocated mapped time-series data block and complete the data block allocation operation.

[0045] Step S22: The allocated mapping time series data blocks are pre-scanned in parallel by each of the parsing threads to parse out the key value corresponding to each record in the mapping time series data. It should be noted that parallel pre-scanning refers to the operation where multiple parsing threads simultaneously perform a rapid scan of their respective assigned mapped time-series data blocks. This scan only targets the key identification information of the data and does not perform complete field parsing. The key-value pair refers to the feature identifier or unique identifier of each record in the mapped time-series data, which is the core basis for determining the computing thread to which the record belongs. Each record corresponds to a unique key-value pair. The purpose of this operation is to quickly extract the core identification information of each record in the mapped time-series data, providing a basis for subsequent determination of data ownership and statistics on the amount of data to be processed by each computing thread. At the same time, the parallel pre-scanning method significantly reduces the data scanning time.

[0046] Specifically, after receiving the allocated mapped time-series data block, each parsing thread simultaneously initiates a pre-scanning process. Following preset key-value parsing rules, it rapidly scans each data record in the mapped time-series data block line by line, extracting preset key-value fields from each record. The extracted key-value fields are then associated with the corresponding record's identifier information and temporarily stored in the parsing thread's local temporary cache. This process continues until the entire allocated mapped time-series data block is scanned and its key-value parsing is complete. All parsing threads perform their scanning operations synchronously. Each parsing thread maintains a simple counter to record the total number of records it scans that will be sent to each computation thread, forming a parallel pre-scanning process. Understandably, this stage is extremely fast because it performs only a minimal amount of parsing. Its sole purpose is to accurately calculate the workload that each computation thread will ultimately need to undertake before actually processing the data.

[0047] Step S23: Determine the computing thread corresponding to each record based on the key value, count the number of data records to be processed corresponding to each computing thread, and obtain the amount of data to be processed corresponding to each computing thread.

[0048] It should be noted that the number of data records to be processed refers to the number of mapped time-series data records that each computing thread needs to process; the amount of data to be processed refers to the workload of each computing thread as represented by the number of data records to be processed.

[0049] In one possible implementation, the system uses a hash mapping rule to establish a correspondence between key-value pairs and computation threads, ensuring a uniform distribution of key-value pairs and further guaranteeing load balancing for computation threads. Simultaneously, each parsing thread's local counter is an array structure, with array indices corresponding one-to-one with computation thread identifiers. The number of records is counted by incrementing the array elements.

[0050] Specifically, the system pre-defines the mapping rules between key-value pairs and computation threads. Each parsing thread reads the parsed key-value pairs and corresponding record information from its local temporary cache. According to the mapping rules, it matches a unique computation thread for each record's key-value pair. At the same time, each parsing thread uses a local counter to count the number of records sent to each computation thread in real time. After all parsing threads have completed the counting of their allocated data blocks, the system collects the statistical results of all parsing threads, sums the number of records corresponding to each computation thread, and obtains the total number of mapped time-series data records that each computation thread needs to process. This number is the amount of data to be processed for each computation thread.

[0051] This embodiment lays a uniform load foundation for parallel processing by logically dividing the mapped time-series data into blocks and allocating corresponding data blocks to the parsing threads. Then, the key value of each record is quickly parsed through parallel pre-scanning of each parsing thread. Finally, the computing thread to which the data belongs is determined based on the key value, and the amount of data to be processed by each computing thread is accurately counted. This achieves fast scanning and load statistics of the mapped time-series data, effectively solving the technical problem of uneven task distribution in traditional parallel processing. At the same time, the parallel processing method greatly improves the efficiency of data scanning and statistics, laying the foundation for the efficient execution of the entire time-series data processing flow.

[0052] In one feasible implementation, based on the amount of data to be processed corresponding to each computing thread, a dedicated memory region is allocated to each computing thread. Each parsing thread performs complete parsing of the mapped timing data in the virtual address space and distributes the parsed mapped timing data to the dedicated memory region of each computing thread. This allows each computing thread to perform independent parallel computation on the mapped timing data in its dedicated memory region, obtaining statistical characteristics of the mapped timing data, including: Step S31: Allocate a matching dedicated memory region for each computing thread according to the amount of data to be processed corresponding to each computing thread; It's important to note that a dedicated memory region refers to a memory area separately allocated by the system for each computing thread, accessible and manipulated only by that thread. These dedicated memory regions are isolated from each other and do not overlap. The purpose of this system operation is to allocate independent working memory spaces for each computing thread, achieving data isolation between threads and laying the foundation for subsequent lock-free parallel computing. Furthermore, precise memory allocation avoids wasting memory resources and prevents data processing failures due to insufficient memory.

[0053] In one possible implementation, the system allocates a dedicated memory region for each computing thread with a pre-defined percentage of redundant space based on the calculated value to handle temporary memory needs during data processing. Simultaneously, the system establishes a management table for the dedicated memory regions of all computing threads, recording the region address, size, corresponding computing thread, and usage status, facilitating unified memory management.

[0054] Specifically, the amount of data to be processed for each computing thread is obtained. Combined with the memory space required after a single record in the mapped time-series data is encapsulated into a work item, the total memory space required for each computing thread to process all the data to be processed is calculated. Then, based on the calculation results, a contiguous memory region of the corresponding size is allocated from the system's free memory for each computing thread. This memory region is marked as the exclusive memory region of the corresponding computing thread. At the same time, the start address, end address, and size of the exclusive memory region are sent to the corresponding computing thread, so that the computing thread can only access and operate its own exclusive memory region.

[0055] Step S32: The mapping time series data is completely parsed by each of the parsing threads, various calculation fields in the mapping time series data are extracted and their formats are converted, and the parsed content is encapsulated into work items; It should be noted that complete parsing refers to the parsing thread performing comprehensive field parsing on each record of the mapped time series data, extracting all field information that can be used for subsequent statistical feature calculations, which is different from parallel pre-scanning that only extracts key values; various calculation fields refer to all fields in the mapped time series data that can reflect data characteristics and can be used for statistical calculations, including identifier fields, time fields, numerical fields, and sequence number fields; format conversion refers to converting text-formatted calculation fields into binary format that computers can directly perform numerical calculations on.

[0056] In one possible implementation, after extracting the calculated fields, the parsing thread verifies the validity of the fields, discarding missing, erroneous, or invalid data to ensure the accuracy of subsequent calculations. In another possible implementation, the system may pre-define multiple work item data structures, allowing users to select the appropriate structure for encapsulation based on the type of time-series data and computational requirements.

[0057] Specifically, all parsing threads start from scratch again, scanning their respective data blocks in parallel. This time, the parsing threads perform a complete parsing of each row of data, extracting all fields that need to be calculated (e.g., serial number, merchant ID, timestamp, amount, etc.), and packaging them into a "work item" data structure. This parsing process is highly optimized, particularly by utilizing the SIMD parallel instruction set. For example, it can process multiple characters simultaneously with a single instruction, achieving extremely fast conversion between time and amount strings, several times more efficient than traditional methods.

[0058] Step S33: Based on the key value corresponding to each record in the mapping time series data, distribute the work item to the dedicated memory area of ​​the computing thread corresponding to the work item; It should be noted that the key value refers to the feature identifier or unique identifier of each record in the mapped time-series data; distribution refers to the process of transferring the work item encapsulated by the parsing thread from the temporary storage area of ​​the parsing thread to the dedicated memory area of ​​the corresponding computing thread; the corresponding computing thread refers to the computing thread that is responsible for processing the data record corresponding to the key value, determined according to the preset mapping rules between the key value and the computing thread.

[0059] The purpose of this system operation is to accurately and efficiently allocate standardized work items to the workspaces of the corresponding computing threads, achieving precise data distribution while ensuring lock-free distribution and avoiding synchronization waits between threads. The implementation idea is that the parsing thread extracts the corresponding key-value from each work item, determines the computing thread corresponding to the work item according to the preset key-value mapping rules, and then directly copies the work item to the specified location in the corresponding dedicated memory area according to the address information of the dedicated memory area of ​​the computing thread. The entire distribution process is executed in parallel by multiple parsing threads, and the distribution target memory areas of different parsing threads do not overlap.

[0060] In one possible implementation, the parsing thread stores the work items destined for the same computation thread in an ordered manner, according to the order in which they were generated, to facilitate sequential processing by the computation thread. Simultaneously, after completing the work item distribution, the parsing thread sends a data-ready notification to the corresponding computation thread, reminding it to begin data processing.

[0061] Specifically, each parsing thread extracts the corresponding key-value pair from each encapsulated work item. Based on the system's preset key-value pair-to-calculation thread mapping rules, it matches each work item with a unique corresponding computation thread. Then, it obtains the starting address and data storage offset of the computation thread's dedicated memory region, directly copies the work item to the specified offset position in the corresponding dedicated memory region, and simultaneously updates the data storage offset of that dedicated memory region. Multiple parsing threads execute this distribution operation in parallel, with each thread targeting different dedicated memory regions or different positions within the same region. This achieves conflict-free and lock-free work item distribution. That is, after parsing, the parsing thread directly copies the packaged "work item" to the target computation thread's dedicated memory region based on the recorded "key" (merchant ID) and the pre-calculated memory location. This is a conflict-free and lock-free "scatter" process.

[0062] Step S34: Each computing thread performs independent parallel computation on the work items in the dedicated memory area, maintains the sliding window statistical state corresponding to each key value, and updates and calculates the statistical characteristics of the mapped time series data.

[0063] It should be noted that independent parallel computing means that each computing thread processes work items only in its own dedicated memory area, with no data interaction or resource contention between threads, and all computing threads perform computational operations synchronously; a sliding window refers to a dynamically updated time window set for statistical analysis of the characteristics of time series data within a specific time range, which can be added, slid, or destroyed according to preset rules; the statistical status of the sliding window refers to the real-time values ​​of various statistical indicators of the time series data within the sliding window at a certain moment, which is the basis for calculating the final statistical features; statistical features refer to the set of indicators that reflect the overall characteristics and changing patterns of the data, obtained by performing multi-dimensional statistical calculations on the mapped time series data, including mean, variance, maximum value, minimum value, standard deviation, sum, etc.

[0064] The purpose of this system operation is to achieve lock-free, high-concurrency statistical feature calculation of time-series data, eliminate the synchronization overhead of multi-core collaboration, improve computational efficiency through efficient algorithms and parallel computing instructions, and avoid redundant calculations. The implementation idea is that each computing thread processes the work items in its own dedicated memory area one by one in batches. The calculation status of different key values ​​is managed by a hash table, and the statistical status of at least one sliding window is maintained for each key value. The window statistical status is updated in real time using online incremental algorithms and queue algorithms. Then, the intermediate statistical values ​​of multiple sliding windows are calculated in parallel using a single instruction multiple data stream instruction set, and the final statistical features are obtained by integration.

[0065] In one possible implementation, the computation thread processes work items in batches, updating the sliding window statistics after each batch is completed, thus improving computational efficiency. Furthermore, the system can configure different numbers and time ranges of sliding windows for different key values ​​according to business needs, enabling personalized statistical calculations.

[0066] Specifically, each computation thread simultaneously initiates the computation process, processing each work item in its own dedicated memory area line by line. A hash table manages the computation status of the keys it is responsible for, maintaining at least one sliding window with a preset time range for each key, and initializing the sliding window's statistical status. For each work item, the corresponding sliding window is found based on its key, and the mean and variance of the sliding window are updated using an online incremental algorithm. A double-ended queue algorithm is used to track the maximum and minimum values ​​of the sliding window, completing the real-time update of the sliding window's statistical status. After updating the statistical status of all sliding windows corresponding to a key, the intermediate statistical values ​​of each sliding window are computed in parallel using a single instruction multiple data stream instruction set to obtain the statistical indicators corresponding to each window. The statistical indicators of all keys and all sliding windows are then integrated to form the statistical characteristics of the mapped time-series data.

[0067] This embodiment achieves strict data isolation between computing threads by allocating dedicated memory regions to each computing thread based on the amount of data to be processed. Multiple parsing threads perform complete parsing, format conversion, and encapsulation of the mapped time-series data into standardized work items, transforming the raw data into a format directly processable by the computing threads, thus improving data preprocessing efficiency. Precise key-value matching distributes work items to the dedicated memory regions of the corresponding computing threads, achieving balanced and conflict-free data distribution. Each computing thread performs independent parallel computation within its dedicated memory region, combined with efficient algorithms and a single-instruction multiple-data-stream instruction set to maintain and update the sliding window statistical state and calculate statistical features, eliminating the synchronization overhead of multi-core collaboration, avoiding redundant computation, and significantly improving the efficiency of statistical feature calculation. The entire process is seamlessly integrated, systematically solving the technical bottlenecks of inefficient data conversion, uneven task allocation, low computational efficiency, and high multi-core collaboration overhead in traditional time-series data processing. It fully utilizes the parallel processing capabilities of hardware, achieving efficient processing of massive amounts of time-series data and ensuring the accuracy and completeness of the calculation results.

[0068] In one feasible implementation, the step of performing complete parsing of the mapped time-series data through each of the parsing threads, extracting various calculated fields from the mapped time-series data and performing format conversion, and encapsulating the parsed content into work items includes: Step S41: The mapped time series data is parsed line by line by each of the parsing threads to extract various calculation fields; It should be noted that the various calculated fields refer to all fields in the mapped time series data that can be used for subsequent statistical feature calculations; they are the core data carriers reflecting the characteristics of the time series data. The purpose of this operation is to accurately extract all the data fields required for subsequent calculations from the mapped time series data, laying the data foundation for format conversion and data encapsulation. At the same time, by using a multi-threaded parallel line-by-line parsing approach, the overall efficiency of field extraction is improved, ensuring that the extracted fields cover all the requirements of subsequent calculations.

[0069] In one possible implementation, the system pre-defines a list of fields to be extracted based on the processing requirements of the time-series data. The parsing thread extracts fields according to the list, avoiding the extraction of invalid fields and the generation of redundant data. During the field extraction process, the parsing thread verifies the completeness of the fields, marking and temporarily storing records missing key fields for later unified processing.

[0070] Specifically, each parsing thread simultaneously performs parsing operations on the allocated mapped time-series data, traversing each row in the row storage order of the mapped time-series data, identifying and extracting all preset calculated fields from each row of data records according to preset field extraction rules and extraction ranges, and temporarily storing the extracted calculated fields in association with the identification information of the corresponding data records, until the full row-by-row parsing and field extraction of the allocated mapped time-series data are completed. All parsing threads execute this operation in parallel.

[0071] Step S42: Use the Single Instruction Multiple Data Stream (SIDML) instruction set to perform format conversion on the extracted calculated fields, converting the text format data into binary format data; It should be noted that Single Instruction Multiple Data (SID) refers to the parallel computing instruction set of modern central processing units, which can perform the same operation on multiple data simultaneously through a single instruction, achieving batch parallel processing of data; text format data refers to raw computation field data stored in character form, which is a format that is easy for humans to read; binary format data refers to computation field data stored in binary number form, which is a format that computers can directly recognize and perform numerical calculations in.

[0072] The purpose of this system operation is to convert text-formatted calculated fields, which are easy to store and read, into a binary format that computers can directly compute. This solves the problem of low efficiency in traditional format conversion. Simultaneously, by leveraging the parallel processing capabilities of the Single Instruction Multiple Data Stream (SMD) instruction set, it enables rapid batch conversion of calculated fields, significantly improving conversion efficiency. In one possible implementation, the system matches different types of calculated fields with corresponding processing instructions from the SMD instruction set, improving conversion accuracy.

[0073] Specifically, the parsing thread retrieves the extracted text format calculated fields from the local temporary storage area, classifies and integrates them according to the data type of the fields, calls the parallel processing instructions corresponding to the single instruction multiple data stream instruction set, and performs format conversion operations on multiple groups of text format calculated fields of the same type simultaneously through a single instruction, uniformly converting all text format calculated fields into corresponding binary format data, and maintaining the association between the converted binary format data and the identification information of the corresponding data record, thus completing the format conversion of all calculated fields.

[0074] Step S43: Encapsulate the various calculated fields after format conversion to form standardized work items.

[0075] It should be noted that encapsulation refers to the operation of integrating scattered binary format computation fields into a unified data unit according to a preset structure and rules; standardization refers to the fact that the encapsulated data unit follows uniform preset rules in terms of data structure, field composition, and storage format, ensuring the consistency of the data unit. The purpose of the system performing this operation is to integrate scattered binary format computation fields into standardized processing units, allowing computation threads to directly read and process them, avoiding the problem of low computational efficiency caused by inconsistent data formats. At the same time, standardized work items can also improve the standardization and convenience of subsequent data distribution and computation. In one possible implementation, the system will preset multiple standardized work item structures according to the processing scenario and computational requirements of time-series data, and the parsing thread can select the corresponding structure for encapsulation according to actual needs.

[0076] Specifically, the parsing thread retrieves all binary format calculated fields after format conversion, integrates all binary format calculated fields associated with the same data record into the same data unit according to the system's preset standardized work item data structure, adds unified identification information to the data unit, forms a single standardized work item, and sequentially completes the encapsulation operation of the binary format calculated fields corresponding to all data records, forming a standardized work item set that corresponds one-to-one with the original mapped time-series data records.

[0077] This embodiment parses the mapped time-series data line by line and extracts various calculated fields through each parsing thread, ensuring the comprehensiveness and accuracy of the data required for subsequent calculations. It utilizes a single instruction multiple data stream (SMILE) instruction set to batch convert text-formatted calculated fields into binary format data, fully leveraging the hardware's parallel computing capabilities and solving the technical problem of inefficiency in traditional data conversion, significantly improving the efficiency of format conversion. The converted calculated fields are encapsulated into standardized work items, achieving unified data integration, allowing subsequent calculation threads to directly read and process them, improving the standardization and convenience of data distribution and statistical feature calculation. These three steps are interconnected and executed in parallel, completing the preprocessing of time-series data and effectively improving the overall efficiency of processing massive amounts of time-series data.

[0078] In one feasible implementation, the step of independently and in parallel computing the work items in the dedicated memory region through each of the computing threads, maintaining the sliding window statistical state corresponding to each key value, and updating and calculating the statistical characteristics of the mapped time-series data includes: Step S51: Obtain the calculation status of the key value managed by each calculation thread using a hash table, and maintain at least one sliding window statistical status for each key value; It should be noted that a hash table is a data structure that enables fast data access through key-value pairs, allowing direct location of the corresponding data's storage location via the key. The computational state refers to the real-time status information of the time-series data corresponding to each key during the statistical computation process, including computation progress and intermediate statistical results. The statistical state of the sliding window refers to the real-time values ​​of various statistical indicators of the time-series data within the sliding window, forming the basis for calculating the final statistical features. The purpose of this system operation is to establish a dedicated computational state management and sliding window statistical system for each key, achieving refined management of the statistical process for different data subjects. The implementation involves allowing each computation thread to manage all the key-value pairs it is responsible for through a hash table, leveraging the hash table's efficient access characteristics to quickly locate the computational state of each key-value pair. Simultaneously, at least one sliding window with a preset time range is configured for each key-value pair, and the statistical state of each sliding window is initialized to ensure the orderly execution of statistical computations.

[0079] In one possible implementation, the system configures different numbers and time ranges of sliding windows for different key values ​​according to business needs, satisfying personalized statistical requirements. Simultaneously, the hash table uses chaining to handle hash collisions, ensuring the uniqueness and accuracy of the mapping between key values ​​and computational states.

[0080] Specifically, each computing thread retrieves all key-value pairs from its own dedicated memory area, establishes a corresponding storage node for each key-value pair based on a hash table data structure, and stores the initial computing state of each key-value pair into the corresponding node, thereby achieving unified management of the computing state of the key-value pairs under its responsibility through the hash table. At the same time, according to the preset statistical requirements, at least one sliding window is created for each key-value pair, the statistical state parameters of each sliding window are initialized, and the sliding window is associated with the hash table storage node of the key-value pair, thereby achieving dedicated maintenance of the statistical state of the sliding window corresponding to each key-value pair, ensuring that the statistical computing process of each key-value pair can be independently tracked and updated.

[0081] Step S52: The mean and variance of the sliding window are updated using an online incremental algorithm, and the maximum and minimum values ​​of the sliding window are tracked using a double-ended queue algorithm. It should be noted that the online incremental algorithm (Welford algorithm) refers to an algorithm that can incrementally update the existing statistical results based on newly input time-series data without recalculating the entire dataset, thus enabling real-time updates of statistical results; the mean refers to the arithmetic mean of the time-series data within the sliding window, and is a core indicator reflecting the central tendency of the data; the variance refers to the degree to which the time-series data deviates from the mean within the sliding window, and is a core indicator reflecting the dispersion of the data; the double-ended queue algorithm refers to an algorithm that uses the characteristic of double-ended queues to perform data enqueue and dequeue operations at both ends to track the maximum and minimum values ​​in the dataset in real time, avoiding repeated traversal of the entire window of data; the maximum value refers to the largest value in the time-series data within the sliding window, and the minimum value refers to the smallest value in the time-series data within the sliding window; both are important indicators reflecting the range of data fluctuation.

[0082] The purpose of this system operation is to achieve efficient real-time updates of the statistical status of the sliding window, solving the problem of low computational efficiency caused by repeated calculations in traditional methods. The implementation idea is to use an online incremental algorithm to update the window mean and variance by using only the new data and the original statistical results when new time series data enters the sliding window. At the same time, a double-ended queue algorithm is used to process the data entering and leaving the window in real time, dynamically maintaining the extreme value data in the queue, thereby quickly tracking the maximum and minimum values ​​of the window.

[0083] In one possible implementation, the online incremental algorithm verifies the validity of newly added data, removes outliers, and then updates the mean and variance to ensure statistical accuracy. In another possible implementation, the double-ended queue algorithm automatically removes extreme value data that exceeds the window's time range according to the sliding window's sliding rules, ensuring real-time extreme value tracking.

[0084] Specifically, when new work item data enters the sliding window corresponding to the key value, each calculation thread calls the online incremental algorithm to extract the numerical data in the work item. Combining this with the existing statistical status parameters of the sliding window, such as the mean, variance, and number of data rows, the mean and variance of the window are updated using the incremental calculation formula to obtain new statistical values ​​for the mean and variance. At the same time, the double-ended queue algorithm is called to add the newly added numerical data to the double-ended queue according to the rules and remove numerical data that exceeds the time range of the sliding window from the queue. By maintaining the order of the queue, the maximum and minimum values ​​within the current sliding window are directly obtained from both ends of the queue, completing the real-time tracking of the maximum and minimum values ​​of the sliding window and synchronously updating the statistical status of the sliding window.

[0085] More specifically, for each "key," the system maintains the statistical state of one or more "sliding windows" (e.g., 15-minute, 2-hour, and 6-hour windows). Two advanced algorithms are used here: The Welford algorithm is used for online, incremental calculation of the mean and variance. When new data arrives, there is no need to recalculate the entire window; a simple update based on the existing data is sufficient, making it both fast and accurate.

[0086] Double-ended queue algorithm: used to track the maximum and minimum values ​​within a window in real time, thus avoiding time-consuming repeated traversal.

[0087] Step S53: The intermediate statistical values ​​of each sliding window are calculated in parallel using a single instruction multiple data stream instruction set to obtain the statistical indicators corresponding to each sliding window, and then integrated to form the statistical features of the mapped time series data.

[0088] It should be noted that Single Instruction Multiple Data (SID) refers to the parallel computing instruction set of modern central processing units, which can perform the same operation on multiple data simultaneously through a single instruction, achieving batch parallel processing of data; Intermediate statistics refer to the basic statistical data such as mean, variance, maximum, minimum, and number of data rows obtained after incremental updates of the sliding window, which are the basis for calculating derived statistical indicators; Statistical indicators refer to various indicators that reflect the characteristics of time-series data, calculated based on intermediate statistics using specific algorithms, including sum, standard deviation, Z-score, etc.; Mapped time-series data refers to the directly accessible time-series data formed after mapping the original time-series data file to the program's virtual address space through memory mapping technology; Statistical features refer to the set of indicators that comprehensively reflect the overall characteristics and patterns of mapped time-series data, formed by integrating all key values ​​and statistical indicators of all sliding windows.

[0089] In one possible implementation, the system will pre-define a list of statistical indicators based on business needs, and only perform parallel calculations on the indicators in the list to reduce invalid calculations.

[0090] Specifically, each computation thread extracts intermediate statistics for each sliding window corresponding to all the key values ​​it is responsible for. It then organizes the same type of intermediate statistics for different key values ​​into consecutive data batches. A single instruction multiple data stream (SIMD) instruction set is invoked to perform parallel computation on each data batch simultaneously using a single instruction, calculating statistical indicators such as the sum, standard deviation, and Z-score for each sliding window. Subsequently, the statistical indicators for each sliding window corresponding to all key values ​​are summarized and categorized according to the dimensions of the key value and the sliding window, forming a complete set of indicators. This set of indicators represents the statistical features of the mapped time-series data. For example, after updating the state of all windows corresponding to a "work item," the system again utilizes the SIMD parallel instruction set to complete the final feature calculation for multiple windows at once. For instance, it can load the intermediate statistics of three different time windows into a dedicated vector register and then use a single instruction to simultaneously calculate the standard deviation or Z-score for these three windows.

[0091] This embodiment achieves refined and independent management of the statistical processes for different data subjects by enabling each computing thread to efficiently manage the computational state of the keys it is responsible for using a hash table and maintaining the sliding window statistical state for each key. By updating the mean and variance of the sliding window through an online incremental algorithm and tracking extreme values ​​using a double-ended queue algorithm, it avoids redundant calculations in traditional methods, achieving efficient real-time updates of the sliding window statistical state and significantly improving computational efficiency. The use of a single-instruction multiple-data-stream instruction set to perform parallel computation of the intermediate statistical values ​​of each sliding window fully leverages the hardware's parallel computing capabilities, further accelerating the calculation speed of statistical indicators. Finally, by integrating all statistical indicators to form statistical features mapping the time-series data, it ensures the comprehensiveness and completeness of the features, effectively solving the technical bottleneck of low computational efficiency in traditional time-series data processing. This achieves efficient extraction of statistical features from massive time-series data while ensuring the accuracy and refinement of the calculation results.

[0092] In one feasible implementation, the step of aggregating the statistical features calculated by each computing thread into a global result pointer array in the order of the original data, and asynchronously writing the completed statistical features in the global result pointer array into a storage device during the parallel computing process of the computing threads, thereby completing the time-series data processing, includes: Step S61: Convert the statistical features obtained by each calculation thread into a text-formatted result string and store it in a dedicated output buffer. It should be noted that the result string refers to the string data of statistical features stored in character form after format conversion; the dedicated output buffer refers to a memory buffer allocated separately by the system for each computing thread, used only by that thread to store text-formatted result strings, and each buffer is independent of the others. The purpose of this operation is to convert the binary format statistical features into a common text format for easier subsequent storage and output, while using dedicated output buffers to isolate the storage of result data from each computing thread and avoid data corruption.

[0093] Specifically, after feature calculation is completed, each calculation thread needs to convert the obtained numerical results (such as sum, mean, standard deviation, etc.) back to text format for output. This process is also highly optimized, using special SIMD instructions to accelerate the conversion from integer to string. Each calculation thread retrieves the binary format mapped time series data statistical features it has calculated, and converts various binary statistical indicators into text format in sequence according to the system's preset format conversion rules. These are then integrated into a standardized result string, and the result strings are stored sequentially in their own dedicated output buffers according to the original data record sequence number. At the same time, the storage address and length information of each result string in the buffer are recorded, thus completing the format conversion and storage of statistical features.

[0094] Step S62: Fill the address and length of the result string in each dedicated output buffer into the corresponding position of the global result pointer array according to the record number of the original data to complete the aggregation of statistical features; It should be noted that the address refers to the starting position of the result string in memory storage within its dedicated output buffer; the length refers to the number of bytes of memory occupied by the result string; the record sequence number of the original data refers to the inherent sequential number of the data records in the original time-series data file, which is the basis for sorting the results; the global result pointer array refers to a global memory array created by the system and oriented towards all computing threads, indexed by the record sequence number of the original data; and aggregation refers to the process of integrating the statistical feature results scattered in the dedicated output buffers of each computing thread into the global array according to the order of the original data.

[0095] The purpose of this operation is to integrate the scattered statistical feature results in the order of the original data, ensuring the consistency between the processing results and the order of the original data. At the same time, the lock-free filling method avoids the synchronization overhead of multi-core collaboration. The implementation idea is that each computing thread reads the address and length of the result string from its own dedicated output buffer, and fills the address and length information into the corresponding index position of the global result pointer array according to the original data record sequence number corresponding to the result. The writing positions of each thread do not overlap, thus completing the aggregation of results.

[0096] Specifically, each computation thread iterates through all result strings in its own dedicated output buffer, extracts the original data record number corresponding to each result string, and the string's storage address and length in the buffer. Based on the record number, it finds the corresponding index position in the global result pointer array and accurately fills in the address and length information at that position. The filling operations of each computation thread are executed in parallel, and the array positions written to do not conflict, until the address and length information of all result strings have been filled. In essence, all threads aggregate the results through a global "result pointer array," which uses the "serial number" of the original data record as an index. Each computation thread only needs to fill in the "address" and length of its calculated result string into the corresponding "serial number" position in the array. Since each thread writes to a different position, this process also does not require locking, thus achieving the aggregation of statistical features.

[0097] Step S63: During the parallel computation process of the computing thread, the global result pointer array is checked in blocks according to the record sequence number of the original data. After confirming that the statistical features in the data blocks have been completed, the statistical features of the data blocks are written to the storage device in batches to complete the time-series data processing.

[0098] It should be noted that the record sequence number of the original data refers to the inherent sequential number of the original time-series data, which is the basis for the order of block partitioning and writing; block partitioning check refers to the operation of dividing the global result pointer array into several data blocks according to the record sequence number and checking whether the information in the block is complete; data block refers to the part corresponding to the continuous index interval in the global result pointer array, which is the basic unit of batch writing; statistical features have been completed means that all index positions in the data block have been filled with valid result string addresses and length information; batch writing refers to the operation of writing the statistical feature results corresponding to the entire data block to the storage device at once; storage device refers to media such as disks and solid-state drives used to store the final time-series data processing results.

[0099] The purpose of this system operation is to achieve asynchronous parallelism between computation and write-back, hiding the time spent on disk write operations within the computation process, thereby improving the overall system throughput. At the same time, block-based batch writing reduces the number of system calls and improves write efficiency. The implementation idea is to start an independent write-back process while the computation thread is performing parallel computation. The global result pointer array is divided into blocks according to the record sequence number. The integrity of the information in each block is checked. After confirmation, the result string is read from the dedicated output buffer according to the address and length of the block and written to the storage device in batches until all data blocks have been written.

[0100] In one possible implementation, the system can preset the data block size based on the performance of the storage device to maximize write efficiency. Simultaneously, the write-back process uses a lightweight loop to wait for incomplete blocks, avoiding excessive consumption of system resources.

[0101] Specifically, while each computing thread performs parallel computation, the system initiates an independent write-back process. This process divides the global result pointer array into several consecutive data blocks according to the record sequence number of the original data. It checks each data block in sequence to see if all index positions within the block have been filled with valid result string addresses and length information. If it is confirmed that the statistical features within a certain data block have been completed, the result string is read from the corresponding dedicated output buffer according to the address and length of each position within the block. The result string of the entire data block is then written to the storage device in batches. The checking and writing operations of all data blocks are completed in sequence until all information in the global result pointer array has been written, thus completing the entire time-series data processing flow.

[0102] This embodiment achieves the generalization of statistical feature format and the isolated storage of results from each thread by converting the statistical features obtained by the computation thread into text format and storing them in a dedicated output buffer. By filling the address and length of the result string into a global result pointer array according to the original data record sequence number, it completes the ordered aggregation of scattered results, ensuring the sequential consistency between the processed results and the original data, and the lock-free filling eliminates the synchronization overhead of multi-core collaboration. By performing block checks on the global result pointer array and batch writing it to the storage device during parallel computation, asynchronous parallelism between computation and write-back is achieved, hiding the disk write operation time within the computation process, and the block batch writing also significantly improves write efficiency. The entire process is seamlessly integrated, effectively solving the problems of disordered result integration and low efficiency caused by serial reading, writing, and computation in traditional time-series data processing. It further optimizes the processing flow of massive time-series data, significantly improving the overall throughput and processing efficiency of the system, while ensuring the accuracy and sequential consistency of the processing results.

[0103] In one feasible implementation, during the parallel computation of the computing thread, the global result pointer array is checked in blocks according to the record sequence number of the original data. After confirming that the statistical features within the data blocks have been completed, the statistical features of the data blocks are written in batches to the storage device to complete the time-series data processing. This includes: Step S71: While the computing thread is performing parallel computing, an independent write-back process is started and the global result pointer array is divided into blocks according to the record number of the original data. Then, the statistical features in the data blocks after the blocks are checked one by one to see if they have all been written into the global result pointer array. It should be noted that the write-back process refers to a dedicated processing flow in the system for writing statistical feature results to storage devices, which is executed concurrently with the calculation process; the record sequence number of the original data refers to the inherent sequential number of the data records in the original time-series data file, which is the core basis for result sorting and block partitioning; the global result pointer array refers to a global memory array indexed by the original data record sequence number, used to store the address and length of the statistical feature result string; block partitioning refers to the operation of dividing the global result pointer array into several data blocks according to consecutive record sequence numbers, with the data block being the basic unit of batch writing; all statistical features have been written means that all index positions within the data block have been filled with valid result string address and length information.

[0104] The purpose of this system operation is to achieve asynchronous parallelism between computation and result write-back, hiding the time consumed by disk write operations within the computation process. Simultaneously, it uses block-based checking to ensure ordered result verification, laying the foundation for subsequent batch writes. The implementation involves initiating an independent write-back process immediately after the computation thread starts parallel computation. The global result pointer array is evenly divided into blocks according to record sequence numbers. Then, the write status of all index positions within each block is checked sequentially to confirm whether statistical features have been written. In one possible implementation, the system can dynamically adjust the data block size based on the storage device's write performance and the total amount of time-series data to maximize write efficiency. Simultaneously, the write-back process verifies the boundaries of data blocks to ensure that blocks are non-overlapping, complete, and correspond perfectly with the record sequence numbers of the original data.

[0105] Specifically, while each computing thread begins executing parallel computation of time-series data, the system immediately initiates an independent write-back process. This process first obtains the index range of the global result pointer array, divides the global result pointer array into several continuous and uniformly sized data blocks according to the record sequence number of the original data, determines the record sequence number range and array index range corresponding to each data block, and then traverses all array index positions within the data block one by one according to the order of the data blocks, checking whether each position has been filled with the address and length information of the valid statistical feature result string, thereby determining whether the statistical features within the data block have been completely written.

[0106] Step S72: If the statistical features in the data block are not all completed, the data block is looped and waited until all statistical features are written. It's important to note that circular waiting refers to a lightweight polling approach used in the write-back process. This intermittently checks the write status of data blocks until all statistical features within the block have been written. Unlike blocking waiting, this avoids excessive consumption of system resources. The purpose of this operation is to ensure the integrity of batch write results, preventing the writing of incomplete statistical features to storage devices. Simultaneously, by using lightweight circular waiting, the system reduces the consumption of computing resources while ensuring real-time verification. The implementation involves pausing subsequent processing of a data block when an incompletely written index position is detected within the data block, entering a pre-set waiting state. After the wait ends, the write status of the data block is checked again, and this process is repeated until all index positions within the data block have completed the writing of statistical features.

[0107] In one possible implementation, the system can dynamically adjust the interval of the loop wait based on the processing speed of the computing thread, balancing real-time verification and resource utilization. During the loop wait, the system monitors the running status of the computing thread; if an abnormality is detected, the system will promptly terminate the wait and trigger an alarm.

[0108] Specifically, when the write-back process detects that there are array index positions in a data block that have not been filled with valid information, and determines that the statistical features in the data block are not all complete, the process will not continue to process subsequent data blocks, nor will it perform a write operation on the data block. Instead, it will enter a lightweight loop waiting state, intermittently checking the write status of all index positions in the data block at preset time intervals. After each check, it will determine whether the condition for complete writing is met. If not, it will continue to wait until all index positions in the data block have been filled with valid statistical feature result string addresses and length information.

[0109] Step S73: If all statistical features in the data block are completed, then the statistical features corresponding to the data block are written to the storage device in batches. It should be noted that "all statistical features completed" means that all array index positions within the data block have been filled with valid result string addresses and length information; "batch write" refers to the operation of writing all statistical feature results corresponding to the entire data block to the storage device at once, which differs from line-by-line writing and can significantly reduce the number of system calls; the storage device refers to the hardware medium used to store the final results of time-series data processing, including solid-state drives, hard disks, etc. The purpose of the system performing this operation is to improve the writing efficiency of statistical feature results, reduce the system overhead of disk I / O, and at the same time ensure the order and integrity of the written results. The implementation idea is that after determining that all statistical features within the data block have been written, the write-back process reads all result strings in batches from the dedicated output buffer of the corresponding calculation thread according to the result string address and length of each index position within the data block, integrates them according to the record sequence number of the original data, and writes the result strings of the entire data block to the storage device at once.

[0110] In one possible implementation, the write-back process performs an integrity check on the read result string before batch writing to ensure that the data is complete and error-free. Furthermore, the system configures a large-capacity I / O buffer for the storage device to further improve the speed of batch writing.

[0111] Specifically, when the write-back process detects that all array index positions within a data block have been filled with valid result string addresses and length information, and determines that all statistical features within the data block are complete, it accurately reads all text-formatted statistical feature result strings from the dedicated output buffer of the corresponding computation thread based on the address and length recorded at each index position within the data block. The result strings are then organized according to the record sequence number of the original data, and subsequently, the entire organized result string of the data block is written to the preset storage device in batches, completing the write-back of the data block's results.

[0112] Step S74: Complete the inspection and writing operations of all data blocks in sequence until all statistical features in the global result pointer array have been written to the storage device, thus completing the time-series data processing.

[0113] It should be noted that "completed sequentially" means that the write-back process checks, waits for, and writes data one by one according to the original data record sequence number corresponding to the data block; the global result pointer array refers to a global array that uses the original data record sequence number as an index and stores the address and length of the result string; "all statistical features have been written to the storage device" means that the statistical feature results corresponding to all index positions in the global result pointer array have been written to the storage device in batches; and time series data processing refers to the entire process from reading and parsing the original time series data file to calculating statistical features and then writing back the results.

[0114] The purpose of this operation is to ensure that all statistical characteristics of the original time-series data are written to the storage device completely and in an orderly manner, thus achieving a closed loop in time-series data processing.

[0115] In one possible implementation, after writing all data blocks, the write-back process performs an integrity check on the result file in the storage device to ensure that the number and order of records in the result are consistent with the original data. After completing the time-series data processing, the system automatically releases all temporarily allocated memory resources, including dedicated memory areas, output buffers, and global result pointer arrays.

[0116] Specifically, after completing the checking, waiting, and batch write operations for a data block, the write-back process immediately performs a write status check operation on the next data block according to the original data record sequence number. This "check-wait-write" processing logic is repeated, sequentially processing all data blocks until the last data block in the global result pointer array completes its batch write. This confirms that all statistical feature results corresponding to all index positions in the global result pointer array have been successfully written to the storage device. At this point, the system automatically releases all temporary memory resources allocated during this time-series data processing, officially completing the entire time-series data processing flow. This design constitutes an efficient "producer-consumer" model: the computation thread is the "producer," continuously generating results; the write-back thread is the "consumer," continuously writing results to disk. These two processes overlap most of the time, thus "hiding" the disk write operation time within the core computation time, greatly improving the overall system throughput.

[0117] This embodiment achieves asynchronous parallelism between computation and write-back by initiating an independent write-back process while the computation thread performs parallel computation. It divides the global result pointer array into blocks according to the original data record sequence number and checks the write status of each block, effectively hiding disk I / O latency. By executing a lightweight loop to wait for incomplete blocks, the integrity of the write results is ensured, while avoiding excessive system resource consumption. Batch writing of completed blocks significantly reduces the number of system calls and improves the efficiency of result write-back. By sequentially checking and writing all data blocks until all global statistical features have been written back, the integrity and order consistency of the results are guaranteed, achieving a closed-loop process for time-series data processing. The entire process effectively solves the technical bottlenecks of low efficiency and high write-back overhead caused by serial reading, writing, and computation in traditional time-series data processing. It further optimizes the processing efficiency of massive time-series data, significantly improves the overall system throughput, and ensures the accuracy, integrity, and orderliness of the processing results, making the entire time-series data processing flow more efficient and stable.

[0118] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0119] This application also provides a timing data processing apparatus, please refer to... Figure 5 The time-series data processing device includes: The acquisition module 51 is used to acquire and map the original timing data file to the virtual address space in virtual memory to obtain the mapped timing data; The scanning module 52 is used to perform parallel pre-scanning of the mapped time series data through multiple parsing threads, and to count the amount of data to be processed corresponding to each computing thread based on the scanning results; The calculation module 53 is used to allocate a dedicated memory area to each calculation thread according to the amount of data to be processed corresponding to each calculation thread, and to completely parse the mapped timing data in the virtual address space through each parsing thread, and distribute the parsed mapped timing data to the dedicated memory area of ​​each calculation thread, so that each calculation thread can independently and in parallel calculate the mapped timing data in the dedicated memory area to obtain the statistical characteristics of the mapped timing data. The writing module 54 is used to aggregate the statistical features calculated by each computing thread into a global result pointer array in the original data order, and asynchronously write the completed statistical features in the global result pointer array to a storage device during the parallel computing process of the computing threads, thereby completing the time-series data processing. The time-series data processing apparatus provided in this application, which adopts the time-series data processing method in the above embodiments, can solve the technical problems in the background art. Compared with the prior art, the beneficial effects of the time-series data processing apparatus provided in this application are the same as the beneficial effects of the time-series data processing method provided in the above embodiments, and other technical features in the time-series data processing apparatus are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0120] This application provides a timing data processing apparatus, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the timing data processing method in the first embodiment described above.

[0121] The following is for reference. Figure 6This document illustrates a structural schematic diagram of a timing data processing device suitable for implementing embodiments of this application. The timing data processing device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The timing data processing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0122] like Figure 6 As shown, the timing data processing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the timing data processing device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the timing data processing device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows timing data processing devices with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0123] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0124] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0125] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the timing data processing methods provided by the methods described above.

[0126] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0127] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0128] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A time-series data processing method, characterized in that, include: The original timing data file is acquired and mapped to the virtual address space in virtual memory to obtain the mapped timing data; The mapped time series data is pre-scanned in parallel by multiple parsing threads, and the amount of data to be processed corresponding to each computing thread is counted based on the scan results. Based on the amount of data to be processed corresponding to each computing thread, a dedicated memory area is allocated to each computing thread. Each parsing thread performs complete parsing of the mapped timing data in the virtual address space and distributes the parsed mapped timing data to the dedicated memory area of ​​each computing thread. Each computing thread performs independent parallel computation on the mapped timing data in the dedicated memory area to obtain the statistical characteristics of the mapped timing data. The statistical features calculated by each computing thread are aggregated into a global result pointer array in the order of the original data. During the parallel computing process of the computing threads, the completed statistical features in the global result pointer array are asynchronously written to the storage device to complete the time-series data processing.

2. The time-series data processing method as described in claim 1, characterized in that, The process involves parallel pre-scanning of the mapped time-series data using multiple parsing threads, and statistical analysis of the amount of data to be processed for each computation thread based on the scan results. The mapping time-series data is logically divided into blocks, and corresponding mapping time-series data blocks are allocated to each of the parsing threads. Each of the aforementioned parsing threads performs a parallel pre-scan of the allocated mapped time-series data blocks to parse out the key value corresponding to each record in the mapped time-series data. The computing thread corresponding to each record is determined based on the key value, and the number of data records to be processed corresponding to each computing thread is counted to obtain the amount of data to be processed for each computing thread.

3. The time-series data processing method as described in claim 1, characterized in that, The process involves allocating a dedicated memory region for each computing thread based on the amount of data to be processed, using each parsing thread to fully parse the mapped timing data in the virtual address space, and distributing the parsed mapped timing data to the dedicated memory region of each computing thread. This allows each computing thread to perform independent parallel computation on the mapped timing data in its dedicated memory region, obtaining the statistical characteristics of the mapped timing data, including: Based on the amount of data to be processed corresponding to each computing thread, a matching dedicated memory area is allocated to each computing thread; The mapping time series data is completely parsed by each of the parsing threads, various calculation fields in the mapping time series data are extracted and their formats are converted, and the parsed content is encapsulated into work items. Based on the key value corresponding to each record in the mapping time series data, the work item is distributed to the dedicated memory area of ​​the computing thread corresponding to the work item; Each of the aforementioned computing threads performs independent parallel computation on the work items in the dedicated memory region, maintains the sliding window statistical state corresponding to each key value, and updates and calculates the statistical characteristics of the mapped time-series data.

4. The time-series data processing method as described in claim 3, characterized in that, The process involves fully parsing the mapped time-series data through each of the parsing threads, extracting various calculated fields from the mapped time-series data and performing format conversion, and encapsulating the parsed content into work items, including: The mapping time series data is parsed line by line by each of the parsing threads to extract various calculation fields; The extracted calculated fields are formatted using a single instruction multiple data stream instruction set, converting text-formatted data into binary-formatted data. The various calculated fields after format conversion are encapsulated to form standardized work items.

5. The time-series data processing method as described in claim 3, characterized in that, The process of independently and in parallel computing each of the aforementioned computing threads on the work items in the dedicated memory region, maintaining the sliding window statistical state corresponding to each key value, and updating and calculating the statistical characteristics of the mapped time-series data includes: Obtain the calculation status of the key values ​​managed by each of the aforementioned calculation threads using a hash table, and maintain at least one sliding window of statistical status for each of the aforementioned key values; An online incremental algorithm is used to update the mean and variance of the sliding window, and a double-ended queue algorithm is used to track the maximum and minimum values ​​of the sliding window. The intermediate statistical values ​​of each sliding window are calculated in parallel using a single instruction multiple data stream instruction set to obtain the statistical indicators corresponding to each sliding window, and then integrated to form the statistical characteristics of the mapped time series data.

6. The time-series data processing method as described in claim 1, characterized in that, The step of aggregating the statistical features calculated by each computing thread into a global result pointer array in the order of the original data, and asynchronously writing the completed statistical features in the global result pointer array to a storage device during the parallel computing process of the computing threads, to complete the time-series data processing, includes: The statistical features obtained by each of the computing threads are converted into text-formatted result strings and stored in a dedicated output buffer. The addresses and lengths of the result strings in each dedicated output buffer are filled into the corresponding positions in the global result pointer array according to the record sequence number of the original data, thus completing the aggregation of statistical features; During the parallel computation process of the computing thread, the global result pointer array is checked in blocks according to the record sequence number of the original data. After confirming that the statistical features in the data blocks have been completed, the statistical features of the data blocks are written to the storage device in batches to complete the time-series data processing.

7. The time-series data processing method as described in claim 6, characterized in that, During the parallel computation process of the computing thread, the global result pointer array is checked in blocks according to the record sequence number of the original data. After confirming that the statistical features within the data blocks have been completed, the statistical features of the data blocks are written to the storage device in batches to complete the time-series data processing, including: While the computing thread performs parallel computing, an independent write-back process is initiated and the global result pointer array is divided into blocks according to the record sequence number of the original data. The statistical features in the data blocks after the blocks are checked one by one to see if they have all been written into the global result pointer array. If not all statistical features within the data block are completed, the data block will be looped and waited until all statistical features are written. If all statistical features within the data block are completed, the statistical features corresponding to the data block are written to the storage device in batches. The process of checking and writing all data blocks is completed sequentially until all statistical features in the global result pointer array have been written to the storage device, thus completing the time-series data processing.

8. A time-series data processing apparatus, characterized in that, include: The acquisition module is used to acquire and map the raw timing data file to the virtual address space in virtual memory to obtain the mapped timing data. The scanning module is used to perform parallel pre-scanning of the mapped time series data through multiple parsing threads, and to count the amount of data to be processed corresponding to each computing thread based on the scanning results; The calculation module is used to allocate a dedicated memory area to each calculation thread according to the amount of data to be processed corresponding to each calculation thread, and to completely parse the mapped timing data in the virtual address space through each parsing thread, and distribute the parsed mapped timing data to the dedicated memory area of ​​each calculation thread, so that each calculation thread can independently and in parallel calculate the mapped timing data in the dedicated memory area to obtain the statistical characteristics of the mapped timing data. The writing module is used to aggregate the statistical features calculated by each computing thread into a global result pointer array in the order of the original data, and asynchronously write the completed statistical features in the global result pointer array into the storage device during the parallel computing process of the computing threads, thereby completing the time-series data processing.

9. A time-series data processing device, characterized in that, The time-series data processing device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the time-series data processing method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the timing data processing method as described in any one of claims 1 to 7.

Images