Highly parallelizable scalable quant backtesting system based on event-driven

Abstract

Embodiments of the present disclosure disclose an event-driven high-parallel scalable quantification backtest system. A specific implementation of the system comprises an event-driven server, a matrix calculation server, a plug-in management server and a virtual resource storage connected in communication with each other, wherein the event-driven server is configured to read an event information queue from a historical data stream and parse the event information queue into a task instruction set; the matrix calculation server is configured to parse atomic operators included in the task instruction set, reorganize logical formulas with the same atomic operators into batch matrix operation expressions to obtain a batch matrix operation expression set, and execute the batch matrix operation expression set to obtain a numerical result matrix set. The implementation achieves the effects of improving the data processing speed of the system and reducing the waste of computing resources.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more particularly to an event-driven, highly parallel, scalable quantization backtesting system. Background Technology

[0002] In the field of computer simulation and system performance analysis, especially in applications requiring high-fidelity and high-efficiency simulation of historical processes with complex event sequences and a large number of computational tasks, it is often necessary to build an event-driven simulation system. The core task of such a system is to drive each functional module to perform corresponding calculations and state updates strictly according to the timestamp sequence of historical events, in order to reproduce and analyze specific processes. Currently, the common approaches to implementing such simulation systems are: a serial processing architecture based on discrete isomorphic operations: traditional simulation systems often use direct traversal and serial computation when handling massive, highly repetitive computational tasks; and a system design and deployment mode based on a tightly coupled monolithic architecture: at the system architecture level, existing solutions often encode multiple functional components such as data acquisition, logic processing, and rule judgment in a tightly coupled manner within a single process.

[0003] However, when using the above system for simulation, the following technical problems often arise: Slow data processing speed leads to distorted simulation results, resulting in wasted computing resources. The use of discretization and serialization to handle massive homogeneous computational tasks fails to effectively utilize the parallel processing capabilities of modern computing hardware. This makes data processing and computation speed the main bottleneck for system performance, limiting the scale and speed of the simulation system. To improve computational efficiency, some solutions employ simplified event handling models or batch approximations, but this sacrifices the simulation's ability to accurately reproduce the precise temporal sequence and causal logic of the original events, leading to distorted simulation results and further wasting computing resources.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure propose an event-driven, highly parallel, scalable simulation backtesting system to address one or more of the technical problems mentioned in the background section above.

[0007] Some embodiments of this disclosure provide an event-driven, highly parallel, scalable quantization backtesting system. The system includes: an event-driven server, a matrix computation server, a plug-in management server, and a virtual resource storage that are interconnected. The event-driven server is used to read an event information queue from a historical data stream and parse the event information queue into a task instruction set. The historical data stream includes at least one of an IoT sensor data stream, a network traffic log data stream, and an item flow data stream. The matrix computation server is used to parse the atomic operators included in the task instruction set, recombine logical formulas with the same atomic operators into batch matrix operation expressions to obtain a batch matrix operation expression set, and execute the batch matrix operation expression set to obtain a numerical result matrix set. The virtual resource storage is used to store the event information queue and the numerical result matrix set.

[0008] The various embodiments of this disclosure have the following beneficial effects: The event-driven, highly parallel, scalable quantization backtesting system based on some embodiments of this disclosure combines precise event timing-driven processing, matrix-based parallel processing of homogeneous computing tasks, and plug-in dynamic management of functional modules to improve system data processing speed and reduce the waste of computing resources. Specifically, the reasons for slow data processing speed, distorted simulation results, and consequently wasted computing resources are as follows: Because the massive homogeneous computing tasks are processed in a discretized and serial manner, the parallel processing capabilities of modern computing hardware cannot be effectively utilized, making data processing and computation speed the main bottleneck of system performance, limiting the scale and data processing speed of the simulation system. To improve computational efficiency, some solutions use simplified event processing models or batch approximation calculations, but this sacrifices the simulation process's ability to reproduce the precise timing and causal logic of the original events, leading to distorted simulation results and consequently wasted computing resources. Based on this, some embodiments of the event-driven, highly parallel, scalable quantization backtesting system disclosed herein include an event-driven server, a matrix computation server, and a virtual resource storage that are interconnected. First, the event-driven server reads an event information queue from a historical data stream and parses the event information queue into a task instruction set. The historical data stream includes at least one of an IoT sensor data stream, a network traffic log data stream, and a goods circulation data stream. This allows for strict scheduling of tasks based on the chronological order of event timestamps, technically eliminating "look-ahead bias" and ensuring accurate reproduction of the real event sequence during simulation, thus improving the fidelity and reliability of the simulation results. Simultaneously, parsing different events into standardized task instructions provides a unified and decoupled driving interface for subsequent modules. Second, the matrix computation server parses the atomic operators included in the task instruction set, reorganizes logical formulas with identical atomic operators into batch matrix operation expressions, obtains a set of batch matrix operation expressions, and executes the set of batch matrix operation expressions to obtain a set of numerical result matrices. Therefore, massive discrete and isomorphic scalar computations can be reorganized into continuous and regular matrix operations, thus fully utilizing the single-instruction multiple-data extended instruction set and cache of the central processing unit (CPU) to achieve hardware-level parallel acceleration of computational tasks and improve the throughput of data processing and numerical computation. Furthermore, the aforementioned virtual resource memory is used to store the event information queue and the numerical result matrix set. This provides an ordered input buffer for event-driven processes and centralized, persistent storage for all intermediate and final states of the entire simulation process. This enables reliable recording and backtracking of simulation states, supports asynchronous aggregation and reuse of computation results, and avoids redundant data computation and transmission. This implementation method achieves the effect of improving system data processing speed and reducing the waste of computing resources. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0010] Figure 1 This is a schematic diagram of the structure of some embodiments of an event-driven, highly parallel, scalable quantization backtesting system according to the present disclosure; Figure 2 This is a schematic diagram showing the horizontal performance comparison of some embodiments of the event-driven, highly parallel, scalable quantization backtesting system according to the present disclosure on multiple quantization evaluation metrics. Figure 3 This is a line graph comparing the cumulative revenue changes of some embodiments of the event-driven, highly parallel, scalable quantization backtesting system according to this disclosure. Detailed Implementation

[0011] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0012] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0013] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0014] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0015] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0016] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0017] Please see Figure 1 , Figure 1 This is a schematic diagram 100 of some embodiments of the event-driven, highly parallel, scalable quantization backtesting system disclosed herein. The event-driven, highly parallel, scalable quantization backtesting system includes: an event-driven server 101, a matrix computation server 102, and a virtual resource storage 103. The aforementioned quantization backtesting is an important method for verifying the validity of algorithm logic using historical data.

[0018] Event-driven server 101 is a server used to read event information queues from historical data streams and parse the event information queues into task instruction sets.

[0019] In some embodiments, the event-driven server 101 can read an event information queue from a historical data stream and parse the event information queue into a task instruction set. The task instructions in the task instruction set include logical formulas and target module identifiers. The logical formulas include atomic operators and at least one operand. The historical data stream includes at least one of IoT sensor data streams, network traffic log data streams, and item transfer data streams. The event-driven server 101 may include, but is not limited to, at least one of the following: a network processing unit, memory, and storage. The network processing unit may be a high-performance smart network interface card (NIC) with multiple queues and hardware offloading capabilities, specifically designed for line-speed reception, classification, and distribution of massive event message packets.

[0020] In some optional implementations of certain embodiments, the event-driven server 101 described above can be configured to perform the following steps: The first step is to obtain an event information queue from the historical data stream. In practice, the aforementioned execution entity can read at least one event information from the historical data stream and, based on the "event timestamp" attribute of the at least one event information, arrange the at least one event information into an ordered queue according to chronological order, thus obtaining the event information queue. The event information queue can include at least one event information. The event information can include a data source identifier, an event timestamp, an event type identifier, and a description. The event information can be a standardized description of an independently occurring event (such as "data update," "timed period arrival," or "command trigger"). The event timestamp can be a numerical value indicating the moment the event information occurred. The data source identifier can be a string format label used to uniquely distinguish different data input sources in the system. The event type identifier can be a label used to distinguish the event information (such as an enumerated value or a specific string). For example, the event type identifier can include data update events, timed period events, and command trigger events. The data update event can indicate that external data has changed. The timed period event can indicate that a preset simulation cycle node has been reached (such as the start or end of each minute or day). The command trigger event can indicate the issuance or cancellation of a command. The aforementioned historical data stream can be a collection of event information arranged in chronological order, serving as the system's input data source. This historical data stream includes at least one of the following: IoT sensor data stream, network traffic log data stream, and item movement data stream.

[0021] As an example, in response to the aforementioned historical data stream being an IoT sensor data stream used for greenhouse monitoring, it is necessary to replay the temperature and humidity sensor data from the past week to verify the effectiveness of a certain ventilation control rule. The aforementioned event-driven server reads event information from the IoT sensor data stream in chronological order of event timestamps and arranges it into an event information queue. For example, the event information could be: "Data source identifier: 'SENSOR.TEMP.GREENHOUSE_01', Event timestamp: '2026-06-20 14:30:00.000', Event type identifier: 'Data update', Description: 'The temperature sensor in greenhouse 1 reported the current temperature as 28.5℃'" or "Data source identifier: 'ACTUATOR.FAN.GREENHOUSE_01', Event timestamp: '2026-06-20 14:31:00.000', Event type identifier: 'Command triggered', Description: 'The system issued a command: Turn on the ventilation fan in greenhouse 1'."

[0022] As an example, in response to the aforementioned historical data stream being a network traffic log data stream, it is necessary to replay the enterprise network firewall logs from the past hour to evaluate the false negative rate of a novel intrusion detection algorithm. The aforementioned event-driven server reads event information from the network traffic log data stream in chronological order of event timestamps and arranges it into an event information queue. For example, the above event information could be: "Data source identifier: 'NETFLOW.EGRESS.VPN_01', Event timestamp: '2026-06-20 15:00:00.123', Event type identifier: 'Data update', Description: 'VPN egress gateway records an outbound packet, source IP 10.0.0.5, destination IP 203.0.113.50'" or "Data source identifier: 'FIREWALL.RULE.DMZ_01', Event timestamp: '2026-06-20 15:00:03.789', Event type identifier: 'Command triggered', Description: 'Firewall enforces rule: Block all subsequent connections from 203.0.113.50'".

[0023] As an example, in response to the aforementioned historical data stream being an item flow data stream, it is necessary to replay the order and inventory change records from the past Double Eleven period to verify the efficiency of a new picking route optimization algorithm. The aforementioned event-driven server reads event information from the item flow data stream in chronological order of event timestamps and arranges it into an event information queue. For example, the above event information could be: "Data source identifier: 'WMS.INVENTORY.SKU_A001', Event timestamp: '2026-06-20 15:00:00.123', Event type identifier: 'Data update', Description: 'WMS system update: The inventory of SKU A001 has changed from 500 units to 450 units'" or "Data source identifier: 'OMS.ORDER.ORD_20260620001', Event timestamp: '2026-06-20 15:00:03.789', Event type identifier: 'Instruction trigger', Description: 'OMS system issues a new order: ORD_20260620001, containing 50 units of SKU A001'".

[0024] The second step is to parse the aforementioned event information queue to generate a task instruction set. In practice, the executing entity can extract the data source identifier and event type identifier for each event in the event information queue, and then query the event information-task instruction mapping table to obtain the corresponding task instruction. This event information-task instruction mapping table can be a predefined mapping table between fields such as the logical formula and target module identifier corresponding to the event type identifier and data source identifier. The task instructions in the task instruction set include logical formulas and target module identifiers. The logical formula can be a structured mathematical expression that can be parsed and executed by a computer. For example, "Revenue = ∑(Price × Quantity)". "Price" and "Quantity" are operands. "×" is an atomic operator. The target module identifier can be an identifier used to specify the functional module that executes the task instructions.

[0025] For example, taking the greenhouse monitoring mentioned above as an example, the response to the above event information is: "Data source identifier: 'ACTUATOR.FAN.GREENHOUSE_01', event timestamp: '2026-06-20 14:31:00.000', event type identifier: 'command triggered', description: 'System issued command: turn on the ventilation fan of greenhouse No. 1'". The corresponding task command obtained by querying the above event information-task command mapping table can be "Target module identifier: 'control execution module', logical formula: 'comfort_index = temperature * 0.4 + humidity * 0.3 + co2_level * 0.3'".

[0026] In addressing the technical challenges mentioned above, the application scenario—a highly reliable and consistent remote management and command synchronization system for industrial IoT device clusters—often presents the following technical issues: In event-driven asynchronous task processing systems, system interruptions, event replays, or unclear task dependencies can lead to task state loss, duplicate execution, disordered execution order, and inconsistent states after system recovery, resulting in wasted computing resources. Considering the following requirements for this application scenario: end-to-end state traceability, consistent processing logic results, and post-fault state reconstruction, we have decided to adopt the following solution: In some alternative implementations of certain embodiments, the event-driven server 101 described above may also be configured to perform the following steps: The first step is to maintain the event cursor table. In practice, the aforementioned execution entity can create and maintain the event cursor table in memory or persistent storage. This event cursor table can be a key-value data structure or a database table used to record the latest event processing status information corresponding to different data source identifiers. For example, whenever an event from a data source is successfully processed (i.e., its corresponding task instruction has been generated and distributed), the server atomically updates the value of the corresponding key in this table. This value records the latest event processing status information for the aforementioned data source, typically a data structure containing the event timestamp of the last processed event, a globally incrementing sequence number, and a checksum of the last processed event content. For example, the globally incrementing sequence number can be a 64-bit unsigned integer (uint64) that monotonically increments from 0, maintained independently for each data source. The checksum can be CRC32 or MD5.

[0027] The second step involves updating the event cursor table in real time in response to the generation of the aforementioned task instruction set. In practice, during the generation of the task instruction set, an update operation is triggered in real time after the executing entity successfully parses an event information and generates a corresponding "task instruction." This update operation can take the event information of the current event (data source identifier, event timestamp, sequence number, and event content) as input, determine the checksum, and write the event information into the entry corresponding to the data source identifier in the event cursor table. As an example, taking the greenhouse monitoring as an example, in response to the event information being: "Data source identifier: 'ACTUATOR.FAN.GREENHOUSE_01', event timestamp: '2026-06-2014:31:00.000', event type identifier: 'instruction triggered', description: 'System issued instruction: turn on the ventilation fan of greenhouse No. 1'", the above event information is concatenated into a string using a preset separator (vertical bar "|"). The above string can be represented as "ACTUATOR.FAN.GREENHOUSE_01|2026-06-20 14:31:00.000|2048|System command: Turn on ventilation fan of greenhouse No. 1". Then, the CRC32 hash function is called on the above string to obtain the checksum.

[0028] The third step is to attach enhanced fields to each task instruction in the aforementioned task instruction set. These enhanced fields include: an input resource handle pointing to the input data, an output resource handle pointing to the output location, a dependent task identifier indicating the preceding task, an idempotent key for task deduplication, and a priority field determining the scheduling order. The input and output resource handles can be Uniform Resource Identifiers (URIs) or memory address pointers. The dependent task identifier can be a unique identifier pointing to the task instruction ID of the preceding task that the current task depends on. The idempotent key can be a fixed-length string generated by a hash function from the event type identifier, the target module identifier, and the input resource handle, used to uniquely identify a computation task. The priority field can be an integer value used to determine the execution order when multiple tasks are ready at the same timestamp.

[0029] The fourth step involves responding to a detected system interruption and restart. Based on the aforementioned event cursor table, at least one unprocessed event information is retrieved from the historical data stream and designated as the second event information queue. In practice, in response to a detected system interruption and restart, the executing entity can read the event cursor table and, for each data source identifier in the event cursor table, use the event timestamp and sequence number as query parameters to call the historical data stream access interface to retrieve at least one event information, which is then designated as the second event information queue. The aforementioned system interruption can be an abnormal termination event that causes service stoppage, such as an unexpected process crash, hardware failure, or unplanned restart. The second event information queue can be an ordered set of unprocessed event information retrieved again from the breakpoint based on the event cursor table after system recovery.

[0030] The fifth step involves parsing the second event information queue to continue generating task instruction sets. By comparing the idempotent keys, successfully completed logical parts are identified and skipped. In practice, the execution entity can maintain a registry of completed task idempotent keys in response to parsing the second event information queue to continue generating task instruction sets. In response to generating a new task instruction set, the execution entity can compare the idempotent keys of the task instructions with the registry of completed task idempotent keys to obtain the query result. If the query result indicates that the key already exists, it means the task instruction generation task was successfully completed before the interruption. The execution entity can skip the successfully completed task instruction generation task to avoid duplicate processing. The registry of completed task idempotent keys can be an index table created and maintained in memory by the event-driven server to implement idempotency. The registry of completed task idempotent keys can use idempotent keys as keys and query results as values. The query result can include 0 or 1. A query result of 0 indicates that the key does not exist. A query result of 1 indicates that the key exists.

[0031] Step 6: Based on the aforementioned dependent task identifiers, construct the dependency graph between the aforementioned task instructions, and determine the scheduling and execution order of the aforementioned task instructions based on the aforementioned priority field. In practice, the execution entity can read the dependent task identifier of each task instruction in the aforementioned task instruction set, and construct a graph structure using the task instructions as nodes and the aforementioned dependent task identifiers as directed edges to obtain the dependency graph. Then, the execution entity can read the priority field of each task instruction and determine the scheduling and execution order of the aforementioned task instructions based on the highest priority. The aforementioned dependency graph can be a directed graph data structure used to represent the pre- and post-constraint relationships between multiple tasks.

[0032] The first to sixth steps and related content described above, as an inventive point of this disclosure, solve the technical problem of "in event-driven asynchronous task processing systems, due to abnormal system interruptions, event replays, or unclear task dependencies, task states are lost, repeated executions occur, execution order is disordered, and the state is inconsistent after system recovery, leading to a waste of computing resources." Factors leading to the waste of computing resources are often as follows: In event-driven asynchronous task processing systems, due to abnormal system interruptions, event replays, or unclear task dependencies, task states are lost, repeated executions occur, execution order is disordered, and the state is inconsistent after system recovery, leading to a waste of computing resources. Solving these factors can avoid repeated executions caused by abnormal system interruptions, reducing the waste of computing resources. To achieve this effect, firstly, an event cursor table is maintained. This provides the system with a persistent, event-level accurate snapshot of the processing progress, serving as a reliable source of system state. Secondly, the event cursor table is updated in real time during the generation of task instruction sets. This ensures that the event processing progress and task generation actions remain atomically consistent, avoiding progress loss or overshoot, and providing an accurate starting point for fault recovery. Third, add enhanced fields to each task instruction in the aforementioned task instruction set. This allows for the injection of key metadata such as idempotent keys, dependent task identifiers, and priorities into each task instance, giving it self-descriptive capabilities and providing structured input for subsequent deduplication, dependency resolution, and scheduling decisions. Fourth, in response to detecting a system abnormal interruption and restart, at least one unprocessed event information is retrieved from the historical data stream based on the aforementioned event cursor table, serving as a second event information queue. This enables the system to "resume interrupted tasks," accurately locating the processing position at the time of the failure and avoiding the huge waste of computing resources and delays caused by replaying all events from the beginning. Fifth, when parsing the aforementioned second event information queue to continue generating task instruction sets, the successfully completed logical parts are identified and skipped by comparing the aforementioned idempotent keys. This ensures idempotency in the processing process, guaranteeing that even if events or tasks are repeatedly generated during the recovery phase, their business logic will only be executed once, ensuring system state consistency. Sixth, based on the aforementioned dependent task identifiers, a dependency graph between the aforementioned task instructions is constructed, and based on the aforementioned priority fields, the scheduling execution order of the aforementioned task instructions is determined. Therefore, after recovery or under normal conditions, the task execution sequence can be intelligently arranged to meet the logical priority constraints between tasks and optimize system resource utilization and overall processing latency. Ultimately, this avoids duplicate execution caused by abnormal system interruptions and reduces the waste of computing resources.

[0033] The matrix calculation server 102 is a server used to parse the atomic operators included in the above task instruction set, reorganize the logical formulas with the same atomic operators into batch matrix operation expressions to obtain a batch matrix operation expression set, and execute the above batch matrix operation expression set to obtain a set of numerical result matrices.

[0034] In some embodiments, the matrix computation server 102 can parse the atomic operators included in the task instruction set, reorganize the logical formulas with the same atomic operators into batch matrix operation expressions to obtain a set of batch matrix operation expressions, and execute the set of batch matrix operation expressions to obtain a set of numerical result matrices. The matrix computation server 102 may include, but is not limited to, at least one of the following: a computation acceleration unit, high-bandwidth memory, a central processing unit, and storage. The computation acceleration unit may integrate one or more massively parallel computing processors, such as GPUs, TPUs, or dedicated AI computing cards, whose hardware architecture is specifically optimized for performing large-scale matrix and tensor operations involved in forward / backward propagation in diffusion transformers. The atomic operators may be the most basic, indivisible mathematical operators. The atomic operators may include, but are not limited to, scalar multiplication, vector dot product, Hadamard product, matrix multiplication, scalar addition, scalar subtraction, scalar division, absolute value operations, and scrolling window operations.

[0035] In some alternative implementations of certain embodiments, the matrix calculation server 102 described above can be configured to perform the following steps: The first step is to identify the atomic operators included in the logical formulas of each task instruction in the aforementioned task instruction set. In practice, the execution entity can perform the following operations on the logical formulas included in each task instruction in the aforementioned task instruction set: First, call the parser to convert the logical formulas into a syntax tree. Second, use depth-first search (DFS) to visit each node of the syntax tree and query the structure of each node in the pattern matching rule set to identify the atomic operators. The parser can be a recursive descent parser or a bottom-up parser (such as an LR parser). The syntax tree can be a tree-like data structure used to represent the syntactic structure of an expression. The syntax tree includes a root node, at least one internal node, and at least one leaf node. The root node can be "=" (assignment). Each internal node can represent an operator (such as addition +, multiplication *) or a function call (such as SUM), and each leaf node can represent an operand (such as the variable "price" or the constant "2.5"). As an example, in the syntax tree corresponding to the logical formula "Profit = SUM(Price * Quantity)", the root node is "=" (assignment). This root node includes the leaf node "Profit" and the internal node "SUM". The internal node "SUM" includes the leaf nodes "Price" and "Quantity". The pattern matching rules in the above set of rules can be used to map a syntax tree node to an atomic operator. For example, the above pattern matching rule could be {operator: '*', left_operand: Matrix, right_operand: Matrix} -> Hadamard product. The above pattern matching rule could also be {function: 'dot', arguments: [Vector,Vector]} -> vector dot product.

[0036] The second step is to recombine logical formulas with identical atomic operators to obtain a batch set of matrix operation formulas. In practice, the above execution entity can use a structure array conversion method to recombine logical formulas with identical atomic operators to obtain a batch set of matrix operation formulas. This structure array conversion method can be an AoS (Array of Structures) to SoA (Structure of Arrays) conversion method. Taking the greenhouse monitoring example, assume there are three logical formulas: comfort_index_1 = temp_1 * 0.4 + humi_1 * 0.3, comfort_index_2 = temp_2 * 0.4 + humi_2 * 0.3, and comfort_index_3 = temp_3 * 0.4 + humi_3 * 0.3. The matrix calculation server described above can extract the atomic operators in each logical formula as follows: mul(temp_1, 0.4), mul(humi_1, 0.3), add(result1, result2), mul(temp_2, 0.4), mul(humi_2, 0.3), add(result3, result4), mul(temp_3, 0.4), mul(humi_3, 0.3), add(result5, result6). Where result1 = mul(temp_1, 0.4), result2 = mul(humi_1, 0.3), result3 = mul(temp_2, 0.4), result4 = mul(humi_2, 0.3), result5 = mul(temp_3, 0.4), result6 = mul(humi_3, 0.3). `mul()` represents scalar multiplication, and `add()` represents scalar addition. The aforementioned execution entity can reorganize the first operand of all scalar multiplications into matrix A: [temp_1, humi_1; temp_2, humi_2; temp_3, humi_3], and the second operand of all scalar multiplications into matrix B: [0.4, 0.3; 0.4, 0.3; 0.4, 0.3], resulting in the batch matrix operation formula C = A⊙B. ⊙ represents element-wise multiplication. Similarly, the aforementioned execution entity can reorganize the first operand of all scalar additions into matrix D: [result1; result3; result5], and the second operand of all scalar additions into matrix E: [result2; result4; result6], resulting in the batch matrix operation formula F = D + E.

[0037] The third step is to pre-allocate contiguous and memory-aligned storage space for the aforementioned batch matrix operation set. In practice, the execution entity can pre-allocate contiguous and memory-aligned storage space for the aforementioned batch matrix operation set by calling the low-level memory management interface provided by the operating system or programming language runtime. As an example, the execution entity can request a large block of contiguous physical memory for the aforementioned batch matrix operation set at once using, for example, the C language's `posix_memalign` function or the C++'s `aligned_alloc` function. This contiguous and memory-aligned storage space can be a storage space where the matrix elements are sequentially arranged in physical memory addresses, and the starting address of the allocated memory is an integer multiple of a specific value (usually a power of 2, such as 16, 32, or 64). For example, the low-level memory management interface can be the C language's `posix_memalign` function or the C++'s `aligned_alloc` function.

[0038] The fourth step is to execute the above set of batch matrix operations to obtain a set of numerical result matrices. As an example, the execution entity can pass the memory addresses (pointers) of the above matrix A and matrix B as input parameters, and the memory address of matrix C as output parameters, to the SIMD instruction set (such as AVX / AVX512) built into a preset mathematical library (such as Intel MKL, OpenBLAS, or Eigen) to obtain a set of numerical result matrices.

[0039] In addressing the technical challenges of the aforementioned background technologies, the application scenario—a highly reliable and consistent remote management and command synchronization system for industrial IoT device clusters (e.g., predictive maintenance and real-time control for smart manufacturing production lines or smart grids)—often presents the following technical problems: When performing batch calculations (such as state prediction or anomaly detection model inference) on large-scale, high-dimensional, and potentially missing heterogeneous matrix sets, the irregular input data, diverse calculation modes, and lack of result management lead to complex processing flows, low computational efficiency, unreliable and difficult-to-trace results, and consequently, increased scheduling and storage overhead. Considering the following requirements for this application scenario: automatic input data regularization and standardization, intelligent adaptation and scheduling of computing units, and end-to-end auditability of the computation process, we have decided to adopt the following solution: In some alternative implementations of certain embodiments, the matrix calculation server 102 described above can be configured to perform the following steps: The first step is to perform dimension consistency correction on the aforementioned batch operation matrix set, resulting in a corrected batch operation matrix set. In practice, the execution entity can determine a target dimension and check whether the aforementioned batch operation matrix set is consistent with the target dimension. In response to inconsistency, the following operations are performed: First, for matrices smaller than the target dimension, pad the edges with specific values. Second, for matrices larger than the target dimension, truncate the portion whose top-left corner matches the target dimension. The target dimension can be the largest number of rows and columns among all matrices, or a preset number of rows and columns. The specific values ​​can be 0 or NaN.

[0040] The second step involves performing a missing value masking operation on the aforementioned corrected batch operation matrix set to obtain a masked batch operation matrix set. In practice, the executing entity can identify the missing value positions of each corrected batch operation matrix in the set, generate a Boolean mask matrix with the same dimensions as the corrected batch operation matrix, and merge the corrected batch operation matrix and the Boolean mask matrix into a masked batch operation matrix set. In this set, the Boolean mask matrix displays a value of True (or 1) at the missing value positions in the corrected batch operation matrix, and a value of False (or 0) at other positions.

[0041] The third step is to normalize the data type of the above-mentioned mask batch operation matrix set to obtain a normalized batch operation matrix set. In practice, the execution entity can convert different numerical data types (such as int8, float16, float32, float64) in the above-mentioned mask batch operation matrix set into a preset data type (e.g., float32 or float64) to obtain a normalized batch operation matrix set.

[0042] Fourth step: Based on the types of atomic operators corresponding to the above-mentioned normalized batch operation matrix set, perform the following operations: Vectorized element-wise arithmetic units are selected for scalar multiplication and Hadamard product. Mathematically, both scalar multiplication and Hadamard product involve independent multiplication and division operations on corresponding elements of a matrix, making vectorized element-wise arithmetic units suitable for highly parallel element-wise computation. These vectorized element-wise arithmetic units can be inline assembly code segments written using CPU Single Instruction Multiple Data Extended Instruction Sets (such as AVX2, AVX-512), compiler built-in functions, or calls to corresponding element-wise multiplication functions in highly optimized math libraries (e.g., the vsMul family of functions in the Intel SVML library, or array-type multiplication operators in the Eigen library).

[0043] Choose basic linear algebra matrix operation units for vector dot product and matrix multiplication. The vector dot product and matrix multiplication described above are typical core linear algebra operations, suitable for basic linear algebra matrix operation units. These basic linear algebra matrix operation units can be the GEMV (General Matrix-Vector Multiplication) or GEMM (General Matrix-Matrix Multiplication) functions from the BLAS library.

[0044] For scrolling window operations, a sliding window operation unit based on a circular buffer is selected. The aforementioned scrolling window operations require local computations (such as convolution, pooling, and moving average) by sliding a window across the matrix, making a sliding window operation unit based on a circular buffer suitable. This minimizes redundant data loading and improves cache utilization. The aforementioned sliding window operation unit based on a circular buffer can be a user-defined function. The aforementioned sliding window operation unit based on a circular buffer can be represented as follows: First, load the first N elements of the input sequence (N being the window size) into a circular buffer. Second, perform window computations (such as summation or variance calculation) on at least one element in the circular buffer to obtain the window result. Third, overwrite the first element of the circular buffer with the next element of the input sequence and update the head pointer of the circular buffer. Fourth, repeat steps two and three until the input sequence has been traversed.

[0045] The fifth step involves performing batch matrix operations on the normalized batch operation matrix set according to the selected computation unit, resulting in a numerical result matrix set. This batch matrix operation can be a mode where multiple matrices are submitted to the computation unit for processing in a single batch. This batch matrix operation reduces function call overhead and improves data throughput and hardware utilization.

[0046] Step 6: Generate and bind metadata for each numerical result matrix in the above numerical result matrix set. This metadata includes: operator type, matrix dimension, generation timestamp, and data verification digest. The matrix dimension can be the size of the numerical result matrix. The generation timestamp can be the time when the numerical result matrix was generated. The data verification digest can be the checksum (e.g., CRC32, MD5) of the numerical result matrix.

[0047] Step 7: Store the set of numerical results with accompanying metadata into the aforementioned virtual resource storage.

[0048] The aforementioned steps one through seven and related content, as an inventive point of this disclosure, solve the technical problem of "when performing batch operations on large-scale, high-dimensional heterogeneous matrix sets that may have missing data, the irregular input data, diverse calculation modes, and lack of result management lead to complex processing flows, low computational efficiency, unreliable results, and difficulty in traceability, thereby increasing scheduling and storage overhead." The factors leading to increased scheduling and storage overhead are often as follows: when performing batch operations on large-scale, high-dimensional heterogeneous matrix sets that may have missing data, the irregular input data, diverse calculation modes, and lack of result management lead to complex processing flows, low computational efficiency, unreliable results, and difficulty in traceability, thereby increasing scheduling and storage overhead. Solving these factors can reduce scheduling and storage overhead. To achieve this effect, firstly, dimensional consistency correction is performed on the aforementioned batch operation matrix set to obtain a corrected batch operation matrix set. This eliminates the inability of computational units to execute or logical errors caused by inconsistent input matrix dimensions, providing standardized, dimensionally unified input for subsequent batch parallel computation. Second, a missing value masking operation is performed on the corrected batch operation matrix set to obtain a masked batch operation matrix set. This explicitly and structurally represents the missing information in the data, preventing it from contaminating the results during computation, while preserving the integrity of the original data and providing a basis for reliability assessment. Third, the data type normalization process is performed on the masked batch operation matrix set to obtain a normalized batch operation matrix set. This unifies the data representation format in memory, eliminating precision loss and performance bottlenecks caused by mixed data types, creating optimal conditions for calling underlying high-performance computing libraries (such as BLAS), and maximizing the utilization of hardware computing units. Fourth, based on the types of atomic operators corresponding to the normalized batch operation matrix set, the following operations are performed: Vectorized element-wise operation units are selected for scalar multiplication and Hadamard product. This enables highly optimized units utilizing the Single Instruction Multiple Data (SIMD) instruction set to achieve maximum throughput for highly parallel element-wise operation modes. Basic linear algebra matrix operation units are selected for vector dot product and matrix multiplication. Therefore, for classic linear algebra operations, the deeply optimized Basic Linear Algebra Subroutine (BLAS) can be invoked to fully utilize CPU cache hierarchy and parallel computing capabilities. For rolling window operations, a sliding window operation unit based on a circular buffer is selected. Thus, for sliding window computations with data locality characteristics, the circular buffer algorithm logically reuses data, significantly reducing memory access overhead and improving cache friendliness and execution efficiency for such operations. Fifth, based on the selected operation unit, batch matrix operations are performed on the above-mentioned normalized batch operation matrix set to obtain a numerical result matrix set.Therefore, all input matrices can be efficiently computed in one go with optimal hardware utilization and memory access mode, significantly improving the throughput and response speed of the overall computing task. Sixth, metadata is generated and bound to each numerical result matrix in the aforementioned numerical result matrix set. This allows each computation result to be attached with a "digital fingerprint" containing information such as data verification summaries, achieving end-to-end traceability, verifiability, and auditability of the computation results, ensuring the reliability of the computation process and the credibility of the results. Seventh, the set of numerical result matrices with attached metadata is stored in the aforementioned virtual resource storage. This allows for the persistent and resource-based management of high-value computation results with complete contextual information, providing standardized and readily accessible data services for downstream data analysis, model training, or decision-making applications. Ultimately, this reduces scheduling and storage overhead.

[0049] In some alternative implementations of some embodiments, the matrix calculation server 102 described above can be configured to perform the following steps: In the first step, the aforementioned matrix computation server is deployed as multiple computation processes. These multiple computation processes can execute the aforementioned batch matrix operation task set in parallel.

[0050] The second step involves data exchange between the aforementioned matrix calculation server and the aforementioned event-driven server via shared memory or zero-copy technology.

[0051] Virtual resource storage 103 is a server used to store the aforementioned event information queue and the aforementioned numerical result matrix set.

[0052] In some embodiments, the virtual resource storage 103 can store the event information queue and the numerical result matrix set. The virtual resource storage 103 may include, but is not limited to, at least one of the following: a large-capacity storage array, a storage controller, and a high-speed data interface. The large-capacity storage array may consist of multiple hard disk enclosures, employing erasure coding or RAID technology, and is specifically designed to provide massive, persistent, and reliable object or block storage space for storing raw image data, generated latent representations, model files, and other cold, warm, and hot data. The storage controller may be equipped with an independent storage processor and a large cache, specifically responsible for storage-specific functions such as data redundancy, lifecycle management, and cross-device data scheduling. The high-speed data interface can provide a high-bandwidth (e.g., 100GbE or higher) network interface, dedicated to large data stream transmission with the matrix computing server 102, etc., with its hardware design goal being optimal throughput rather than low latency.

[0053] Optionally, the system may also include a plug-in management server 104, which is used to receive the numerical result matrix set and send the numerical result matrix set to the functional modules for corresponding processing operations.

[0054] In some embodiments, the plug-in management server 104 can receive the numerical result matrix set and send it to the functional module for corresponding processing operations. For example, the processing operations may include, but are not limited to, turning on the heater or the ventilation fan. The plug-in management server 104 may include, but is not limited to, at least one of the following: at least one functional module, a management bus interface, a central processing unit (CPU), and memory. The CPU and memory may be a combination of a general-purpose CPU and large-capacity system memory. The management bus interface can provide a reliable internal communication channel (e.g., via shared memory or a high-speed internal network) for receiving module control commands from the event-driven server 101 and returning status. The functional module may be an independent software binary component that supports dynamic loading, linking, execution, monitoring, unloading, and management at runtime. For example, the functional module may include, but is not limited to, a data module, an algorithm logic module, a flow module, or a control execution module. The data module can be used to acquire, preprocess, and provide standardized format data from various data sources. For example, the data module may be the open-source financial data interface library TuShare. The algorithm logic module can be used to perform tasks such as strategy calculation and signal generation. For example, the algorithm logic module may be the open-source event-driven backtesting framework Backtrader. The aforementioned workflow module can be used to process generated instructions and simulate or actually execute order flow operations. For example, the workflow module could be the order execution engine (nautilus-execution) of the open-source high-performance algorithmic trading platform NautilusTrader.

[0055] As an example, taking the aforementioned greenhouse monitoring as an example, the aforementioned plug-in management server can receive the aforementioned numerical result matrix set and send it to the control execution module to execute control commands such as ventilation and heating. Assume that the numerical result matrix F in the aforementioned numerical result matrix set is [85.2; 62.7; 91.5], where 85.2, 62.7, and 91.5 represent the comfort indices of greenhouses 1, 2, and 3, respectively. The aforementioned control execution module searches the control command mapping table to obtain the control command. The aforementioned control command mapping table can be represented as: {Condition: "Comfort index < 70.0", Control command: "Turn on heater"; Condition: "Comfort index ≥ 70.0 and ≤ 90.0", Control command: "Maintain status"; Condition: "Comfort index > 90.0", Control command: "Turn on ventilation fan"} In some alternative implementations of certain embodiments, the plug-in management server 104 is configured to perform the following steps: The first step is to integrate the aforementioned functional modules into the system using pre-defined standardized interfaces. These pre-defined standardized interfaces include: initialization functions, daily cycle start functions, daily cycle end functions, minute cycle start functions, and minute cycle end functions. These pre-defined standardized interfaces can be a set of application programming interface specifications predefined during the system design phase, which all functional modules must adhere to. The initialization functions can be functions automatically called once by the computer during system startup or module loading to perform one-time preparatory work. For example, the initialization functions can use the `__attribute__((constructor)` attribute of a dynamic link library in a Linux environment. The daily cycle start and end functions can be functions called sequentially by the computer when the system time or logical time enters a new natural day (or business day) and at the end of a natural day (or business day) to handle tasks with a daily cycle. For example, the daily cycle start and end functions can refer to the `on_start_of_day` and `on_end_of_day` callback methods in the open-source backtesting framework Backtrader. The aforementioned minute cycle start and end functions can be functions frequently called by the computer when the system time enters each minute (or a specific minute interval), used to handle fine-grained tasks requiring near real-time response and computation. For example, the aforementioned minute cycle start and end functions can refer to the handle_data method in the Zipline framework or the next method in the Backtrader framework.

[0056] The second step is to configure the aforementioned functional modules as dynamic link library files. In practice, the execution entity can use a development toolchain to configure these functional modules as dynamic link library files. As an example, the development toolchain can use preset compilation and linking options (such as -shared -fPIC) to compile the source code of the functional modules implementing the preset standardized interfaces into dynamic link library files. This development toolchain can be a collection of software tools, consisting of a compiler, linker, archiver, and other auxiliary tools (such as build systems and debuggers), used to automatically convert source code into executable code or library files. As an example, this development toolchain can include, but is not limited to, the GNU compiler collection, the Clang / LLVM toolchain, and the Microsoft Visual C++ toolchain. The dynamic link library file can be a binary executable file format that is loaded into memory and linked at runtime (not compile time) (often with the extension .so under Linux / Unix, and .dll under Windows).

[0057] The third step involves dynamically loading the aforementioned dynamic link library (DLL) file from the disk into system memory and establishing a call link between the system and the DLL file. In practice, the executing entity can call the first system API to dynamically load the DLL file from the disk into system memory. Then, the executing entity can call the second system API. By specifying the function name (such as "initialization function"), the entry memory address of the corresponding function is found and obtained from the DLL file in system memory. Finally, the entry memory address is recorded in the internal management table, thus establishing the call link between the system and the DLL file. As an example, the first system API could be the Linux `dlopen()` function. The second system API could be the Linux `dlsym()` function. The internal management table can be a key data structure created and maintained by the computer in memory to record the metadata of the loaded DLL file and its exported functions. As an example, the internal management table could be a hash table, an array, or a dictionary.

[0058] Optionally, the aforementioned event-driven, highly parallel, scalable quantization backtesting system may further include a parameter optimization scheduling server 105. This parameter optimization scheduling server 105 is used to implement fully automated, distributed, and highly reliable parameter optimization. The parameter optimization scheduling server 105 may include, but is not limited to, at least one of the following: a task scheduling processor and a high-speed network communication unit. The task scheduling processor may be at least one multi-core CPU, used to efficiently execute complex parameter space partitioning, job dependency analysis, resource allocation, and dynamic rescheduling algorithms. The high-speed network communication unit is equipped with a high-performance network interface card supporting remote direct data access protocols, used to distribute job instructions to distributed computing nodes at high speed and low latency, and to collect the status heartbeats and result metadata of each distributed computing node in real time. The parameter optimization scheduling server 105 can transform the parameter optimization process, which originally required high manual intervention, was slow and error-prone, into a standardized technical process that is fully automated by a computer, parallel, efficient, robust, and reliable, thereby reducing the time required for parameter tuning.

[0059] In some alternative implementations of certain embodiments, the parameter optimization scheduling server 105 described above can be configured to perform the following steps: The first step is to receive the parameter space and decompose it into at least one parameter subset. In practice, the execution entity can receive the parameter space. Then, according to a preset partitioning algorithm, the parameter space is decomposed into at least one parameter subset. The parameter space can be a multi-dimensional vector space consisting of all possible values ​​of the parameters. This parameter space typically includes parameter names, types, minimum values, maximum values, step sizes, or a list of discrete values. The preset partitioning algorithm can be the Latin hypercube sampling method. The parameter subset can be an independent range of parameters partitioned from the entire parameter space for parallel processing and allocated to a single computing node for complete backtesting.

[0060] The second step involves encapsulating at least one subset of the aforementioned parameters and the corresponding backtesting configuration information into an optimization job instruction set. In practice, the executing entity can package each subset of parameters along with the backtesting configuration information required for backtesting (such as backtesting start and end times, target range) into a single optimization job instruction set. The backtesting configuration information can be the environmental configuration information required to drive a backtesting job. For example, the backtesting configuration information may include the data source identifier, backtesting time interval, initial funds, or basic transaction costs. The optimization job instruction can be a structured data object generated by the parameter optimization scheduling server, used to start and manage an independent backtesting job on distributed computing nodes.

[0061] The third step involves sending the aforementioned optimized job instruction set to at least one compute node of the event-driven server to asynchronously start at least one backtest job instance. In practice, the execution entity can send each optimized job instruction in the aforementioned optimized job instruction set to at least one idle compute node of the event-driven server. Then, the at least one idle compute node starts the backtest job instance corresponding to the optimized job instruction in a non-blocking manner. The backtest job instance can be a program generated in a distributed computing environment after the parameter optimization scheduling server instantiates an "optimized job instruction".

[0062] Fourth, continuously monitor the execution status of at least one backtesting job instance, and perform the following operations on each of the at least one backtesting job instances: In response to detecting a backtest job instance failure or timeout, the corresponding optimization job instruction will be rescheduled to another available compute node. The execution status can be the stage of the backtest job instance's lifecycle, typically including waiting, running, success, failure, and timeout.

[0063] In response to the detection that the backtest job instance has been successfully executed, the corresponding set of numerical results matrix is ​​retrieved from the aforementioned virtual resource storage.

[0064] The fifth step involves normalizing and aggregating all the acquired numerical result matrices and generating a visual summary. In practice, the execution entity can use the Z-score method to normalize and aggregate all the acquired numerical result matrices, obtaining a normalized numerical result matrix. Then, a graphics rendering engine is used to transform the normalized numerical result matrix into a visual summary such as a line graph or histogram. This visual summary can be presented as follows: Figure 3 As shown above. Figure 3 This could be a line graph comparing three curves, showing the cumulative revenue changes of three different algorithmic logic modules over a time series spanning several months. (The above...) Figure 2 A horizontal comparison was made of the performance of three different algorithm logic modules (test_strategy_0, test, test_strategy_1) across multiple dimensions, including quantitative evaluation indicators such as Sharpe ratio, maximum drawdown, turnover rate, win rate, and volatility. As an example, the three different algorithm logic modules could include: Algorithm logic module A: triggering a cooling command when the temperature exceeds 30℃ based on a fixed threshold rule; Algorithm logic module B: dynamically adjusting ventilation intensity based on temperature deviation using a PID control algorithm; and Algorithm logic module C: predicting future temperature changes and adjusting accordingly based on historical temperature and humidity data using a machine learning prediction model.

[0065] The various embodiments of this disclosure have the following beneficial effects: The event-driven, highly parallel, scalable quantization backtesting system based on some embodiments of this disclosure combines precise event timing-driven processing, matrix-based parallel processing of homogeneous computing tasks, and plug-in dynamic management of functional modules to improve system data processing speed and reduce the waste of computing resources. Specifically, the reasons for slow data processing speed, distorted simulation results, and consequently wasted computing resources are as follows: Because the massive homogeneous computing tasks are processed in a discretized and serial manner, the parallel processing capabilities of modern computing hardware cannot be effectively utilized, making data processing and computation speed the main bottleneck of system performance, limiting the scale and data processing speed of the simulation system. To improve computational efficiency, some solutions use simplified event processing models or batch approximation calculations, but this sacrifices the simulation process's ability to reproduce the precise timing and causal logic of the original events, leading to distorted simulation results and consequently wasted computing resources. Based on this, some embodiments of the event-driven, highly parallel, scalable quantization backtesting system disclosed herein include an event-driven server, a matrix calculation server, a plug-in management server, and a virtual resource storage that are interconnected. First, the event-driven server reads an event information queue from a historical data stream and parses the event information queue into a task instruction set. The historical data stream includes at least one of an IoT sensor data stream, a network traffic log data stream, and a goods circulation data stream. This allows for strict scheduling of tasks based on the chronological order of event timestamps, technically eliminating "look-ahead bias" and ensuring accurate reproduction of the real event sequence during simulation, thus improving the fidelity and reliability of the simulation results. Simultaneously, parsing different events into standardized task instructions provides a unified and decoupled driving interface for subsequent modules. Second, the matrix calculation server parses the atomic operators included in the task instruction set, reorganizes logical formulas with identical atomic operators into batch matrix operation expressions to obtain a set of batch matrix operation expressions, and executes the set of batch matrix operation expressions to obtain a set of numerical result matrices. This allows massive discrete, isomorphic scalar computations to be reorganized into continuous, regular matrix operations, fully utilizing the CPU's Single Instruction Multiple Data Extended Instruction Set and cache to achieve hardware-level parallel acceleration of computational tasks and improve the throughput of data processing and numerical computation. Then, the aforementioned plug-in management server receives the numerical result matrix set and sends it to the functional modules for corresponding processing operations. This enables forced decoupling through pre-defined standardized lifecycle interfaces and allows functional modules to be compiled into independent dynamic link libraries, achieving runtime dynamic loading, hot replacement, and fault isolation of modules.This greatly enhances the system's scalability and maintainability, supports rapid integration of new functions and independent upgrades of existing modules, while ensuring the security of core module code and the overall stability of the system. Furthermore, the aforementioned virtual resource storage is used to store the aforementioned event information queue and the aforementioned numerical result matrix set. This provides an ordered input buffer for event-driven processes and centralized, persistent storage for all intermediate and final states of the entire simulation process. This enables reliable recording and backtracking of simulation states, supports asynchronous aggregation and reuse of calculation results, and avoids redundant data calculation and transmission. This implementation improves system data processing speed and reduces waste of computing resources. The above description is merely an explanation of some preferred embodiments of this disclosure and the principles of the applied technology. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to the technical solutions formed by specific combinations of the above-mentioned technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, technical solutions formed by substituting the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. An event-driven, highly parallel, scalable quantization backtesting system, characterized in that, The system includes: an event-driven server, a matrix calculation server, a plug-in management server, and a virtual resource storage that are interconnected. The event-driven server is used to read an event information queue from a historical data stream and parse the event information queue into a task instruction set. The task instructions in the task instruction set include logical formulas and target module identifiers. The logical formulas include atomic operators. The historical data stream includes at least one of IoT sensor data stream, network traffic log data stream, and item flow data stream. The matrix calculation server is used to parse the atomic operators included in the task instruction set, and recombine logical formulas with the same atomic operators into batch matrix operation expressions to obtain a set of batch matrix operation expressions, and execute the set of batch matrix operation expressions to obtain a set of numerical result matrices. The virtual resource storage is used to store the event information queue and the numerical result matrix set.

2. The system according to claim 1, characterized in that, The event-driven server is configured to: Obtain an event information queue from a historical data stream, wherein the event information queue includes at least one event information, and the event information includes a data source identifier, an event timestamp, an event type identifier, and a description; The event information queue is parsed to generate a task instruction set, wherein the task instructions in the task instruction set include logical formulas and target module identifiers.

3. The system according to claim 1, characterized in that, The matrix calculation server is configured to: Identify the atomic operators included in the logical formulas of each task instruction in the task instruction set, wherein the atomic operators include at least one of scalar multiplication, vector dot product, Hadamard product, matrix multiplication, and scroll window operations; Reorganize logical formulas with the same atomic operators to obtain a set of batch matrix operation formulas; Pre-allocate contiguous and memory-aligned storage space for the set of batch matrix operations; Execute the set of batch matrix operations to obtain a set of numerical result matrices.

4. The system according to claim 1, characterized in that, The system also includes: a plug-in management server, and The plugin management server is configured as follows: Functional modules are connected to the system through a pre-defined standardized interface; Configure the aforementioned functional modules as dynamic link library files; The dynamic link library file is dynamically loaded from the disk into the system memory, and a calling link is established between the system and the dynamic link library file.

5. The system according to claim 1, characterized in that, The matrix calculation server is further configured to: The matrix computation server is deployed as multiple computation processes, wherein the multiple computation processes execute the batch matrix operation task set in parallel; The matrix calculation server and the event-driven server exchange data through shared memory or zero-copy technology.

6. The system according to claim 1, characterized in that, The system also includes a parameter optimization scheduling server, and The parameter optimization scheduling server is configured to: Receive a parameter space and decompose the parameter space into at least one subset of parameters; The at least one subset of parameters and the corresponding backtesting configuration information are encapsulated into optimization job instructions to obtain an optimization job instruction set; The optimized job instruction set is sent to at least one compute node of the event-driven server to asynchronously start at least one backtest job instance; Continuously monitor the execution status of the at least one backtest job instance, and perform the following operations on each of the at least one backtest job instances: In response to detecting that a backtest job instance has failed or timed out, the optimized job instruction corresponding to the backtest job instance will be rescheduled to other available computing nodes; In response to the detection that the backtest job instance has been successfully executed, the corresponding numerical result matrix set is retrieved from the virtual resource storage; The obtained numerical result matrix set is normalized, aggregated, and a visual summary is generated.

Images