An analysis processing system of heterogeneous data sources

Abstract

The application discloses an analysis processing system of heterogeneous data sources, a data source abstraction layer, a uniform encapsulation of heterogeneous data sources by using an iterator design mode, a provision of a standard access interface set, a data processing layer for obtaining original data from the data source abstraction layer and converting the original data into structured data rows, a process execution layer for encapsulating heterogeneous execution units by using a uniform WorkerBase base class, the base class providing an initialization interface, an input / output port acquisition interface, a message notification interface, a work mode setting interface and a type acquisition interface, a process builder for configuring a connection relationship between workers, distributing tasks to a thread pool, adopting a work stealing mechanism for load balancing, and supporting the merging of continuous tasks. The heterogeneous data sources are uniformly encapsulated through the iterator interface and the structured data rows are output, data zero-reconstruction access is realized, meanwhile, the work stealing and task merging mechanism of the process builder and the encapsulation of the multi-modal execution units by using the WorkerBase base class achieve seamless compatibility and efficiency improvement of the heterogeneous analysis tools.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to an analysis processing system of heterogeneous data sources. BACKGROUND

[0002] In the field of embedded device applications, especially in scenarios involving complex system testing, monitoring and control, embedded devices will continuously generate massive amounts of test data. These data are usually obtained through various different collectors and recorded in various storage media. After data collection and recording, the core step is to effectively analyze and interpret these data to assess device status, identify abnormalities or verify performance. The typical analysis and interpretation process includes data screening, parameter analysis, data interpretation, report generation and data playback.

[0003] However, the current data analysis and interpretation system faces serious heterogeneity challenges in actual application, which seriously restricts efficiency and flexibility: first, the data sources are significantly heterogeneous. The collected data can be stored in various types of files or different database systems. Traditional solutions usually use hard-coded methods to directly interface each specific data source. This method causes the data reading logic and subsequent data screening rules and parameter analysis logic to be highly coupled. As a direct result, the analysis software developed for different data sources cannot reuse the core screening and analysis functions, and similar modules must be repeatedly developed for each data source, resulting in huge development and maintenance costs and making it difficult to adapt to dynamic expansion of data source types. Second, the data interpretation process relies heavily on criteria established by analysts. Due to differences in technical background and usage habits among analysts, the criteria themselves also exhibit high heterogeneity, with various forms including but not limited to text descriptions, script languages, dynamic link libraries or static libraries. The differences in the form of criteria make it difficult for the system to provide a unified execution environment. Different forms of criteria require the development of special analysis and execution engines, and the portability and reusability of the criteria themselves are extremely poor. Analysts have difficulty sharing and reusing existing criterion knowledge bases, and the system also has difficulty integrating new criterion types flexibly. Third, users often expect to automate and link the entire analysis process from data parsing and storage to interpretation and report generation. However, each work node in the process may need to run function modules implemented in different technology stacks, such as scripts, dynamic libraries, static libraries or independent applications. The heterogeneity of the implementation technology of the process nodes makes it extremely difficult to build a unified, flexible and programmable workflow engine. The system needs to solve complex problems such as communication between heterogeneous nodes, data transmission, state management, error handling and resource scheduling, resulting in cumbersome process construction, difficult maintenance and difficulty in dynamically adjusting the process structure according to requirements.

[0004] In summary, the prior art in processing embedded device data analysis interpretation is limited by the three core challenges of data source heterogeneity, criterion heterogeneity and process node heterogeneity. There are generally high system coupling, poor functional reuse, high development and maintenance costs, rigid process construction, and difficulty in adapting to rapidly changing requirements, which seriously hinder the improvement of data analysis efficiency and the development of technology. Therefore, there is an urgent need for a data analysis interpretation method and system that can effectively respond to the above heterogeneity challenges and achieve efficient, flexible and scalable data analysis. SUMMARY

[0005] The purpose of the present application is to overcome the technical defects of the prior art and provide an analysis processing system for heterogeneous data sources. The system uniformly encapsulates heterogeneous data sources through an iterator interface and outputs structured data rows to achieve data zero-reconstruction access. Meanwhile, the system uses a WorkerBase base class to encapsulate multi-modal execution units and a work stealing and task merging mechanism of the process builder to achieve seamless compatibility and efficiency improvement of heterogeneous analysis tools.

[0006] The purpose of the present application is achieved by the following technical solutions:

[0007] In a first aspect, the present application provides an analysis processing system for heterogeneous data sources, which includes:

[0008] A data source abstraction layer that uniformly encapsulates heterogeneous data sources using an iterator design pattern and provides a standard access interface set, including a First interface, a Next interface, an IsEnd interface and a Current interface. The First interface locates the first data, the Next interface jumps to the next data, the IsEnd interface judges the termination of data, and the Current interface outputs the current data row.

[0009] A data processing layer that includes a parameter parsing iterator to obtain raw data from the data source abstraction layer and convert it into a structured data row. The structured data row includes a data pointer, a total byte size, an element number, an element size and a data type enumeration value.

[0010] A process execution layer that encapsulates heterogeneous execution units through a unified WorkerBase base class. The base class provides initialization interfaces, input / output port acquisition interfaces, message notification interfaces, work mode setting interfaces and type acquisition interfaces.

[0011] A process builder that is used to configure the connection relationship between workers, allocate tasks to a thread pool, use a work stealing mechanism for load balancing, and support merging of consecutive tasks.

[0012] In one possible implementation, the data source abstraction layer includes:

[0013] File Iterator: First interface positions file handle to file head, Next interface moves handle by data block length, IsEnd interface detects file tail mark;

[0014] Database Iterator: First interface, Next interface and IsEnd interface are realized by standard database cursor;

[0015] Current interface outputs intermediate structure containing original data pointer.

[0016] In a possible implementation, the message notification interface of the WorkerBase base class is configured to send a finish_msg message to a superior node when a unit of execution completes data processing, so as to trigger the superior node to push new data to the message queue.

[0017] In a possible implementation, the flow constructor comprises:

[0018] A connection configuration module is configured to define data flow between Workers by creating an edge relationship interface.

[0019] A template loading module is configured to load corresponding templates according to Worker types, and the corresponding templates comprise a Python script template, a dynamic library template and an executable program template.

[0020] A dynamic scheduling module is configured to set serial / parallel identification of an interface according to a working mode, and allocate thread resources.

[0021] In a possible implementation, the work stealing mechanism is that each thread maintains a local task queue, and when it is detected that the local queue is empty, a task to be processed is stolen from a queue of the remaining threads at random.

[0022] In a possible implementation, the analysis processing system further comprises a report generation module configured to receive an interpretation result output by the flow execution layer, and automatically generate a traceable report containing abnormal data marks and criterion execution paths.

[0023] In a possible implementation, the data processing layer further comprises a data playback iterator configured to receive an abnormal data position marked by the flow execution layer, extract original data of an abnormal period from the data source abstraction layer in a directional manner, and generate a reproducible data stream.

[0024] In a possible implementation, the data type enumeration values of the structured data row comprise integer, single-precision floating point, double-precision floating point and string.

[0025] The above main scheme and each further selected scheme of the application can be freely combined to form multiple schemes, which are all the schemes that can be used and claimed by the application; and the application can also be freely combined between each non-conflicting selection and between and other selections. Those skilled in the art can understand that there are many combinations according to the prior art and common knowledge after understanding the schemes of the application, which are all the technical schemes claimed by the application, and are not enumerated here.

[0026] The application discloses an analysis processing system of heterogeneous data sources, a data source abstraction layer, a uniform encapsulation of heterogeneous data sources using an iterator design pattern, a set of standard access interfaces, a data processing layer that obtains raw data from the data source abstraction layer and converts it into structured data rows, a process execution layer that encapsulates heterogeneous execution units through a uniform WorkerBase base class, which provides initialization interfaces, input / output port acquisition interfaces, message notification interfaces, work mode setting interfaces, and type acquisition interfaces; a process builder for configuring the connection relationship between Workers, assigning tasks to a thread pool, using a work stealing mechanism for load balancing, and supporting the merging of consecutive tasks. By uniformly encapsulating heterogeneous data sources through the iterator interface and outputting structured data rows, data zero-reconstruction access is achieved, and by encapsulating multi-modal execution units with the WorkerBase base class and the work stealing and task merging mechanism of the process builder, seamless compatibility and efficiency improvement of heterogeneous analysis tools are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical schemes of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0028] Figure 1 The structure schematic diagram of the analysis processing system of heterogeneous data sources proposed by the embodiments of the application is shown.

[0029] Figure 2 The flowchart of data access to report generation proposed by the embodiments of the application is shown. DETAILED DESCRIPTION

[0030] The embodiments of the application are described below through specific and concrete examples, and those skilled in the art can easily understand other advantages and effects of the application from the disclosure. The application can also be implemented or applied through other different specific embodiments, and each detail in the specification can be modified or changed based on different views and applications without departing from the spirit of the application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0031] All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor fall within the scope of protection of the present application.

[0032] To solve the problems in the prior art, the embodiments of the present application propose an analysis processing system of heterogeneous data sources, please refer to Figure 1 , Figure 1 The structure diagram of the analysis processing system of heterogeneous data sources proposed by the embodiments of the present application is shown, and the system comprises:

[0033] The data source abstraction layer adopts the iterator design pattern to uniformly encapsulate the heterogeneous data sources, and provides a standard access interface set, the access interface set comprising a First interface, a Next interface, an IsEnd interface and a Current interface, the First interface locating the first data, the Next interface jumping to the next data, the IsEnd interface judging the termination of data, and the Current interface outputting the current data row;

[0034] In actual application environment, the time series data source presents the characteristics of diversification and heterogeneity, covering relational databases (such as MySQL, Oracle), time series databases, binary files, CSV semantic files and other forms. In order to realize the unified access and management of these heterogeneous data sources, the present application constructs a data source abstraction layer, which skillfully uses the iterator design pattern to abstractly process the data sources. By carefully designing the abstract iterator interface, the many differences of the underlying data sources are successfully shielded, and the data source management layer only provides four key interfaces of First, Next, IsEnd and Current, wherein the Current interface is responsible for outputting the data row at the current position, and provides a standardized data access mode for the upper layer application.

[0035] The data processing layer comprises a parameter parsing iterator, which obtains the original data from the data source abstraction layer and converts it into a structured data row.

[0036] The structured data row comprises a data pointer, a total byte number, an element number, an element size and a data type enumeration value;

[0037] The parameter analysis iterator integrated in the data processing layer first stably obtains raw data from the data source abstraction layer. The raw data has been orderly read through the unified access interface set (First, Next, IsEnd, Current) via the standardized processing of the data source abstraction layer. Subsequently, the parameter analysis iterator performs comprehensive and accurate analysis and conversion operations on the raw data according to the established analysis rules and data format requirements, and forges it into a structured data row. This structured data row is not simply a data combination, but contains rich information and strict structure: the data pointer accurately locates the storage position of the raw data in the memory, providing a quick channel for subsequent data access; the total byte number clearly identifies the allocated memory capacity; the element number explicitly records the number of stored data elements; the element size details the number of bytes occupied by a single data element, assisting accurate data parsing operations; and the data type enumeration value indicates the data type in a standardized form, such as integer, single-precision floating point, double-precision floating point, and string, ensuring accurate communication of data semantics.

[0038] The flow execution layer encapsulates heterogeneous execution units through a unified WorkerBase base class, which provides initialization interfaces, input / output port acquisition interfaces, message notification interfaces, work mode setting interfaces, and type acquisition interfaces;

[0039] The flow execution layer encapsulates and manages various heterogeneous execution units (i.e., Workers) through a unified WorkerBase base class, ensuring that different types of Workers can efficiently collaborate under a unified framework. The WorkerBase base class, as the common parent class of all Workers, provides consistent interface specifications and behavior frameworks for different types of Workers, including initialization interfaces, input / output port acquisition interfaces, message notification interfaces, work mode setting interfaces, and type acquisition interfaces. The initialization interface is used to initialize the Worker, set initial parameters, configure the running environment, and allocate system resources, ensuring that the Worker is in an executable state; the input / output port acquisition interface is used to obtain the input and output ports of the Worker, establishing data flow and connection relationships between Workers; the message notification interface sends messages to related nodes after the Worker completes data processing, achieving synchronization and data flow control between Workers in the process; the work mode setting interface allows users to flexibly set the work mode of the Worker as serial or parallel, improving data processing efficiency; and the type acquisition interface is used to obtain the type information of the Worker.

[0040] Flow builder, used to configure the connection relationship between workers, allocate tasks to thread pools, load balance with work stealing mechanism, and support merging continuous tasks.

[0041] The flow builder is responsible for comprehensively configuring the connection relationship between workers, accurately defining the flow path of data between different workers, and ensuring the smoothness and efficiency of the entire data processing process. At the same time, the flow builder reasonably allocates tasks to thread pools, and according to the task characteristics and system load status, adopts the work stealing mechanism to achieve efficient load balancing. When the local task queue of a thread is empty, the thread will actively steal tasks from the queue of other busy threads, thus eliminating thread idleness and maximizing the utilization of system resources. In addition, the flow builder also has intelligent optimization strategies, which can identify and merge continuous small tasks, effectively reducing thread scheduling overhead.

[0042] The data source abstraction layer includes:

[0043] File iterator: the First interface positions the file handle to the file head, the Next interface moves the handle by the data block length, and the IsEnd interface detects the file tail identifier;

[0044] Database iterator: the First interface, Next interface and IsEnd interface are implemented through the standard database cursor;

[0045] The Current interface outputs an intermediate structure containing the original data pointer.

[0046] For file-type data sources, a file iterator is specially designed to adapt to them. The file iterator accurately positions the file handle to the file head position through the First interface, and the Next interface moves the file handle according to the preset data block length, each time moving exactly across a complete data record, thus realizing the traversal reading of file data. The IsEnd interface monitors the position state of the file handle in real time, accurately judges whether it has reached the file end identifier, so as to ensure that the system can timely perceive whether the file data has been completely processed. The Current interface reads a data from the specific position when the file handle is located at the position, and outputs it after converting it into a standard data row format.

[0047] The database iterator is used to connect the database type data source. The database iterator uses the standard database cursor mechanism to efficiently implement the functions of the First, Next, and IsEnd interfaces. The First interface uses the standard interface provided by the database to initialize and accurately position the database cursor to the start of the data set, enabling the data reading to start in order from the first record. The Next interface uses the moving function of the database cursor to move the cursor forward to the position of the next data record each time, sequentially accessing each record in the data set to achieve the retrieval of each record in the database data. The IsEnd interface detects the state of the database cursor to determine whether the current cursor position has reached the end of the data set. If the cursor cannot move forward, it indicates that the data set has been completely read, and the system can decide whether to terminate the reading operation based on the result returned by the interface.

[0048] The Current interface obtains the data record at the specific record position of the database cursor through the relevant operations of the database, converts it into a data row format that meets the system's internal standard specifications, and outputs it, completing the conversion of the original database data into the system's unified data format.

[0049] Both the file iterator and the database iterator output a intermediate structure containing an original data pointer through their Current interfaces. The original data pointer directly points to the original data block stored in the underlying storage medium, and the system can quickly locate and access the actual data content through this pointer without the need for tedious data copying or conversion operations. The system can perform corresponding parsing, conversion, and other operations on the original data based on different data source types and data formats, ultimately generating standardized data rows that meet the system's internal specifications.

[0050] The message notification interface of the WorkerBase base class is configured to send a finish_msg message to the upper node when the execution unit completes data processing, triggering the upper node to push new data to the message queue.

[0051] The message notification interface of the WorkerBase base class ensures the efficiency and smoothness of the data processing flow. When a Worker (execution unit) completes the processing task of a piece of data, it immediately sends a finish_msg message to the upper node through this message notification interface. This finish_msg message serves as a signal, and when the upper node receives this message, it knows that the Worker is idle and has the ability to process new data. Therefore, the upper node will accordingly put new data into the message queue, waiting for the Worker to process it. In this way, the process execution layer can monitor the processing status of each Worker in real time and dynamically allocate tasks.

[0052] The flow constructor comprises:

[0053] The connection configuration module is configured to define the data flow between the Workers by creating the edge relationship interface.

[0054] The template loading module is configured to load the corresponding template according to the Worker type, and the corresponding template comprises a Python script template, a dynamic library template and an executable program template.

[0055] The dynamic scheduling module is configured to allocate thread resources according to the serial / parallel identification of the work mode setting interface.

[0056] The connection configuration module defines the data flow between the Workers by creating the edge relationship interface. When configuring the flow, the user clearly specifies which output of a Worker is connected to the input of another Worker by means of the module, thereby constructing the skeleton of the entire data processing flow.

[0057] The template loading module loads the corresponding template according to the Worker type. The system provides a variety of templates for different types of Workers, such as Python script templates, dynamic library templates and executable program templates. When a Worker of a specific type needs to be created, the template loading module automatically loads the corresponding template according to user configuration or system default settings. These templates not only encapsulate the execution logic of different types of Workers, but also provide a convenient parameter passing method for users, enabling different types of Workers to be quickly deployed and run under a unified framework.

[0058] The dynamic scheduling module closely depends on the serial / parallel identification of the work mode setting interface to allocate thread resources. For Workers configured in serial mode, the dynamic scheduling module reasonably arranges their tasks to be executed in sequence in the same thread, ensuring the order and consistency of data processing. For Workers configured in parallel mode, the module fully utilizes the advantages of multi-threading to allocate independent thread resources for them, enabling them to process tasks in parallel with other Workers, thereby significantly improving the efficiency and throughput of data processing.

[0059] The work-stealing mechanism is as follows: each thread maintains a local task queue, and when it detects that the local queue is empty, it randomly selects the queue of the remaining threads to steal the tasks to be processed.

[0060] The work stealing mechanism can ensure that all threads can fully utilize computing resources, avoid some threads being idle due to insufficient tasks, and improve the execution efficiency of the entire system. Each thread maintains a local task queue for storing the tasks assigned to the thread. During task execution, the thread first obtains a task from its local task queue and processes it. When a thread detects that its local task queue is empty, it indicates that the thread currently has no tasks to process, and there is a risk of idle resources. At this time, the thread will not be idle and wait, but will actively select the task queue maintained by other threads and steal one or more tasks from the task queue to its local queue. After stealing the tasks, the thread can continue to process the newly obtained tasks, thereby keeping busy and fully utilizing computing resources.

[0061] The analysis processing system further comprises a report generation module for receiving the interpretation results output by the process execution layer and automatically generating a traceable report containing abnormal data markers and criterion execution paths.

[0062] The report generation module receives tasks from the interpretation results output by the process execution layer. After obtaining these interpretation results, the report generation module will automatically generate detailed report documents according to pre-set rules and format requirements. The generated report not only contains the analysis results of the original data, but also specially marks the specific location and characteristics of the abnormal data, so that users can quickly locate and focus on the abnormal situation in the data. In addition, the report also records the execution path of the criterion, that is, how each criterion is applied and executed in the entire data processing process, including the trigger condition, execution order and related parameter settings of the criterion.

[0063] The data processing layer further comprises a data playback iterator for receiving the abnormal data location marked by the process execution layer, directing the extraction of abnormal period original data from the data source abstraction layer, and generating a replayable data stream.

[0064] The data playback iterator can realize accurate positioning and playback processing of abnormal data. During system operation, the data playback iterator first receives the abnormal data location information marked by the process execution layer. Once the key position coordinates are obtained, the data playback iterator cooperates closely with the data source abstraction layer to extract the original data fragments corresponding to the abnormal data from the massive data source. Subsequently, the data playback iterator integrates the extracted original data fragments to generate a replayable data stream. This replayable data stream is a special format and timing characteristic data copy that not only retains the content and characteristics of the original data, but also has the ability to be played back multiple times. Through this replayable data stream, users can repeatedly view, analyze and verify the abnormal data, and deeply explore the root cause of the abnormality, and verify the rationality of the data processing process.

[0065] The data type enumeration values of the structured data row include integer, single-precision floating point, double-precision floating point, and string.

[0066] The data row is presented in a structured data format, containing a raw data pointer, total capacity (bytes), current element quantity, single element size (bytes), and data type. The data type supports four types of integer, single-precision floating point, double-precision floating point, and string. Table 1 shows the data row:

[0067] Table 1

[0068]

[0069] In a possible implementation manner, Figure 2 A flowchart of data access to report generation according to an embodiment of the present application is shown. The system accesses heterogeneous data sources, and converts the data sources through a data source abstraction layer and a data processing layer into uniform structured data rows. Subsequently, a data access flow engine is accessed, and multiple types of workers (application type, script type, and dynamic library type) in the engine work cooperatively. A flow constructor is responsible for worker connection configuration and task allocation, and uses a work stealing mechanism to load balance and combine continuous tasks to improve efficiency. Finally, the system generates a traceable data analysis report containing abnormal data markers and criterion execution paths, and completes the whole process of data processing and analysis.

[0070] Compared with the prior art, the embodiment of the present application has the following beneficial effects:

[0071] First, through the iterator design pattern and the structured data row format, seamless compatibility for multi-source heterogeneous data is embodied. The design shields the underlying differences of database (MySQL / Oracle), time series database, binary file, CSV, and other data sources, accesses new data sources in cycles, shortens the weeks of traditional hard coding mode, and significantly improves system scalability.

[0072] Second, through template criterion configuration and visual process arrangement, the zero coding development capability of business experts is embodied. Domain experts can directly reuse standard operators and domain criteria to construct analysis processes without paying attention to underlying technical implementation, thereby improving business implementation efficiency.

[0073] Third, through the WorkerBase base class unified encapsulation of heterogeneous execution units and dynamic loading mechanism, the fusion capability of "adapting business to platform" is embodied. Combined with a unified data model and a rule engine, the whole process traceability from data cleaning to report generation is ensured.

[0074] Fourth, through the work of the flow builder algorithm, continuous task merging optimization and finish_msg message driven mechanism, the real-time processing capability of the thousand-node flow is embodied, the resource utilization is improved, the time delay is reduced, and the data processing of ten thousand per second is supported; Modular design makes new sensors / analysis tools pluggable access.

[0075] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An analytical processing system of heterogeneous data sources, characterized in that, The system comprises: a data source abstraction layer, which uniformly encapsulates heterogeneous data sources by using an iterator design pattern, and provides a standard access interface set, the access interface set comprising a First interface, a Next interface, an IsEnd interface and a Current interface, the First interface locating first data, the Next interface jumping to next data, the IsEnd interface judging data termination, and the Current interface outputting a current data row; a data processing layer, which contains a parameter analysis iterator, obtains original data from the data source abstraction layer and converts the original data into a structured data row, the structured data row comprising a data pointer, a total byte number, an element number, an element size and a data type enumeration value; a process execution layer, which encapsulates heterogeneous execution units by using a unified WorkerBase base class, the base class providing an initialization interface, an input / output port acquisition interface, a message notification interface, a work mode setting interface and a type acquisition interface; a process builder, which is used for configuring connection relationships between Workers, allocating tasks to a thread pool, using a work stealing mechanism for load balancing, and supporting merging of continuous tasks.

2. The analysis processing system of claim 1, wherein, The data source abstraction layer comprises: a file iterator, which positions a file handle to a file head through the First interface, moves the handle according to a data block length through the Next interface, and detects a file tail identifier through the IsEnd interface; a database iterator, which implements the First interface, the Next interface and the IsEnd interface through a standard database cursor; the Current interface outputs an intermediate structure body comprising an original data pointer.

3. The analysis processing system of claim 1, wherein, The message notification interface of the WorkerBase base class is configured as follows: when an execution unit completes data processing, the execution unit sends a finish_msg message to an upper node, triggering the upper node to push new data to a message queue.

4. The analysis processing system of claim 1, wherein, The process builder comprises: a connection configuration module, which is used for defining data flow directions between Workers by creating edge relationship interfaces; a template loading module, which is used for loading corresponding templates according to Worker types, the corresponding templates comprising a Python script template, a dynamic library template and an executable program template; a dynamic scheduling module, which is used for allocating thread resources according to serial / parallel identifiers of the work mode setting interface.

5. The analysis processing system of claim 1, wherein, The work stealing mechanism is as follows: each thread maintains a local task queue, and when it is detected that the local queue is empty, the thread randomly selects a queue of a remaining thread to steal a task to be processed.

6. The analysis processing system of claim 1, wherein, The analysis processing system further comprises a report generation module, which is used for receiving an interpretation result output by the process execution layer, and automatically generating a traceable report comprising abnormal data markers and criterion execution paths.

7. The analysis processing system of claim 1, wherein, The data processing layer further comprises a data playback iterator, which is used for receiving an abnormal data position marked by the process execution layer, directionally extracting original data of an abnormal period from the data source abstraction layer, and generating a reproducible data stream.

8. The analysis processing system of claim 1, wherein The data type enumeration value of the structured data row comprises an integer type, a single-precision floating point type, a double-precision floating point type and a string type.

Images