Data processing method and device, electronic equipment and storage medium

By identifying and connecting the time series relationships of data processing nodes in a multi-level processing system, the problems of time series consistency and coordination are solved, and the efficiency of software development is improved.

CN119088358APending Publication Date: 2024-12-06GUANGZHOU AUTOMOBILE GROUP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411096190.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In multi-level processing systems, the consistency and coordination of time series are difficult to coordinate, resulting in the complexity of data multi-source and processing links, increasing the difficulty of time series alignment and analysis, and thus reducing the efficiency of software development and iteration.

Method used

By identifying multiple target output data corresponding to each sequence number from the output data set of multiple data processing nodes, determining their time series relationships, and performing chain connections, storing them into a preset chain table form structure, and obtaining the sending and Receive timestamps to determine the time series relationship and visualize the vertical and horizontal data association relationships.

Benefits of technology

It effectively solves the problem of demand matching of data event sequence acquisition and analysis, accelerates the iterative change cycle of software development, and improves the efficiency of software iterative development.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088358A_ABST
    Figure CN119088358A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of intelligent driving, in particular to a data processing method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting an output data set of a plurality of data processing nodes, the output data set comprising output data corresponding to a plurality of serial numbers, and each serial number corresponding to a plurality of output data; identifying a plurality of target output data corresponding to each serial number from the output data set, and determining a time sequence relationship of the plurality of target output data corresponding to each serial number; and based on the time sequence relationship of the multiple pieces of target output data corresponding to each serial number, performing chain type series connection on the time sequence relationship of the multiple pieces of target output data corresponding to each serial number, and storing a chain type series connection result to a preset chain type table structure. Therefore, the requirement matching problem of data event sequence collection and analysis is solved, the iteration change period of software development is effectively shortened, and the software iteration development efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] With the development of intelligent driving technology, adaptation to more complex scenarios has become an important research goal, and complementary identification and processing of diverse data sources have become one of the effective means to deal with complex scenarios. In a multi-level processing system, ensuring the consistency and coordination of time series has become the key. In order to effectively respond to the challenges of time consistency and coordination in complex scenarios, data collection and analysis based on time series have become the key to solving complex problems. However, due to the complexity of multiple data sources and processing links, there are many difficulties in time series alignment and analysis that are difficult to coordinate.

[0003] In related technologies, in order to effectively analyze the coordination and consistency of multi-source data and multi-level processing links in the intelligent driving process, the method usually adopted is to add time series to the data transmitted between each level of processing nodes.

[0004] However, the methods in related technologies have the problem of unpredictable demand. New time series statistics and collection needs require the addition of new time series collection points and data structures, which greatly reduces the efficiency of function development and software iteration and needs to be solved urgently. Summary of the invention

[0005] The present application provides a data processing method, device, electronic device and storage medium to solve the problem of demand matching for data event sequence collection and analysis, effectively speed up the iterative change cycle of software development, and improve the efficiency of software iterative development.

[0006] The first embodiment of the present application provides a data processing method, comprising the following steps:

[0007] Collecting output data sets of multiple data processing nodes, wherein the output data sets include output data corresponding to multiple sequence numbers, and each sequence number corresponds to multiple output data;

[0008] Identifying a plurality of target output data corresponding to each of the sequence numbers from the output data set, and determining a time series relationship between the plurality of target output data corresponding to each of the sequence numbers;

[0009] Based on the time series relationship of the multiple target output data corresponding to each serial number, the time series relationship of the multiple target output data corresponding to each serial number is chained in series, and the chained series result is stored in a preset chained table structure.

[0010] According to an embodiment of the present application, determining the time series relationship of the plurality of target output data corresponding to each sequence number includes:

[0011] Obtaining a sending timestamp and a receiving timestamp of each target output data among the multiple target output data corresponding to each sequence number;

[0012] Based on the sending timestamp and the receiving timestamp of each target output data, the time series relationship of the multiple target output data corresponding to each sequence number is determined in chronological order.

[0013] According to an embodiment of the present application, after storing the chain concatenation result in the preset chain table structure, the method further includes:

[0014] Get data key field information;

[0015] The target sequence number is determined according to the data key field information, and the vertical data association relationship and / or the horizontal data association relationship of multiple output data corresponding to the target sequence number are visually displayed.

[0016] According to an embodiment of the present application, the visual display of the vertical data association relationship and / or the horizontal data association relationship of the multiple output data corresponding to the target sequence number includes:

[0017] The upstream and downstream complete data chains of the multiple output data corresponding to the target sequence number are chain-indexed, and based on the upstream and downstream complete data chains, the multiple output data corresponding to the target sequence number are displayed according to the time occurrence order; the upstream and downstream complete data chains are the association relationship and time relationship between the reception and transmission between each module through which the data flows from entering the system to leaving the system;

[0018] And / or, by chain-indexing the time information of the key data of the aggregation node corresponding to the multiple output data, and horizontally expanding the non-key data of the aggregation node according to the time information, the output data information closest to the time series is selected and indexed upstream, and the horizontal data association relationship of the multiple output data corresponding to the target sequence number is displayed.

[0019] According to an embodiment of the present application, the sending timestamp, receiving timestamp and sequence number of each output data are stored in the data structure of the corresponding output data, and the data structure of all output data is integrated in the Header load of the preset data transmission middleware.

[0020] According to the data processing method of the embodiment of the present application, multiple target output data corresponding to each sequence number are identified from the output data sets of multiple data processing nodes, and the time series relationship of the multiple target output data corresponding to each sequence number is determined, and the time series relationship of the multiple target output data corresponding to each sequence number is chained and connected, and the chain connection result is stored in a preset chain table structure. Thus, the demand matching problem of data event sequence collection and analysis is solved, the iterative change cycle of software development is effectively accelerated, and the efficiency of software iterative development is improved.

[0021] A second aspect of the present application provides a data processing device, including:

[0022] A collection module, used for collecting output data sets of multiple data processing nodes, wherein the output data sets include output data corresponding to multiple sequence numbers, and each sequence number corresponds to multiple output data;

[0023] A time series relationship determination module, used to identify the multiple target output data corresponding to each sequence number from the output data set, and determine the time series relationship of the multiple target output data corresponding to each sequence number;

[0024] A processing module is used to chain the time series relationship of the multiple target output data corresponding to each serial number based on the time series relationship of the multiple target output data corresponding to each serial number, and store the chain series result in a preset chain table structure.

[0025] According to one embodiment of the present application, the time series relationship determination module is used to:

[0026] Obtaining a sending timestamp and a receiving timestamp of each target output data among the multiple target output data corresponding to each sequence number;

[0027] Based on the sending timestamp and the receiving timestamp of each target output data, the time series relationship of the multiple target output data corresponding to each sequence number is determined in chronological order.

[0028] According to an embodiment of the present application, after storing the chain concatenation result in the preset chain table structure, the processing module is further used to:

[0029] Get data key field information;

[0030] The target sequence number is determined according to the data key field information, and the vertical data association relationship and / or the horizontal data association relationship of multiple output data corresponding to the target sequence number are visually displayed.

[0031] According to one embodiment of the present application, the processing module is used to:

[0032] The upstream and downstream complete data chains of the multiple output data corresponding to the target sequence number are chain-indexed, and based on the upstream and downstream complete data chains, the multiple output data corresponding to the target sequence number are displayed according to the time occurrence order; the upstream and downstream complete data chains are the association relationship and time relationship between the reception and transmission between each module through which the data flows from entering the system to leaving the system;

[0033] And / or, by chain-indexing the time information of the key data of the aggregation node corresponding to the multiple output data, and horizontally expanding the non-key data of the aggregation node according to the time information, the output data information closest to the time series is selected and indexed upstream, and the horizontal data association relationship of the multiple output data corresponding to the target sequence number is displayed.

[0034] According to an embodiment of the present application, the sending timestamp, receiving timestamp and sequence number of each output data are stored in the data structure of the corresponding output data, and the data structure of all output data is integrated in the Header load of the preset data transmission middleware.

[0035] According to the data processing device of the embodiment of the present application, multiple target output data corresponding to each sequence number are identified from the output data sets of multiple data processing nodes, and the time series relationship of the multiple target output data corresponding to each sequence number is determined, and the time series relationship of the multiple target output data corresponding to each sequence number is chained and connected, and the chained connection result is stored in a preset chained table structure. Thus, the demand matching problem of data event sequence collection and analysis is solved, the iterative change cycle of software development is effectively accelerated, and the efficiency of software iterative development is improved.

[0036] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data processing method as described in the above embodiment.

[0037] The fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to implement the data processing method as described in the above embodiment.

[0038] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0040] Figure 1 A flowchart of a data processing method provided according to an embodiment of the present application;

[0041] Figure 2 is a schematic diagram of a time series acquisition method according to an embodiment of the present application;

[0042] Figure 3 A schematic diagram of a data structure definition of a time series acquisition system according to an embodiment of the present application;

[0043] Figure 4 A schematic diagram of a time series connection of processing nodes according to an embodiment of the present application;

[0044] Figure 5 A schematic diagram of visualizing the association of time series of acquisition nodes according to an embodiment of the present application;

[0045] Figure 6 is a block diagram of a data processing device according to an embodiment of the present application;

[0046] Figure 7 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limitations of the present invention. In order to provide a deep understanding of the present application, many specific details will be included in the following description. The present application may also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present application, some specific details will be omitted in the description. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0048] In the related technology, the hash serialization method is used to serialize and synchronize the collected data of multiple sensors, which can achieve the time series alignment of multi-sensor data. However, this method does not explain the data time series concatenation method in the subsequent processing unit, and cannot solve the single-frame and multi-frame synchronization of data time series between multiple processing nodes, and cannot completely solve the time series alignment problem of full-process data collection.

[0049] In response to the above-mentioned problems and the demand matching pain points of data event sequence collection and analysis mentioned in the background technology, the present application provides a data processing method, in which multiple target output data corresponding to each sequence number are identified from the output data sets of multiple data processing nodes, and the time series relationship of the multiple target output data corresponding to each sequence number is determined, and the time series relationship of the multiple target output data corresponding to each sequence number is chained, and the chained series result is stored in a preset chained table structure. In this way, the demand matching problem of data event sequence collection and analysis is solved, the iterative change cycle of software development is effectively accelerated, and the efficiency of software iterative development is improved.

[0050] Specifically, Figure 1 A flowchart of a data processing method provided in an embodiment of the present application.

[0051] In this embodiment, if Figure 2 As shown, the data processing method includes the data processing node marking and concatenating the data time, and the data acquisition node performing the horizontal and vertical association and visualization of the data.

[0052] like Figure 1 As shown, the data processing method includes the following steps:

[0053] In step S101, output data sets of multiple data processing nodes are collected, wherein the output data sets include output data corresponding to multiple serial numbers, and each serial number corresponds to multiple output data; wherein, the data processing node of the embodiment of the present application is composed of multiple nodes, including data perception, positioning, fusion, prediction, decision-making, planning, control, and data abstraction processing. For example, some data processing nodes are responsible for acquiring data from sensors, some data processing nodes are responsible for data abstraction, some data processing nodes are responsible for aggregating multiple data, some data processing nodes are responsible for data conversion processing, and some data processing nodes are responsible for data output.

[0054] The output data set includes the data of each functional node.

[0055] Specifically, the embodiments of the present application can utilize data acquisition nodes (such as the final data acquisition node on the vehicle side) to subscribe to data (i.e., output data sets) published by various data processing nodes through data communication middleware, such as original sensor data sent by data source nodes, sensor abstract data sent by data abstraction nodes, multi-sensor fusion data sent by data aggregation nodes, and processed data sent by other data processing nodes.

[0056] In step S102, a plurality of target output data corresponding to each sequence number is identified from the output data set, and a time series relationship of the plurality of target output data corresponding to each sequence number is determined.

[0057] Further, in some embodiments, determining the time series relationship of multiple target output data corresponding to each serial number includes: obtaining a sending timestamp and a receiving timestamp of each target output data among the multiple target output data corresponding to each serial number; and determining the time series relationship of the multiple target output data corresponding to each serial number in chronological order based on the sending timestamp and the receiving timestamp of each target output data.

[0058] The sending timestamp refers to the timestamp of the sending time assigned to the data when the data is sent from the processing node, and the receiving timestamp refers to the timestamp of the receiving time assigned to the data when the data is received by the processing node.

[0059] In some embodiments, the sending timestamp, receiving timestamp and sequence number of each output data are stored in the data structure corresponding to the output data, such as Figure 3 As shown, the data structure is used in the data transmission process of all nodes in this system. The data structure has a Stamp header information, which includes a sending timestamp identifier, a receiving timestamp identifier and a Seq Id (sequence number) identifier, and the data structure of all output data is integrated in the Header load of the preset data transmission middleware, and does not exist independently in the data structure (Data Payload). The data transmission middleware can obtain the above identification information by providing a valid API (Application Programming Interface) to the data processing node.

[0060] Therefore, the sending timestamp and receiving timestamp of the data are automatically completed by the data transmission middleware when the data is sent and received, carried in the data structure header of the data transmitted across nodes, and an acquisition interface is provided for the data processing node, thereby realizing the unconscious processing of timestamps by the data processing node and eliminating the need for the processing node to pay attention to the assigned timestamps.

[0061] Specifically, Figure 4 As shown, the previous data processing node assigns a sending timestamp to the data, the current data processing node assigns a receiving timestamp to the data, and the current data processing node performs single-frame or multi-frame association.

[0062] Among them, the previous data processing node assigns a sending timestamp to the data, which means that after the data processing node completes the data processing, when sending the data sequence to the next processing node, the instantaneous time information at this moment is added to the data header. Preferably, this operation can be completed by the data transmission middleware. When the data processing node calls the data transmission middleware to send the data, the data transmission middleware automatically records the instantaneous time of sending the frame data in the middleware load Header. Therefore, the actual transmission time of the data is recorded more accurately in this way, and the accuracy of the time series is improved.

[0063] Furthermore, this data processing node assigns a receiving timestamp to the data, which means that when the next data processing node (here referred to as this node) that is associated with the previous data processing node receives the data sent by the previous data processing node, the instantaneous time information at this moment is added to the received data header. Preferably, this operation can be completed by the data transmission middleware. When the data transmission middleware triggers the data reception callback API of this node, the instantaneous time of receiving the frame data is automatically recorded in the middleware load Header. Therefore, the actual time of data transmission is recorded more accurately in this way, and the accuracy of the time series is improved.

[0064] Specifically, after receiving various types of output data from the data processing node, the data acquisition node of the embodiment of the present application can obtain the sending timestamp and receiving timestamp information of each output data from the data structure, and can establish a one-to-one mapping relationship between the corresponding data structure and the time series through Map. More preferably, in a system that uses middleware to record timestamps, the data acquisition node uses the timestamp acquisition API interface provided by the middleware to obtain more accurate timestamp information and establish a one-to-one mapping relationship between the corresponding data structure and the time series.

[0065] In addition, after receiving various output data from the data processing node, the data collection node also obtains the serial number information of each output data from the data structure. Preferably, in a system that uses middleware to record serial numbers, the data collection node uses the serial number query interface provided by the middleware to obtain the serial number information of the corresponding data.

[0066] Furthermore, multiple target output data corresponding to each sequence number are identified from the output data set of each data processing node. For each target output data, its time sequence on the corresponding node is determined according to its sending timestamp and receiving timestamp, and the time series relationship of multiple target output data corresponding to each sequence number on each data processing node is determined in chronological order. If a target output data is recorded on multiple nodes (i.e., it has been processed by multiple nodes), its complete time series relationship on these nodes is determined according to the timestamp.

[0067] In step S103, based on the time series relationship of the multiple target output data corresponding to each sequence number, the time series relationship of the multiple target output data corresponding to each sequence number is chained and concatenated, and the chained concatenation result is stored in a preset chained table structure.

[0068] Specifically, the data acquisition node of the embodiment of the present application establishes a preset linked table form structure with the serial number of the output data as the key, and chains the frames with the same serial number in the data structure collected from each data processing node in the order of occurrence according to the time series relationship indicated by the timestamp, and records them in the preset linked table form structure.

[0069] Furthermore, in some embodiments, after storing the chain concatenation results in a preset chain table form structure, it also includes: obtaining data key field information; determining the target sequence number based on the data key field information, and visually displaying the vertical data association relationship and / or horizontal data association relationship of multiple output data corresponding to the target sequence number.

[0070] Optionally, the embodiment of the present application can use data visualization tools in the prior art (such as Matplotlib, Seaborn, Tableau, etc.) to select appropriate visualization charts according to the type of data association relationship to visualize the data association relationship of the target output data.

[0071] Specifically, the embodiments of the present application can obtain data key field information according to actual needs, and further determine the target sequence number according to the data key field information, and visualize the vertical data association relationship of multiple output data corresponding to the target sequence number, or visualize the horizontal data association relationship of the target output data, or visualize both the vertical data association relationship and the horizontal data association relationship of the target output data.

[0072] Furthermore, in some embodiments, the vertical data association relationship and / or the horizontal data association relationship of multiple output data corresponding to the target sequence number are visualized, including: chain-indexing the upstream and downstream complete data chains of the multiple output data corresponding to the target sequence number, and based on the upstream and downstream complete data chains, displaying the multiple output data corresponding to the target sequence number according to the time order of occurrence; the upstream and downstream complete data chain is the association relationship and time relationship between the reception and transmission of each module through which the data flows from entering the system to leaving the system; and / or, chain-indexing the time information of the key data of the aggregation node corresponding to the multiple output data, and horizontally expanding the non-key data of the aggregation node according to the time information, selecting the output data information with the closest time series and indexing it upstream, and displaying the horizontal data association relationship of the multiple output data corresponding to the target sequence number.

[0073] Specifically, the data acquisition node of the embodiment of the present application uses the key field information of the data that the developer is concerned about as the trigger source, and uses chain indexing to index the complete upstream and downstream data chain of the frame data (that is, the multiple output data corresponding to the target sequence number), and arranges and expands the time sequence of the multiple output data to form a vertical data association relationship between the multiple output data corresponding to the target sequence number, and visualizes the vertical data association relationship between the multiple output data corresponding to the target sequence number.

[0074] In addition, the data collection node can also use the key field information of the data that the developer is concerned about as the trigger source, and use chain indexing to obtain the time information of the key data of the aggregation node corresponding to the frame data (that is, the multiple output data corresponding to the target sequence number), and expand it horizontally in other non-key data according to the time sending relationship, select the frame information closest to the time series and index it upstream, so as to obtain the horizontal data association relationship of the multiple output data corresponding to the target sequence number, and visualize the horizontal data association relationship of the multiple output data corresponding to the target sequence number.

[0075] It should be noted that the embodiments of the present application can also chain-index the complete upstream and downstream data chains of multiple output data corresponding to the target sequence number, and based on the complete upstream and downstream data chains, arrange and expand the time occurrence order of the multiple output data corresponding to the target sequence number, and visualize the vertical data association relationship of the multiple output data corresponding to the target sequence number. At the same time, chain-index the time information of the key data of the aggregation node corresponding to the multiple output data, and horizontally expand the non-key data of the aggregation node according to the time information, select the output data information with the closest time series and index it upstream, and visualize the horizontal data association relationship of the multiple output data corresponding to the target sequence number.

[0076] Therefore, the data collection node serializes the data stream in a chain according to the serial number, receiving timestamp, and sending timestamp. After the serial number is connected in series, a topological relationship between the data receiving and sending time of the upstream and downstream nodes in the entire chain is formed from the time when the data enters the domain controller to the time when the data is sent from the domain controller. Then, the transmission delay, calculation delay, multi-frame delay, etc. of the entire link can be calculated.

[0077] In order to facilitate those skilled in the art to more clearly understand the data collection and visualization of the data collection node of this application, the following is combined with Figure 5 Provide detailed explanation.

[0078] like Figure 5 As shown, the data collection and visualization of the data collection node includes: obtaining the data sending and receiving timestamp, obtaining the data Seq Id, chaining the data in series according to the Seq Id, and realizing the horizontal and vertical visualization of the data.

[0079] Furthermore, in order to facilitate those skilled in the art to more clearly and intuitively understand the data processing node of the embodiment of the present application, the working content of the data processing node of the embodiment of the present application is described in detail below.

[0080] Specifically, the data processing nodes can concatenate the Seq Ids of the receiving side data and the sending side data of a single frame or multiple frames of data according to the needs of the respective nodes for data processing, that is, concatenate the received data and the sent data in time series. The concatenation of time series refers to reading the Seq Id from the receiving side data structure and adding it to the Seq Id of the sending side data structure. More preferably, in the method implemented by the data transmission middleware, the Seq Id is located in the Header information of the data transmission middleware, and the data transmission middleware provides a Seq Id reading API and a Seq Id assignment API. The data processing module acquires the Seq Id of the upstream data and assigns the Seq Id of the downstream data by calling the API.

[0081] Further exemplary, the data processing node concatenates the upstream and downstream data, which may include: ① The original data acquisition node acquires data from the sensor and assigns an original sending timestamp. ② The data abstraction node associates the Seq Id of the abstracted data with the Seq Id of the original sensor data. ③ The data aggregation processing node associates the Seq Id of the aggregated data with the Seq Id of the key data, where the key data is the data that plays a decisive role in multiple aggregated source data.

[0082] Preferably, the data processing node can also establish an association relationship between multiple frames of data according to processing needs, that is, the processing node associates the Seq Id of the sent data with the Seq Id of the received data multiple frames ago. This operation is used to identify the time series association relationship between the initial change time of the data input source and the initial change time of the output data when the data processing node performs multi-frame data smoothing or threshold control such as Kalman filtering.

[0083] In addition, when the data processing node performs Seq Id association, it can perform data association across multiple frames based on the multi-frame relationship of the data, thereby identifying the multi-frame relationship of the data, realizing the time series binding of the multi-frame association relationship, and providing a time series basis for the data collection node to restore the multi-frame relationship of the data.

[0084] Therefore, this application makes the single-frame and multi-frame relationships of data flowing in each processing node explicit, improves the efficiency of problem analysis, provides an integrated and convenient data association method for cloud data analysis, improves the efficiency of cloud visualization development, and provides each processing node with a convenient data association method, improves the iteration efficiency of the node development process, and at the same time improves the consistency of each processing node in extracting and assigning timestamps for sending and receiving data.

[0085] According to the data processing method of the embodiment of the present application, multiple target output data corresponding to each sequence number are identified from the output data sets of multiple data processing nodes, and the time series relationship of the multiple target output data corresponding to each sequence number is determined, and the time series relationship of the multiple target output data corresponding to each sequence number is chained and connected, and the chain connection result is stored in a preset chain table structure. Thus, the demand matching problem of data event sequence collection and analysis is solved, the iterative change cycle of software development is effectively accelerated, and the efficiency of software iterative development is improved.

[0086] Next, the data processing device proposed according to the embodiment of the present application is described with reference to the accompanying drawings.

[0087] Figure 6 It is a block diagram of a data processing device according to an embodiment of the present application.

[0088] like Figure 6 As shown, the data processing device 10 includes: a collection module 100 , a time series relationship determination module 200 and a processing module 300 .

[0089] Among them, the collection module 100 is used to collect the output data sets of multiple data processing nodes, wherein the output data sets include output data corresponding to multiple serial numbers, and each serial number corresponds to multiple output data; the time series relationship determination module 200 is used to identify multiple target output data corresponding to each serial number from the output data set, and determine the time series relationship of the multiple target output data corresponding to each serial number; the processing module 300 is used to chain the time series relationship of the multiple target output data corresponding to each serial number based on the time series relationship of the multiple target output data corresponding to each serial number, and store the chain series result in a preset chain table structure.

[0090] Furthermore, in some embodiments, the time series relationship determination module 200 is used to: obtain a sending timestamp and a receiving timestamp of each target output data among multiple target output data corresponding to each serial number; and determine the time series relationship of the multiple target output data corresponding to each serial number in chronological order based on the sending timestamp and the receiving timestamp of each target output data.

[0091] Furthermore, in some embodiments, after storing the chain concatenation results in a preset chain table structure, the processing module 300 is also used to: obtain data key field information; determine the target sequence number based on the data key field information, and visualize the vertical data association relationship and / or horizontal data association relationship of multiple output data corresponding to the target sequence number.

[0092] Further, in some embodiments, the processing module 300 is used to: chain-index the complete upstream and downstream data chains of multiple output data corresponding to the target sequence number, and based on the complete upstream and downstream data chains, display the multiple output data corresponding to the target sequence number in the order of time occurrence; the complete upstream and downstream data chains are the association relationship and time relationship between the reception and transmission of each module through which the data flows from entering the system to leaving the system; and / or, chain-index the time information of the key data of the aggregation node corresponding to the multiple output data, and horizontally expand the non-key data of the aggregation node according to the time information, select the output data information with the closest time series and index it upstream, and display the horizontal data association relationship of the multiple output data corresponding to the target sequence number.

[0093] Furthermore, in some embodiments, the sending timestamp, receiving timestamp and sequence number of each output data are stored in the data structure corresponding to the output data, and the data structure of all output data is integrated in the Header load of the preset data transmission middleware.

[0094] It should be noted that the above explanation of the data processing method embodiment is also applicable to the data processing device of this embodiment, and will not be repeated here.

[0095] According to the data processing device of the embodiment of the present application, multiple target output data corresponding to each sequence number are identified from the output data sets of multiple data processing nodes, and the time series relationship of the multiple target output data corresponding to each sequence number is determined, and the time series relationship of the multiple target output data corresponding to each sequence number is chained and connected, and the chained connection result is stored in a preset chained table structure. Thus, the demand matching problem of data event sequence collection and analysis is solved, the iterative change cycle of software development is effectively accelerated, and the efficiency of software iterative development is improved.

[0096] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0097] A memory 701 , a processor 702 , and a computer program stored in the memory 701 and executable on the processor 702 .

[0098] When the processor 702 executes the program, the data processing method provided in the above embodiment is implemented.

[0099] Furthermore, the electronic device further comprises:

[0100] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0101] The memory 701 is used to store computer programs that can be executed on the processor 702 .

[0102] The memory 701 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0103] If the memory 701, the processor 702 and the communication interface 703 are implemented independently, the communication interface 703, the memory 701 and the processor 702 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0104] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0105] The processor 702 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0106] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above data processing method is implemented.

[0107] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0108] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0109] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A data processing method, characterized in that: The following steps are involved: Collecting output data sets of multiple data processing nodes, wherein the output data sets include output data corresponding to multiple sequence numbers, and each sequence number corresponds to multiple output data; Identifying a plurality of target output data corresponding to each of the sequence numbers from the output data set, and determining a time series relationship between the plurality of target output data corresponding to each of the sequence numbers; Based on the time series relationship of the multiple target output data corresponding to each serial number, the time series relationship of the multiple target output data corresponding to each serial number is chained in series, and the chained series result is stored in a preset chained table structure.

2. The method according to claim 1, characterized in that Determining the time series relationship of the plurality of target output data corresponding to each sequence number includes: Obtaining a sending timestamp and a receiving timestamp of each target output data among the multiple target output data corresponding to each sequence number; Based on the sending timestamp and the receiving timestamp of each target output data, the time series relationship of the multiple target output data corresponding to each sequence number is determined in chronological order.

3. The method according to claim 1, characterized in that After storing the chain concatenation result in the preset chain table structure, the method further includes: Get data key field information; The target sequence number is determined according to the data key field information, and the vertical data association relationship and / or the horizontal data association relationship of multiple output data corresponding to the target sequence number are visualized.

4. The method according to claim 3, characterized in that The visual display of the vertical data association relationship and / or the horizontal data association relationship of the plurality of output data corresponding to the target sequence number includes: The upstream and downstream complete data chains of the multiple output data corresponding to the target sequence number are chain-indexed, and based on the upstream and downstream complete data chains, the multiple output data corresponding to the target sequence number are displayed according to the time occurrence order; the upstream and downstream complete data chains are the association relationship and time relationship between the reception and transmission between each module through which the data flows from entering the system to leaving the system; And / or, by chain-indexing the time information of the key data of the aggregation node corresponding to the multiple output data, and horizontally expanding the non-key data of the aggregation node according to the time information, the output data information closest to the time series is selected and indexed upstream, and the horizontal data association relationship of the multiple output data corresponding to the target sequence number is displayed.

5. The method according to any one of claims 1 to 4, characterized in that The sending timestamp, receiving timestamp and sequence number of each output data are stored in the data structure of the corresponding output data, and the data structure of all output data is integrated in the Header load of the preset data transmission middleware.

6. A data processing device, characterized in that: include: A collection module, used for collecting output data sets of multiple data processing nodes, wherein the output data sets include output data corresponding to multiple sequence numbers, and each sequence number corresponds to multiple output data; A time series relationship determination module, used to identify the multiple target output data corresponding to each sequence number from the output data set, and determine the time series relationship of the multiple target output data corresponding to each sequence number; A processing module is used to chain the time series relationship of the multiple target output data corresponding to each serial number based on the time series relationship of the multiple target output data corresponding to each serial number, and store the chain series result in a preset chain table structure.

7. The device according to claim 6, characterized in that The time series relationship determination module is used to: Obtaining a sending timestamp and a receiving timestamp of each target output data among the multiple target output data corresponding to each sequence number; Based on the sending timestamp and the receiving timestamp of each target output data, the time series relationship of the multiple target output data corresponding to each sequence number is determined in chronological order.

8. The device according to claim 6, characterized in that After storing the chain concatenation result in the preset chain table structure, the processing module is further used to: Get data key field information; The target sequence number is determined according to the data key field information, and the vertical data association relationship and / or the horizontal data association relationship of multiple output data corresponding to the target sequence number are visually displayed.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data processing method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the data processing method according to any one of claims 1 to 5.