Transaction data flow acquisition method and device

Abstract

The present invention provides a transaction data stream acquisition method and device, which relates to the field of big data processing and can be applied to the financial field and other fields. The method comprises: receiving transaction information, and obtaining transaction process data according to the transaction information analysis; collecting monitoring data from a background database, extracting change data information of a corresponding time period in the monitoring data according to the start and end time in the transaction information by means of primary key extraction, and sorting the change data information in the order of occurrence time to generate a data stream set; and matching and associating the transaction process data with the data stream set in order according to the business elements in the transaction information to generate a transaction data stream.

Description

Technical Field

[0001] The present invention relates to the field of big data processing and can be applied to the financial field and other fields, and in particular to a method and device for obtaining transaction data streams. Background Art

[0002] The front-end transactions of a computer software system often correspond to the complex data flows and associations in the back-end database. In complex distributed systems, especially in online trading systems with complex architectures, it has always been difficult to sort out the correspondence between transaction processes and back-end data flows. However, for online trading systems with increasingly complex distributed architectures, the sorting out of such correspondences is very meaningful and necessary for the sorting out of the overall links, troubleshooting, and asset accumulation.

[0003] At present, when combing such links and extracting corresponding relationships, the commonly used methods are mainly: 1. (Pure manual extraction) Combination of manual business logic combing and manual data flow combing. This method requires experts in the field who have a deep understanding of business transactions and data architecture, and also requires a lot of manual intervention. 2. (Semi-automatic extraction) Manual business logic combing and automatic data architecture matching. Compared with the previous method, this method is a semi-automatic method, but it requires the foreknowledge of the data structures such as the database tables, stored procedures, indexes, etc. behind the transaction logic in advance. It also requires a lot of knowledge accumulation and precipitation of developers, and there is a possibility of omissions. Summary of the invention

[0004] The purpose of the present invention is to provide a transaction data stream acquisition method and device, which uses the monitoring data automatically generated by the background database, and forms a set of data streams that do not require human intervention through the combined use of time link collection, primary key transfer tracking, noise processing and other technologies. The matched data streams can be gradually improved and dynamically adjusted through the background learning mechanism, which is much more efficient than traditional manual and semi-automatic implementation methods, and avoids data omissions caused by human intervention.

[0005] To achieve the above-mentioned purpose, the transaction data stream acquisition method provided by the present invention specifically includes: receiving transaction information, and obtaining transaction process data based on transaction information analysis; collecting monitoring data from the background database, extracting the change data information of the corresponding time period in the monitoring data according to the start and end time in the transaction information by primary key extraction, and sorting the change data information in the order of occurrence time to generate a data stream set; according to the business elements in the transaction information, the transaction process data and the data stream set are matched and associated in order to generate a transaction data stream.

[0006] In the above-mentioned transaction data flow acquisition method, preferably, before the transaction process data is matched and associated with the data flow set in sequence according to the business elements in the transaction information, it also includes: when there is a transaction link data flow in the data flow set and the primary key changes during the transmission process, the primary key jump position is obtained through the primary key transmission tracking technology; according to the primary key jump position, the changed primary key is associated with the primary key of the primary key jump position and then stored in the data flow set.

[0007] In the above transaction data flow acquisition method, preferably, obtaining transaction process data based on transaction information analysis includes: analyzing the transaction information to obtain transaction elements, and classifying the transaction elements to generate transaction process data.

[0008] In the above transaction data stream acquisition method, preferably, the monitoring data includes query text, execution time and corresponding table name.

[0009] In the above-mentioned transaction data stream acquisition method, preferably, extracting the change data information of the corresponding time period in the monitoring data by extracting the primary key according to the start and end time in the transaction information includes: obtaining the primary key information in the data table structure in the monitoring data according to the start and end time in the transaction information; and extracting the change data information of the corresponding time period in the monitoring data through the primary key information.

[0010] In the above-mentioned transaction data stream acquisition method, preferably, generating a data stream set by sorting the change data information in the order of occurrence includes: when the change data information comes from databases with different time settings, calculating the standard time of the change data information by a time conversion formula; and sorting the change data information in the order of occurrence according to the standard time to generate a data stream set.

[0011] In the above-mentioned transaction data stream acquisition method, preferably, before sequentially matching and associating the transaction process data with the data stream set according to the business elements in the transaction information, it also includes: extracting relevant information associated with the transaction through business element collection at different preset periods, and performing noise reduction processing on the data stream set through the relevant information.

[0012] The present invention also provides a transaction data stream acquisition device, which includes a transaction collection module, a technical data stream processing module and a summary module; the transaction collection module is used to receive transaction information and obtain transaction process data based on transaction information analysis; the technical data stream processing module is used to collect monitoring data from a background database, extract change data information of a corresponding time period in the monitoring data by primary key extraction based on the start and end time in the transaction information, and sort the change data information in the order of occurrence to generate a data stream set; the summary module is used to match and associate the transaction process data with the data stream set in order based on the business elements in the transaction information to generate a transaction data stream.

[0013] In the above-mentioned transaction data stream acquisition device, preferably, the technical data stream processing module also includes a primary key transfer tracking module and a noise processing module; the primary key transfer tracking module is used to obtain the primary key jump position through the primary key transfer tracking technology when the primary key of the transaction link data stream in the data stream set changes during the transmission process; according to the primary key jump position, the changed primary key is associated with the primary key of the primary key jump position and then stored in the data stream set; the noise processing module is used to extract relevant information associated with the transaction through business elements of different preset periods, and perform noise reduction processing on the data stream set through the relevant information.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.

[0015] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing the above method.

[0016] The beneficial technical effects of the present invention are: greatly reducing the time cost and experience threshold requirements of manual combing, and only requiring a small amount of manual entry during the initial sample collection period; the automatically generated data flow information provides an effective basis for analysis and troubleshooting for the monitoring and tracking of full-link transactions; and the monitoring data is continuously and automatically refreshed to enrich the content, forming a complete set of data flow information asset precipitation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0018] Figure 1 A schematic diagram of a flow chart of a transaction data stream acquisition method provided by an embodiment of the present invention;

[0019] Figure 2A schematic diagram of a process for obtaining change data information provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a process for generating a set of data streams at different times provided by an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of a process of extracting a time link provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of a process of primary key transfer tracking provided by an embodiment of the present invention;

[0023] Figure 6 A schematic diagram of the principle of primary key transfer tracking provided by an embodiment of the present invention;

[0024] Figure 7 A schematic diagram of the principle of noise processing provided by an embodiment of the present invention;

[0025] Figure 8 A schematic diagram of the structure of a transaction data stream acquisition device provided by an embodiment of the present invention;

[0026] Fig. 9 A schematic diagram of the application structure of a transaction data stream acquisition device provided by an embodiment of the present invention;

[0027] Fig.10 The figure is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will describe the implementation methods of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly. It should be noted that as long as there is no conflict, the various embodiments of the present invention and the various features in the embodiments can be combined with each other, and the technical solutions formed are all within the protection scope of the present invention.

[0029] In addition, the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be performed in an order different from that shown here.

[0030] Please refer to Figure 1 As shown, the transaction data stream acquisition method provided by the present invention specifically includes:

[0031] S101 receives transaction information and obtains transaction process data based on analysis of the transaction information;

[0032] S102 collects monitoring data from a backend database, extracts change data information of a corresponding period in the monitoring data in a primary key extraction manner according to the start and end time in the transaction information, and sorts the change data information in order of occurrence to generate a data stream set;

[0033] S103: Match and associate the transaction process data with the data flow set in order according to the business elements in the transaction information to generate a transaction data flow.

[0034] In the above embodiment, the monitoring data may include query text, i.e., SQL text, execution time, and corresponding table name and other element information. Obtaining transaction process data according to transaction information analysis includes: analyzing the transaction information to obtain transaction elements, and classifying the transaction elements to generate transaction process data. Specifically, the transaction information mainly includes three elements in actual work: transaction name, start time, and end time; the start time and end time can be achieved by dotting or snapshot, which simplifies manual operations as much as possible. Thereafter, the collected transaction information elements are classified and provided to the subsequent step S103 for association.

[0035] Please refer to Figure 2 As shown, in one embodiment of the present invention, extracting the change data information of the corresponding period in the monitoring data in a primary key extraction manner according to the start and end time in the transaction information includes:

[0036] S201 obtains primary key information from the data table structure in the monitoring data according to the start and end time in the transaction information;

[0037] S202 extracts the change data information of the corresponding time period in the monitoring data through the primary key information.

[0038] Specifically, in this embodiment, time link extraction is mainly performed, that is, from the monitoring data, the change data information within the time period determined in step S101 is extracted, and the extraction method is adopted according to the primary key (the primary key can be obtained in the structure of the data table), and the information is arranged in the order of time occurrence (according to the timestamp in the database, in the database table of the current standardized distributed transaction design, the transaction table generally has timestamp information).

[0039] For further information, please refer to Figure 3 As shown, in one embodiment of the present invention, generating a data stream set by sorting the change data information in the order of occurrence may include:

[0040] S301: when the change data information comes from databases with different time settings, calculate the standard time of the change data information by using a time conversion formula;

[0041] S302 sorts the time sequence of the changed data information according to the standard time to generate a data stream set.

[0042] Please refer to Figure 4 As shown in the figure, the processing flow in actual work is mainly to summarize the change information collected from each distributed database and rearrange it in the order of timestamps (assuming that the time of each distributed database adopts a synchronization mechanism and is the same time.) If the time of each database is different (which exists in some databases used for testing), it is necessary to use a conversion formula to correspond to the natural time before rearranging and combining. The information after arrangement and combination is summarized into a temporary information database for further processing.

[0043] Please refer to Figure 5 As shown, in one embodiment of the present invention, before sequentially matching and associating the transaction process data with the data flow set according to the business elements in the transaction information, it also includes:

[0044] S501, when there is a transaction link data flow in the data flow set and the primary key changes during the transmission process, the primary key jump position is obtained by using the primary key transmission tracking technology;

[0045] S502 associates the changed primary key with the primary key at the primary key jump position according to the primary key jump position and stores the resultant associations into the data stream set.

[0046] The above embodiment is mainly applicable to the situation where the primary key changes during the transmission of the entire transaction link data flow, that is, the primary key "jumps", such as jumping from the group number to a specific transaction serial number. In this process, this technology is used to track the primary key jump, avoid data flow tracking interruption, and protect its transmission integrity. For details, please refer to Figure 6As shown, in actual work, the data source of primary key transmission tracking is the set of ordered data in the temporary information library processed by step S102. In these data, the primary key information of each data is extracted one by one for analysis. Assuming that there are n data in the database, each primary key data is compared with the remaining n-1 primary key data through a for loop. If a common primary key is found and there is the same data value on this primary key, it can be regarded as an associated data, that is, a "ring" in the data flow is connected in series, which can be included in the data link. If the associated primary key is not found, the information of the jump primary key is searched. The principle of the jump primary key search is: assuming that there is a primary key A1 in the A data, and a primary key B1 in the B data, A1 is not equal to B1, but by scanning other intermediate tables, it is found that there is a table TAB1, and the primary key association relationship from A1 to B1 can be established. Then the TAB1 table can be used as an intermediate conversion to form the series information between A1 and B1. This situation may be: A1 and B1 are in a one-to-many relationship, or A1 and B1 are in a one-to-one relationship, and there is also a many-to-one relationship between A1 and B1; this can be included in the data link; if a jump primary key is found, it can be identified as suspicious noise data and filtered out.

[0047] In one embodiment of the present invention, before sequentially matching and associating the transaction process data with the data stream set according to the business elements in the transaction information, it also includes: extracting relevant information associated with the transaction through business element collection at different preset periods, and performing noise reduction processing on the data stream set through the relevant information. This embodiment mainly targets the noise part in the acquired data stream set, repeatedly extracts information that is directly related to the transaction by collecting business elements at different time periods, and reduces the impact of noise on the formation of data stream links. For details, please refer to Figure 7 As shown, in actual work, the data link can be processed according to the two dimensions of x-axis and y-axis. The x-axis stores the data link information as different data sets according to the business period, and the y-axis stores the unique value of the data link. According to the arrangement relationship between the horizontal axis and the vertical axis, the data link can be extracted into three categories: A: data flow that exists in the entire business period, B: data flow that appears occasionally, and C: data flow that only appears once. Among these three types of data, type A data directly enters the extraction information library, type B data enters the suspicious information library, and type C data is discarded. It should be noted that the data of the business period is a dynamic and incremental process. As more and more data flows are collected in different business periods, the ABC classification of the data link becomes more and more accurate, until the suspicious information is finally analyzed and extracted one by one and included in the extraction library or the discard library.

[0048] In order to facilitate a clearer understanding of the specific implementation process and method of the transaction data stream acquisition method provided by the present invention, the following takes a transaction with a business name of Trade 1 as an example to integrate the above embodiments. Relevant technical personnel in the field should know that this example does not impose any limitation on it.

[0049] 1. Trade1 is collected during business periods: T1, T2, and T3, and included in the database.

[0050] 2. Extract the change data information of the T1, T2, and T3 periods from the monitoring database. Assume that table A1 of database A has added n data (insert operation), table B1 of database B has modified or updated n data (update operation), and tables C1 and C2 of database C have added m data (insert operation) respectively.

[0051] 3. Extract the timestamp relationship between the data and find that the time sequence is: B1 updated -> A1 added -> C1 added -> C2 added.

[0052] 4. Table B1 has primary key B1pk1, table A1 has primary key A1pk1, table C1 has primary key C1pk1, and table C2 has primary key C2pk1. After business logic comparison, it is found that B1pk1 is equal to A1pk1, and C1pk1 is equal to C2pk1.

[0053] 5. Scan the data structures of A, B, and C databases respectively, and find that in the B2 table of database B, there is an association between B1pk1 and C1pk1, thus satisfying the condition of primary key jump, and the link is generated: B1pk1->A1pk1->C1pk1->C2pk1.

[0054] 6. According to the link in 5, establish data flow links for n data contents one by one, and store n data flow information in the data link information library: dataflow1...dataflow n.

[0055] 7. Establish the corresponding storage relationship between data flow and business period, such as:

[0056] T1: dataflow 1…dataflow x (all have the same data link B1pk1->A1pk1->C1pk1->C2pk1);

[0057] T2: dataflow x+1…dataflow y (all have the same data link B1pk1->A1pk1->C1pk1->C2pk1);

[0058] T3: dataflow y+1…dataflow n (all have the same data link B1pk1->A1pk1->C1pk1->C2pk1);

[0059] 8. Since the data of the same data link exists in the three time periods of T1, T2, and T3, it is considered to be a data link closely related to Trade1 and enters the data link extraction library.

[0060] 9. In the data extracted from the monitoring, another data link was found. It is assumed to be: A2 table of library A, to B2 table of library B, and C3 table of library C. The data link is: A2pk1->B2pk1->C2pk1.

[0061] 10. If the data link only exists in the T1 period and is not found in the T2 and T3 periods, it will enter the suspicious database and wait for more period samples T4...Tn to be stored in the database for further analysis.

[0062] Please refer to Figure 8 As shown, the present invention also provides a transaction data stream acquisition device, which includes a transaction collection module, a technical data stream processing module and a summary module; the transaction collection module is used to receive transaction information and obtain transaction process data based on transaction information analysis; the technical data stream processing module is used to collect monitoring data from a background database, extract change data information of a corresponding time period in the monitoring data according to the start and end time in the transaction information by primary key extraction, and sort the change data information in the order of occurrence to generate a data stream set; the summary module is used to match and associate the transaction process data with the data stream set in order according to the business elements in the transaction information to generate a transaction data stream.

[0063] In the above embodiment, the technical data flow processing module also includes a primary key transfer tracking module and a noise processing module; the primary key transfer tracking module is used to obtain the primary key jump position through the primary key transfer tracking technology when the primary key of the transaction link data flow in the data flow set changes during the transmission process; according to the primary key jump position, the changed primary key is associated with the primary key of the primary key jump position and then stored in the data flow set; the noise processing module is used to extract relevant information associated with the transaction through business elements of different preset periods, and perform noise reduction processing on the data flow set through the relevant information.

[0064] For a clearer explanation, the specific structure of the transaction data stream acquisition device provided by the present invention and the relationship between them are as follows. Fig. 9 As shown, the specific principles of each component of the above-mentioned transaction data flow acquisition device in actual application work are further explained.

[0065] The transaction data flow acquisition device provided by the present invention mainly includes three modules in actual work: a transaction collection module, a technical data flow processing module and a summary module.

[0066] The transaction collection module includes two units: transaction information entry and transaction process collection:

[0067] Transaction information entry, the transaction information entry mainly includes three elements: transaction name, start time, and end time. The start time and end time are realized by dotting or snapshot, which simplifies manual operations as much as possible.

[0068] Transaction process collection, classify the collected transaction information elements, and send them to the information aggregation unit.

[0069] Among them, the technical data flow processing module includes four units: monitoring data acquisition, time link extraction, primary key transfer tracking and noise processing:

[0070] Monitoring data acquisition, the data source is the backend database monitoring data, which includes SQL text, execution time, corresponding table name and other element information.

[0071] Time link extraction, extracts the change data information within the time period obtained in submodule 1 from the monitoring data, adopts the method of extraction by primary key (the primary key can be obtained in the structure of the data table), and arranges the information in chronological order (according to the timestamp in the database, in the database table of the current standardized distributed transaction design, the transaction table generally has timestamp information).

[0072] Primary key transfer tracking: This module is an optional module and is generally applicable to situations where primary keys change during the entire transaction link data flow transmission process, that is, the primary key "jumps", such as jumping from the group number to a specific transaction serial number. In this process, this technology is used to track the primary key jumps to avoid data flow tracking breaks and protect its transmission integrity.

[0073] Noise processing: For the noise part in the acquired data stream set, by collecting business elements at different time periods, repeatedly extract the information that is directly related to the transaction, and reduce the impact of noise on the formation of data stream links.

[0074] The summary module includes two units: information summary and transaction process data flow relationship extraction:

[0075] Information aggregation: Stores transaction process collection and time link extraction, primary key transfer tracking, and two-way data source information transmitted after noise processing.

[0076] Transaction process data flow relationship: business elements and data flow information are matched in an orderly manner. The matching basis is the set of ordered data streams extracted within the business time period after time link extraction, primary key transfer tracking, and noise processing, and displayed to the outside for query and retrieval.

[0077] The beneficial technical effects of the present invention are: greatly reducing the time cost and experience threshold requirements of manual combing, and only requiring a small amount of manual entry during the initial sample collection period; the automatically generated data flow information provides an effective basis for analysis and troubleshooting for the monitoring and tracking of full-link transactions; and the monitoring data is continuously and automatically refreshed to enrich the content, forming a complete set of data flow information asset precipitation.

[0078] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method when executing the computer program.

[0079] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for executing the above method.

[0080] like Fig.10 As shown, the electronic device 600 may further include: a communication module 110, an input unit 120, an audio processing unit 130, a display 160, and a power supply 170. It is worth noting that the electronic device 600 does not necessarily have to include Fig.10 In addition, the electronic device 600 may also include Fig.10 For components not shown, reference may be made to the prior art.

[0081] like Fig.10 As shown, the central processor 100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices. The central processor 100 receives inputs and controls the operations of various components of the electronic device 600.

[0082] The memory 140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 100 may execute the program stored in the memory 140 to implement information storage or processing.

[0083] The input unit 120 provides input to the CPU 100. The input unit 120 is, for example, a key or a touch input device. The power supply 170 is used to provide power to the electronic device 600. The display 160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0084] The memory 140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 140 may also be some other type of device. The memory 140 includes a buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application / function storage unit 142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 600 through the central processor 100.

[0085] The memory 140 may also include a data storage unit 143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 144 of the memory 140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0086] The communication module 110 is a transmitter / receiver 110 that transmits and receives signals via an antenna 111. The communication module (transmitter / receiver) 110 is coupled to the central processor 100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0087] Based on different communication technologies, multiple communication modules 110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module (transmitter / receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide an audio output via the speaker 131 and receive an audio input from the microphone 132, thereby realizing a common telecommunication function. The audio processor 130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 130 is also coupled to the central processor 100, so that the sound can be recorded on the local machine through the microphone 132, and the sound stored on the local machine can be played through the speaker 131.

[0088] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A transaction data flow acquisition method, characterized in that: The method comprises: Receive transaction information, and obtain transaction process data based on transaction information analysis; Collect monitoring data from the backend database, extract the change data information of the corresponding period in the monitoring data by primary key extraction according to the start and end time in the transaction information, and sort the change data information in the order of occurrence to generate a data stream set; According to the business elements in the transaction information, the transaction process data is sequentially matched and associated with the data flow set to generate a transaction data flow; Obtaining transaction process data based on transaction information analysis includes: analyzing the transaction information to obtain transaction elements, and classifying the transaction elements to generate transaction process data; wherein the monitoring data includes query text, and the query text includes SQL text, execution time and corresponding table name.

2. The transaction data flow acquisition method according to claim 1, characterized in that: Before sequentially matching and associating the transaction process data with the data flow set according to the business elements in the transaction information, the following steps are also included: When there is a transaction link data flow in the data flow set and the primary key changes during the transmission process, the primary key jump position is obtained by using the primary key transmission tracking technology; According to the primary key jump position, the changed primary key is associated with the primary key at the primary key jump position and then stored in the data stream set.

3. The transaction data flow acquisition method according to claim 1, characterized in that: The monitoring data includes query text, execution time and corresponding table name.

4. The transaction data flow acquisition method according to claim 3, characterized in that: Extracting the change data information of the corresponding period in the monitoring data in a manner of extracting the start and end time in the transaction information by primary key includes: Obtaining primary key information from the data table structure in the monitoring data according to the start and end time in the transaction information; The change data information of the corresponding time period in the monitoring data is extracted through the primary key information.

5. The transaction data flow acquisition method according to claim 3, characterized in that: The data stream set generated by sorting the change data information in the order of occurrence time includes: When the change data information comes from databases with different time settings, the standard time of the change data information is calculated by a time conversion formula; The change data information is sorted according to the standard time to generate a data stream set.

6. The transaction data stream acquisition method according to any one of claims 1 to 5, characterized in that: Before sequentially matching and associating the transaction process data with the data flow set according to the business elements in the transaction information, the following steps are also included: Relevant information associated with the transaction is collected and extracted through business elements of different preset periods, and noise reduction processing is performed on the data stream set based on the relevant information.

7. A transaction data flow acquisition device, characterized in that: The device comprises a transaction collection module, a technical data stream processing module and a summary module; The transaction collection module is used to receive transaction information and obtain transaction process data based on transaction information analysis; The technical data stream processing module is used to collect monitoring data from the backend database, extract the change data information of the corresponding period in the monitoring data in the primary key extraction method according to the start and end time in the transaction information, and sort the change data information in the order of occurrence to generate a data stream set. The monitoring data includes query text, and the query text includes SQL text, execution time and corresponding table name; The aggregation module is used to match and associate the transaction process data with the data flow set in order according to the business elements in the transaction information to generate a transaction data flow; Obtaining transaction process data based on transaction information analysis includes: analyzing the transaction information to obtain transaction elements, and classifying the transaction elements to generate transaction process data.

8. The transaction data flow acquisition device according to claim 7, characterized in that: The technical data stream processing module also includes a primary key transfer tracking module and a noise processing module; The primary key transfer tracking module is used to obtain the primary key jump position through the primary key transfer tracking technology when the primary key of the transaction link data flow in the data flow set changes during the transfer process; according to the primary key jump position, the changed primary key is associated with the primary key of the primary key jump position and then stored in the data flow set; The noise processing module is used to collect and extract relevant information associated with the transaction through business elements of different preset periods, and perform noise reduction processing on the data stream set through the relevant information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for causing a computer to execute the method according to any one of claims 1 to 6.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images