A stream processing method and apparatus

Abstract

Provided are a stream processing method and device, related to the financial field, the method comprising: acquiring one or more input streams, each input stream comprising an event for which processing is requested, each event relating to data in one or more data tables; mapping each input stream as a transaction object, the time of occurrence and type of each event for which processing is requested being mapped as a timestamp and a label in the corresponding transaction object, respectively; writing the one or more transaction objects into a transaction event stream in the order of the time of occurrence of the events; for a first transaction object in the transaction event stream, determining a corresponding transaction function from a plurality of preset transaction functions, the first transaction object being the first transaction object to be processed in the transaction event stream; and processing the first transaction object based on the determined transaction function to obtain a processing result. By converting the input stream into a transaction object and selecting a corresponding transaction function for processing, the efficiency of processing is improved.

Description

Technical Field

[0001] This application relates to the financial field, and more particularly to a stream processing method and apparatus. Background Technology

[0002] Streaming technology is rapidly evolving, and its applications can be found across various business sectors. As stock exchanges shift from floor-to-ceiling trading to electronic trading, streaming technology has also emerged in the financial field.

[0003] A transaction often involves data from multiple databases, or even multiple data tables. For example, a money transfer transaction involves data from the accounts of both the sender and recipient. To ensure consistency, it is necessary to update the data in these multiple databases, or even multiple data tables, involved in the transaction.

[0004] In practice, transactions are input into the business system in the form of streams, and different streams involve different data and tables. Therefore, how to perform stream processing efficiently is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This application provides a stream processing method and apparatus that improves processing efficiency by converting an input stream into a transaction object and selecting a corresponding transaction function for processing.

[0006] In a first aspect, this application provides a stream processing method, comprising: acquiring one or more input streams, each input stream including an event to be processed, each event involving data in one or more data tables; mapping each of the one or more input streams to a transaction object, wherein the occurrence time of the event to be processed by each input stream is mapped to a timestamp in the corresponding transaction object, and the type of the event to be processed by each input stream is mapped to a tag in the corresponding transaction object; writing the transaction objects corresponding to the one or more input streams into a transactional event stream, the transactional event stream including multiple transaction objects ordered in chronological order of the occurrence time of the events; determining a corresponding transaction function from a set of preset transaction functions for a first transaction object in the transactional event stream, the first transaction object being the first transaction object to be processed in the transactional event stream; and processing the first transaction object based on the determined transaction function to obtain a processing result.

[0007] Based on the above scheme, input streams involving data from one or more tables are converted into transaction objects. At the transaction object level, corresponding transaction functions are selected to process these objects. Furthermore, the transaction objects corresponding to one or more input streams are written into a transactional data stream according to the event occurrence time, allowing for sequential processing of these objects. This ensures orderly processing of one or more input streams, avoiding errors and improving processing efficiency. Each processing of a transaction object involves updating one or more pieces of data related to that object, guaranteeing the atomicity, consistency, isolation, and durability of stream processing. Therefore, the atomicity, consistency, isolation, and durability of stream processing are achieved, resulting in overall improved stream processing efficiency.

[0008] Optionally, writing the transaction objects corresponding to the one or more input streams into the transactional event stream includes: writing the transaction objects corresponding to the one or more input streams into the transactional event stream in chronological order based on the timestamps included in each transaction object.

[0009] Optionally, determining the corresponding transaction function from a set of preset transaction functions for the first transaction object in the transactional event stream includes: determining the transaction function corresponding to the label included in the first transaction object in the transactional event stream from a set of preset transaction functions.

[0010] Optionally, before processing the first transaction object based on the determined transaction function, the method further includes: reading the original data values ​​of the data in the one or more data tables involved in the event of the corresponding input stream request processing included in the first transaction object;

[0011] The original data value is placed in the cache.

[0012] Optionally, the step of processing the first transaction object based on a determined transaction function to obtain a processing result includes: processing the first transaction object according to a preset logic in the determined transaction function based on the original data value put into the cache to obtain a processing result.

[0013] Optionally, the processing result includes processing success and processing failure.

[0014] Optionally, when the processing result includes successful processing, the processing result also includes a data value calculated based on the original data value placed in the cache; and the first transaction object is processed according to the preset logic in the determined transaction function based on the original data value placed in the cache, including: updating the calculated data value to the corresponding position of the original data value.

[0015] Optionally, when the processing result includes processing failure, the processing result may also include the reason for the processing failure.

[0016] Secondly, this application provides a stream processing apparatus, the apparatus comprising a mapping module and a transaction processing module. The mapping module is used to acquire one or more input streams, each input stream including an event to be processed, each event involving data in one or more data tables; mapping each of the one or more input streams to a transaction object, wherein the occurrence time of the event to be processed by each input stream is mapped to a timestamp in the corresponding transaction object, and the type of the event to be processed by each input stream is mapped to a tag in the corresponding transaction object; writing the transaction objects corresponding to the one or more input streams into a transactional event stream, the transactional event stream including multiple transaction objects ordered in chronological order of the occurrence time of the events; the transaction processing module is used to determine a corresponding transaction function from a plurality of preset transaction functions for a first transaction object in the transactional event stream, the first transaction object being the first transaction object to be processed in the transactional event stream; and processing the first transaction object based on the determined transaction function to obtain a processing result.

[0017] Thirdly, this application provides a stream processing apparatus including a processor. The processor is coupled to a memory and can be used to execute a computer program in the memory to implement the methods of the first aspect and any possible implementation thereof.

[0018] Optionally, the apparatus in the third aspect also includes a memory.

[0019] Optionally, the apparatus in the third aspect also includes a communication interface, to which the processor is coupled.

[0020] Fourthly, this application provides a chip system including at least one processor for supporting the implementation of the functions involved in the first aspect and any possible implementation of the first aspect, such as acquiring or processing data involved in the above methods.

[0021] In one possible design, the chip system also includes a memory for storing program instructions and data, which may be located within or outside the processor.

[0022] The chip system can consist of chips or include chips and other discrete components.

[0023] Fifthly, this application provides a computer-readable storage medium storing a computer program (also referred to as code or instructions) that, when executed by a processor, causes the methods described in the first aspect and any possible implementation thereof to be performed.

[0024] Sixthly, this application provides a computer program product comprising: a computer program (also referred to as code or instructions) that, when run, causes the methods described in the first aspect and any possible implementation thereof to be executed.

[0025] It should be understood that the second to sixth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a scenario applicable to an embodiment of this application;

[0027] Figure 2 This is a schematic flowchart of a stream processing method provided in an embodiment of this application;

[0028] Figure 3 This is a schematic block diagram of a stream processing apparatus provided in an embodiment of this application;

[0029] Figure 4 This is a schematic block diagram of another stream processing apparatus provided in the embodiments of this application;

[0030] Figure 5 This is a schematic block diagram of another stream processing device provided in the embodiments of this application. Detailed Implementation

[0031] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0033] To facilitate understanding of the embodiments of this application, some terms or vocabulary involved in this application will be briefly explained below.

[0034] 1. Streaming data: Streaming data is a sequence of data that arrives in a sequential, large, fast and continuous manner.

[0035] 2. Input Streams and Output Streams: In some programming languages, such as Java, different types of input and output sources can be abstracted as streams, where the input and output data can be called data streams. A data stream is a channel for an application to acquire and send data, and it can include input streams and output streams. Typically, applications use input streams to read data and output streams to write data. The characteristic of streaming input and output is that data acquisition and transmission occur sequentially along the data sequence. An output stream can be understood as writing data to a storage medium or data channel, while an input stream can be understood as reading data from a storage medium or data channel.

[0036] Generally speaking, the characteristics of streams include the following:

[0037] First-in, first-out (FIFO): the data written to the output stream first is read by the input stream first.

[0038] Sequential access allows you to write a string of bytes into the stream one after another, and read the stream in the same order as when it was written. You cannot access the data in between randomly.

[0039] Each stream can only be either an input stream or an output stream; it cannot have both functions simultaneously. In a data transmission channel, if data needs to be written and read at the same time, two separate streams must be provided.

[0040] 3. ACID semantics:

[0041] Atomicity (A): Generally, an atom refers to something that cannot be further divided into its smallest unit of granularity. Atomicity can mean that all actions involved in a transaction are either executed or not executed at all; it does not end at some intermediate stage.

[0042] Consistency (C): can refer to the period from the start to the end of a transaction during which the integrity of data objects is not compromised.

[0043] Isolation (I): refers to a transaction being independent of other concurrent transactions, with no mutual interference between transactions.

[0044] Durability (D): This refers to the fact that after a transaction is completed, the modifications to the data will be permanently effective and will not be lost even if a system failure occurs.

[0045] 4. Stream processing: Stream processing is a computer programming paradigm, equivalent to data stream programming, event stream processing, and reactive programming. Stream processing makes it easier for certain applications to take advantage of limited forms of parallel processing.

[0046] Streaming technology is rapidly evolving, and its applications can be found across various business sectors. As stock exchanges shift from floor-to-ceiling trading to electronic trading, streaming technology has also emerged in the financial field.

[0047] A transaction often involves data from multiple data tables, or even multiple data records. For example, a money transfer transaction involves data from the accounts of the sender and recipient. To ensure consistency, the data from these multiple data tables needs to be updated. While many known stream processing systems provide strong consistency guarantees, these guarantees are only applicable to specific types of applications that update only a single key at a time. In other words, these methods are suitable for streaming data that only needs to update a single piece of data at a time.

[0048] In practice, transactions are input into the business system in the form of streams, and different streams involve different data and tables. That is, if an application only updates the balance of a single bank account each time, it can be achieved using currently known stream processing methods. However, if the application needs to update multiple keys and multiple tables each time, such as transferring funds from one bank account to another, the processing efficiency becomes relatively low, and achieving strong consistency becomes difficult. Therefore, how to perform stream processing efficiently is a technical problem that urgently needs to be solved.

[0049] This application provides a stream processing method and apparatus, which aims to improve processing efficiency by converting a data stream involving data from one or more data tables into a transaction object, selecting the corresponding transaction function, and processing the data sequentially according to the occurrence time of events.

[0050] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0051] Figure 1 This is a schematic diagram of a scenario applicable to an embodiment of this application. Figure 1 Multiple input streams are shown, such as input stream 1, input stream 2, ..., input stream L (L≥1, L is an integer), as well as the computer and the output stream. These input streams can be understood as data sequences generated based on user operations. The computer can process these input streams, and the computer can write the processing results to the output stream.

[0052] It should be understood that Figure 1 This is merely an example and should not be construed as limiting the scope of this application.

[0053] The following uses a computer as an example to describe in detail the stream processing method proposed in the embodiments of this application.

[0054] It should be noted that the stream processing method and apparatus provided in the embodiments of this application can be applied to the financial field, or to any field other than the financial field, and this application does not limit them.

[0055] Figure 2 This is a schematic flowchart of a stream processing method provided in an embodiment of this application.

[0056] like Figure 2 As shown, the stream processing method 200 includes steps 210 to 250. The steps in method 200 are described in detail below.

[0057] In step 210, the computer acquires one or more input streams.

[0058] Each input stream can include an event for request processing, and each event relates to data in one or more data tables.

[0059] Each event involves data from one or more data tables. This can be understood as each event involving one piece of data from one data table, or each event involving multiple pieces of data from one data table, or each event involving one piece of data from each of multiple data tables, or each event involving multiple pieces of data from each of multiple data tables. The specific case can be determined based on the design of the data tables involved in each event, and this application does not impose any restrictions on this.

[0060] For example, in the financial field, events can include deposits, withdrawals, transfers, and purchases. Depending on the user's action, each input stream can correspond to a different event. For instance, if a user performs a transfer, the input stream acquired by the computer could include a transfer event, which could involve the source account, the source account's balance, the destination account, the destination account's balance, the transfer amount, and the transfer time.

[0061] In step 220, the computer maps each of the one or more input streams to a transaction object.

[0062] Computers can map input streams one-to-one to transaction objects; that is, one input stream can be mapped to one transaction object, and L (L≥1, L is an integer) input streams can be mapped to L transaction objects. A transaction object can include data related to the corresponding event. It should be understood that since one input stream can be mapped to one transaction object, one transaction object can correspond to one input stream; since one input stream can include one event to be processed, one input stream corresponds to one event. Therefore, it can also be said that one transaction object corresponds to one event.

[0063] In this context, the occurrence time of the event requested by each input stream in one or more input streams is mapped to a timestamp in the corresponding transaction object, and the type of the event requested by each input stream is mapped to a label in the corresponding transaction object.

[0064] As mentioned above, for example in the financial field, events can be deposits, withdrawals, transfers, and consumption. In this application, the types of events can include, but are not limited to, deposits, withdrawals, transfers, and consumption. When a computer maps an input stream to a transaction object, it can map the type of the event to a label in the transaction object. The label can be used to indicate which type of event the data contained in this transaction object corresponds to.

[0065] The time of a user's action can be considered the time the event occurred. For example, when a user performs a transfer operation, the time it takes to generate and process the transfer request based on the user's action can be considered the time when this transfer event occurred. When a computer maps an input stream to a transaction object, it can map the time of an event's occurrence to a timestamp within the transaction object, with the timestamp corresponding to the time of the event associated with that transaction object.

[0066] In step 230, the computer writes the transaction objects corresponding to one or more input streams into a transactional event stream.

[0067] A transactional event stream can include multiple transaction objects ordered in chronological order of their occurrence.

[0068] Computers can write transaction objects into a transactional event stream according to the time the event occurs.

[0069] In one possible implementation, the computer can write one or more transaction objects corresponding to input streams into a transactional event stream in chronological order, based on the timestamps included in each transaction object.

[0070] As mentioned above, the timestamp in a transaction object is mapped from the occurrence time of the event. Therefore, the computer can directly write transaction objects into the transactional event stream in sequence according to the timestamp in the transaction object. In other words, the transaction object corresponding to the event that occurred earlier will be written into the transactional event stream earlier, and its order in the transactional event stream will be earlier accordingly.

[0071] In step 240, the computer determines the corresponding transaction function from a set of preset transaction functions for the first transaction object in the transactional event stream.

[0072] The first transaction object can be the first pending transaction object in a transactional event stream. Transaction objects in a transactional event stream are written in sequentially according to their timestamps, and they can be processed in the order they are listed.

[0073] A computer can pre-store various transaction functions for handling different events. The computer can determine the transaction function to handle the event corresponding to the first transaction object from the pre-stored various transaction functions based on the type of event corresponding to the first transaction object.

[0074] In one possible implementation, the computer can determine the transaction function corresponding to the label from a set of preset transaction functions for the label included in the first transaction object in the transactional event stream.

[0075] As mentioned above, a transaction object corresponds to an event, and a transaction function can handle one type of event. Therefore, a transaction function can handle one type of event. The label in the transaction object is mapped from the type of the event. Thus, the computer can determine the transaction function corresponding to the label from a set of pre-defined transaction functions based on the label in the first transaction object.

[0076] In step 250, the computer processes the first transaction object based on a defined transaction function to obtain the processing result.

[0077] The transaction function includes the processing logic for handling the corresponding event, so the computer can process the first transaction object based on the determined transaction function to obtain the processing result.

[0078] Optionally, the computer may read the raw data values ​​of data from one or more data tables involved in the event of the corresponding input stream request processing included in the first transaction object; and put the raw data values ​​into a cache.

[0079] In other words, the computer can pre-read the original data values ​​of the relevant data involved in the event corresponding to the first transaction object and put these original data values ​​into the cache for later use.

[0080] For example, if the event corresponding to the first transaction object is a transfer event, the computer can query and read the balance of the source account from the table based on the identifier of the source account, and can query and read the original data values ​​of the balance of the destination account based on the identifier of the destination account, and store the original data values ​​of the source account balance and the destination account balance, etc., in the cache.

[0081] After obtaining the processing result, the computer can clear the data in the cache.

[0082] In one possible implementation, the computer can process the first transaction object based on the original data value stored in the cache, according to the preset logic in the determined transaction function, and obtain the processing result.

[0083] Different transaction functions have different preset processing logic. The computer can process the first transaction object according to the preset logic in the transaction function corresponding to the first transaction object. For example, it can perform logical judgment and / or calculation on the original data value to obtain the corresponding processing result.

[0084] Optionally, the processing result may include processing success and processing failure.

[0085] The processing result can also be understood as an output stream. The processing result can include success and failure; that is, the output stream can be divided into a success output stream and a failure output stream. Depending on the specific processing status of the transaction function on the first transaction object, the computer can output a success output stream or a failure output stream.

[0086] In one possible implementation, when the processing result includes successful processing, the processing result also includes a data value calculated based on the original data value placed in the cache; and the computer can update the corresponding position of the original data value with the calculated data value.

[0087] In other words, when a transaction function processes the first transaction object, the processing result includes successful processing, or when the computer outputs a successful processing output stream. The successful processing output stream can also include data values ​​calculated by the computer based on the original data values ​​put into the cache. Furthermore, the computer can update the corresponding positions of the original data values ​​in the relevant data tables with the calculated data values ​​to update the data in the corresponding data tables.

[0088] In one possible implementation, when the processing result includes processing failure, the processing result may also include the reason for the processing failure.

[0089] In this possible implementation, when the computer outputs a processing failure output stream, the processing failure output stream can also include the reason for the processing failure. For example, when the event corresponding to the first transaction object is a transfer event, assuming that according to the preset logic in the determined transaction function, the computer determines that the balance of the source account is less than the transfer amount, and therefore the transfer cannot be processed, the computer can output a processing failure output stream, and the reason for the processing failure included in the processing failure output stream can be insufficient balance.

[0090] Based on the above scheme, input streams involving data from one or more tables are converted into transaction objects. At the transaction object level, corresponding transaction functions are selected to process these objects. Furthermore, the transaction objects corresponding to one or more input streams are written into a transactional data stream according to the event occurrence time, allowing for sequential processing of these objects. This ensures orderly processing of one or more input streams, avoiding errors and improving processing efficiency. Each processing of a transaction object involves updating one or more pieces of data related to that object, guaranteeing the atomicity, consistency, isolation, and durability of stream processing. Therefore, the atomicity, consistency, isolation, and durability of stream processing are achieved, resulting in overall improved stream processing efficiency.

[0091] Figure 3 This is a schematic block diagram of a stream processing apparatus provided in an embodiment of this application.

[0092] like Figure 3 As shown, the stream processing device 300 may include a mapping module 310 and a transaction processing module 320. The stream processing device 300 can be used to implement the functions of the computer in the method 200 described above. The mapping module 310 can be used to acquire one or more input streams, each input stream including an event to be processed, and each event involving data in one or more data tables. The mapping module 310 can be used to map each of the one or more input streams to a transaction object, wherein the occurrence time of the event requested by each input stream is mapped to a timestamp in the corresponding transaction object, and the type of the event requested by each input stream is mapped to a tag in the corresponding transaction object. The mapping module 310 can also be used to write the transaction objects corresponding to the one or more input streams into a transactional event stream, the transactional event stream including multiple transaction objects ordered according to the chronological order of the events' occurrence times. The transaction processing module 320 can be used to determine a corresponding transaction function from a set of preset transaction functions for a first transaction object in the transactional event stream, the first transaction object being the first transaction object to be processed in the transactional event stream. The transaction processing module 320 can also be used to process the first transaction object based on the determined transaction function to obtain a processing result.

[0093] Optionally, the mapping module 310 can be used to write the transaction objects corresponding to the one or more input streams into the transactional event stream according to the time sequence based on the timestamp included in each transaction object.

[0094] Optionally, the transaction processing module 320 may be used to determine the transaction function corresponding to the label from a set of preset transaction functions for the label included in the first transaction object in the transactional event stream.

[0095] Optionally, the transaction processing module 320 can also be used to read the original data values ​​of the data in one or more data tables involved in the events of the corresponding input stream request processing included in the first transaction object; and put the original data values ​​into a cache.

[0096] Optionally, the transaction processing module 320 can be used to process the first transaction object based on the original data value put into the cache, according to the preset logic in the determined transaction function, to obtain the processing result.

[0097] Optionally, the processing result includes processing success and processing failure.

[0098] Optionally, when the processing result includes successful processing, the processing result also includes a data value calculated based on the original data value placed in the cache; and the transaction processing module 320 can be specifically used to update the calculated data value to the corresponding position of the original data value.

[0099] Optionally, when the processing result includes processing failure, the processing result may also include the reason for the processing failure.

[0100] Figure 4 This is a schematic block diagram of another stream processing apparatus provided in the embodiments of this application.

[0101] like Figure 4As shown, the stream processing device 400 may include: an acquisition module 410, a mapping module 420, a writing module 430, a determination module 440, and a processing module 450. The stream processing device 400 can be used to implement the functions of the computer in the method 200 described above. The acquisition module 410 can acquire one or more input streams, each input stream including an event to be processed, and each event involving data in one or more data tables. The mapping module 420 can map each of the one or more input streams to a transaction object, wherein the occurrence time of the event requested by each input stream is mapped to a timestamp in the corresponding transaction object, and the type of the event requested by each input stream is mapped to a tag in the corresponding transaction object. The writing module 430 can write the transaction objects corresponding to the one or more input streams into a transactional event stream, the transactional event stream including multiple transaction objects ordered according to the chronological order of the events' occurrence times. The determining module 440 can determine a corresponding transaction function from a set of preset transaction functions for a first transaction object in the transactional event stream, the first transaction object being the first transaction object to be processed in the transactional event stream. The processing module 450 can process the first transaction object based on the determined transaction function to obtain a processing result.

[0102] Optionally, the writing module 430 can be specifically used to write the transaction objects corresponding to the one or more input streams into the transactional event stream according to the time sequence based on the timestamp included in each transaction object.

[0103] Optionally, the determining module 440 may be specifically used to determine the transaction function corresponding to the label from a set of preset transaction functions for the label included in the first transaction object in the transactional event stream.

[0104] Optionally, the processing module 450 can also be used to read the original data values ​​of the data in one or more data tables involved in the event of the corresponding input stream request processing included in the first transaction object; and put the original data values ​​into a cache.

[0105] Optionally, the processing module 450 can be specifically used to process the first transaction object based on the original data value put into the cache, according to the preset logic in the determined transaction function, to obtain the processing result.

[0106] Optionally, the processing result includes processing success and processing failure.

[0107] Optionally, when the processing result includes successful processing, the processing result also includes a data value calculated based on the original data value placed in the cache; and the processing module 450 can specifically be used to update the calculated data value to the corresponding position of the original data value.

[0108] Optionally, when the processing result includes processing failure, the processing result may also include the reason for the processing failure.

[0109] It should be understood that Figure 3 and Figure 4 The module division of the stream processing device described herein is merely exemplary. In practical applications, different functional modules can be divided according to different functional requirements. This application does not impose any limitations on the form or number of functional modules in practical applications. Figure 3 and Figure 4 This application cannot impose any limitations.

[0110] Figure 5 This is a schematic block diagram of another stream processing device provided in the embodiments of this application.

[0111] The stream processing device 500 can be used to implement the functions of the computer in the method 200 described above. The stream processing device 500 can be a chip system. In this embodiment, the chip system can be composed of chips, or it can include chips and other discrete devices.

[0112] like Figure 5 As shown, the stream processing device 500 may include at least one processor 510 for implementing the functions of the computer in the method 200 provided in the embodiments of this application.

[0113] For example, when the stream processing device 500 is used to implement the method 200 provided in the embodiments of this application, the processor 510 can be used to acquire one or more input streams, each input stream including an event to be processed, each event involving data in one or more data tables; map each of the one or more input streams to a transaction object, wherein the occurrence time of the event to be processed by each input stream is mapped to a timestamp in the corresponding transaction object, and the type of the event to be processed by each input stream is mapped to a tag in the corresponding transaction object; write the transaction objects corresponding to the one or more input streams into a transactional event stream, the transactional event stream including multiple transaction objects ordered according to the chronological order of the occurrence time of the events; for the first transaction object in the transactional event stream, determine a corresponding transaction function from a plurality of preset transaction functions, the first transaction object being the first transaction object to be processed in the transactional event stream; based on the determined transaction function, process the first transaction object to obtain a processing result. See the detailed description in the method example for specific details, which will not be repeated here.

[0114] The stream processing device 500 may further include at least one memory 520, which can be used to store multiple transaction functions as well as instructions and / or data. The memory 520 is coupled to the processor 510. The coupling in this embodiment is an indirect coupling or communication connection between devices, units, or modules, and can be electrical, mechanical, or other forms, used for information exchange between devices, units, or modules. The processor 510 may operate in conjunction with the memory 520. The processor 510 may execute program instructions stored in the memory 520. At least one of the at least one memories may be included in the processor.

[0115] The stream processing device 500 may further include a communication interface 530 for communicating with other devices via a transmission medium, thereby enabling the stream processing device 500 to communicate with other devices. The communication interface 530 may be, for example, a transceiver, interface, bus, circuit, or a device capable of transmitting and receiving functions. The processor 510 may utilize the communication interface 530 to transmit and receive data and / or information, and to implement... Figure 2 The corresponding embodiment describes the method 200 executed by the computer.

[0116] This application embodiment does not limit the specific connection medium between the processor 510, memory 520, and communication interface 530. This application embodiment... Figure 5 The processor 510, memory 520, and communication interface 530 are connected via bus 540. Bus 540 is... Figure 5The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0117] This application also provides a chip system, the chip system including at least one processor for implementing the above. Figure 2 The functions involved in the computer-executed method described in the illustrated embodiment.

[0118] In one possible design, the chip system also includes a memory for storing program instructions and data, which may be located within or outside the processor.

[0119] The chip system can consist of chips or include chips and other discrete components.

[0120] This application also provides a computer program product, the computer program product comprising: a computer program (also referred to as code or instructions), which, when run, causes a computer to perform the following actions: Figure 2 The method of the illustrated embodiment.

[0121] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is executed, it causes the computer to perform actions such as... Figure 2 The method of the illustrated embodiment.

[0122] It should be understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0123] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0124] The terms “unit”, “module”, etc., used in this specification may be used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution.

[0125] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. In the several embodiments provided in this application, it should be understood that the disclosed apparatus, devices, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0126] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more units can be integrated into one module.

[0128] In the above embodiments, the functions of each functional module can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs), etc.

[0129] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0130] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A stream processing method, characterized in that, The method includes: Retrieve one or more input streams, each input stream including an event for request processing, and each event relating to data in one or more data tables; Each of the one or more input streams is mapped to a transaction object, wherein the occurrence time of the event requested by each input stream is mapped to a timestamp in the corresponding transaction object, and the type of the event requested by each input stream is mapped to a label in the corresponding transaction object; Based on the timestamp included in each transaction object, the transaction objects corresponding to the one or more input streams are written into a transactional event stream in chronological order. The transactional event stream includes multiple transaction objects sorted in chronological order of the occurrence time of the events. For the tags included in the first transaction object in the transactional event stream, a transaction function corresponding to the tag is determined from a plurality of preset transaction functions, and the first transaction object is the first transaction object to be processed in the transactional event stream; Reading the original data values ​​of the data in one or more data tables involved in the events corresponding to the input stream request processing of the first transaction object; The original data value is placed in the cache; Based on the original data value put into the cache, the first transaction object is processed according to the preset logic in the determined transaction function to obtain the processing result.

2. The method as described in claim 1, characterized in that, The processing results include successful processing and processing failure.

3. The method as described in claim 2, characterized in that, When the processing result includes successful processing, the processing result also includes a data value calculated based on the original data value placed in the cache; as well as The process of processing the first transaction object based on the original data value placed in the cache and according to the preset logic in the determined transaction function includes: The calculated data value is then updated to the corresponding position of the original data value.

4. The method as described in claim 2, characterized in that, When the processing result includes processing failure, the processing result also includes the reason for the processing failure.

5. A stream processing apparatus, characterized in that, The device includes: A mapping module is used to acquire one or more input streams, each input stream including an event to be processed, and each event involving data in one or more data tables; map each of the one or more input streams to a transaction object, wherein the occurrence time of the event to be processed by each input stream is mapped to a timestamp in the corresponding transaction object, and the type of the event to be processed by each input stream is mapped to a label in the corresponding transaction object; and write the transaction objects corresponding to the one or more input streams into a transactional event stream, wherein the transactional event stream includes multiple transaction objects ordered in chronological order of the occurrence time of the events; The transaction processing module is used to determine the transaction function corresponding to the label from a plurality of preset transaction functions for the label included in the first transaction object in the transactional event stream, wherein the first transaction object is the first transaction object to be processed in the transactional event stream; and to process the first transaction object based on the determined transaction function to obtain the processing result. The mapping module is specifically used to write the transaction objects corresponding to one or more input streams into the transactional event stream according to the time sequence based on the timestamp included in each transaction object; The transaction processing module is further configured to read the original data values ​​of the data in the one or more data tables involved in the events of the corresponding input stream request processing included in the first transaction object; and put the original data values ​​into a cache; The transaction processing module is specifically used to process the first transaction object based on the original data value put into the cache, according to the preset logic in the determined transaction function, and obtain the processing result.

6. A stream processing apparatus, characterized in that, Includes a processor that executes computer execution instructions to implement the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, Includes a computer program that, when run, causes a computer to perform the method as described in any one of claims 1 to 4.

Images