Data real-time calculation method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a data real-time calculation method and device, equipment and a storage medium. The method comprises the steps that a logic log corresponding to a database is acquired; wherein the logic log is generated by triggering an operation aiming at data in the database; the logic log comprises the type of the operation and the data after the operation; analyzing the logic log to obtain a first operation message; determining a second operation message according to historical data in the database and the first operation message; the second operation message comprises data before operation, data after operation and an operation type; and calculating data in the database in real time according to the second operation message.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a real-time data calculation method, device, equipment and storage medium. Background Art

[0002] In today's environment where data is generated and flows rapidly, enterprises have an increasing demand for real-time data processing. At present, the commonly used real-time data computing method is based on streaming data. This computing method can perform streaming computing and analysis on real-time data streams when the data source continuously generates continuous data streams, so as to achieve rapid processing of real-time data.

[0003] However, in the computing method based on streaming data, the data involved in the calculation are mainly new data or data in a specific window period with a short recent window period. There is less attention paid to historical data, and the modification of historical data is not included in the scope of regular calculation. Therefore, there are certain problems with the calculations involved in the modification of historical data, and it is impossible to fully meet the real-time computing needs when historical data changes. Summary of the invention

[0004] The embodiments of the present disclosure provide a real-time data calculation method, device, equipment and storage medium to solve the problem in the related art that the real-time calculation requirements when the historical data changes cannot be fully met due to the failure to include the modification of the historical data into the scope of conventional calculation.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for real-time data calculation, the method comprising: Obtaining a logical log corresponding to a database; wherein the logical log is generated by triggering an operation on data in the database; the logical log includes the type of operation and the data after the operation; Parsing the logic log to obtain a first operation message; Determine a second operation message according to the historical data in the database and the first operation message; the second operation message includes data before the operation, data after the operation, and the type of operation; According to the second operation message, real-time calculation is performed on the data in the database.

[0006] In a second aspect, an embodiment of the present disclosure provides a real-time data computing device, the device comprising: An acquisition module, used for acquiring a logical log corresponding to a database; wherein the logical log is generated by triggering an operation on data in the database; the logical log includes the type of operation and the data after the operation; A parsing module, used for parsing the logic log to obtain a first operation message; A determination module, configured to determine a second operation message according to the historical data in the database and the first operation message; the second operation message includes data before the operation, data after the operation, and a type of operation; A calculation module is used to perform real-time calculation on the data in the database according to the second operation message.

[0007] In a third aspect, an embodiment of the present disclosure provides a real-time data computing device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, enable the processor to implement the steps of the method described in the first aspect above.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the steps of the method described in the first aspect when executed by a processor.

[0009] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] At least one of the above technical solutions provided by the embodiments of the present invention can achieve the following technical effects: In an embodiment of the present invention, a logical log generated by an operation trigger on data in a database can be first obtained, wherein the logical log can include the type of the operation and the data after the operation. Then, the logical log corresponding to the obtained database can be parsed to obtain a first operation message, and based on the historical data in the database and the first operation message, a second operation message including the data before the operation, the data after the operation and the operation type can be determined. Based on the second operation message, the data in the database can be calculated in real time.

[0011] It can be known from the above-mentioned embodiments of the invention that the embodiments of the present invention can first obtain the logical log corresponding to the database when performing real-time calculation on the data in the database, and parse the logical log to obtain the first operation message. Since the logical log includes the type of this operation and the data after the operation, the first operation message obtained can include the type of this operation and the data after the operation. Then, the embodiments of the present invention will further combine the historical data in the database to complete the first operation message, that is, complete the data before the operation that is not in the first operation message, and obtain a second operation message including the data before the operation, the data after the operation, and the type of the operation, and perform real-time calculation on the data in the database according to the completed second operation message. Since the data can be combined with the historical data in the database for real-time calculation, there is a high degree of attention to the historical data. When the historical data is modified, the data can be calculated in real time based on the modified historical data. Therefore, it can well solve the problem that the real-time calculation requirements when the historical data changes cannot be fully met due to the failure to include the modification of the historical data in the conventional calculation scope in the related technology, and effectively improve the application scope and accuracy of the real-time calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate one or more embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor. Figure 1 One of the flow charts of a method for real-time data calculation provided by an embodiment of the present invention; Figure 2 A second flow chart of a method for real-time data calculation provided by an embodiment of the present invention; Figure 3 A third flow chart of a method for real-time data calculation provided by an embodiment of the present invention; Figure 4 A fourth flowchart of a method for real-time data calculation provided by an embodiment of the present invention; Figure 5 A schematic diagram of the module composition of a real-time data computing device 500 provided in one embodiment of the present invention; Figure 6 A schematic diagram of the hardware structure of a real-time data computing device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the technical parties in this technical field better understand the technical aspects of one or more embodiments of the present disclosure, and to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in combination with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0014] The technical solutions provided by various embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0015] See also Figure 1 , Figure 1 One of the flow charts of the real-time data calculation method provided by one embodiment of the present invention is as follows: Figure 1 As shown, the method comprises the following steps: Step 102: Obtain the logical log corresponding to the database; wherein the logical log is generated by triggering an operation on the data in the database; the logical log includes the type of operation and the data after the operation.

[0016] Step 104: Parse the logical log to obtain a first operation message.

[0017] Step 106: Determine a second operation message according to the historical data in the database and the first operation message; the second operation message includes data before the operation, data after the operation, and the type of operation.

[0018] Step 108: Perform real-time calculation on the data in the database according to the second operation message.

[0019] In the embodiment of the present invention, a logical log acquisition tool can be used to acquire the logical log corresponding to the database in real time, wherein the logical log is generated by triggering an operation on the data in the database, and the operation on the data in the database may include a data modification operation. The acquired logical log may include the type of the operation and the data after the operation.

[0020] In one embodiment of the invention, considering that for some single-system real-time data computing scenarios, there may be a problem of incomplete logical log data. For example, when the database is an informix database, in order to reduce the impact on the production process, the archive log cannot be in full log mode. The logical log generated by the update operation triggers only records the updated data, which causes great trouble to the real-time computing.

[0021] For example, in an informix database, when a customer modifies his or her contact information, a logical log of the modification operation will be generated. However, the logical log only records the updated data and the update operation, and does not record the data before the update.

[0022] After the logical log is obtained, the first operation message can be obtained. According to the database type and the logical log capture tool, the data delay and the message structure of the first operation message may be different. Figure 2 As shown, for different databases, such as relational databases, using different parsing tools can generate messages with different structures.

[0023] In one embodiment of the invention, a message queue can also be introduced to shield the heterogeneity of the data source. After obtaining the logical log, the logical log can be written into the first message queue, and the logical log in the first message queue can be obtained in real time. Then, the obtained logical log is parsed to obtain the first operation message. After obtaining the first operation message, the first operation message can be stored in a pre-set preprocessing library, such as Figure 3 shown.

[0024] After obtaining the first operation message, the second operation message can be determined according to the historical data in the database and the first operation message, wherein the second operation message may include the data before the operation, the data after the operation, and the type of operation.

[0025] In one embodiment of the invention, at least one data table may be obtained from a database according to a first operation message, and then a second operation message may be determined according to historical data in the obtained at least one data table and the first operation message.

[0026] In one embodiment of the invention, a preprocessing library may be pre-set, and the second operation message may be determined according to the historical data in at least one data table and the first operation message acquired through the processing program of the preprocessing library.

[0027] When acquiring at least one data table from a database, the acquisition process of acquiring the historical data corresponding to one data table and acquiring multiple data tables is different. Specifically, in the case of acquiring one data table, the data before the operation can be acquired according to the historical data in the acquired data table, and the second operation message can be determined according to the data before the operation and the first operation message; and in the case of acquiring multiple data tables, the multiple data tables can be associated according to the common fields in the acquired multiple data tables, and the data before the operation can be acquired according to the historical data in the associated multiple data tables, and then the second operation message can be determined according to the data before the operation and the first operation message.

[0028] In one example, the first operation message that does not include the data before the operation can be completed by the processing program of the preprocessing library to obtain the second operation message that includes the data before the operation. The processing program of the preprocessing library can include the following functions: If a single table can be used to complete the data and obtain the second operation message, the historical data in the preprocessing library can be accessed to obtain the historical data or old version data, and the missing data before the change in the message can be completed to form a complete calculation message of before and after.

[0029] When obtaining a data table, the processing program of the preprocessing library will access the historical data of the single table in the preprocessing library. These historical data contain the previous version or old version of the data of the table. In some cases, the first operation message may lack the data before the change. For example, in certain operations of the database, only the updated data may be recorded, but not the data before the update. The function of the processing program of the preprocessing library is to obtain these historical data or old version data, compare and integrate them with the current data, and complete the missing data before the change in the message. Through such operations, a complete calculation message containing "before" and "after" is finally formed. Among them, the "before" part contains the completed data before the change, and the "after" part contains the current latest data.

[0030] Such a complete calculation report can provide a more comprehensive and accurate data basis for subsequent calculations, making the calculation results more reliable and accurate. For example, when performing data comparison, difference analysis or other calculations that need to consider the situation before and after the data change, such a complete report is very important.

[0031] If a single table cannot complete the data, the latest report and historical data from multiple tables can be combined to complete the association of data from multiple tables and form a complete calculation report required for complex calculations.

[0032] When complex calculation logic is involved, the processing program of the preprocessing library can play an important role. Specifically, it combines the latest message currently obtained with the historical data of multiple tables for processing.

[0033] In actual applications, data from multiple tables can be associated with each other. For example, data from different tables can be associated with each other through some common fields to reflect the overall picture of the business or meet specific computing needs. The processing program of the preprocessing library can use these associations to integrate and associate historical data from multiple tables.

[0034] In this way, the processing program of the preprocessing library can form data packets that meet the needs of complex calculations. These packets contain multi-table data that have been associated and integrated, providing the necessary data support for subsequent complex calculations.

[0035] For example, when conducting some comprehensive analysis or calculation, data from multiple tables may be involved, and there may be complex relationships between these data. The processing program of the preprocessing library can ensure that these complex calculations can be carried out smoothly by completing the multi-table data association, providing an accurate and complete data foundation.

[0036] In addition, the latest report contains the latest status and change information of the current data, and can provide current real-time information that may not be fully covered by historical data in multiple tables. Combined with the real-time details provided by the latest report, the data can be more complete.

[0037] After the second operation message is obtained, the data in the database may be calculated in real time according to the second operation message.

[0038] In one embodiment of the invention, in order to shield message differences and improve warehousing efficiency, standardized messages can be obtained. In order to implement real-time data calculation based on the standardized messages, a standard message structure can be set in advance, and the second operation message can be standardized to obtain a third operation message that meets the standard message structure. Then, the data in the database can be calculated in real time based on the third operation message.

[0039] In one example, if Figure 4 As shown, the processing program of the pre-set preprocessing library can be used to standardize the second operation message according to the preset standard message structure to obtain a third operation message that meets the standard message structure. The standardized third operation message can meet the requirements of multiple types of calculations and maximize the standardization, scalability and robustness of the system.

[0040] like Figure 2 and Figure 3 As shown, in an embodiment of the present invention, after obtaining the logical logs corresponding to different databases through the logical log acquisition tool, the first message queue of the logical log protocol can be used, and then the logical logs can be obtained from the first message queue in real time, and the obtained logical logs can be parsed according to different parsing tools to obtain a heterogeneous first operation message, and the first operation message can be stored in the pre-processing library. The processing program of the pre-processing library can obtain historical data through at least one data table, and when necessary, perform multi-table data association to obtain a second operation message that completes the data before the operation, and then, the second operation message can be standardized according to the preset standard message result to obtain a third operation message that meets the standard message structure. At this time, the pre-processing library can include historical data and a standardized third operation message.

[0041] like Figure 3As shown, after obtaining the third operation message, the third operation message can be written into the second message queue, the operation message in the second message queue is obtained in real time, and the data in the database is calculated in real time according to the operation message obtained in real time.

[0042] In real-time calculation, calculations can be performed according to pre-defined indicators, and calculations can be performed through the stream processing module to obtain calculation results, which can be stored in the database or processed later as needed.

[0043] like Figure 4 As shown, during real-time computing, real-time computing can be performed according to defined indicators through the real-time computing engine FLINK or other real-time computing engines.

[0044] In an embodiment of the present invention, a logical log generated by an operation trigger on data in a database can be first obtained, wherein the logical log can include the type of the operation and the data after the operation. Then, the logical log corresponding to the obtained database can be parsed to obtain a first operation message, and based on the historical data in the database and the first operation message, a second operation message including the data before the operation, the data after the operation and the operation type can be determined. Based on the second operation message, the data in the database can be calculated in real time.

[0045] It can be known from the above-mentioned embodiments of the invention that the embodiments of the present invention can first obtain the logical log corresponding to the database when performing real-time calculation on the data in the database, and parse the logical log to obtain the first operation message. Since the logical log includes the type of this operation and the data after the operation, the first operation message obtained can include the type of this operation and the data after the operation. Then, the embodiments of the present invention will further combine the historical data in the database to complete the first operation message, that is, complete the data before the operation that is not in the first operation message, and obtain a second operation message including the data before the operation, the data after the operation, and the type of the operation, and perform real-time calculation on the data in the database according to the completed second operation message. Since the data can be combined with the historical data in the database for real-time calculation, there is a high degree of attention to the historical data. When the historical data is modified, the data can be calculated in real time based on the modified historical data. Therefore, it can well solve the problem that the real-time calculation requirements when the historical data changes cannot be fully met due to the failure to include the modification of the historical data in the conventional calculation scope in the related technology, and effectively improve the application scope and accuracy of the real-time calculation.

[0046] Corresponding to the above-mentioned real-time data calculation method, the embodiment of the present invention further provides a real-time data calculation device, Figure 5 A schematic diagram of the module composition of the data real-time computing device 500 provided in an embodiment of the present invention is shown in FIG. Figure 5 As shown, the data real-time calculation device 500 includes: The acquisition module 501 is used to acquire the logical log corresponding to the database; wherein the logical log is generated by triggering an operation on the data in the database; the logical log includes the type of operation and the data after the operation; The parsing module 502 is used to parse the logic log to obtain a first operation message; A determination module 503, configured to determine a second operation message according to the historical data in the database and the first operation message; the second operation message includes data before the operation, data after the operation, and a type of operation; The calculation module 504 is used to perform real-time calculation on the data in the database according to the second operation message.

[0047] Optionally, the determining module 503 is used to: According to the first operation message, obtaining at least one data table from the database; A second operation message is determined according to the acquired historical data in at least one data table and the first operation message.

[0048] Optionally, the determining module 503 is used to: In the case of acquiring a data table, acquiring data before the operation according to historical data in the acquired data table, and determining a second operation message according to the data before the operation and the first operation message; In the case of acquiring multiple data tables, the multiple data tables are associated according to the common fields in the acquired multiple data tables, and the data before the operation is acquired according to the historical data in the associated multiple data tables, and the second operation message is determined according to the data before the operation and the first operation message.

[0049] Optionally, the calculation module 504 is used to: According to a preset standard message structure, the second operation message is standardized to obtain a third operation message that meets the standard message structure; According to the third operation message, real-time calculation is performed on the data in the database.

[0050] Optionally, the determining module 503 is used to: Storing the first operation message in a preset preprocessing library; Obtaining at least one data table from the database according to the first operation message through a processing program of the preprocessing library; Determining, by a processing program of the preprocessing library, a second operation message according to historical data in at least one data table obtained and the first operation message; The calculation module 504 is used for: The second operation message is standardized by the processing program of the preprocessing library according to a preset standard message structure to obtain a third operation message that meets the standard message structure.

[0051] Optionally, the parsing module 502 is used to: Writing the logical log into a first message queue; Acquire the logic log in the first message queue in real time, and parse the acquired logic log to obtain a first operation message; Storing the first operation message in the preprocessing library; The calculation module 504 is used for: Writing the third operation message into the second message queue; The operation message in the second message queue is obtained in real time, and the data in the database is calculated in real time according to the operation message obtained in real time.

[0052] In an embodiment of the present invention, a logical log generated by an operation trigger on data in a database can be first obtained, wherein the logical log can include the type of the operation and the data after the operation. Then, the logical log corresponding to the obtained database can be parsed to obtain a first operation message, and based on the historical data in the database and the first operation message, a second operation message including the data before the operation, the data after the operation and the operation type can be determined. Based on the second operation message, the data in the database can be calculated in real time.

[0053] It can be known from the above-mentioned embodiments of the invention that the embodiments of the present invention can first obtain the logical log corresponding to the database when performing real-time calculation on the data in the database, and parse the logical log to obtain the first operation message. Since the logical log includes the type of this operation and the data after the operation, the first operation message obtained can include the type of this operation and the data after the operation. Then, the embodiments of the present invention will further combine the historical data in the database to complete the first operation message, that is, complete the data before the operation that is not in the first operation message, and obtain a second operation message including the data before the operation, the data after the operation, and the type of the operation, and perform real-time calculation on the data in the database according to the completed second operation message. Since the data can be combined with the historical data in the database for real-time calculation, there is a high degree of attention to the historical data. When the historical data is modified, the data can be calculated in real time based on the modified historical data. Therefore, it can well solve the problem that the real-time calculation requirements when the historical data changes cannot be fully met due to the failure to include the modification of the historical data in the conventional calculation scope in the related technology, and effectively improve the application scope and accuracy of the real-time calculation.

[0054] Corresponding to the above-mentioned real-time data calculation method, an embodiment of the present invention further provides a real-time data calculation device, Figure 6 A schematic diagram of the hardware structure of a real-time data computing device provided by one embodiment of the present invention.

[0055] The data real-time computing device may be a terminal device or a server for real-time computing data provided in the above-mentioned embodiment.

[0056] The data real-time computing device may have relatively large differences due to different configurations or performances, and may include one or more processors 601 and memory 602, and one or more storage applications or data may be stored in the memory 602. Among them, the memory 602 may be a short-term storage or a persistent storage. The application stored in the memory 602 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the data real-time computing device. Furthermore, the processor 601 may be configured to communicate with the memory 602 and execute a series of computer executable instructions in the memory 602 on the data real-time computing device. The data real-time computing device may also include one or more power supplies 603, one or more wired or wireless network interfaces 604, one or more input and output interfaces 605, and one or more keyboards 606.

[0057] Specifically in this embodiment, the data real-time computing device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer executable instructions in the data real-time computing device, and is configured to be executed by one or more processors.

[0058] In an embodiment of the present invention, a logical log generated by an operation trigger on data in a database can be first obtained, wherein the logical log can include the type of the operation and the data after the operation. Then, the logical log corresponding to the obtained database can be parsed to obtain a first operation message, and based on the historical data in the database and the first operation message, a second operation message including the data before the operation, the data after the operation and the operation type can be determined. Based on the second operation message, the data in the database can be calculated in real time.

[0059] It can be known from the above-mentioned embodiments of the invention that the embodiments of the present invention can first obtain the logical log corresponding to the database when performing real-time calculation on the data in the database, and parse the logical log to obtain the first operation message. Since the logical log includes the type of this operation and the data after the operation, the first operation message obtained can include the type of this operation and the data after the operation. Then, the embodiments of the present invention will further combine the historical data in the database to complete the first operation message, that is, complete the data before the operation that is not in the first operation message, and obtain a second operation message including the data before the operation, the data after the operation, and the type of the operation, and perform real-time calculation on the data in the database according to the completed second operation message. Since the data can be combined with the historical data in the database for real-time calculation, there is a high degree of attention to the historical data. When the historical data is modified, the data can be calculated in real time based on the modified historical data. Therefore, it can well solve the problem that the real-time calculation requirements when the historical data changes cannot be fully met due to the failure to include the modification of the historical data in the conventional calculation scope in the related technology, and effectively improve the application scope and accuracy of the real-time calculation.

[0060] Another embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the above process when executed by a processor.

[0061] The storage medium in the embodiment of the present disclosure can implement each process of the above-mentioned data real-time calculation method embodiment and achieve the same effect and function, which will not be repeated here.

[0062] Another embodiment of the present disclosure further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above process is implemented.

[0063] The computer program product in the embodiment of the present disclosure can implement each process of the above-mentioned data real-time calculation method embodiment and achieve the same effect and function, which will not be repeated here.

[0064] In various embodiments of the present disclosure, the computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0065] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0066] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0067] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0068] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0069] It should be understood by those skilled in the art that one or more embodiments of the present disclosure may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0073] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0074] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0075] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0076] The above description is only an embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A real-time data calculation method, characterized in that: The method comprises: Obtaining a logical log corresponding to a database; wherein the logical log is generated by triggering an operation on data in the database; the logical log includes the type of operation and the data after the operation; Parsing the logic log to obtain a first operation message; Determine a second operation message according to the historical data in the database and the first operation message; the second operation message includes data before the operation, data after the operation, and the type of operation; According to the second operation message, real-time calculation is performed on the data in the database.

2. The method according to claim 1, characterized in that The determining the second operation message according to the historical data in the database and the first operation message includes: According to the first operation message, obtaining at least one data table from the database; A second operation message is determined according to the acquired historical data in at least one data table and the first operation message.

3. The method according to claim 2, characterized in that The determining the second operation message according to the acquired historical data in at least one data table and the first operation message includes: In the case of acquiring a data table, acquiring data before the operation according to historical data in the acquired data table, and determining a second operation message according to the data before the operation and the first operation message; In the case of acquiring multiple data tables, the multiple data tables are associated according to the common fields in the acquired multiple data tables, and the data before the operation is acquired according to the historical data in the associated multiple data tables, and the second operation message is determined according to the data before the operation and the first operation message.

4. The method according to claim 3, characterized in that The performing real-time calculation on the data in the database according to the second operation message includes: According to a preset standard message structure, the second operation message is standardized to obtain a third operation message that meets the standard message structure; According to the third operation message, real-time calculation is performed on the data in the database.

5. The method according to claim 4, characterized in that The acquiring at least one data table from the database according to the first operation message includes: Storing the first operation message in a preset preprocessing library; Obtaining at least one data table from the database according to the first operation message through a processing program of the preprocessing library; The determining the second operation message according to the acquired historical data in at least one data table and the first operation message includes: Determining, by a processing program of the preprocessing library, a second operation message according to historical data in at least one data table obtained and the first operation message; The step of performing standard processing on the second operation message according to a preset standard message structure to obtain a third operation message that satisfies the standard message structure includes: The second operation message is standardized by the processing program of the preprocessing library according to a preset standard message structure to obtain a third operation message that meets the standard message structure.

6. The method according to claim 5, characterized in that The step of parsing the logic log to obtain a first operation message includes: Writing the logical log into a first message queue; Acquire the logic log in the first message queue in real time, and parse the acquired logic log to obtain a first operation message; Storing the first operation message in the preprocessing library; The performing real-time calculation on the data in the database according to the third operation message includes: Writing the third operation message into the second message queue; The operation message in the second message queue is obtained in real time, and the data in the database is calculated in real time according to the operation message obtained in real time.

7. A real-time data computing device, characterized in that: The device comprises: An acquisition module is used to acquire a logical log corresponding to a database; wherein the logical log is generated by triggering an operation on data in the database; the logical log includes the type of operation and the data after the operation; A parsing module, used for parsing the logic log to obtain a first operation message; A determination module, configured to determine a second operation message according to the historical data in the database and the first operation message; the second operation message includes data before the operation, data after the operation, and a type of operation; A calculation module is used to perform real-time calculation on the data in the database according to the second operation message.

8. A real-time data computing device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method according to any one of claims 1 to 6 when executed by the processor.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images