Log-based data correction method, device, electronic device and storage medium

Abstract

The present invention relates to artificial intelligence technology, and discloses a log-based data correction method, including: obtaining stock data in a data source, and obtaining an operation log corresponding to the stock data; determining whether the operation log is a complete field log; when the operation log is a complete field log, using the operation log to perform data backtracking on the stock data to obtain backtracking data; when the operation log is not a complete field log, using the stock data to complete the field of the operation log, and returning to the step of performing data backtracking on the stock data using the operation log to obtain backtracking data; using the backtracking data to correct the stock data to obtain corrected data. In addition, the present invention also relates to blockchain technology, and the stock data can be stored in a node of the blockchain. The present invention also proposes a log-based data correction device, equipment and medium. The present invention can improve the overall quality of the collected data.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a log-based data correction method, device, electronic device, and computer-readable storage medium. Background Art

[0002] With the rise of big data technology, more and more companies are embracing digital transformation. They are collecting data from various business systems and channels to build data lakes and data warehouses for intelligent business analysis, providing decision-making information and driving business digitally. The consistency between the data collected in data lakes or data warehouses and the data stored in the data source plays a key role in subsequent data analysis.

[0003] Current data collection solutions generally use tools such as Sqoop and DataX for offline collection. Offline incremental collection solutions have a natural flaw, namely, strong reliance on time fields. When the data source does not update the time field value according to the rules, omissions or over-invalidation may occur, resulting in inaccurate or low real-time performance of the collected data. Consequently, the overall quality of the collected data is low. Summary of the Invention

[0004] The present invention provides a log-based data correction method, device and computer-readable storage medium, the main purpose of which is to solve the problem of low overall quality of collected data.

[0005] To achieve the above objectives, the present invention provides a log-based data correction method, comprising:

[0006] Obtaining stock data in a preset data source, and obtaining operation logs corresponding to the stock data;

[0007] Determine whether the operation log is a complete field log;

[0008] When the operation log is a complete field log, backtracking the stock data using the operation log to obtain backtracking data;

[0009] When the operation log is not a complete field log, the operation log is completed using the stock data, and the operation log is used to perform data backtracking on the stock data to obtain backtracking data.

[0010] The stock data is corrected using the retrospective data to obtain corrected data.

[0011] Optionally, obtaining the operation log corresponding to the stock data includes:

[0012] Obtain a log generation request corresponding to the stock data in the preset data source;

[0013] Performing operation log address resolution on the log capture request to obtain the background operation log address corresponding to the stock data;

[0014] The operation log of the stock data is captured using the background operation log address.

[0015] Optionally, performing operation log address resolution on the log capture request to obtain a background operation log address corresponding to the stock data includes:

[0016] Traversing the log generation request and determining the position of field separators in the log generation request;

[0017] Splitting the log generation request into fields according to the position of the field separator to obtain a plurality of split fields;

[0018] Mark the multiple split fields with serial numbers according to the splitting order;

[0019] Selecting a split field with a preset sequence number as the target data field;

[0020] Perform field parsing on the target data field to obtain the background operation log address corresponding to the stock data.

[0021] Optionally, determining whether the operation log is a complete field log includes:

[0022] Extracting null value fields for each operation in the operation log;

[0023] If a null value field can be extracted from each operation step in the operation log, it is determined that the operation log is not a complete field log;

[0024] If there are operation steps in the operation log from which a null value field cannot be extracted, it is determined that the operation log is a complete field log.

[0025] Optionally, the backtracking the stock data using the operation log to obtain the backtracking data includes:

[0026] Selecting the result of one of the operation steps in the operation log one by one as the target result;

[0027] Searching the original data corresponding to the target result in the operation log in the stock data to obtain the stock data corresponding to the original data;

[0028] Determine that the target result is back-test data of the stock data corresponding to the original data.

[0029] Optionally, the using the stock data to complete fields of the operation log includes:

[0030] Selecting one of the operation steps from the operation log one by one as the target step;

[0031] Perform field segmentation on the target step to obtain operation fields;

[0032] Identifying a primary key field within the operation field;

[0033] Determine that the value corresponding to the primary key field in the stock data is the original field value, and fill the original field value into the target step.

[0034] Optionally, the using the retrospective data to correct the stock data to obtain corrected data includes:

[0035] Selecting one of the retrospective data one by one from the retrospective data, and determining that the stock data corresponding to the selected retrospective data is the data to be corrected;

[0036] Retrieving all to-be-amended fields containing the to-be-amended data in the stock data;

[0037] The data to be corrected in each of the fields to be corrected is replaced by the backtracking data to obtain corrected data.

[0038] In order to solve the above problems, the present invention further provides a log-based data correction device, the device comprising:

[0039] A log acquisition module is used to obtain the stock data in the preset data source and obtain the operation log corresponding to the stock data;

[0040] A type determination module, configured to determine whether the operation log is a complete field log;

[0041] A first backtracking module is configured to, when the operation log is a complete field log, perform data backtracking on the stock data using the operation log to obtain backtracking data;

[0042] a second backtracking module configured to, when the operation log is not a complete field log, complete the fields of the operation log using the stock data, and return to the step of backtracking the stock data using the operation log to obtain backtracking data;

[0043] The data correction module is used to correct the stock data using the retrospective data to obtain corrected data.

[0044] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0045] at least one processor; and,

[0046] a memory communicatively connected to the at least one processor; wherein,

[0047] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the log-based data correction method described above.

[0048] In order to solve the above problem, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned log-based data correction method.

[0049] The log-based data correction method, device, electronic device, and computer-readable storage medium proposed in the present invention can solve the problem of low overall quality of collected data. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flow chart of a log-based data correction method according to an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of a data table included in the inventory data provided in one embodiment of the present invention;

[0052] Figure 3 A schematic diagram of an operation record table for operating a data table provided in one embodiment of the present invention;

[0053] Figure 4 A schematic diagram of an updated data table provided in one embodiment of the present invention;

[0054] Figure 5 A schematic diagram of a complete field log provided by an embodiment of the present invention;

[0055] Figure 6 A schematic diagram of an operation record table for operating some fields in a data table provided by one embodiment of the present invention;

[0056] Figure 7 A schematic diagram of a complete data record obtained by operating on some fields in a data table according to an embodiment of the present invention;

[0057] Figure 8 A schematic diagram of an incomplete field log provided by an embodiment of the present invention;

[0058] Figure 9A functional module diagram of a log-based data correction device provided by an embodiment of the present invention;

[0059] Figure 10 A schematic structural diagram of an electronic device for implementing the log-based data correction method provided by an embodiment of the present invention.

[0060] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0062] The embodiment of the present application provides a log-based data correction method. The execution subject of the log-based data correction method includes but is not limited to at least one of the electronic devices such as the server, the terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the log-based data correction method can be executed by software or hardware installed on the terminal device or the server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0063] Reference Figure 1 FIG. 1 is a flow chart of a log-based data correction method according to an embodiment of the present invention. In this embodiment, the log-based data correction method includes:

[0064] S1. Obtaining stock data in a preset data source, and obtaining an operation log corresponding to the stock data.

[0065] In the embodiment of the present invention, the preset data source may be any preset database (such as a MySql database, an Oracle database, or a PostgreSQL database), a data processing system, or other software or hardware with a data storage function.

[0066] In detail, computer statements with data capture functions (such as Java statements, Python statements, etc.) can be used to capture the existing data of the data source from the preset data source, wherein the existing data refers to the data currently stored in the data source.

[0067] In one of the practical application scenarios of the present invention, due to the influence of many external factors during the data collection process (such as the insertion, update or deletion of the stock data in the preset data source during collection), the collected stock data may be inconsistent with the data stored in the preset data source. Therefore, in order to accurately collect the data in the preset data source, the operation log corresponding to the stock data can be obtained at the same time, which facilitates the subsequent analysis of the operation log to realize the retrospective correction of the collected stock data and obtain the accurate data in the preset data source.

[0068] In an embodiment of the present invention, obtaining the operation log corresponding to the stock data includes:

[0069] Obtain a log generation request corresponding to the stock data in the preset data source;

[0070] Performing operation log address resolution on the log generation request to obtain a background operation log address corresponding to the stock data;

[0071] The operation log of the stock data is captured using the background operation log address.

[0072] In detail, the log generation request is a data request automatically generated by the background system of the preset data source when generating an operation log, or a request to open a DML operation log for a MySql database, an Oracle database, or a PostgreSQL database.

[0073] The present invention implements the use of an ASM enhanced bytecode filter to obtain log generation requests corresponding to the stock data in a preset data source. The ASM enhanced bytecode filter is a code analysis tool based on the Java bytecode level. The ASM enhanced bytecode filter is used to monitor and obtain log generation requests, which can improve the success rate of obtaining log generation requests.

[0074] Specifically, performing operation log address resolution on the log generation request to obtain the background operation log address corresponding to the stock data includes:

[0075] Traversing the log generation request and determining the position of field separators in the log generation request;

[0076] Splitting the log generation request into fields according to the position of the field separator to obtain a plurality of split fields;

[0077] Mark the multiple split fields with serial numbers according to the splitting order;

[0078] Selecting a split field with a preset sequence number as the target data field;

[0079] Perform field parsing on the target data field to obtain the background operation log address corresponding to the inventory data.

[0080] For example, the data packaging request is "qwe<rt<yuio<pl", where < is the field delimiter. Then, the data packaging request is split into fields according to the positions of the field delimiters in the data packaging request: "qwe", "rt", "yuio", and "pl" are the four split fields, and the multiple split fields are numbered according to the split order as: 1 "qwe", 2 "rt", 3 "yuio", and 4 "pl". For example, if the preset serial number is 2, then the field "rt" is determined as the target data field.

[0081] Generally, the positions of each field in the log generation request are fixed. Therefore, the target field indicating the background operation log address corresponding to the background system can be obtained through the screening of the preset serial number.

[0082] Furthermore, in the embodiments of the present invention, a parser can be used to perform field parsing on the target data field to obtain the background operation log address corresponding to the inventory data. The parser includes Carakan C / C++, SquirrelFish C++, and SquirrelFish Extreme C++, etc.

[0083] In other embodiments of the present invention, the operation logs corresponding to the inventory data can also be collected by a preset log collection component. Among them, the log collection component includes but is not limited to: Flume component, Logstash component, and Filebeat component.

[0084] S2. Determine whether the operation log is a complete field log.

[0085] In one actual application scenario of the present invention, the operation log can often be divided into a complete field log and an incomplete field log (partial field log). Among them, the complete field log records the complete process of operating on the inventory data, and the incomplete log records the result of operating on the inventory data. Therefore, in order to implement the retrospective correction of the inventory data using the operation log, it is necessary to analyze the complete field log and the incomplete field log针对性地.

[0086] In the embodiments of the present invention, the determination of whether the operation log is a complete field log includes:

[0087] Extract null value fields for each operation in the operation log;

[0088] If null value fields can be extracted for each operation in the operation log, it is determined that the operation log is not a complete field log;

[0089] If there are operation steps in the operation log from which a null value field cannot be extracted, it is determined that the operation log is a complete field log.

[0090] In detail, since the incomplete field log records the results of each operation step of the existing data, each operation step recorded in the incomplete field log contains an empty value field, that is, the original state of the operation object is omitted. Therefore, the empty value field can be extracted for each operation step in the operation log, and then the operation log can be judged as a complete field log based on the empty value field.

[0091] Specifically, a Java statement with a specific field extraction function can be used to extract null value fields for each operation in the operation log, or a pre-built regular expression can be used to extract null value fields for each operation in the operation log, wherein the regular expression is a pre-compiled statement that can extract fields in a specific format.

[0092] S3. When the operation log is a complete field log, use the operation log to perform data backtracking on the stock data to obtain backtracking data.

[0093] In an embodiment of the present invention, when the operation log is a complete field log, since the complete field log records all steps of operating the stock data, the operation log can be directly analyzed to achieve data backtracking of the stock data and obtain backtracking data, wherein the data backtracking refers to replacing the data in the stock data according to the operation results in the operation log to find the true value of each data in the stock data.

[0094] In the embodiment of the present invention, the step of using the operation log to perform data backtracking on the stock data to obtain the backtracking data includes:

[0095] Selecting the result of one of the operation steps in the operation log one by one as the target result;

[0096] Searching the original data corresponding to the target result in the operation log in the stock data to obtain the stock data corresponding to the original data;

[0097] Determine that the target result is back-test data of the stock data corresponding to the original data.

[0098] For example, Figure 2As shown, the stock data includes a data table named invent_sample, wherein the data table contains four fields, namely, id (primary key), name, patent, and update_time, and has three data records.

[0099] Now execute the following on the above data table Figure 3 The operation shown.

[0100] That is, at 2021-01-13 10:21:15, the operation was executed: Update Figure 2 The patent field of the data with the primary key (id) of 3 in the invent_sample data table is 'Lili-NO.1 patent'; at 2021-01-13 11:15:22, the operation is executed: the data with the primary key (id) of 4 is inserted into the invent_sample data table, and the values ​​of the id (primary key), name, patent, and update_time fields of the data with the primary key (id) of 4 are: 4, 'Tim', 'Tim-NO.6 patent', '2021-01-13 11:15:22'; at 2021-01-13 12:05:34, the operation is executed: the data with the primary key (id) of 1 in the invent_sample data table is deleted; at 2021-01-13 12:16:15, the operation is executed: the patent field of the data with the primary key (id) of 3 in the invent_sample data table is updated to 'Lili-NO.2 patent'.

[0101] It can be seen that the updated data table of invent_sample data table is as follows Figure 4 shown.

[0102] Furthermore, it can be seen that according to the above steps, the complete field log is obtained as follows Figure 5 shown.

[0103] The field after "before" is used to record the stock data before modification in the invent_sample data table, and the field after "after" is used to record the stock data after execution in the invent_sample data table. Figure 2 The data generated after the operation in the table; the position field is used to record the order of data operations; op_type is the data operation type, such as (insert / update / delete).

[0104] In detail, since the operation log is a complete field log, the operation log records all the steps of operating the existing data. The data in the existing data that is different from the operation result can be replaced according to the operation result of the operation log to obtain backtracking data.

[0105] For example, Figure 6 As shown in the invent_sample data table, the data with primary key (id) 3 has Figure 6 The two operations recorded in.

[0106] Furthermore, according to the operation steps recorded in the complete field log, in the data with the primary key (id) of 3, the after field value is used to replace the before field value to obtain a complete data record, such as Figure 7 shown.

[0107] S4. When the operation log is not a complete field log, use the stock data to complete the fields of the operation log, and return to S3.

[0108] In an embodiment of the present invention, when the operation log is not a complete field log, the operation log mostly records the results of the operations. Therefore, it is impossible to directly know what the original data corresponding to each operation result in the existing data is, and further, it is impossible to directly use the operation log to trace the existing data.

[0109] In detail, regardless of whether any one or more operations such as updating, inserting, deleting, etc. are performed on the existing data, each operation record in the operation log of the incomplete field will contain part of the content of the object being operated. Therefore, the operation log can be completed based on the repeated fields in each step of the operation log and the existing field.

[0110] In the embodiment of the present invention, the using of the stock data to complete the fields of the operation log includes:

[0111] Selecting one of the operation steps from the operation log one by one as the target step;

[0112] Perform field segmentation on the target step to obtain operation fields;

[0113] Identifying a primary key field within the operation field;

[0114] Determine that the value corresponding to the primary key field in the stock data is the original field value, and fill the original field value into the target step.

[0115] For example, therefore, according to S3 Figure 4 The incomplete field logs obtained are as follows: Figure 8 shown.

[0116] The field after "after" is used to record the execution of the existing data in the invent_sample data table. Figure 2 The data generated after the operation is generated. "before" is a null value field. The content omitted by its null value is the existing data of the field after "after" in the invent_sample data table.

[0117] For example, the result of step 1 is:

[0118] {{"patent":"Lili-NO.1 patent"},"position":1,"op_type":"update"}

[0119] It can be seen that after the field segmentation of the above results, it can be identified that the primary key field in the above results is NO.1. Therefore, it can be seen that this operation step is an operation on the data with the primary key of 1 in the stock data, that is, the value of the data with the primary key of 1 in the stock data is the original field value, and the original field value can be filled in Figure 8 Inside the "before" in step 1.

[0120] In an embodiment of the present invention, when all operation steps in the operation log complete the field completion operation, the operation log is a complete field log. Therefore, the operation log can be used to backtrack the existing data and obtain the backtracked data for processing.

[0121] S5. Correct the stock data using the retrospective data to obtain corrected data.

[0122] In one practical application scenario of the present invention, since the backtracking data is the final operation result of each stock data obtained by analyzing the operation log, the backtracking data can be used to correct the stock data to obtain corrected data.

[0123] In the embodiment of the present invention, the step of using the retrospective data to correct the stock data to obtain the corrected data includes:

[0124] Selecting one of the retrospective data one by one from the retrospective data, and determining that the stock data corresponding to the selected retrospective data is the data to be corrected;

[0125] Retrieving all to-be-amended fields containing the to-be-amended data in the stock data;

[0126] The data to be corrected in each of the fields to be corrected is replaced by the backtracking data to obtain corrected data.

[0127] In other embodiments of the present invention, a CREATE INDEX statement in SQL may be used to retrieve all data records to be corrected that contain the data to be corrected in the stock data, and then the data to be corrected may be corrected according to the data records to obtain corrected data.

[0128] In other embodiments of the present invention, full data correction may be performed based on the retrospective data and the stock data in a join manner to correct erroneous data in the stock data set to obtain corrected data.

[0129] In other embodiments of the present invention, the stock data may be fully processed by window calculation, such as in the form of (row_number), to obtain corrected data.

[0130] In other embodiments of the present invention, data lake components, such as Apache Iceberg / Apache Hudi / DeltaLake, can also be used to provide upsert / update / delete capabilities to correct erroneous data in the existing data by performing streaming, micro-batch, offline correction of the entire data, and other operations to obtain corrected data.

[0131] Then, each field to be corrected is replaced with the backtracking data, so that each field to be corrected is corrected according to the operation in the operation log, thereby achieving correction of the data.

[0132] The log-based data correction method proposed in the present invention can solve the problem of low overall quality of collected data.

[0133] like Figure 9 , which is a functional module diagram of a log-based data correction device provided by an embodiment of the present invention.

[0134] The log-based data correction device 100 of the present invention can be installed in an electronic device. Depending on the functions to be implemented, the log-based data correction device 100 may include a log acquisition module 101, a type determination module 102, a first backtracking module 103, a second backtracking module 104, and a data correction module 105. The module of the present invention, also referred to as a unit, refers to a series of computer program segments that can be executed by an electronic device processor and can perform a fixed function, and is stored in the memory of the electronic device.

[0135] In this embodiment, the functions of each module / unit are as follows:

[0136] The log acquisition module 101 is used to acquire stock data in a preset data source and to acquire operation logs corresponding to the stock data;

[0137] The type determination module 102 is configured to determine whether the operation log is a complete field log;

[0138] The first backtracking module 103 is configured to, when the operation log is a complete field log, use the operation log to perform data backtracking on the stock data to obtain backtracking data;

[0139] The second backtracking module 104 is configured to, when the operation log is not a complete field log, complete the fields of the operation log using the stock data, and return to the step of backtracking the stock data using the operation log to obtain backtracking data;

[0140] The data correction module 105 is configured to correct the stock data using the retrospective data to obtain corrected data.

[0141] In detail, each module in the log-based data correction device 100 according to the embodiment of the present invention adopts the same Figures 1 to 8 The technical means are the same as the log-based data correction method described in , and can produce the same technical effects, so I will not go into details here.

[0142] like Figure 10 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a log-based data correction method provided by an embodiment of the present invention.

[0143] The electronic device 1 may include a processor 10 , a memory 11 , a communication bus 12 , and a communication interface 13 , and may further include a computer program stored in the memory 11 and executable on the processor 10 , such as a log-based data correction program.

[0144] In some embodiments, the processor 10 may be comprised of an integrated circuit, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting various components of the entire electronic device using various interfaces and circuits. It executes or runs programs or modules stored in the memory 11 (e.g., executing a log-based data correction program) and accesses data stored in the memory 11 to perform various functions of the electronic device and process data.

[0145] The memory 11 includes at least one type of readable storage medium, including flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Furthermore, the memory 11 may include both an internal storage unit of the electronic device and an external storage device. The memory 11 can be used not only to store application software installed in the electronic device and various types of data, such as the code of a log-based data correction program, but also to temporarily store data that has been output or is about to be output.

[0146] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0147] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, and includes a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0148] The figure only shows an electronic device with components. Those skilled in the art will understand that the structure shown in the figure does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0149] For example, although not shown, the electronic device may further include a power source (e.g., a battery) to power various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby enabling functions such as charge management, discharge management, and power consumption management via the power management device. The power source may further include any of one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not further detailed here.

[0150] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0151] The log-based data correction program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When executed in the processor 10, it can achieve the following:

[0152] Obtaining stock data in a preset data source, and obtaining operation logs corresponding to the stock data;

[0153] Determine whether the operation log is a complete field log;

[0154] When the operation log is a complete field log, backtracking the stock data using the operation log to obtain backtracking data;

[0155] When the operation log is not a complete field log, the operation log is completed using the stock data, and the operation log is used to perform data backtracking on the stock data to obtain backtracking data.

[0156] The stock data is corrected using the retrospective data to obtain corrected data.

[0157] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0158] Furthermore, if the modules / units integrated into the electronic device 1 are implemented as software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. The computer-readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0159] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0160] Obtaining stock data in a preset data source, and obtaining operation logs corresponding to the stock data;

[0161] Determine whether the operation log is a complete field log;

[0162] When the operation log is a complete field log, backtracking the stock data using the operation log to obtain backtracking data;

[0163] When the operation log is not a complete field log, the operation log is completed using the stock data, and the operation log is used to perform data backtracking on the stock data to obtain backtracking data.

[0164] The stock data is corrected using the retrospective data to obtain corrected data.

[0165] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is merely a logical function division, and other division methods may be used in actual implementation.

[0166] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0167] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0168] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0169] Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0170] Blockchain, as used in this article, refers to a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of online transactions, used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0171] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0172] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. Terms such as "first" and "second" are used to indicate names and do not imply any particular order.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A log-based data correction method, characterized in that: The method comprises: Obtain a log generation request corresponding to the stock data in a preset data source, traverse the log generation request and determine the position of a field separator in the log generation request; Performing field splitting on the log generation request according to the position of the field separator to obtain a plurality of split fields; Marking the multiple split fields with serial numbers according to the splitting order; Select the split field with the preset sequence number as the target data field; Performing field parsing on the target data field to obtain the background operation log address corresponding to the stock data; Using the background operation log address to capture the operation log of the stock data; Determine whether the operation log is a complete field log; When the operation log is a complete field log, backtracking the stock data using the operation log to obtain backtracking data; When the operation log is not a complete field log, the field of the operation log is completed by using the stock data, and the operation log is used to perform data backtracking on the stock data to obtain the backtracking data. The stock data is corrected using the retrospective data to obtain corrected data.

2. The log-based data correction method according to claim 1, characterized in that: The determining whether the operation log is a complete field log includes: Extracting null value fields for each operation in the operation log; If a null value field can be extracted from each operation step in the operation log, it is determined that the operation log is not a complete field log; If there are operation steps in the operation log from which a null value field cannot be extracted, it is determined that the operation log is a complete field log.

3. The log-based data correction method according to claim 1, characterized in that: The step of performing data backtracking on the stock data by using the operation log to obtain backtracking data includes: Selecting the result of one of the operation steps in the operation log one by one as the target result; Retrieving the original data corresponding to the target result in the operation log in the stock data to obtain the stock data corresponding to the original data; Determine that the target result is the back-tracing data of the stock data corresponding to the original data.

4. The log-based data correction method according to claim 1, characterized in that: The using the stock data to complete the fields of the operation log includes: Selecting one of the operation steps from the operation log one by one as the target step; Perform field segmentation on the target step to obtain operation fields; identifying a primary key field within the operation field; Determine that the value corresponding to the primary key field in the stock data is the original field value, and fill the original field value into the target step.

5. The log-based data correction method according to any one of claims 1 to 4, characterized in that: The method of using the retrospective data to correct the stock data to obtain the corrected data includes: Selecting one of the backtracking data one by one from the backtracking data, and determining that the stock data corresponding to the selected backtracking data is the data to be corrected; Retrieving all the fields to be corrected that contain the data to be corrected in the stock data; The data to be corrected in each of the fields to be corrected are replaced with the traceback data to obtain corrected data.

6. A log-based data correction device, used to execute the log-based data correction method according to any one of claims 1 to 5, characterized in that: The device comprises: A log acquisition module is used to acquire stock data in a preset data source, and to acquire operation logs corresponding to the stock data; A type determination module, used to determine whether the operation log is a complete field log; A first backtracking module, configured to, when the operation log is a complete field log, perform data backtracking on the stock data using the operation log to obtain backtracking data; A second backtracking module is used for, when the operation log is not a complete field log, completing the fields of the operation log using the stock data, and returning to the step of backtracking the stock data using the operation log to obtain backtracking data; The data correction module is used to correct the stock data using the retrospective data to obtain corrected data.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the log-based data correction method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the log-based data correction method according to any one of claims 1 to 5 is implemented.

Images