Data processing method and device

Abstract

The present application discloses a data processing method and device. The method comprises: obtaining first stock trading data at a first time; obtaining first offline data at a second time, wherein the first offline data corresponds to the first stock trading data; in the case where an error occurs in the first stock trading data, repairing the first stock trading data using the first offline data; wherein the first object targeted by the first stock trading data is the same as the second object targeted by the first offline data.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a data processing method and device. Background Art

[0002] In the securities industry, securities trading data is the basic data of all data analysis systems. Changes in the quotation information of each security are the data reflection of market transactions. These transaction data are used as a reference for investors during the investment process, thereby directly affecting investors' investment behavior.

[0003] Currently, in the stock trading market, stock trading data can be calculated based on the real-time flow data of stock changes for investors' reference.

[0004] However, in the event of stock trading data errors (eg, transaction errors, loss, duplication, manual price adjustments, business adjustments, program problem repairs, etc.), real-time repair of stock trading data is inefficient. Summary of the Invention

[0005] The embodiments of the present application provide a data processing method and apparatus to solve the problem of low efficiency in real-time repair of stock trading data in the prior art.

[0006] In a first aspect, the present application provides a data processing method, the method comprising:

[0007] Get the first stock trading data at the first time;

[0008] Acquire first offline data at a second time, where the first offline data corresponds to the first stock trading data;

[0009] In the event that an error occurs in the first stock transaction data, repairing the first stock transaction data using the first offline data;

[0010] The first object targeted by the first stock trading data and the second object targeted by the first offline data are the same.

[0011] In a second aspect, the present application provides a data processing device, comprising:

[0012] A first acquisition module, configured to acquire first stock transaction data at a first time;

[0013] a second acquiring module, configured to acquire first offline data at a second time, wherein the first offline data corresponds to the first stock trading data;

[0014] a processing module, configured to repair the first stock trading data using the first offline data when an error occurs in the first stock trading data;

[0015] The first object targeted by the first stock trading data and the second object targeted by the first offline data are the same.

[0016] In a third aspect, the present application provides a server comprising: a processor, a memory, and a program or instruction stored on the memory and running on the processor, wherein the program or instruction implements the steps of the method of the first aspect when executed by the processor.

[0017] In a fourth aspect, the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0018] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0019] In an embodiment of the present application, first stock trading data at a first time is obtained; first offline data at a second time is obtained, the first offline data corresponding to the first stock trading data; if an error occurs in the first stock trading data, the first stock trading data is repaired using the first offline data; wherein the first object targeted by the first stock trading data and the second object targeted by the first offline data are the same. In this way, if an error occurs in the first stock trading data at the first time, the first stock trading data can be repaired by obtaining the first offline data corresponding to the first stock trading data, thereby improving the efficiency of repairing the first stock trading data, thereby solving the problem of low efficiency in real-time repair of stock trading data in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0022] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a stock cost flow sheet in a data processing method provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of a stock holding cost statement in a data processing method provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a data application layer table in a data processing method provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of a data processing method according to an embodiment of the present invention;

[0027] Figure 7 A structural block diagram of a data processing device provided in an embodiment of the present application;

[0028] Figure 8 A structural block diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0031] The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0032] Figure 1 This is a flow chart of a data processing method provided in an embodiment of the present application, with reference to Figure 1 , the data processing method provided in the embodiment of the present application may include:

[0033] Step 110, obtaining first stock transaction data at a first time;

[0034] The first time may be after the market closes on the first target date, or during the stock market opening period on the first target date; the first stock trading data may be real-time data of stock trading on the first target date. For example, the first target date may be the day before the stock trading day. For example, if the stock trading day for Bank of China is Friday, the day before the stock trading day may be Thursday. In other words, the first target date may be Thursday, and the first time may be any time between the market close on Thursday and the market opening on Friday, or during the market opening period on Thursday.

[0035] In this embodiment of the present application, the first stock trading data may include data such as the diluted cost of the stock and the purchase cost of the stock. Cost dilution can be a method of averaging investments and spreading risk. For example, after purchasing a certain stock, if you discover that the stock has fallen, you buy more. The average cost price is naturally lower, which is called cost dilution, but the actual effect is not significant. Furthermore, buying the same stock or fund again after it has fallen below the purchase price is called covering a position. Covering a position allows you to buy the stock at a lower price, lowering the unit cost price, in the hope that it will rebound after covering a position, and then sell it. The profit earned from the purchased stock will offset the loss of the higher-priced stock. If the stock originally purchased at a high price has fallen too far and is unlikely to return to its original price, covering a position allows you to break even without having to return to the original high price. While covering a position can dilute the cost price, the market is unpredictable, and the stock price may continue to fall after covering a position, exacerbating losses. In practical applications, covering a position can be performed when the stock price has fallen sharply, resulting in significant losses, or when the stock price is expected to rise or rebound.

[0036] Step 120: Acquire first offline data at a second time, where the first offline data corresponds to the first stock trading data;

[0037] The second time may be before the opening of the second target date, or the second time may be during the stock opening period on the same day as the first time; the second target date may be the day after the first target date. For example, the second target date may be a stock trading day. For example, if the stock trading day for Bank of China is Friday, the second target date may be Friday, and the day before the stock trading day may be Thursday. The second time may be any time between the opening of Thursday and before the opening of Friday. Specifically, the second time may be before the opening of Friday, or both the first time and the second time may be during the opening period on Thursday.

[0038] In an embodiment of the present application, the first time may be the time after the closing of the first target date, the second time may be the time before the opening of the second target date, and the second target date may be the day after the first target date; for example, the first target date may be the day before the stock trading day, and the second target date may be the day of the stock trading day, then the first time may be the time after the closing of the day before the stock trading day, and the second time may be the time before the opening of the day of the stock trading day. Alternatively, the first time and the second time may be the time during the opening period of the stock on the same day; for example, the first time and the second time may both be the time during the opening period of the day before the stock trading day. It should be understood that the period for data processing may be one working day, that is, the period from the opening of the current day to the opening of the next day.

[0039] The first object targeted by the first stock trading data and the second object targeted by the first offline data may be the same. It should be understood that the first object and the second object may be the same stock or multiple stocks. For example, the first object targeted by the first stock trading data and the second object targeted by the first offline data may both be Sina stocks, or the first object targeted by the first stock trading data and the second object targeted by the first offline data may both be multiple stocks such as Sina, Tencent, Baidu, and Alibaba.

[0040] In an embodiment of the present application, the first offline data may be the stock trading data after the opening of the previous day of the stock trading day, which is synchronized to the data warehouse through a data migration tool before the opening of the stock trading day, and then calculated using the data warehouse.

[0041] It can be understood that a data warehouse (DW / DWH) can be a subject-oriented, integrated, time-varying, but relatively stable collection of data that can be used to support management decision-making processes. A data warehouse is a single data store created for analytical reporting and decision support. For enterprises requiring business intelligence, a data warehouse can guide business process improvements, monitor time, cost, quality, and control.

[0042] The data warehouse architecture can be divided into three layers: the Operation Data Store (ODS), the Common Data Model (CDM), and the Application Data Store (ADS). The Operation Data Store can be a raw data layer, storing raw data in the data warehouse system with minimal processing. Its structure is essentially the same as the source system, and it serves as the data preparation area of ​​the data warehouse. This layer's primary responsibility is to synchronize and store raw data. The Application Data Store can store statistical indicator data customized for data products, generated based on processing by the Common Data Model and the Operation Data Store. The Common Data Store can store detailed fact data, dimension table data, and common indicator summary data. Detailed fact data and dimension table data are generally generated based on data processed by the Operation Data Store; common indicator summary data is generally generated based on processing of dimension table data and detailed fact data. The Common Data Store can also be divided into a Data Warehouse Detail (DWD) layer and a Data Warehouse Summary (DWS) layer. The detailed data layer can use the business process as a modeling driver, constructing the finest-grained detailed fact tables based on the characteristics of each specific business process; it can also make certain important attribute fields appropriately redundant, i.e., wide-table processing. The summary data layer can use the subject of analysis as a modeling driver, constructing a summary indicator table of common granularity based on the indicator requirements of upper-layer applications and products; it can also physicalize the model using wide-table methods, constructing statistical indicators with standardized naming and consistent caliber, providing common indicators for the upper layer, and establishing summary wide tables and detailed fact tables.

[0043] Step 130: When an error occurs in the first stock trading data, the first offline data is used to repair the first stock trading data.

[0044] In an embodiment of the present application, when an error occurs in the first stock trading data, the server that records the first stock trading data will issue an alarm prompt, and human intervention can be carried out to troubleshoot the problem. Possible problems include, for example, flow errors, losses, duplications of stock trading data, or manual price adjustments, business adjustments, program problem repairs, etc., and then the first offline data is used to repair the first stock trading data, thereby improving the efficiency of repairing the first stock trading data.

[0045] It should be understood that the first stock trading data can be stored on a first server, and the first offline data can be stored on a second server, and the first server and the second server can be different. In other words, the first stock trading data can be stored on a server responsible for real-time computing, and the first offline data can be stored on another server responsible for offline computing.

[0046] The data processing method provided in the embodiment of the present application obtains first stock trading data at a first time; obtains first offline data at a second time, wherein the first offline data corresponds to the first stock trading data; and in the event that an error occurs in the first stock trading data, repairs the first stock trading data using the first offline data; wherein the first object targeted by the first stock trading data and the second object targeted by the first offline data are the same. In this way, in the event that an error occurs in the first stock trading data at the first time, the first stock trading data can be repaired by obtaining the first offline data corresponding to the first stock trading data, thereby improving the efficiency of repairing the first stock trading data, thereby solving the problem of low efficiency in real-time repair of stock trading data in the prior art.

[0047] Optionally, in one embodiment of the present application, the data processing method may further include: in the event that an error occurs in the first offline data, repairing the first offline data to obtain target offline data; step 130 may specifically include: in the event that an error occurs in the first stock trading data, using the target offline data to repair the first stock trading data.

[0048] The target offline data may be correct offline data obtained by repairing the first offline data after the user manually intervenes to find the cause of the error in the first offline data.

[0049] In an embodiment of the present application, the process of manual intervention by the user to find the cause of the error in the first offline data can be: first, by comparing the stock transaction data of the day before the stock trading day with the stock transaction data on the offline server, if the stock transaction data is already wrong, then correcting the stock transaction data; then manually calculating the stock holding cost data of the two days before the stock trading day and the stock transaction data of the day before the stock trading day through the stock transaction data, to determine whether the first offline data calculated offline is correct.

[0050] In this way, when an error occurs in the first offline data, the first offline data can be repaired first to obtain the correct target offline data after repair. Then, when an error occurs in the first stock trading data, the target offline data can be used to repair the first stock trading data, thereby better repairing the first stock trading data.

[0051] Optionally, in one embodiment of the present application, the data processing method may further include: determining whether the first stock trading data is consistent with the first offline data; if the first stock trading data is consistent with the first offline data, backing up the first offline data or the first stock trading data.

[0052] It is understandable that since the data of the stock market is accumulated periodically, if no error occurs in the process of calculating the first stock trading data, that is, if the first stock trading data and the first offline data are both normal trading data, the first offline data or the first stock trading data can be backed up as the basic data before the opening of the stock trading day, and then the trading on the stock trading day can be carried out.

[0053] In this way, when the first stock trading data is consistent with the first offline data, that is, when there is no error in the first stock trading data, the first offline data or the first stock trading data can be backed up as the basic data before the opening of the stock trading day, thereby improving the efficiency of data processing.

[0054] In an embodiment of the present application, a Go server can be used to determine the consistency between stock trading data and offline data. The Go server can be a microservice based on the Go language; the Go language (also known as Golang) is a statically strongly typed, compiled, concurrent programming language developed by Google that has garbage collection capabilities; the microservice is a software development technology - a variant of the service-oriented architecture (SOA) architecture style. Microservices advocate dividing a single application into a group of small services, which coordinate and cooperate with each other to provide ultimate value to users. The Go server can include a timed synchronization offline data module and a real-time calculation module. The timed synchronization offline data module can pull offline data before each stock trading market opens, overwriting or correcting the data calculated in real time on the previous day; the real-time calculation module can monitor the message queue of the stock flow based on the offline data of the previous day after each stock trading market opens, and calculate the stock cost in real time.

[0055] The message queue can be RabbitMQ (Rabbit Message Queue), an open source message broker and queue server used to share data between different applications (across platforms and languages) via common protocols. RabbitMQ is written in Erlang and implemented based on the Advanced Message Queuing Protocol (AMQP). The real-time calculated stock costs can be stored in a PostgreSQL database, a free software object-relational database management system (ORDBMS) with a comprehensive set of features. PostgreSQL supports most Structured Query Language (SQL) standards and provides many other modern features, such as complex queries, foreign keys, triggers, views, transaction integrity, multi-version concurrency control, etc. The database can also be expanded in many ways, such as by adding new data types, functions, operators, aggregate functions, indexing methods, procedural languages, etc. It is understandable that the real-time stock costs can be the stock trading data calculated after the market opens that day. Similarly, daily real-time stock trading data can be stored in a real-time cost database.

[0056] In this way, through real-time calculation and storage of daily trading data, stock trading data can be processed cyclically to improve data processing efficiency.

[0057] In an embodiment of the present application, the first time may be the time after the closing of the first target date, the second time may be the time before the opening of the second target date, and the second target date may be the day after the first target date; the specific process of obtaining the first offline data at the second time in step 120 may be implemented in various different ways.

[0058] A specific implementation example is given below. Please note that the following is only an example and is not intended to be limiting.

[0059] See Figure 2 The specific process of obtaining the first offline data at the second time in step 120 may include: step 210, step 220 and step 230. These three steps are explained below.

[0060] Step 210, obtaining second stock trading data before the opening of the first target date;

[0061] The second stock trading data may be real-time data of stock trading conducted on the day before the first target date. For example, if the first target date is the stock trading day, the second stock trading data may be real-time data of stock trading conducted between the opening of the market on the day before the stock trading day and the opening of the market on the stock trading day.

[0062] Step 220: Acquire operation data during the opening process of the first target date, wherein the operation data is data used to perform operations based on the second stock trading data;

[0063] The operation data may be data used to perform operations such as calculations on the second stock trading data during the opening of the first target date.

[0064] In an embodiment of the present application, step 210 and step 220 can be performed on an Amazon Web Services Simple Storage Service (AWS S3) file system. The Amazon Web Services Simple Storage Service can be a network-oriented storage service that can support users to store and retrieve data of any size at any location on the website at any time.

[0065] It is understood that steps 210 and 220 can be performed at the data ingestion layer of the data warehouse. That is, at the data ingestion layer of the data warehouse, the second stock transaction data before the opening of the first target date and the operation data during the opening of the first target date are synchronized to Amazon Simple Storage Service for storage and stored as business data. For example, the business data may include stock orders, pending orders for new shares, corporate actions, stock position adjustments, stock transfers, manual price adjustments, closing prices, and the like.

[0066] Step 230: Obtain first offline data at the second time by calculation based on the operation data and the second stock transaction data.

[0067] In the embodiment of the present application, step 230 can be performed in the data public layer and the data application layer of the data warehouse. Specifically, in the data public layer of the data warehouse, the business data of the data introduction layer can be sorted, and based on the operation data and the second stock transaction data, calculations can be performed to obtain data such as the diluted cost of the stock and the purchase cost of the stock. These data can then be sorted into various tables, such as a stock cost flow table and a stock holding cost detail table. For example, Figure 3 As shown, the stock cost flow table can be a table that records stock flow data, that is, the data is not calculated; Figure 4As shown, the stock holding cost detailed table can be a data table obtained by calculating the flow data in the stock cost flow table. Finally, in the data application layer of the data warehouse, the data obtained from the data public layer can be calculated again based on the stock holding cost detailed table to obtain a data application layer table (such as Figure 5 As shown), thereby obtaining the first offline data at the second time.

[0068] It should be understood that through the stock holding cost details table in the data public layer, the data in the table can be adjusted again based on various human factors before the market opens on the day, or the task output cost data in the target stock table can be directly used as the first offline data without adjustment.

[0069] In this embodiment of the present application, step 230 may specifically include: obtaining the first offline data at the second time by dividing the computation among multiple servers based on the operation data and the second stock trading data. In this manner, the computation can be divided among multiple servers, leveraging the high efficiency of clustering capabilities to rapidly compute large amounts of offline data, thereby improving data processing efficiency.

[0070] To facilitate understanding, a specific example may be given to illustrate the calculation process based on the operation data and the second stock transaction data in the data common layer of the data warehouse:

[0071] For example, the scheduling engine of Amazon Web Services Step Functions (AWS Step Functions) and the computing engine of Presto can be introduced. The scheduling engine of AWS Step Functions can then send a computing task to the Presto computing engine at a preset time, such as 2 minutes. After receiving the computing task, the Presto computing engine quickly calculates the first offline data based on the operation data and the second stock trading data.

[0072] The scheduling engine of the Amazon Web Services Step service can add flexible workflow automation capabilities to applications in minutes without writing code. The workflows built using step functions can include built-in error handling, parameter passing, recommended security settings, and state management capabilities, thereby reducing the amount of code that needs to be written and maintained. The Presto computing engine can be an open source distributed structured query language query engine suitable for interactive analytical queries, supporting data volumes ranging from gigabytes (GB) to petabytes (PB).

[0073] In this way, the first offline data can be calculated through the cooperation between the scheduling engine and the calculation engine, thereby improving the efficiency of data processing.

[0074] In addition, the scheduling engine of the Amazon website's Step service can also schedule a directed acyclic graph (DAG), a linear representation that can more intuitively reflect changes in stock trading data. Specifically, the scheduling engine can extract, transform, and load business data from the data ingestion layer into the data common layer of the data warehouse for subsequent calculations to obtain first offline data, which can then be represented by a directed acyclic graph. The purpose of this extraction, transformation, and loading process is to integrate scattered, disorganized, and non-standardized business data.

[0075] The data processing method provided in the embodiment of the present application can synchronize the second stock trading data before the opening of the first target date and the operation data during the opening process of the first target date to the three-tier architecture of the data warehouse for offline processing, thereby obtaining the first offline data at the second time, thereby quickly calculating large quantities of offline data and improving the efficiency of calculating the first offline data.

[0076] The following is combined with the actual application scenario and Figure 6 , the data processing method provided in the embodiments of the present application is further introduced in detail with examples.

[0077] For example, Figure 6 As shown, before the opening of the stock trading day, the data warehouse can be used to calculate the first offline data of the previous day. Specifically, the stock trading data before the opening of the stock trading day and the operation data during the opening of the stock trading day can be synchronized to the file system (AWS S3) through the data migration tool (DataX) in the data introduction layer of the data warehouse. These data may include: stock orders, orders for new shares, corporate actions, stock holding adjustments, stock transfers, manual price adjustments, closing prices, etc. Then, the data synchronized in the data introduction layer can be sorted in the data public layer of the data warehouse, and based on the operation data and the second stock trading data, that is, the stock flow data, calculations can be performed to obtain data such as the diluted cost of the stock and the purchase cost of the stock, and then these data can be sorted into a stock cost flow table and a stock holding cost detail table. Finally, in the data application layer of the data warehouse, the stock holding cost details table in the data public layer can be combined with the account adjustment and labor cost adjustment data before each market opens on that day to calculate once before each market opens to obtain offline data corresponding to the stock trading data; and if there are no errors in the offline data, the offline data will be used as the basic data before the opening of the stock trading day.

[0078] like Figure 6 As shown, after the market opens, based on the offline data generated by the big data offline task, the Go server can begin consuming the second stock trading data pulled from the message queue (Rabbit MQ). It calculates the diluted cost and purchase cost for each transaction in the order of the messages, which is the first stock trading data, and writes the calculation results in real time to a real-time database (PostgreSQL) table for stock holding costs. Furthermore, users can access various stock trading data in real time through the mobile client. That is, the mobile client can request the presentation layer state transfer network interface (RestAPI) exposed by the Go server in real time to obtain the stock trading data for display to the user. After the market closes and before the next day's opening, the Go server can also begin reading the first offline data generated by the offline task for the stock trading day and verify the consistency between the real-time first stock trading data and the first offline data. If any inconsistency occurs, an alarm is issued, and manual intervention is required to investigate and resolve the issue. If the stock trading data is consistent with the offline data, the offline data calculated by the offline server can be synchronized with the server responsible for calculating the real-time stock trading data as the base data.

[0079] In the event of manual intervention to troubleshoot and repair data or logic, the offline server can leverage the cluster to rerun historical data in large quantities, without affecting the real-time computing module in the Go server, which can calculate and display stock trading data. Once the offline task has repaired the data, if the market has not yet closed for the day, the real-time computing server can recalculate the current day's message queue data based on the previous day's offline data to repair the erroneous stock trading data. This is highly efficient and does not affect the user experience. If the problem is fixed after the market has closed for the day, the repaired offline data can be synchronized to the real-time computing server, and the calculation can begin the next day based on the correct offline data.

[0080] The data processing method provided in the embodiment of the present application can, when an error occurs in the first stock trading data at the first time, repair the first stock trading data by acquiring the first offline data corresponding to the first stock trading data, thereby improving the efficiency of repairing the first stock trading data, thereby solving the problem of low efficiency in real-time repair of stock trading data in the existing technology.

[0081] Figure 7 This is a structural block diagram of a data processing device provided in an embodiment of the present application. Figure 7 A data processing device 700 provided in an embodiment of the present application may include: a first acquisition module 710, a second acquisition module 720 and a processing module 730.

[0082] The first acquisition module 710 is configured to acquire first stock transaction data at a first time;

[0083] The second acquisition module 720 is configured to acquire first offline data at a second time, where the first offline data corresponds to the first stock trading data;

[0084] The processing module 730 is configured to repair the first stock trading data using the first offline data when an error occurs in the first stock trading data;

[0085] The first object targeted by the first stock trading data and the second object targeted by the first offline data are the same.

[0086] The data processing device provided in the embodiment of the present application obtains first stock trading data at a first time; obtains first offline data at a second time, wherein the first offline data corresponds to the first stock trading data; and in the event that an error occurs in the first stock trading data, repairs the first stock trading data using the first offline data; wherein the first object targeted by the first stock trading data and the second object targeted by the first offline data are the same. In this way, in the event that an error occurs in the first stock trading data at the first time, the first stock trading data can be repaired by obtaining the first offline data corresponding to the first stock trading data, thereby improving the efficiency of repairing the first stock trading data, thereby solving the problem of low efficiency in real-time repair of stock trading data in the prior art.

[0087] Optionally, in one embodiment, the processing module 730 can also be used to: in the event that an error occurs in the first offline data, repair the first offline data to obtain target offline data; the processing module 730 can be specifically used to: in the event that an error occurs in the first stock trading data, use the target offline data to repair the first stock trading data.

[0088] Optionally, in one embodiment, the first time is the time after the closing of the first target date, the second time is the time before the opening of the second target date, and the second target date is the day after the first target date; or, the first time and the second time are the time during the stock opening period on the same day.

[0089] Optionally, in one embodiment, the processing module 730 can also be used to: determine whether the first stock trading data is consistent with the first offline data; if the first stock trading data is consistent with the first offline data, back up the first offline data or the first stock trading data.

[0090] Optionally, in one embodiment, the first time is the time after the closing of the first target date, the second time is the time before the opening of the second target date, and the second target date is the day after the first target date; the second acquisition module 720 can be specifically used to: obtain the second stock trading data before the opening of the first target date; obtain operation data during the opening process of the first target date, the operation data is data used to perform operations based on the second stock trading data; according to the operation data and the second stock trading data, obtain the first offline data at the second time by calculation.

[0091] Optionally, in one embodiment, the second acquisition module 720 may also be configured to obtain the first offline data at the second time by dividing the work and calculating among multiple servers based on the operation data and the second stock transaction data.

[0092] Optionally, in one embodiment, the first stock transaction data is stored in a first server, and the first offline data is stored in a second server, and the first server is different from the second server.

[0093] It should be noted that the data processing device provided in the embodiment of the present application corresponds to the data processing method mentioned above. For relevant content, please refer to the description of the data processing method above, and no further details will be given here.

[0094] In addition, if Figure 8 As shown, an embodiment of the present application further provides a server 800, comprising: a processor 810, a memory 820, and a program or instruction stored in the memory 820 and executed on the processor 810, wherein the program or instruction, when executed by the processor 810, implements the steps of any of the methods described above. For example, when the program is executed by the processor 820, it implements the following process: obtaining first stock trading data at a first time; obtaining first offline data at a second time, wherein the first offline data corresponds to the first stock trading data; in the event that an error occurs in the first stock trading data, repairing the first stock trading data using the first offline data; wherein the first object targeted by the first stock trading data and the second object targeted by the first offline data are the same. In this way, in the event that an error occurs in the first stock trading data at a first time, the first stock trading data can be repaired by obtaining the first offline data corresponding to the first stock trading data, thereby improving the efficiency of repairing the first stock trading data, thereby solving the problem of low efficiency in real-time repair of stock trading data in the prior art.

[0095] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by the processor 810, the steps of any of the methods described above are implemented. For example, when the program is executed by the processor 810, the following process is implemented: obtaining first stock trading data at a first time; obtaining first offline data at a second time, the first offline data corresponding to the first stock trading data; in the event that an error occurs in the first stock trading data, repairing the first stock trading data using the first offline data; wherein the first object targeted by the first stock trading data and the second object targeted by the first offline data are the same. In this way, in the event that an error occurs in the first stock trading data at the first time, the first stock trading data can be repaired by obtaining the first offline data corresponding to the first stock trading data, thereby improving the efficiency of repairing the first stock trading data, thereby solving the problem of low efficiency in real-time repair of stock trading data in the prior art.

[0096] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0100] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0101] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0102] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0103] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0104] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Get the first stock trading data at the first time; Acquire first offline data at a second time, the first offline data corresponding to the first stock trading data; In the case where an error occurs in the first stock transaction data, repairing the first stock transaction data using the first offline data; The first object targeted by the first stock trading data and the second object targeted by the first offline data are the same; The first time is the time after the closing of the first target date, the second time is the time before the opening of the second target date, and the second target date is the day after the first target date; The acquiring of the first offline data at the second time includes: Acquire second stock trading data before the opening of the first target date; Acquire operation data during the opening process of the first target date, wherein the operation data is data used to perform operations based on the second stock trading data; obtaining first offline data at the second time by calculation according to the operation data and the second stock transaction data; The process of determining the first offline data includes: in the data introduction layer of the data warehouse, the stock before the opening of the stock trading day on the previous day The transaction data and the operation data during the opening of the stock market the day before the stock trading day are synchronized to the file system through the data migration tool; In the data common layer of the data warehouse, the data synchronized by the data import layer are sorted, and based on the operation data and the second stock transaction data, i.e., the flow data of the stock, a calculation is performed to obtain the diluted cost of the stock and the purchase cost data of the stock, and the diluted cost and the purchase cost data are sorted into a stock cost flow table and a stock holding cost detail table; In the data application layer of the data warehouse, through the stock holding cost details table of the data public layer, combined with the account adjustment and labor cost adjustment data before each market opens on the day, the offline data corresponding to the stock trading data is calculated once before each market opens.

2. The data processing method according to claim 1, characterized in that: The method further comprises: When an error occurs in the first offline data, repair the first offline data to obtain target offline data; When an error occurs in the first stock transaction data, repairing the first stock transaction data by using the first offline data includes: In the event that an error occurs in the first stock trading data, the first stock trading data is repaired using the target offline data.

3. The data processing method according to claim 1, characterized in that: The first time is the time after the closing of the first target date, the second time is the time before the opening of the second target date, and the second target date is the day after the first target date; Alternatively, the first time and the second time are the stock opening time on the same day.

4. The data processing method according to claim 1, characterized in that: The method further comprises: determining whether the first stock transaction data is consistent with the first offline data; When the first stock trading data is consistent with the first offline data, the first offline data or the first stock trading data is backed up.

5. The data processing method according to claim 1, characterized in that: The first offline data at the second time obtained by calculation according to the operation data and the second stock transaction data comprises: According to the operation data and the second stock transaction data, the first offline data at the second time is obtained by calculation through multiple servers.

6. The data processing method according to claim 1, characterized in that: The first stock transaction data is stored in a first server, and the first offline data is stored in a second server, and the first server is different from the second server.

7. A data processing device, characterized in that: The device comprises: A first acquisition module, used to acquire first stock transaction data at a first time; A second acquisition module, configured to acquire first offline data at a second time, wherein the first offline data corresponds to the first stock trading data; a processing module, configured to repair the first stock trading data by using the first offline data when an error occurs in the first stock trading data; The first object targeted by the first stock trading data and the second object targeted by the first offline data are the same; The first time is the time after the closing of the first target date, the second time is the time before the opening of the second target date, and the second target date is the day after the first target date; The second acquisition module is specifically used to acquire second stock trading data before the opening of the first target date; Acquire operation data during the opening process of the first target date, wherein the operation data is data used to perform operations based on the second stock trading data; obtaining first offline data at the second time by calculation according to the operation data and the second stock transaction data; The process of determining the first offline data includes: in the data introduction layer of the data warehouse, the stock before the opening of the stock trading day on the previous day The transaction data and the operation data during the opening of the stock market the day before the stock trading day are synchronized to the file system through the data migration tool; In the data common layer of the data warehouse, the data synchronized by the data import layer are sorted, and based on the operation data and the second stock transaction data, i.e., the flow data of the stock, a calculation is performed to obtain the diluted cost of the stock and the purchase cost data of the stock, and the diluted cost and the purchase cost data are sorted into a stock cost flow table and a stock holding cost detail table; In the data application layer of the data warehouse, through the stock holding cost details table of the data public layer, combined with the account adjustment and labor cost adjustment data before each market opens on the day, the offline data corresponding to the stock trading data is calculated once before each market opens.

8. A server, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and running on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images