A high-frequency market data verification method and device

Abstract

The present invention provides a method and device for checking high-frequency market data: the method comprises: step 102: obtaining high-frequency market data, which includes snapshot data, transaction data, transaction data and K-line market data; step 104: if the maximum business sequence number under different channel codes in the transaction data and the transaction data does not correspond to the total number of the transaction data and the transaction data, the high-frequency market data is judged to be erroneous; step 106: when the high-frequency market data is erroneous, an error record is generated and an email is sent to the corresponding recipient. The method and device for checking high-frequency market data provided by the present invention checks whether the high-frequency market data obtained through different channels are missing, garbled, etc., thereby improving data quality and greatly reducing labor costs.

Description

Technical Field

[0001] The present invention relates to the field of data verification technology, and in particular to a high-frequency market data verification method and device. Background Art

[0002] In order to ensure the accuracy of data during transmission, the data sender is usually required to perform a series of calculations on the data to obtain and introduce the corresponding check code, which is attached to the end of the valid data. The data receiver uses the algorithm function according to the agreement to recalculate the received data and generate a check code, and then compares it with the check code sent by the data sender. If the check code is correct, the data is submitted to the application for further processing; if the check code is wrong, it is considered that there are certain errors in the data.

[0003] At present, in the existing technology, the widely used check codes mainly include exclusive or check (Xor), cumulative sum check (Add), cyclic redundancy check (CRC), cumulative sum check (CheckSum), etc. Both the exclusive or check (Xor) and the cumulative sum check (Add) check methods use bytes as units to check the data. Their algorithms are relatively simple, but due to the single check form, errors and omissions are prone to occur, and the accuracy of data integrity check is not high. Cyclic redundancy check (CRC) and cumulative sum check (CheckSum), which have lower error rates, have problems such as complicated algorithms and coding steps.

[0004] Therefore, it is necessary to provide a data verification method and device that can solve the above problems. Summary of the invention

[0005] In view of the problems and shortcomings of the prior art, the present invention provides a high-frequency market data verification method and device to verify whether the high-frequency market data obtained through different channels has missing or garbled characters, thereby improving data quality and greatly reducing labor costs.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] The present invention provides a high-frequency market data verification method, the high-frequency market data verification method comprising:

[0008] Step 102: Obtain high-frequency market data, including snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data;

[0009] Step 104: if the maximum business sequence number under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data does not correspond to the total number of the transaction-by-transaction data and the transaction-by-transaction data, then the high-frequency market data is judged to be erroneous;

[0010] Step 106: When the high-frequency market data is erroneous, an error record is generated and an email is sent to the corresponding recipient.

[0011] Preferably, before step 102, the method further includes:

[0012] Step 101: retrieve the daily code value table through the API interface, and retrieve the high-frequency market data according to the scene number corresponding to the daily code value table.

[0013] Preferably, in step 101, retrieving the high-frequency market data according to the scene number corresponding to the daily code value table also includes obtaining the file size of the high-frequency market data once at a first preset time interval. If the file size of the high-frequency market data obtained this time is the same as the file size of the high-frequency market data obtained last time, the download of the high-frequency market data is completed. If the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, the high-frequency market data is obtained again at the next time interval until the second preset time.

[0014] Preferably, if the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, the previously downloaded high-frequency market data is automatically deleted, the repaired data is re-downloaded, and the high-frequency market data is divided into blocks.

[0015] Preferably, the obtaining of high-frequency market data in step 102 includes if the high-frequency market data is read successfully, it indicates that the high-frequency market data is normal, if the high-frequency market data is read unsuccessfully, it indicates that the high-frequency market data is abnormal, and when the high-frequency market data is abnormal, an error record is generated and an email is sent to the corresponding recipient.

[0016] Preferably, after step 104, the method further includes:

[0017] Step 105: Generate the K-line market data of the day according to the transaction data, and compare the K-line market data of the day with the official K-line market data. If the K-line market data of the day cannot correspond to the official K-line market data one by one, indicating that the transaction data is missing, an error record is generated and an email is sent to the corresponding recipient.

[0018] Preferably, the K-line market data for the day includes yesterday's closing price, opening price, latest price, highest price, lowest price, transaction amount, transaction volume and average price of the corresponding security.

[0019] Preferably, the error records are stored in a daily verification table of high-frequency data, and the error records in the daily verification table of high-frequency data include ID, trading market, trading day, verification time, verification result, error type, error type description, specific error content, release time, update time and update ID.

[0020] Preferably, if the snapshot data and the K-line market data do not correspond, indicating that the snapshot data and the daily market transaction amount do not correspond, an error record is generated and an email is sent to the corresponding recipient.

[0021] The present invention also provides a high-frequency market data verification device, the high-frequency market data verification device comprising:

[0022] The data acquisition module is used to obtain high-frequency market data, including snapshot data, transaction data, transaction data and K-line market data;

[0023] A data judgment module, which is used to judge that if the maximum business sequence number under different channel codes in the transaction data and the transaction data does not correspond to the total number of the transaction data and the transaction data, then the high-frequency market data is judged to be wrong;

[0024] The data error reporting module is used to generate an error record and send an email to the corresponding recipient when the high-frequency market data is erroneous.

[0025] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0026] The high-frequency market data verification method and device provided in the embodiment of the present invention determines whether the high-frequency market data is accurate by comparing the maximum business sequence number under different channel codes in the transaction data and the transaction data to see whether they correspond to the total number of the transaction data and the transaction data. When the high-frequency market data is wrong, an error record is generated and an email is sent to the corresponding recipient, thereby greatly improving the accuracy of high-frequency market data verification while reducing technical difficulty.

[0027] Further, the K-line market data of the day is generated according to the transaction data, and the K-line market data of the day is compared with the official K-line market data. If the K-line market data of the day and the official K-line market data cannot correspond one-to-one, indicating that the transaction data is missing, an error record is generated and an email is sent to the corresponding recipient, thereby further ensuring the accuracy of the high-frequency market data verification;

[0028] Furthermore, if the snapshot data and the K-line market data do not correspond, indicating that the snapshot data and the daily market transaction amount do not correspond, an error record is generated and an email is sent to the corresponding recipient, so that the relevant recipient can be notified in time, further ensuring the accuracy of the high-frequency market data verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a flow chart of a high-frequency market data verification method provided by an embodiment of the present invention;

[0030] Figure 2 A flowchart of a high-frequency market data verification method provided by another embodiment of the present invention;

[0031] Figure 3 A schematic diagram of the structure of a high-frequency market data verification device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0033] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0034] Based on the problems existing in the existing technology, such as Figure 1 As shown, the present invention provides a high-frequency market data verification method, and the high-frequency market data verification method includes:

[0035] Step 102: Obtain high-frequency market data, including snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data;

[0036] Step 104: if the maximum business sequence number under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data does not correspond to the total number of the transaction-by-transaction data and the transaction-by-transaction data, then the high-frequency market data is judged to be erroneous;

[0037] Step 106: When the high-frequency market data is erroneous, an error record is generated and an email is sent to the corresponding recipient.

[0038] In step 102, the snapshot data includes the exchange's cumulative transaction amount, transaction volume, and the last latest transaction price.

[0039] In step 104, the business sequence number is a field set by the exchange for the convenience of sorting the order time. The order data and the transaction data are arranged in sequence according to the channel code, and the business sequence number in the order data will not be repeated in the business sequence number of the transaction data. Therefore, the maximum business sequence number under different channel codes in the order data and the transaction data should correspond to the total number of the order data and the transaction data, which means that there is no missing phenomenon in the high-frequency market data. If there is a mismatch, the high-frequency market data is judged to be wrong, and an error record needs to be generated and an email needs to be sent to the corresponding recipient.

[0040] For example, if high-frequency market data is missing on multiple different trading days, an error record is generated: the data type is the transaction-by-transaction data and the transaction-by-transaction data, and the verification result is 0, indicating that the high-frequency market data is wrong, and the error type is described as: transaction-by-transaction data and transaction-by-transaction data are missing. At the same time, an email is sent to the corresponding recipient to process the high-frequency market data accordingly. In addition, the updated data is automatically downloaded, and the verification process of step 104 is repeated.

[0041] In a specific implementation, if the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, the previously downloaded high-frequency market data is automatically deleted, the repaired data is downloaded again, and the high-frequency market data is divided into blocks. The division of the high-frequency market data into blocks includes dividing the high-frequency market data into bond, fund, stock data, etc. according to the securities type of each security, so that investors can obtain the desired data. In a specific implementation, the acquisition of high-frequency market data in step 102 includes that if the high-frequency market data is read successfully, it means that the high-frequency market data is normal, and if the high-frequency market data is read unsuccessfully, it means that the high-frequency market data is abnormal. When the high-frequency market data is abnormal, an error record is generated and an email is sent to the corresponding recipient.

[0042] For example, if high-frequency market data is wrong on multiple different trading days, an error record is generated: the data type is transaction-by-transaction data, and the verification result is 0, indicating that the high-frequency market data is wrong. The error type is described as: the file contains garbled characters. At the same time, an email is sent to the corresponding recipient to process the high-frequency market data accordingly.

[0043] View now Figure 2The present invention provides a high-frequency market data verification method, the high-frequency market data verification method comprising:

[0044] Step 101: retrieve the daily code value table through the API interface, and retrieve the high-frequency market data according to the scene number corresponding to the daily code value table;

[0045] Step 102: Obtain high-frequency market data, including snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data;

[0046] Step 104: if the maximum business sequence number under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data does not correspond to the total number of the transaction-by-transaction data and the transaction-by-transaction data, then the high-frequency market data is judged to be erroneous;

[0047] Step 105: Generate the K-line market data of the day according to the transaction data, compare the K-line market data of the day with the official K-line market data, if the K-line market data of the day and the official K-line market data cannot correspond one to one, indicating that the transaction data is missing, then generate an error record and send an email to the corresponding recipient;

[0048] Step 106: When the high-frequency market data is erroneous, an error record is generated and an email is sent to the corresponding recipient.

[0049] In a specific implementation, the daily code value table includes information such as the official code of the securities of the exchange, the abbreviation of the securities, the type of securities, the trading market, etc., and the code value specifically refers to the official code of the securities. The scene number includes snapshot data, transaction data, entrustment data and K-line market data for different scene numbers. In step 101, the high-frequency market data is retrieved according to the scene number corresponding to the daily code value table, and the file size of the high-frequency market data is obtained once every first preset time interval. If the file size of the high-frequency market data obtained this time is the same as the file size of the high-frequency market data obtained last time, the download of the high-frequency market data is completed. If the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, the high-frequency market data is obtained again at the next time interval until the second preset time.

[0050] The first preset time is usually 1 hour, and the second preset time is usually 23:00. If high-frequency market data that meets the above 17-hour acquisition requirements is not obtained until 23:00 of the second preset time of the day, it is considered that no high-frequency market data is generated on that day. If high-frequency market data that is consistent with the previous size is not obtained, it is considered that no accurate high-frequency market data is generated on that day.

[0051] For example, the first high-frequency market data is obtained at 17:00 every day. After a batch of data is obtained, one hour after the first preset time, high-frequency market data is obtained again according to the first preset time, that is, 18:00. One hour after the first preset time, high-frequency market data is obtained again according to the first preset time, that is, 19:00, and so on, until the last high-frequency market data is obtained at the second preset time 23:00.

[0052] In step 105, the K-line market data for the day includes yesterday's closing price, opening price, latest price, highest price, lowest price, transaction amount, transaction volume and average price of the corresponding securities. If the official K-line market data is missing, the error record is stored in the high-frequency data daily inspection table, and an email is sent to the corresponding recipient.

[0053] For example, multiple trading markets have high-frequency market data, but the official K-line market data is missing. An error record is generated: the data type is K-line market data, and the verification result is 0, which means that the high-frequency market data is wrong. The error type is described as: high-frequency market data exists, but the official K-line market data is missing. At the same time, an email is sent to the corresponding recipient to process the high-frequency market data accordingly.

[0054] In a specific implementation, the error records are stored in the high-frequency data daily verification table, and the error records in the high-frequency data daily verification table include ID, trading market, trading day, verification time, verification result, error type, error type description, error specific content, release time, update time and update ID. The details are shown in the following table:

[0055] Monitoring information Notes Monitoring information Notes ID ID ErrorType Error type ExchangeCode Trading Market ErrorDesc Error type description FileType File Type Details Specific content of the error TradingDay Trading Day InsertTime Release time CheckTime Verification time UpdateTime Update time Result Verification Results UPID Update ID

[0056] In a specific implementation, if the snapshot data and the K-line market data cannot correspond, indicating that the snapshot data and the daily market transaction amount cannot correspond, an error record is generated and an email is sent to the corresponding recipient.

[0057] For example, in multiple trading markets, if the snapshot data and the K-line market data cannot correspond, an error record is generated: the data type is transaction-by-transaction data and transaction-by-transaction data, and the verification result is 0, indicating that the high-frequency market data is wrong, and the error type is described as: the snapshot data and the K-line market data cannot correspond. At the same time, an email is sent to the corresponding recipient to process the high-frequency market data accordingly.

[0058] Based on the problems existing in the existing technology, such as Figure 3 As shown, the present invention also provides a high-frequency market data verification device 30, and the high-frequency market data verification device 30 includes:

[0059] The data acquisition module 301 is used to acquire high-frequency market data, including snapshot data, transaction data, transaction data and K-line market data;

[0060] A data judgment module 302 is used to judge that if the maximum business sequence number under different channel codes in the transaction data and the transaction data does not correspond to the total number of the transaction data and the transaction data, then the high-frequency market data is judged to be wrong;

[0061] The data error reporting module 303 is used to generate an error record and send an email to the corresponding recipient when the high-frequency market data is erroneous.

[0062] In summary, the high-frequency market data verification method and device provided in the embodiment of the present invention determines whether the high-frequency market data is accurate by comparing whether the maximum business sequence number under different channel codes in the transaction data and the transaction data corresponds to the total number of the transaction data and the transaction data. When the high-frequency market data is wrong, an error record is generated and an email is sent to the corresponding recipient, thereby greatly improving the accuracy of high-frequency market data verification while reducing technical difficulty;

[0063] Further, the K-line market data of the day is generated according to the transaction data, and the K-line market data of the day is compared with the official K-line market data. If the K-line market data of the day and the official K-line market data cannot correspond one-to-one, indicating that the transaction data is missing, an error record is generated and an email is sent to the corresponding recipient, thereby further ensuring the accuracy of the high-frequency market data verification;

[0064] Furthermore, if the snapshot data and the K-line market data do not correspond, indicating that the snapshot data and the daily market transaction amount do not correspond, an error record is generated and an email is sent to the corresponding recipient, so that the relevant recipient can be notified in time, further ensuring the accuracy of the high-frequency market data verification.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-frequency market data verification method, characterized in that: The high-frequency market data verification method comprises: Step 101: retrieve the daily code value table through the API interface, and retrieve the high-frequency market data according to the scene number corresponding to the daily code value table; obtain the file size of the high-frequency market data once at a first preset time interval, if the file size of the high-frequency market data obtained this time is the same as the file size of the high-frequency market data obtained last time, then the download of the high-frequency market data is completed; if the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, then wait for the next time interval to obtain the high-frequency market data again until the second preset time; Step 102: Obtain high-frequency market data, including snapshot data, transaction-by-transaction data, transaction-by-transaction data, and K-line market data; Step 104: if the maximum business sequence number under different channel codes in the transaction-by-transaction data and the transaction-by-transaction data does not correspond to the total number of the transaction-by-transaction data and the transaction-by-transaction data, then the high-frequency market data is judged to be erroneous; Step 106: When the high-frequency market data is erroneous, an error record is generated and an email is sent to the corresponding recipient.

2. The high-frequency market data verification method according to claim 1, characterized in that: If the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, the previously downloaded high-frequency market data is automatically deleted, the repaired data is re-downloaded, and the high-frequency market data is divided into blocks.

3. The high-frequency market data verification method according to claim 1, characterized in that: The obtaining of high-frequency market data in step 102 includes: if the high-frequency market data is read successfully, it indicates that the high-frequency market data is normal; if the high-frequency market data is read unsuccessfully, it indicates that the high-frequency market data is abnormal; when the high-frequency market data is abnormal, an error record is generated and an email is sent to the corresponding recipient.

4. The high-frequency market data verification method according to claim 1, characterized in that: After step 104, the method further includes: Step 105: Generate the K-line market data of the day according to the transaction data, and compare the K-line market data of the day with the official K-line market data. If the K-line market data of the day cannot correspond to the official K-line market data one by one, indicating that the transaction data is missing, an error record is generated and an email is sent to the corresponding recipient.

5. The high-frequency market data verification method according to claim 4, characterized in that: The K-line market data for the day include yesterday's closing price, opening price, latest price, highest price, lowest price, transaction amount, transaction volume and average price of the corresponding securities.

6. The high-frequency market data verification method according to claim 1, 3 or 4, characterized in that: The error records are stored in the high-frequency data daily verification table. The error records in the high-frequency data daily verification table include ID, trading market, trading day, verification time, verification result, error type, error type description, specific error content, release time, update time and update ID.

7. The high-frequency market data verification method according to claim 1, characterized in that: If the snapshot data and the K-line market data do not correspond, indicating that the snapshot data and the daily market transaction amount do not correspond, an error record is generated and an email is sent to the corresponding recipient.

8. A high-frequency market data verification device, characterized in that: The high-frequency market data verification device comprises: The data acquisition module is used to obtain high-frequency market data, including snapshot data, transaction data, transaction data and K-line market data; A data judgment module, which is used to judge that if the maximum business sequence number under different channel codes in the transaction data and the transaction data does not correspond to the total number of the transaction data and the transaction data, then the high-frequency market data is judged to be wrong; A data error reporting module, which is used to generate an error record and send an email to the corresponding recipient when the high-frequency market data is erroneous; Before obtaining the high-frequency market data, a daily code value table is obtained through an API interface, and the high-frequency market data is retrieved according to the scene number corresponding to the daily code value table; the file size of the high-frequency market data is obtained once at a first preset time interval, and if the file size of the high-frequency market data obtained this time is the same as the file size of the high-frequency market data obtained last time, the download of the high-frequency market data is completed; if the file size of the high-frequency market data obtained this time is different from the file size of the high-frequency market data obtained last time, the high-frequency market data is obtained again at the next time interval until the second preset time.

Images