Transaction bill data processing method and device and server

Abstract

The invention provides a transaction bill data processing method, a transaction bill data processing device and a server, which can be applied to the technical field of big data. Based on the method, a generation request of a target transaction bill business curve can be received and responded, a kettle cluster is called according to a first script file, target sample business data related to the target transaction bill business curve is obtained by executing a first batch of kettle tasks, and the target sample business data is integrated to obtain a target business wide table; according to the second script file, calling a kettle cluster to execute a second batch of kettle tasks and perform preset service curve generation operation by using the target service wide table to generate a target transaction bill service curve; and storing the target transaction bill business curve into an application data layer of a data warehouse. Therefore, the method can be well adapted to a big data processing scene related to the transaction bill, and related data processing based on the transaction bill is efficiently and accurately completed.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and more particularly to a method, device and server for processing transaction bill data. Background Art

[0002] In financial business scenarios, transaction bills (e.g., commercial bills) are an important payment tool and financing method. Transaction bill business curves (e.g., commercial bill yield curves) are important tools for evaluating and analyzing related business products involving transaction bills.

[0003] However, since the above-mentioned business products often involve relatively complex and tedious transaction processes related to transaction bills, the existing methods, when relying on staff to manually construct transaction bill business curves, have problems such as large data processing volume, low processing efficiency, and easy occurrence of errors.

[0004] To address the above problems, no effective solution has been proposed yet. Summary of the invention

[0005] The present specification provides a transaction bill data processing method, device and server, which can be well adapted to big data processing scenarios involving transaction bills with complicated processing procedures and large data processing volumes, and can efficiently and accurately complete relevant data processing based on transaction bills to obtain a target transaction bill business curve that meets the requirements.

[0006] This specification provides a transaction ticket data processing method, which is applied to a server of a transaction business data processing system, and the method includes:

[0007] Receiving a target transaction bill service curve generation request initiated by a user terminal; wherein the target transaction bill service curve generation request carries at least identification information of a target transaction bill package;

[0008] In response to the request for generating the target transaction bill business curve, the corresponding kettle task is determined according to the first script file; and the kettle cluster is called to obtain the target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks;

[0009] Receive and integrate the target sample business data into the corresponding target business wide table based on the execution results of the first batch of kettle tasks; and store the target business wide table in the data detail layer of the data warehouse;

[0010] According to the second script file, the kettle cluster is called to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and a preset business curve generation operation is performed according to the target business wide table to generate a corresponding target transaction bill business curve;

[0011] Receive and based on the execution results of the second batch of kettle tasks, when it is determined that a target transaction bill business curve that meets the requirements is generated, store the target transaction bill business curve in the application data layer of the data warehouse; wherein the user terminal is connected to the application data layer of the data warehouse through a corresponding data interface, and obtains the target transaction bill business curve through the data interface.

[0012] In one embodiment, the first script file includes a shell script for data collection and storage, the second script file includes a python script for data calculation, and the data warehouse includes a database based on an MPP architecture.

[0013] In one embodiment, the kettle cluster is called to obtain target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks, including:

[0014] Call the kettle cluster, execute the corresponding first batch of kettle tasks, query the data source table in the operation data layer of the data warehouse according to the identification information of the target transaction bill package, and obtain the initial sample business data;

[0015] Call the kettle cluster to detect and remove the first type of abnormal data in the initial sample business data by executing the corresponding first batch of kettle tasks, and obtain the intermediate sample business data; and store the intermediate sample business data in a temporary table;

[0016] Call the kettle cluster to detect and remove the second type of abnormal data in the temporary table by executing the corresponding first batch of kettle tasks to obtain the corresponding target sample business data.

[0017] In one embodiment, the second type of abnormal data includes: transaction business data whose deviation does not meet the requirements;

[0018] The first type of abnormal data includes at least: transaction data whose amount ratio does not meet the requirements and transaction data whose term difference does not meet the requirements;

[0019] When the business category of the target transaction bill business curve is rediscount business, the first category of abnormal data also includes: internal transaction business data, back-to-back transaction business data, and bridge transaction business data.

[0020] In one embodiment, the method further comprises:

[0021] According to the preset protocol rules, the kettle cluster is called to collect initial transaction business data from multiple data sources by executing the third batch of kettle tasks and using the corresponding secure data channel; and the initial transaction business data is stored in the cache layer;

[0022] Call the kettle cluster to read the initial transaction business data from the cache layer by executing the third batch of kettle tasks; and preprocess the initial transaction business data to obtain preprocessed transaction business data;

[0023] The kettle cluster is called to execute the third batch of kettle tasks to store the pre-processed transaction business data into the corresponding data source table in the operation data layer of the data warehouse.

[0024] In one embodiment, a preset business curve generation operation is performed according to the target business wide table to generate a corresponding target transaction bill business curve, including:

[0025] Call the kettle cluster to determine multiple key deadlines by executing the corresponding second batch of kettle tasks; and use the multiple key deadlines to divide the target sample business data in the target business wide table into multiple intervals;

[0026] Call the kettle cluster to execute the corresponding second batch of kettle tasks, use the target sample business data in each interval, and determine the target business value of each key deadline through the first linear regression;

[0027] Call the kettle cluster, execute the corresponding second batch of kettle tasks, use the data of key deadlines to perform curve fitting, and determine the target business values ​​of other deadlines;

[0028] Call the kettle cluster and execute the corresponding second batch of kettle tasks to generate the corresponding target transaction bill business curve based on the target business value of the key deadline and the target business value of other deadlines.

[0029] In one embodiment, when the curve type of the target transaction bill business curve is a derivative curve, after calling the kettle cluster, executing the corresponding second batch of kettle tasks, using the target sample business data in each interval, and determining the target business value of each key deadline through a first linear regression, the method further includes:

[0030] According to the target business value of the key deadline point of each interval and the target transaction business data in each interval, the interval regression line segment of each interval is determined through the second linear regression;

[0031] According to the interval regression line segments of each interval, the slope parameters of each key deadline point are calculated and used to determine the first intermediate business value of each key deadline point through the first linear regression;

[0032] According to the reference curve parameters and the reference threshold parameters, the target business value of each key deadline is obtained by performing result control on the first intermediate business value of each key deadline;

[0033] Use the data of key deadlines to perform curve fitting to determine the target business values ​​of other deadlines;

[0034] According to the target business value of the key term point and the target business value of other term points, the corresponding target transaction bill business curve is generated.

[0035] In one embodiment, after storing the target transaction bill business curve in the application data layer of the data warehouse, the method further includes:

[0036] Receiving a target transaction note valuation request initiated by a user terminal; wherein the target transaction note is a transaction note involved in a target transaction note business curve;

[0037] In response to the target transaction bill valuation request, according to the fourth script file, the kettle cluster is called, and the target transaction bill business curve is used in combination with the relevant data source table to determine the valuation result of the target transaction bill;

[0038] The valuation results of the target transaction notes are stored in the application data layer of the data warehouse.

[0039] This specification also provides a transaction ticket data processing device, which is applied to a server of a transaction business data processing system, and the device includes:

[0040] A receiving module, configured to receive a request for generating a target transaction bill service curve initiated by a user terminal; wherein the request for generating a target transaction bill service curve carries at least identification information of a target transaction bill package;

[0041] The first calling module is used to respond to the request for generating the target transaction bill business curve, determine the corresponding kettle task according to the first script file, and call the kettle cluster to obtain the target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks;

[0042] The integration module is used to receive and integrate the target sample business data into the corresponding target business wide table according to the execution results of the first batch of kettle tasks; and store the target business wide table into the data detail layer of the data warehouse;

[0043] The second calling module is used to call the kettle cluster according to the second script file to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and perform a preset business curve generation operation according to the target business wide table to generate a corresponding target transaction bill business curve;

[0044] The processing module is used to receive and store the target transaction bill business curve in the application data layer of the data warehouse based on the execution results of the second batch of kettle tasks when it is determined that a target transaction bill business curve that meets the requirements is generated; wherein the user terminal is connected to the application data layer of the data warehouse through a corresponding data interface, and obtains the target transaction bill business curve through the data interface.

[0045] The present specification also provides a server, comprising a processor and a memory for storing processor executable instructions, wherein the processor implements the relevant steps of the transaction ticket data processing method when executing the instructions.

[0046] The present specification also provides a computer-readable storage medium on which computer instructions are stored. When the instructions are executed by a processor, the relevant steps of the transaction ticket data processing method are implemented.

[0047] Based on the transaction ticket data processing method, device and server provided in this specification, before the specific implementation, a transaction business data processing system adapted to the big data processing scenario involving transaction tickets can be constructed first; wherein, the system includes a kettle cluster, and a data warehouse including at least a data detail layer and an application data layer; at the same time, a first script file responsible for data collection and storage and a second script file responsible for data calculation that are adapted to kettle task scheduling are also pre-deployed. In specific implementation, the transaction business data processing system based on the above structure can receive and respond to the request for generating the target transaction bill business curve, first determine the corresponding kettle task according to the first script file; and call the kettle cluster to obtain the target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks; then receive and integrate the target sample business data into the corresponding target business wide table according to the execution results of the first batch of kettle tasks; and store the target business wide table in the data detail layer of the data warehouse; then, according to the second script file, call the kettle cluster to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and perform the preset business curve generation operation according to the target business wide table to generate the corresponding target transaction bill business curve; finally, receive and store the target transaction bill business curve in the application data layer of the data warehouse according to the execution results of the second batch of kettle tasks, so that the user terminal can download and obtain it. Therefore, by jointly using a variety of related technical components, it can be better adapted to big data processing scenarios involving transaction bills with complicated processing procedures and large data processing volumes, and can automatically and efficiently complete relevant data processing based on transaction bills to obtain target transaction bill business curves that meet the requirements. Furthermore, based on the above-mentioned target transaction bill business curves, further transaction bill analysis and processing such as transaction bill valuation can be automatically and efficiently completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of this specification, the drawings required for use in the embodiments will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 It is a flowchart of a data processing method for a transaction ticket provided by an embodiment of the present specification;

[0050] Figure 2It is a schematic diagram of an embodiment of a data processing method for transaction tickets provided by an embodiment of this specification, in a scenario example;

[0051] Figure 3 It is a schematic diagram of an embodiment of a data processing method for transaction tickets provided by an embodiment of this specification, in a scenario example;

[0052] Figure 4 It is a schematic diagram of an embodiment of a data processing method for transaction tickets provided by an embodiment of this specification, in a scenario example;

[0053] Figure 5 It is a schematic diagram of an embodiment of a data processing method for transaction tickets provided by an embodiment of this specification, in a scenario example;

[0054] Figure 6 It is a schematic diagram of an embodiment of a data processing method for transaction tickets provided by an embodiment of this specification, in a scenario example;

[0055] Figure 7 It is a schematic diagram of an embodiment of a data processing method for transaction tickets provided by an embodiment of this specification, in a scenario example;

[0056] Figure 8 It is a schematic diagram of the structure of a server provided by an embodiment of this specification;

[0057] Fig. 9 It is a schematic diagram of the structure of a data processing device for transaction tickets provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0059] It should be noted that the user-related information and data involved in the embodiments of this specification are all information and data authorized by the user or fully authorized by relevant parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users or relevant parties to choose to authorize or refuse.

[0060] It should also be noted that in the embodiments of this specification, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0061] See also Figure 1 As shown, the embodiment of this specification provides a data processing method for transaction notes. Among them, the method is specifically applied to the server side of the transaction business data processing system. When implemented specifically, the method may include the following contents:

[0062] S101: receiving a request for generating a target transaction bill service curve initiated by a user terminal; wherein the request for generating a target transaction bill service curve carries at least identification information of a target transaction bill package;

[0063] S102: responding to a request for generating a target transaction bill business curve, determining a corresponding kettle task according to the first script file; and calling a kettle cluster to obtain target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks;

[0064] S103: receiving and integrating the target sample business data into a corresponding target business wide table according to the execution results of the first batch of kettle tasks; and storing the target business wide table into the data detail layer of the data warehouse;

[0065] S104: According to the second script file, the kettle cluster is called to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and a preset business curve generation operation is performed according to the target business wide table to generate a corresponding target transaction bill business curve;

[0066] S105: Receive and based on the execution results of the second batch of kettle tasks, when it is determined that a target transaction bill business curve that meets the requirements is generated, store the target transaction bill business curve in the application data layer of the data warehouse; wherein the user terminal is connected to the application data layer of the data warehouse through a corresponding data interface, and obtains the target transaction bill business curve through the data interface.

[0067] The target transaction bill business curve can be specifically understood as a curve image that can reflect the changes in relevant data of transactions involving transaction bills and can provide a reference for users' decisions. For example, a transaction bill yield curve, or a transaction bill market price change curve, etc. The following mainly takes the transaction bill yield curve (commercial bill yield curve) as an example for specific explanation.

[0068] The above-mentioned target sample business data can be specifically understood as transaction business data with smaller errors, higher reference value, better effects, obtained after data conversion, data screening, etc. based on the initial transaction business provided by the data source, and suitable for constructing the corresponding target transaction bill business curve.

[0069] The above-mentioned initial transaction business data can be specifically understood as transaction business data involving the target transaction bill collected through a data source (e.g., a transaction system, etc.). Usually, the amount of initial transaction business data is relatively large, and there are often some abnormal data that are not suitable for constructing the target transaction bill business curve.

[0070] Among them, the above-mentioned transaction notes can be specifically understood as a kind of negotiable and tradable valuable certificate, such as bills of exchange, bank drafts and checks.

[0071] Taking a bill of exchange as an example, the above-mentioned bill of exchange can specifically be a note issued by the drawer, entrusting the payee to unconditionally pay a certain amount of resource data (for example, funds, etc.) to the payee or holder on a specified date.

[0072] Furthermore, the above-mentioned transaction bills may specifically include at least one of the following: bank bills, commercial bills, financial bills, etc.

[0073] Specifically, the above-mentioned silver bill can be understood as a bill of exchange issued by a bank with the bank as the acceptor. The above-mentioned financial bill can be understood as a bill of exchange issued by a financial company with the financial company as the acceptor. The above-mentioned commercial bill can be understood as a bill of exchange issued by an enterprise with the enterprise as the acceptor.

[0074] It should be noted that the above-mentioned different types of bills of exchange may differ in liquidity, risk, profitability (for example, discount rate), etc. due to different acceptors.

[0075] Of course, it should be noted that the transaction notes listed above are only a schematic illustration. In specific implementation, according to specific application scenarios and processing requirements, the transaction notes may also include other types of notes. This specification does not limit this.

[0076] In addition, transactions involving trading bills may also correspond to different business categories, such as discount business or rediscount business.

[0077] Specifically, the above-mentioned discounting business can be understood as the bill behavior in which the holder of the transaction bill transfers the relevant rights of the transaction bill to the bank by paying a certain interest before the bill matures in order to obtain relevant resource data such as funds.

[0078] The above-mentioned rediscount business can be understood as the act of a bank, in certain circumstances, transferring discounted but unexpired transaction bills to other banks or related discounting institutions for discounting in order to obtain relevant resource data such as funds.

[0079] It should be noted that, for transactions corresponding to the above-mentioned different business categories, due to differences in the participants and data processing procedures involved, there will also be differences in the analysis and processing of relevant data involving transaction documents.

[0080] Among them, the above-mentioned transaction business data processing system can be specifically understood as a transaction service platform that is pre-constructed by using a kettle cluster, various types of storage structures (including: Greenplum database, MYSQL database, Redis cache), and pre-prepared various types of script files and other technical components. It supports efficient and automatic generation of relevant transaction bill business curves, as well as calculation of indicators about transaction bills based on the transaction bill business curves (for example, valuation calculation of transaction bills, prediction of repurchase rate of transaction bills, etc.). Data processing system.

[0081] For details, see Figure 2 As shown, the transaction business data processing system may include at least a data middle platform module and a service application module. The data middle platform module may be connected to multiple different data sources such as a transaction system, ECDS (Electronic Commercial Draft System) and the like through corresponding data interfaces, so as to collect and obtain relatively comprehensive transaction business data involving transaction bills.

[0082] Specifically, the above-mentioned middle platform module may include two parts: a data warehouse and a cache layer.

[0083] The cache layer is connected to multiple data sources through corresponding data interfaces, and is used to collect, cache, back up, and clean the initial transaction business data directly from the data source. The cache layer can specifically be a cache structure based on Redis.

[0084] The data warehouse is connected to the cache layer and is used to store, convert, filter, calculate and process the initial transaction business data from the cache layer. The data warehouse can be a database based on the MPP architecture. Specifically, the data warehouse can be a Greenplum database.

[0085] Specifically, the Greenplum database can be understood as a shared-nothing MPP-based database that can run on a virtualized x86 server.

[0086] Furthermore, the above-mentioned MPP (Massively Parallel Processing) architecture can specifically be a massively parallel processing system, which is composed of many loosely coupled processing units (e.g., SMP nodes), and the CPU in each processing unit can have its own independent processing resources, such as bus, memory, hard disk, etc. That is, the CPU of each processing unit cannot access the memory of other processing units, and the information interaction between processing units can be achieved through the node interconnection network. Based on the above architecture, related transaction business data can be stored in multiple processing units to achieve localization and distributed computing of transaction business data; and each processing unit has its own independent processing resources, which are not shared with other processing units, and can support the processing of multiple different tasks at the same time, effectively improving the overall processing efficiency; at the same time, even if a processing unit fails, it will not affect other processing units, ensuring the stability and reliability of the overall operation.

[0087] For details, see Figure 2 As shown, the above data warehouse can be specifically divided into: operational data layer (Operational Data Store, ODS layer), data detail layer (Data Warehouse Detail, DWD layer), and application data layer (Application Data Store, ADS layer).

[0088] The operation data layer is connected to the cache layer and is used to store the initial transaction business data output from the cache layer after data cleaning.

[0089] The data detail layer is connected to the operation data layer, and is used to store sample business data output from the operation data layer and after data conversion and other related processing, which can be used for calculations on transaction tickets.

[0090] The above-mentioned application data layer is connected to the data detail layer, and is used to store the calculation result data obtained after the relevant calculation output from the data detail layer, such as the transaction bill business curve, and the transaction bill valuation result based on the transaction bill business curve and other related benchmark indicator data involving the transaction bill, or the basic result data such as intraday statistics involving the transaction bill. In addition, the above-mentioned application data layer can also store statistical data products involving the transaction bill that are further processed based on the above-mentioned basic result data and combined with the actual needs of users, such as daily statistics, monthly statistics, in-depth analysis reports, etc., to meet the diverse needs of users.

[0091] For details, see Figure 2 As shown, the service application module may at least include a business management database (eg, Mysql) and a front-end application layer (eg, Application).

[0092] The above-mentioned business management database can specifically be a Mysql database (abbreviated as Mysql), which can store user data, authority data, monitoring rules, configuration parameters and other data that are not directly related to transaction business data, and is used to support the system for user management, authority management, system monitoring, and support parameter settings of user terminals and management terminals.

[0093] The Mysql database may be a relational database management system (RDBMS) that manages or operates data using SQL (Structured Query Language).

[0094] The front-end application layer can be a JAVA WEB front-end, which can be connected to the user terminal and the management terminal to provide corresponding services to the user terminal and the management terminal. In addition, the front-end application layer can also provide a corresponding data interface to the user terminal and the management terminal, and the data interface can be connected to the application data layer of the data warehouse, so that the user terminal and the management terminal can download and obtain the required data from the application data layer through the data interface.

[0095] The user terminal and management terminal may specifically include a front end applied to the user or the administrator of the transaction service platform, which can realize functions such as data collection and data transmission. Specifically, the user terminal and management terminal may be electronic devices such as desktop computers, tablet computers, laptop computers, smart phones, etc. Alternatively, the user terminal and management terminal may also be software applications that can run on the above electronic devices.

[0096] A kettle cluster can also be deployed in the above-mentioned transaction business data processing system. Specifically, the above-mentioned kettle can be understood as an ETL (Extract-Transform-Load, i.e., the process of data extraction, transformation, and loading) tool. The cluster deployed with kettle can support efficient calling and management of different types of data storage structures, and support parallel kettle tasks to utilize related data in the data storage structure to complete batch task processing of large amounts of data.

[0097] The above Kettle cluster can be specifically composed of a master carte server and multiple slave carte servers, which is similar to the master-slave structure. The difference is that the master handles specific tasks, while the master carte server is only responsible for task distribution and collecting task execution results. After receiving the request, the master carte server can divide the task into multiple parts and assign it to the slave carte server for execution. After the slave carte server completes the execution, the result is assigned to the master carte server for aggregation, and then the master carte server returns the final result. Using Kettle cluster has the following advantages: 1) Support multi-server operation, speed up processing speed, and it is more obvious for operations with large amounts of data; 2) Prevent single point failure, and other servers can still run after one server fails.

[0098] In addition, a preset script file library is also deployed in the above-mentioned transaction business data processing system. Among them, the preset script file library can pre-prepare script files for different processing flows and adapted to the kettle cluster, including: a first script file, a second script file, a third script file, and a fourth script file. The above-mentioned script files may contain processing rules for different task flows and corresponding task instructions.

[0099] Specifically, the first script file may be a shell script for data collection and storage, which is used to control the execution of the corresponding first batch of kettle tasks to complete the conversion of the initial transaction business data in the cache layer to obtain sample business data that meets the requirements.

[0100] The above-mentioned second script file can specifically be a Python script for data calculation, which is used to control the execution of the corresponding second batch of kettle tasks to complete the calculation and processing of sample business data to obtain the business curve required by the user, such as transaction bill business curve.

[0101] The third script file can be used to control the execution of the corresponding third batch of kettle tasks to complete the collection of initial transaction business data from the data source and perform corresponding preprocessing on the initial transaction business data. Specifically, the third script file can also be a shell script.

[0102] The fourth script file can be used to control the execution of the corresponding fourth batch of kettle tasks to further calculate the transaction bill business curve to obtain result data such as the valuation result of the transaction bill, the repurchase rate prediction result, etc. Specifically, the fourth script file can also be a python script.

[0103] The shell script is a scripting language used for automated system management tasks, and the python script is a scripting language suitable for powerful libraries and a wide range of application calculations. Accordingly, the first script file and the third script file constructed using the shell script are relatively more suitable for simple processing and storage management of the accessed data; the second script file and the fourth script file constructed using the python script are relatively more suitable for efficiently calling related data and performing various and complex operations using the related data.

[0104] Specifically, the above script file can not only reflect the specific kettle tasks of the corresponding data processing flow, but also includes two sub-scripts for kettle task supervision, namely: transformation script and job script. Among them, the transformation script is used to complete the relevant data processing, as well as the collection and feedback of data processing results; the job script is used to realize the control of the entire workflow of the corresponding data processing flow, as well as the collection and feedback of the workflow working status. Accordingly, when the script file is called to run, the data processing results and workflow working status can be obtained at the same time through the above two sub-scripts; and the above two types of data are combined to obtain the execution results of the corresponding kettle task; and then according to the execution results of the kettle task, accurately judge whether the corresponding data processing flow has been executed; or, according to the execution results of the kettle task, make corresponding feedback adjustments to the execution of the kettle task of the kettle cluster to ensure that the data processing flow can be executed smoothly and accurately.

[0105] Before specific implementation, a preset script file library can be constructed in the following manner: historical processing records of data processing related to transaction tickets on the transaction service platform are collected; according to different data processing processes, the above historical processing records are split into multiple processing record data groups; wherein each processing record data group corresponds to a data processing process, and contains multiple historical processing records of the corresponding data processing processes; multiple processing record data groups are clustered to obtain multiple groups of processing features; then, in combination with the calling protocol of the kettle cluster, processing rules corresponding to different data processing processes are sorted out according to the multiple groups of processing features; then, according to the processing rules and in combination with the data processing process, specific kettle tasks are determined, and the preset instruction statement template is used to configure the task instructions corresponding to each kettle task, thereby obtaining multiple script files corresponding to different data processing processes (including a first script file, a second script file, a third script file, and a fourth script file); multiple script files are combined to obtain a preset script file library; and a corresponding relationship between script files and data processing processes is established.

[0106] In specific implementation, the above data warehouse can use the Master / Standby mechanism to perform master node fault tolerance management. Specifically, the Master node and the Standby node are respectively associated with all storage nodes in the data warehouse. Among them, each storage node is respectively provided with 2 primany segments and 2 mirror segments. In this way, based on the above mechanism, when one of the storage nodes fails due to some unexpected circumstances, the data warehouse can continuously and stably provide data storage and management services by performing master node fault tolerance management.

[0107] The above business management database can adopt a dual-master mutual backup deployment strategy, that is, two servers using Master-Master replication are both masters and slaves of the other server. In this way, any changes made by either party will be applied to the database of the other party through replication. In addition, the above business management database can also use druid to perform necessary load balancing on the two Mysql data sources, thereby greatly improving the high availability of the business management database while ensuring the overall data processing performance of the system.

[0108] Based on the above transaction business data processing system, when a user needs to obtain a corresponding target transaction bill business curve, the user can use the user terminal held by the user to initiate a request for generating the corresponding target transaction bill business curve to the server of the transaction business data processing system.

[0109] The server can respond to the generation request and, according to the current data processing flow, first retrieve the first script file from the preset script file library; and determine the corresponding kettle task according to the first script file. Among them, the above-mentioned kettle task can be a task that covers the entire data processing flow of generating the target transaction bill business curve. That is, the above-mentioned kettle task can include multiple kettle tasks corresponding to different data processing flows when generating the target transaction bill business curve. Specifically, the above-mentioned kettle task includes at least the first batch of kettle tasks and the second batch of kettle tasks.

[0110] In the specific implementation, first, the server can call the kettle cluster to execute the first batch of kettle tasks in parallel to obtain the corresponding target sample business data; at the same time, the server receives and monitors the execution status of the current data processing flow based on the execution results of the first batch of kettle tasks; after determining that the current data processing flow has been completed, the above target business data can be integrated into a target business wide table; and then the target business wide table is stored in the data detail layer of the data warehouse.

[0111] Next, the server can call the kettle cluster and continue to execute the second batch of kettle tasks in parallel, read and use the above-mentioned target business wide table to perform preset business curve generation operations to generate the target transaction bill business curve; at the same time, the server receives and monitors the execution status of the current data processing flow according to the execution results of the second batch of kettle tasks; after determining that the current data processing flow has been completed, the target transaction bill business curve can be stored in the application data layer of the data warehouse, and a data interface can be established between the user terminal and the target transaction bill business curve in the application data layer of the data warehouse.

[0112] The server may send a download prompt regarding the target transaction ticket to the user terminal; wherein the download prompt carries at least an interface identifier of the data interface and a download credential.

[0113] The user terminal can receive and respond to the download prompt, determine the corresponding data interface according to the corresponding interface identifier, and then based on the data interface, use the download credential to connect to the application data layer of the data warehouse to download and obtain the target transaction bill business curve.

[0114] Specifically, the user terminal can access the application data layer of the data warehouse through the corresponding data interface according to the corresponding interface identifier; and initiate a download request for the target transaction bill business curve in the application data layer; wherein the download request carries at least the user's identity identifier (for example, the user's user name, user number, etc.), identity credentials (for example, the user's user password login), and download credentials.

[0115] Correspondingly, after receiving the download request, the server can verify the identity of the user by querying the business management database based on the identity identifier and identity certificate; if it is determined that the identity verification is passed, the server can also obtain the user's authority data by querying the business management database based on the user's identity identifier; and verify the user's authority based on the user's authority data and the security level of the target transaction bill to determine whether the user has the authority to download the target transaction bill business curve; if it is determined that the user has the authority to download the target transaction bill business curve, the target transaction bill business curve indicated by the download request is matched with the data interface used by the user to determine whether the two match; at the same time, the validity of the download certificate is verified to determine whether the download certificate is authentic and valid; only when it is determined that the target transaction business curve matches the data interface used by the user and the download certificate is authentic and valid, will the download request be responded to, and the user terminal will be allowed to download and obtain the target transaction bill business curve through the data interface.

[0116] Therefore, by jointly using a variety of related components, it can better adapt to big data processing scenarios involving transaction bills with complicated processing procedures and large data processing volumes, efficiently and accurately complete relevant data processing based on transaction bills, and obtain the target transaction bill business curve that meets the requirements.

[0117] In some embodiments, the first script file may specifically include a shell script regarding data collection and storage, the second script file may specifically include a python script regarding data operations, and the data warehouse may specifically include a database based on an MPP architecture, etc.

[0118] In some embodiments, see Figure 3 As shown, the above calling of the kettle cluster obtains the target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks. The specific implementation may include the following contents:

[0119] S1: Call the kettle cluster, execute the corresponding first batch of kettle tasks, query the data source table in the operation data layer of the data warehouse according to the identification information of the target transaction bill package, and obtain the initial sample business data;

[0120] S2: Call the kettle cluster to detect and remove the first type of abnormal data in the initial sample business data by executing the corresponding first batch of kettle tasks, and obtain the intermediate sample business data; and store the intermediate sample business data in a temporary table;

[0121] S3: Call the kettle cluster to detect and remove the second type of abnormal data in the temporary table by executing the corresponding first batch of kettle tasks to obtain the corresponding target sample business data.

[0122] Among them, in the operation data layer of the above-mentioned data warehouse, the relevant transaction business data can be pre-cleaned, sorted and classified, and then stored in different data source tables such as institution information table, bill information table, transaction and bill association table, discount behavior table, and rediscount behavior table.

[0123] Correspondingly, during the specific implementation, the business category involved in the target transaction bill can be determined first. When it is determined to be a discount business, the kettle cluster can be called, and by executing the corresponding first batch of kettle tasks, according to the identification information of the target transaction bill package, the institution information table, bill information table, transaction and bill association table, and discount behavior table in the operation data layer of the data warehouse are queried to obtain the corresponding transaction business data as the initial sample business data. When it is determined to be a rediscount business, the kettle cluster can be called, and by executing the corresponding first batch of kettle tasks, according to the identification information of the target transaction bill package, the institution information table, bill information table, transaction and bill association table, and rediscount behavior table in the operation data layer of the data warehouse are queried to obtain the corresponding transaction business data as the initial sample business data.

[0124] Furthermore, the kettle cluster is called again to execute the corresponding first batch of kettle tasks to detect and remove the first type of abnormal data and the second type of abnormal data in the initial sample business data in turn to obtain the target sample business data that meets the requirements.

[0125] Based on the above embodiment, the first script file can be used to call the kettle cluster to execute the corresponding first batch of kettle tasks, so as to efficiently and accurately complete the data processing process involving data conversion and screening in a parallel manner to obtain target sample business data that meets the requirements.

[0126] In some embodiments, the second type of abnormal data may specifically include: transaction business data whose deviation does not meet the requirements, etc.;

[0127] The first type of abnormal data may at least include: transaction data whose amount ratio does not meet the requirements, transaction data whose term difference does not meet the requirements, etc.;

[0128] Furthermore, when the business category of the target transaction bill business curve is rediscount business, the first type of abnormal data may also include: internal transaction business data, back-to-back transaction business data, bridge transaction business data, etc.

[0129] In specific implementation, the target transaction bill involved and the bill information of the target transaction bill can be determined based on the target sample business data and the relevant data source table. The bill information can specifically include at least one of the following: bill denomination, acceptor information, drawee information, bill category, bill maturity date, discounter information, etc.

[0130] The bill denomination, acceptor information and bill category in the bill information of the target transaction bill are then combined to determine the characteristic combination type of the target transaction business data; and according to the characteristic combination type of the target transaction business data, the preset curve basic table is queried to obtain the corresponding target curve basic parameters; wherein the preset curve basic table stores multiple preset curve basic parameters corresponding to the multiple characteristic combination types; according to the target curve basic parameters, it is detected whether there is a first type of abnormal data and / or a second type of abnormal data in the initial sample business data.

[0131] Among them, the preset curve basic table may include multiple preset curve basic parameters; wherein each preset curve basic parameter corresponds to at least one feature combination type, and each preset curve basic parameter includes the critical reference value of the relevant parameters of the transaction bill business curve of the corresponding feature combination type, for example, the amount ratio threshold, the term difference threshold, the spread critical threshold, the standard deviation multiple of the deviation, etc.

[0132] In specific implementation, different business categories can be distinguished first, the first type of abnormal data in the initial sample business data can be detected and eliminated to obtain the corresponding intermediate sample business data; and the intermediate sample business data can be stored in a temporary table; then based on the temporary table, the second type of abnormal data can be detected and eliminated, so that the target sample business data that meets the requirements can be obtained more efficiently.

[0133] In some embodiments, see Figure 4 As shown, when the method is implemented specifically, it may also include the following contents:

[0134] S1: According to the preset protocol rules, the kettle cluster is called to collect initial transaction business data from multiple data sources by executing the third batch of kettle tasks and using the corresponding secure data channel; and the initial transaction business data is stored in the cache layer;

[0135] S2: Call the kettle cluster to read the initial transaction business data from the cache layer by executing the third batch of kettle tasks; and preprocess the initial transaction business data to obtain preprocessed transaction business data;

[0136] S3: Call the kettle cluster to execute the third batch of kettle tasks to store the pre-processed transaction business data into the corresponding data source table in the operation data layer of the data warehouse.

[0137] The above preprocessing may specifically include operations such as data cleaning, missing value completion, and unified format. The above preset protocol rules may specifically be data interaction protocol rules between different data sources.

[0138] During specific implementation, the kettle cluster can be called according to the preset protocol rules, and the data status of multiple different data sources can be monitored simultaneously by executing the third batch of kettle tasks; and when data updates are found, the initial transaction business data of multiple data sources can be collected using the corresponding secure data channels; and the initial transaction business data can be stored in the cache layer as the original data; and the collection log of the cache layer can be synchronously updated.

[0139] Next, the kettle cluster can be called to execute the third batch of kettle tasks to read and preprocess the initial transaction business data to obtain the preprocessed transaction business data, and then sort and classify the preprocessed transaction business data and store them in the corresponding data source tables in the operation data layer of the data warehouse; at the same time, the preprocessed transaction business data will be backed up to create corresponding backup files to facilitate subsequent backtracking queries.

[0140] Based on the above embodiments, the cache layer can be used to collect and process a large amount of initial transaction business data from multiple different data sources in a relatively efficient and stable manner.

[0141] In specific implementation, the above cache layer can adopt the sentinel mode to achieve high availability of Redis. Specifically, based on the sentinel mode, the above cache layer can include Master nodes and Slaver nodes. Among them, the Master node is used for writing, and the Slaver node is used for reading, and cannot be written or modified; and one Master node can correspond to multiple Slaver nodes. After the Master node writes data, it can be synchronized to the Slaver node by asynchronous replication, but there will be basically no delay. In this case, if the Slaver node fails, it will not affect the entire storage structure. Only the Slaver node cannot read, but the reading and writing of the user terminal and the management terminal are not affected, and they can be added after restarting. The Sentinel sentinel mechanism is used for the Master node to monitor the Master node. When it fails, other Slaver nodes can be automatically switched to the new Master node. When the original Master node restarts, it automatically becomes the new Master node. Therefore, the above cache layer can be used to efficiently and stably collect initial transaction business data with huge data volumes from different data sources.

[0142] In some embodiments, see Figure 5 As shown, for the target transaction bill business curve whose curve type belongs to the basic curve, the preset business curve generation operation is performed according to the target business wide table to generate the corresponding target transaction bill business curve. When implemented specifically, the following contents may be included:

[0143] S1: Call the kettle cluster to determine multiple key deadlines by executing the corresponding second batch of kettle tasks; and use the multiple key deadlines to divide the target sample business data in the target business wide table into multiple intervals;

[0144] S2: Call the kettle cluster to execute the corresponding second batch of kettle tasks, use the target sample business data in each interval, and determine the target business value of each key deadline through the first linear regression;

[0145] S3: Call the kettle cluster to execute the corresponding second batch of kettle tasks, use the data of key deadlines to perform curve fitting, and determine the target business values ​​of other deadlines;

[0146] S4: Call the kettle cluster to execute the corresponding second batch of kettle tasks, and generate the corresponding target transaction bill business curve according to the target business value of the key deadline and the target business value of other deadlines.

[0147] Among them, the above-mentioned key deadlines can be determined by learning and sorting out the historical transaction bill business curves and combining them with the transaction characteristics of the target transaction bills.

[0148] Based on the above embodiment, the kettle cluster can be called to execute the corresponding second batch of kettle tasks, and the first linear regression and curve fitting can be performed in sequence to obtain a target transaction bill business curve with a high reference value and a relatively small error.

[0149] In specific implementation, the target sample business data in each interval can be used to perform linear regression based on iterative repeated weighted least squares (IRLS) to achieve the first linear regression, so as to accurately determine the target business value of each key deadline.

[0150] Among them, the iterative repeated weighted least squares method can be specifically understood as an iterative algorithm for solving specific optimization problems, which approximates the solution of the original problem by iteratively solving a series of weighted least squares problems. Specifically, in each iteration, weights are calculated based on the current estimated solution, and these weights are used to solve a new least squares problem to update the estimate of the solution. This method is particularly useful when dealing with data containing outliers or noise, because it can reduce the impact of outliers on the final result by adjusting the weights.

[0151] In specific implementation, the Hermite interpolation model can be used to efficiently determine the intermediate business values ​​of other deadlines by performing curve fitting based on the target business values ​​of key deadlines.

[0152] Specifically, taking the commercial bill yield curve as the target transaction bill business curve as an example, please refer to Figure 6 As shown. The target business value may specifically be a rate of return. In the figure, the horizontal axis represents the term, the vertical axis represents the rate of return, and the circles represent sample scatter points generated based on the relevant target sample business data.

[0153] Accordingly, in specific implementation, the target sample business data in each interval can be used in the following manner to determine the target business value of each key deadline through the first linear regression:

[0154] First, for multiple sample scatter points determined based on the target sample business data in the current interval, an iterative weighted least squares method can be used to construct a straight line function y(x)=a*x+b; then, by solving the parameters a and b, the following formula (1) obtains the minimum value:

[0155]

[0156] Where X(i) represents the weighted average remaining term of the product numbered i, Y(i) represents the discount rate or rediscount rate of the product numbered i, a and b are the parameters to be solved, and n is the total number of sample scatter points in the current interval.

[0157] In the formula, and, s = MAD(δ) / 0.6745.

[0158] Among them, δ is the residual obtained by the least square method, t is the adjustment constant, which is set to 1.345 by default, and MAD(δ) is the median absolute deviation of the residual, which can be calculated according to the following formula: MAD(δ) = median(|δ-median(δ)|). Among them, median is the median.

[0159] Then, we can use iterative calculation to find a set of weights w i The initial value of w0 can be obtained by the least squares method. The weight w is obtained by the above method. i After that, the final parameters a and b can be obtained according to the following formula:

[0160]

[0161] Thus, the straight-line function y(x)=a*x+b can be determined; then the straight-line function is used to calculate the corresponding discount rate or rediscount rate according to the remaining term of each key term point as the intermediate business value; then the intermediate business value is substituted into the conversion model of the corresponding target business value, and the yield of the corresponding key term point is calculated and determined as the target business value.

[0162] In specific implementation, the following formula can be used as a conversion model for the target business value to calculate the target business value at other deadlines:

[0163]

[0164] Among them, R YTM is the yield to maturity (target business value), r is the discount rate (intermediate business value), and t is the term.

[0165] Furthermore, in the specific implementation, curve fitting based on the Hermite interpolation model can be performed according to the data of the key term points (for example, the discount rate of the key term points) to determine the intermediate business data of other term points; and then the conversion model of the target business value can be used to calculate the target business values ​​of other term points according to the intermediate business values ​​of other term points.

[0166] Specifically, the discount rate (or rediscount rate) at other term points can be determined by curve fitting based on the Hermite interpolation model according to the discount rate at the key term point in the following manner.

[0167] Let 0 = x1 < x2 < ... < x n =6, given (x i ,y i )(x i+1 ,y i+1 ), i∈[1,n]. i and i Corresponding to the key term point, the term and interest rate of other term points, find any x i ≤x≤x n When the corresponding y(x) is obtained, the monotone cubic Hermite polynomial interpolation model can be used for fitting construction according to the following formula:

[0168] y(x)=y i H1+y i+1 H2+d i H3+d i+1 H4

[0169] in,

[0170] Among them, d j =y'(x),j=i,i+1, representing the association slope.

[0171] During the specific calculation, the associated slope can be initially assigned according to the following rules:

[0172] definition Where k = 1, ..., n-1, n;

[0173] When k∈[2,n-1],

[0174] When k = 1,

[0175] When k = n,

[0176] Based on the above method, we can finally determine that y(x)=y i H1+y i+1 H2+d i H3+d i+1 H4. Substituting the terms of other term points into the above Hermite interpolation model, the intermediate business values ​​of other term points can be determined; and the target business values ​​of other term points can be calculated based on the intermediate business values ​​of other term points using the conversion model of the target business value.

[0177] Finally, the target business value of the key term point and the target business value of other term points can be used together to generate the target transaction bill business curve.

[0178] In some embodiments, for the post-to-cash business, when the curve type of the target transaction bill business curve is a derivative curve, after calling the kettle cluster, executing the corresponding second batch of kettle tasks, using the target sample business data in each interval, and determining the target business value of each key deadline through the first linear regression, the method may further include the following when implemented:

[0179] S1: According to the target business value of the key deadline point of each interval and the target transaction business data in each interval, the interval regression line segment of each interval is determined through the second linear regression;

[0180] S2: Calculate and use the slope parameters of each key deadline point according to the interval regression line segments of each interval, and determine the first intermediate business value of each key deadline point through a first linear regression;

[0181] S3: According to the reference curve parameters and the reference threshold parameters, the target business value of each key deadline is obtained by performing result control on the first intermediate business value of each key deadline;

[0182] S4: Using the data of key deadlines to perform curve fitting, determine the target business values ​​of other deadlines;

[0183] S5: Generate a corresponding target transaction bill business curve based on the target business value at the key term point and the target business value at other term points.

[0184] Based on the above embodiments, for the discounting business, the kettle cluster can be called to execute the corresponding second batch of kettle tasks, and the first linear regression, the second linear regression, and the curve fitting are performed in sequence to obtain a target transaction bill business curve with relatively high reference value, relatively small error, and belonging to the derivative curve type.

[0185] During specific implementation, the target business values of the key maturity points in each interval and the target transaction business data in each interval can be utilized to perform least squares linear regression to achieve the second linear regression and determine the interval regression line segments in each interval.

[0186] During specific implementation, the result control can be performed on the first intermediate business value at the key maturity point according to the credit spread between the derivative curve and the basic curve of the corresponding same target transaction bill, as well as the upper and lower limits of the target business value at the key maturity point, so as to obtain the corresponding target business value.

[0187] Specifically, the result control can be carried out according to the following rules:

[0188] First, let standar(i) represent the value of a certain key maturity point of the derivative curve (for example, the second intermediate business value), standar(1) represent the value of the corresponding maturity point of the basic curve, diff(i) represent the spread between this maturity point and the corresponding maturity point of the basic curve, upper(i) represent the upper limit amplitude of this maturity point, and lower(i) represent the lower limit amplitude of this maturity. standar(i)0 represents the initial value of this maturity point calculated by the least squares method (for example, the first intermediate business value).

[0189] If: standar(i)0 <= standar(1) + diff(i) + upper(i) and standar(i)0 >= standar(1) + diff(i) - lower(i); then standar(i) = standar(i)0.

[0190] If: standar(i)0 > standar(1) + diff(i) + upper(i); then standar(i) = standar(1) + diff(i) + upper(i).

[0191] If: standar(i)0 < standar(1) + diff(i) - lower(i); then standar(i) = standar(1) + diff(i) - lower(i).

[0192] According to the above method, the second intermediate business value of each key deadline can be determined; and then the target business value of the key deadline can be obtained by processing the second intermediate business value of the key deadline using the conversion model of the target business value.

[0193] In some embodiments, see Figure 7 As shown, after the target transaction bill business curve is stored in the application data layer of the data warehouse, the method may further include the following contents when it is specifically implemented:

[0194] S1: receiving a target transaction note valuation request initiated by a user terminal; wherein the target transaction note is a transaction note involved in a target transaction note business curve;

[0195] S2: respond to the target transaction bill valuation request, call the kettle cluster according to the fourth script file, use the target transaction bill business curve, and combine with the relevant data source table to determine the valuation result of the target transaction bill;

[0196] S3: Store the valuation result of the target transaction note into the application data layer of the data warehouse.

[0197] Based on the above embodiment, based on the transaction business data processing system, the generated target transaction bill business curve can also be used for further in-depth analysis and processing to obtain result data with relatively higher reference value, such as valuation results of the target transaction bill.

[0198] Specifically, the server can respond to the target analysis and prediction request initiated by the user terminal, first call the pre-trained curve image recognition model to process the target transaction bill business curve, and obtain the target analysis result that can reflect the changes in relevant data (for example, yield) of the relevant transactions involving the target transaction bill indicated by the target transaction bill business curve at various key time points in history, as well as the reasons for the changes; further, combine the target analysis result and the target transaction bill business curve to obtain the combined data; and use the pre-trained curve trend prediction model to process the above-mentioned combined data to obtain the target prediction result that can reflect the changes in relevant data of the relevant transactions involving the target transaction bill indicated by the target transaction bill business curve in a future time period (for example, the next month); then use the Sangshu target analysis result and the target combination result in combination to generate a corresponding target analysis and prediction report; and feed the target analysis and prediction report back to each user terminal.

[0199] In addition, the server can also use the above-mentioned target transaction business curve to further analyze and provide the user terminal with the repurchase rate prediction results of the target transaction bill, the risk prediction results of the target transaction bill, etc., to meet the user's diverse scenario needs.

[0200] In some embodiments, the fourth script file may call the kettle cluster, use the target transaction note business curve, and determine the valuation result of the target transaction note in combination with the relevant data source table in the following manner.

[0201] Taking the target transaction bill business curve as a commercial bill yield curve as an example, the value (or price) of the target transaction bill is determined according to the target transaction bill business curve; and then according to the reasonable value judgment rule, the transaction business data with effective value is screened out from the transaction business data involving the target transaction bill. In addition, the benchmark interest rate of the target transaction bill can also be determined according to the business curve of the target transaction bill.

[0202] During specific screening, for the undiscounted target transaction bills, it is possible to detect and determine whether the proportion of transaction bills in the same group (benchmark yield curve maintenance - same credit rating of the accepting bank) in the transaction business data is less than the set value (benchmark yield curve maintenance - proportion of the same bill grouping). If so, the value of the transaction business data is judged to be invalid; based on the upper and lower limits of the target transaction bill business curve configuration, determine whether the value is reasonable. If not, the value of the transaction business data is invalid.

[0203] When specifically testing and judging whether the value is reasonable, it can be judged whether the proportion of bills in the same group (grouped according to the content maintained by the "benchmark yield curve" and bills not covered by the maintenance content belong to the same group if their accepting bank credit rating) in the target transaction bill package is less than the set value. If so, the price is invalid. And judge whether the value is reasonable according to the content maintained by the "benchmark yield effective price range". If it is judged to be reasonable, the interest rate spread = discount rate - benchmark yield is calculated.

[0204] Then, by substituting the discounted yield of the valid transaction business data and the transaction bill into the yield curve, we can get the spread between the benchmark yields. Then, we aggregate these spreads to get the weighted spread, i.e. the spread for the day.

[0205] Among them, the overall rule of aggregation processing is: calculate the weighted average spread according to the total face value of the bill transaction (a single transaction is calculated only once, that is, if the price signals of two related bills come from the same transaction, they are calculated only once when calculating the weighted spread).

[0206] Specifically, the interest rate spread for the day can be calculated according to the following formula:

[0207]

[0208] Then, using the interest rate spread of the day, the average interest rate spread of a specified time period is calculated according to the following formula: Average interest rate spread = (interest rate spread of the day + ... + interest rate spread of the previous N-1 days) / N. Where N represents the number of days in the specified time period.

[0209] Then, the average interest rate spread and the corresponding formula can be used to calculate the price of the bill to be discounted in the future discount and the price of the bill to be rediscounted in the future rediscount, so as to realize the valuation of the target transaction bill and obtain the valuation result of the target transaction bill. Specifically, the valuation result of the target transaction bill (j) can be calculated according to the following formula:

[0210]

[0211] It should be noted that, in theory, the target transaction bill yield curve can be used to directly determine and use the discount rate and rediscount rate to value the target transaction bill. However, the above target transaction bill yield curve is usually only calculated for sample business data at key maturity points, while the yields at other non-key maturity points are actually obtained through fitting methods. This means that these valuations based on fitting may have certain deviations from the actual market level. Therefore, calculating and using the average spread for valuation here can make the valuation results relatively more reliable.

[0212] In specific implementation, after obtaining the average interest rate spread, anomaly detection can also be performed. Specifically, it can be detected whether the absolute value of the difference between the interest rate spread of the target transaction bill on the previous working day and the current day is greater than the threshold set based on the average interest rate spread. If it is greater, it is determined to be abnormal, the valuation process of the bill is stopped, and it is recorded as an abnormal bill.

[0213] In addition, during the specific implementation, if the target transaction bill business curve does not exist in advance, a reference bill of matching type can be determined from the reference bills of the existing business curve; and then the reference bill business curve can be used to determine the benchmark interest rate of the target transaction bill.

[0214] For the discounted target transaction bills, when screening, you can determine whether it is an internal transaction or a reverse transaction. If so, the value is invalid. Determine whether the target transaction bills in the target transaction bill package are less than the set value in the same group (grouped according to the content maintained by the "benchmark yield curve". Bills not covered by the maintenance content belong to the same group if their accepting bank credit rating and discount are the same). If so, the price is invalid. And determine whether the value is reasonable according to the content maintained by the "benchmark yield effective price range". If it is judged to be reasonable, calculate the interest rate spread = rediscount rate - benchmark yield.

[0215] The interest rate spread is then calculated using the following formula: interest rate spread = rediscount rate - benchmark yield; and the calculated interest rate spread is then used to calculate the interest rate spread for the day using the following formula:

[0216]

[0217] Furthermore, the average interest rate spread can be calculated: average interest rate spread = (interest rate spread on the current day + ... + interest rate spread on the previous N-1 days) / N.

[0218] Then, add the benchmark rate of return to the interest rate spread of the day to get the rediscount rate of return of the bill to be valued. Finally, substitute this rate of return into the following calculation to get the valuation result of the target transaction bill:

[0219]

[0220] It can be seen from the above that, based on the data processing method of transaction tickets provided in the embodiment of this specification, before the specific implementation, a transaction business data processing system suitable for the big data processing scenario involving transaction tickets can be constructed first; wherein, the system includes a kettle cluster, and a data warehouse including at least a data detail layer and an application data layer; at the same time, a first script file responsible for data collection and storage and a second script file responsible for data calculation that are compatible with kettle task scheduling are prepared. In specific implementation, the transaction business data processing system based on the above structure can receive and respond to the request for generating the target transaction bill business curve. First, according to the first script file, the corresponding kettle task is determined; and the kettle cluster is called to obtain the target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks; then the target sample business data is received and integrated into the corresponding target business wide table according to the execution results of the first batch of kettle tasks; and the target business wide table is stored in the data detail layer of the data warehouse; then according to the second script file, the kettle cluster is called to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and a preset business curve generation operation is performed according to the target business wide table to generate the corresponding target transaction bill business curve; finally, according to the execution results of the second batch of kettle tasks, when it is determined that the target transaction bill business curve that meets the requirements is generated, the target transaction bill business curve is stored in the application data layer of the data warehouse for acquisition by the user terminal. Therefore, by jointly using a variety of related components, it can better adapt to big data processing scenarios involving transaction bills with complicated processing procedures and large data processing volumes, and efficiently and accurately complete the relevant data processing based on transaction bills, and obtain a target transaction bill business curve that meets the requirements, as well as a variety of result data based on the target transaction bill business curve, such as valuation results of transaction bills, repurchase rate prediction results of transaction bills, etc., to meet the diverse scenario needs of users.

[0221] This specification embodiment provides a server, referring to Figure 8 The server includes a network communication port 801, a processor 802 and a memory 803, and the above structures are connected through internal cables so that each structure can perform specific data interaction.

[0222] The network communication port 801 may be specifically used to receive a request for generating a target transaction bill service curve initiated by a user terminal; wherein the request for generating a target transaction bill service curve carries at least identification information of a target transaction bill package.

[0223] The processor 802 can be specifically used to respond to a request for generating a target transaction bill business curve, determine a corresponding kettle task according to a first script file, and call a kettle cluster to obtain target sample business data related to the target transaction bill business curve by executing a first batch of kettle tasks; receive and integrate the target sample business data into a corresponding target business wide table according to the execution results of the first batch of kettle tasks; and store the target business wide table in the data detail layer of the data warehouse; call the kettle cluster according to a second script file to read the target business wide table from the data detail layer of the data warehouse by executing a second batch of kettle tasks; and perform a preset business curve generation operation according to the target business wide table to generate a corresponding target transaction bill business curve; receive and store the target transaction bill business curve in the application data layer of the data warehouse according to the execution results of the second batch of kettle tasks when it is determined that a target transaction bill business curve that meets the requirements is generated; wherein the user terminal is connected to the application data layer of the data warehouse through a corresponding data interface, and obtains the target transaction bill business curve through the data interface.

[0224] The memory 803 may be specifically used to store corresponding instruction programs, and related data such as the execution results of the first batch of kettle tasks, the execution results of the second batch of kettle tasks, etc.

[0225] Based on the above method, the relevant structural performance of the server can be effectively utilized, the data processing speed of the electronic device can be improved, and the data processing of the transaction ticket can be efficiently realized.

[0226] In this embodiment, the network communication port 801 can be a virtual port that is bound to different communication protocols so that different data can be sent or received. For example, the network communication port can be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for email data communication. In addition, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM, CDMA, etc.; it can also be a Wifi chip; it can also be a Bluetooth chip.

[0227] In this embodiment, the processor 802 may be implemented in any appropriate manner. For example, the processor may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc. This specification does not limit this.

[0228] In this embodiment, the memory 803 may include multiple levels. In a digital system, anything that can store binary data can be a memory; in an integrated circuit, a circuit with a storage function but no physical form is also called a memory, such as RAM, FIFO, etc.; in a system, a storage device with a physical form is also called a memory, such as a memory stick, TF card, etc.

[0229] The embodiment of the present specification also provides a computer-readable storage medium based on the data processing method of the transaction ticket, wherein the computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed, the following are achieved: receiving a request for generating a target transaction ticket business curve initiated by a user terminal; wherein the request for generating the target transaction ticket business curve carries at least identification information of a target transaction ticket package; responding to the request for generating the target transaction ticket business curve, according to a first script file, determining a corresponding kettle task; and calling a kettle cluster to obtain target sample business data related to the target transaction ticket business curve by executing the first batch of kettle tasks; receiving and, according to the execution results of the first batch of kettle tasks, converting the target sample business data into the target transaction ticket business curve; and The business data is integrated into a corresponding target business wide table; and the target business wide table is stored in the data detail layer of the data warehouse; according to the second script file, the kettle cluster is called to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and a preset business curve generation operation is performed according to the target business wide table to generate a corresponding target transaction bill business curve; receiving and based on the execution results of the second batch of kettle tasks, when it is determined that a target transaction bill business curve that meets the requirements is generated, the target transaction bill business curve is stored in the application data layer of the data warehouse; wherein the user terminal is connected to the application data layer of the data warehouse through a corresponding data interface, and obtains the target transaction bill business curve through the data interface.

[0230] In this embodiment, the storage medium includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a cache, a hard disk (HDD), or a memory card. The memory may be used to store computer program instructions. The network communication unit may be an interface for network connection communication set in accordance with the standard specified by the communication protocol.

[0231] In this embodiment, the functions and effects specifically implemented by the program instructions stored in the computer-readable storage medium can be explained in comparison with other implementations and will not be described in detail here.

[0232] The embodiment of the present specification also provides a computer program product, which at least includes a computer program. When the computer program is executed by a processor, the following method steps are implemented: receiving a request for generating a target transaction bill business curve initiated by a user terminal; wherein the request for generating a target transaction bill business curve carries at least identification information of a target transaction bill package; responding to the request for generating a target transaction bill business curve, determining a corresponding kettle task according to a first script file; and calling a kettle cluster to obtain target sample business data related to the target transaction bill business curve by executing a first batch of kettle tasks; receiving and integrating the target sample business data into corresponding target business wide table; and storing the target business wide table in the data detail layer of the data warehouse; according to the second script file, calling the kettle cluster to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and performing a preset business curve generation operation according to the target business wide table to generate a corresponding target transaction bill business curve; receiving and based on the execution results of the second batch of kettle tasks, when it is determined that a target transaction bill business curve that meets the requirements is generated, storing the target transaction bill business curve in the application data layer of the data warehouse; wherein the user terminal is connected to the application data layer of the data warehouse through a corresponding data interface, and obtains the target transaction bill business curve through the data interface.

[0233] See also Fig. 9 As shown, the embodiment of this specification also provides a data processing device for transaction tickets, which may specifically include the following structural modules:

[0234] The receiving module 901 may be specifically configured to receive a request for generating a target transaction bill service curve initiated by a user terminal; wherein the request for generating a target transaction bill service curve carries at least identification information of a target transaction bill package;

[0235] The first calling module 902 may be specifically used to respond to a request for generating a target transaction bill business curve, determine a corresponding kettle task according to the first script file, and call the kettle cluster to obtain target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks;

[0236] The integration module 903 may be specifically used to receive and integrate the target sample business data into a corresponding target business wide table according to the execution results of the first batch of kettle tasks; and store the target business wide table into the data detail layer of the data warehouse;

[0237] The second calling module 904 may be specifically used to call the kettle cluster according to the second script file to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and perform a preset business curve generation operation according to the target business wide table to generate a corresponding target transaction bill business curve;

[0238] The processing module 905 can be specifically used to receive and store the target transaction bill business curve into the application data layer of the data warehouse based on the execution results of the second batch of kettle tasks, when it is determined that a target transaction bill business curve that meets the requirements is generated; wherein the user terminal is connected to the application data layer of the data warehouse through a corresponding data interface, and obtains the target transaction bill business curve through the data interface.

[0239] In some embodiments, the first script file may specifically include a shell script regarding data collection and storage, the second script file may specifically include a python script regarding data operations, and the data warehouse may specifically include a database based on an MPP architecture.

[0240] In some embodiments, when the above-mentioned first calling module 902 is implemented, the kettle cluster can be called in the following manner to obtain the target sample business data related to the target transaction note business curve by executing the first batch of kettle tasks: calling the kettle cluster, querying the data source table in the operation data layer of the data warehouse according to the identification information of the target transaction note package by executing the corresponding first batch of kettle tasks, and obtaining the initial sample business data; calling the kettle cluster, detecting and eliminating the first type of abnormal data in the initial sample business data by executing the corresponding first batch of kettle tasks, and obtaining the intermediate sample business data; and storing the intermediate sample business data in a temporary table; calling the kettle cluster, detecting and eliminating the second type of abnormal data in the temporary table by executing the corresponding first batch of kettle tasks, so as to obtain the corresponding target sample business data.

[0241] In some embodiments, the second category of abnormal data may specifically include: transaction business data whose deviation does not meet the requirements; the first category of abnormal data may at least include: transaction business data whose amount ratio does not meet the requirements, and transaction business data whose maturity difference does not meet the requirements; when the business category of the target transaction bill business curve is rediscount business, the first category of abnormal data may also include: internal transaction business data, back-to-back transaction business data, and bridge transaction business data.

[0242] In some embodiments, when the device is implemented, it can also be used to: call the kettle cluster according to the preset protocol rules, and collect initial transaction business data from multiple data sources by executing the third batch of kettle tasks and using the corresponding secure data channel; and store the initial transaction business data in the cache layer; call the kettle cluster, and read the initial transaction business data from the cache layer by executing the third batch of kettle tasks; and pre-process the initial transaction business data to obtain pre-processed transaction business data; call the kettle cluster, and store the pre-processed transaction business data in the corresponding data source table in the operation data layer of the data warehouse by executing the third batch of kettle tasks.

[0243] In some embodiments, when the above-mentioned processing module 905 is implemented, the preset business curve generation operation can be performed according to the target business wide table in the following manner to generate the corresponding target transaction bill business curve: call the kettle cluster, and determine multiple key deadlines by executing the corresponding second batch of kettle tasks; and use the multiple key deadlines to divide the target sample business data in the target business wide table into multiple intervals; call the kettle cluster, and determine the target business value of each key deadline by using the target sample business data in each interval through the first linear regression by executing the corresponding second batch of kettle tasks; call the kettle cluster, and determine the target business value of other deadlines by using the data of the key deadlines for curve fitting by executing the corresponding second batch of kettle tasks; call the kettle cluster, and generate the corresponding target transaction bill business curve according to the target business value of the key deadline and the target business value of other deadlines by executing the corresponding second batch of kettle tasks.

[0244] In some embodiments, when the curve type of the target transaction bill business curve is a derivative curve, after calling the kettle cluster, executing the corresponding second batch of kettle tasks, and using the target sample business data in each interval to determine the target business value of each key deadline through a first linear regression, the above-mentioned processing module 905, when implemented, can also be used to: determine the interval regression line segment of each interval through a second linear regression according to the target business value of the key deadline of each interval and the target transaction business data in each interval; calculate and use the slope parameters of each key deadline according to the interval regression line segment of each interval, and determine the first intermediate business value of each key deadline through a first linear regression; obtain the target business value of each key deadline by performing result control on the first intermediate business value of each key deadline according to the reference curve parameters and the reference threshold parameters; determine the target business value of other deadlines by performing curve fitting using the data of the key deadline; generate the corresponding target transaction bill business curve according to the target business value of the key deadline and the target business value of the other deadline.

[0245] In some embodiments, after storing the target transaction note business curve in the application data layer of the data warehouse, the device can also be used to: receive a target transaction note valuation request initiated by a user terminal; wherein the target transaction note is a transaction note involved in the target transaction note business curve; respond to the target transaction note valuation request, call the kettle cluster according to the fourth script file, use the target transaction note business curve, and determine the valuation result of the target transaction note in combination with the relevant data source table; store the valuation result of the target transaction note in the application data layer of the data warehouse.

[0246] It should be noted that the units, devices or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described separately by functions divided into various modules. Of course, when implementing this specification, the functions of each module can be implemented in the same or more software and / or hardware, or the modules that implement the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0247] It can be seen from the above that the data processing device for transaction tickets provided in the embodiments of this specification can be better adapted to big data processing scenarios involving transaction tickets with complicated processing procedures and large data processing volumes through the combined use of multiple related components, and can efficiently and accurately complete relevant data processing based on transaction tickets to obtain a target transaction ticket business curve that meets the requirements.

[0248] In a specific scenario example, the transaction business data processing system and transaction bill data processing method provided in this specification can be used to implement the valuation of commercial bills. For the specific implementation process, please refer to the following content.

[0249] When valuing pre-discount bills (e.g., undiscounted target transaction bills), at the end of each day, the transaction business data processing system can be used to screen out related bills with similar credit types for each undiscounted bill according to the rules of the bill association module, and specify a benchmark yield curve for the newly added undiscounted bill according to the rules set by the "benchmark yield curve maintenance" function in the valuation management function. The screening rules of the association module can be found in Table 1.

[0250] Table 1

[0251]

[0252] Check whether the discount yield curve for the day has been calculated. If it has been calculated, determine the benchmark yield for each note based on the yield curve.

[0253] Based on the discount transactions of the day, find a list of bills with reasonable discount prices on that day.

[0254] For non-newly added undiscounted bills on the valuation day, first find the reasonable price of the bill based on the discount of each bill itself and the Class I related bills on the valuation day. If a reasonable price exists, calculate the weighted average spread according to the total discounted face value (because there may be multiple reasonable prices for multiple related bills). If a reasonable price does not exist for a bill itself and the Class I related bills, find the reasonable price of the Class II related bills of the bill. If it exists, calculate the weighted average spread according to the total discounted face value. If not, the bill will continue to use its own valuation spread from the previous business day.

[0255] For newly added undiscounted bills on the valuation day, first find the reasonable price of the bill based on the discount of each bill itself and the Class I related bills on the valuation day (whether the price is reasonable is judged by the "reasonable price judgment module"). If there is a reasonable price, calculate the weighted average spread based on the discounted total face value (because there may be multiple reasonable prices for multiple related bills). If there is no reasonable price for a bill itself and the Class I related bills, find the reasonable price of the Class II related bills of the bill. If there is a reasonable price, calculate the weighted average spread based on the discounted total face value. If there is no reasonable price for the Class II related bills on the valuation day, the weighted average valuation spread of the related bills on the previous working day will be used according to the priority of Class I-II (that is, the total face value of the related bills on the previous day).

[0256] Valuation is performed in the valuation module according to the valuation formula based on the benchmark yield and average spread determined in the above steps.

[0257] It should be noted that when valuing the undiscounted bills on day T, the existing bills on day T-1 should be valued first, and then the newly added undiscounted bills on that day should be valued.

[0258] When valuing discounted bills (for example, discounted target transaction bills), at the end of each day, the transaction business data processing system can be used to screen out associated bills with similar credit types for each discounted bill according to the rules of the bill association module, and specify a benchmark yield curve for the newly added discounted bills according to the rules set by the "Benchmark Yield Curve Maintenance" function in the valuation management function.

[0259] Check whether the rediscount yield curve for the day has been calculated. If it has been calculated, determine the benchmark yield for each note based on the yield curve.

[0260] Based on the rediscount transactions of the day, find out the list of bills with reasonable transaction prices on that day.

[0261] For non-newly added discounted bills on the valuation day, first find the reasonable price of the bill based on the transaction status of each bill itself and the Class I related bills on the valuation day. If a reasonable price exists, calculate the weighted average spread according to the total face value of the transaction (because there may be multiple reasonable prices for multiple related bills); if a certain bill does not have a reasonable price both itself and the Class I related bills, find the reasonable price of the Class II related bills of the bill. If so, calculate the weighted average spread according to the total face value of the transaction. If there is no reasonable price for the Class II related bills on the valuation day, find the reasonable price of the Class III related bills of the bill. If not, the bill will continue to use its own valuation spread from the previous business day.

[0262] For newly added discounted bills on the valuation day, first find the reasonable price of the bill based on the transaction of each bill and the Class I related bills on the valuation day. If there is a reasonable price, calculate the weighted average spread based on the total face value of the transaction (because there may be multiple reasonable prices for multiple related bills). If there is no reasonable price for a bill itself and the Class I related bills, find the reasonable price of the Class II related bills of the bill. If there is a reasonable price, calculate the weighted average spread based on the total face value of the transaction. If there is no reasonable price for the Class II related bills on the valuation day, the weighted average valuation spread of the related bills on the previous working day will be used according to the priority of Class I-II (that is, the total face value of the related bills on the previous working day). If neither exists, the spread will be calculated using the discount rate of the bill.

[0263] Valuation is performed in the valuation module according to the valuation formula based on the benchmark yield and average spread determined in the above steps.

[0264] It should be noted that when valuing the discounted bills on day T, the existing bills on day T-1 should be valued first, and then the newly discounted bills on that day should be valued.

[0265] Through the above scenario examples, it is verified that the transaction business data processing system and transaction ticket data processing method provided in this specification can be well adapted to the big data processing scenarios involving transaction tickets, and can efficiently and accurately complete the relevant data processing based on transaction tickets.

[0266] Although the present specification provides method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent a unique execution order. When the device or client product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or device. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or device including the elements. The first, second, etc. words are used to represent the name, and do not represent any particular order.

[0267] Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code, the controller can be made to implement the same function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices for implementing various functions included therein can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0268] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer-readable storage media including storage devices.

[0269] Through the description of the above embodiments, it can be known that those skilled in the art can clearly understand that the present specification can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present specification can essentially be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in each embodiment of the present specification or some parts of the embodiments.

[0270] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. This specification can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0271] Although the present specification is described through embodiments, those skilled in the art will appreciate that there are many modifications and changes to the present specification without departing from the spirit of the present specification, and it is intended that the appended claims include these modifications and changes without departing from the spirit of the present specification.

Claims

1. A data processing method for a transaction ticket, characterized in that: The method is applied to a server of a transaction business data processing system, and comprises: Receiving a target transaction bill service curve generation request initiated by a user terminal; wherein the target transaction bill service curve generation request carries at least identification information of a target transaction bill package; In response to the request for generating the target transaction bill business curve, the corresponding kettle task is determined according to the first script file; and the kettle cluster is called to obtain the target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks; Receive and integrate the target sample business data into the corresponding target business wide table based on the execution results of the first batch of kettle tasks; and store the target business wide table in the data detail layer of the data warehouse; According to the second script file, the kettle cluster is called to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and a preset business curve generation operation is performed according to the target business wide table to generate a corresponding target transaction bill business curve; Receive and based on the execution results of the second batch of kettle tasks, when it is determined that a target transaction bill business curve that meets the requirements is generated, store the target transaction bill business curve in the application data layer of the data warehouse; wherein the user terminal is connected to the application data layer of the data warehouse through a corresponding data interface, and obtains the target transaction bill business curve through the data interface.

2. The method according to claim 1, characterized in that The first script file includes a shell script for data collection and storage, the second script file includes a python script for data calculation, and the data warehouse includes a database based on the MPP architecture.

3. The method according to claim 1, characterized in that Call the kettle cluster and execute the first batch of kettle tasks to obtain the target sample business data related to the target transaction bill business curve, including: Call the kettle cluster, execute the corresponding first batch of kettle tasks, query the data source table in the operation data layer of the data warehouse according to the identification information of the target transaction bill package, and obtain the initial sample business data; Call the kettle cluster to detect and remove the first type of abnormal data in the initial sample business data by executing the corresponding first batch of kettle tasks, and obtain the intermediate sample business data; and store the intermediate sample business data in a temporary table; Call the kettle cluster to detect and remove the second type of abnormal data in the temporary table by executing the corresponding first batch of kettle tasks to obtain the corresponding target sample business data.

4. The method according to claim 3, characterized in that The second type of abnormal data includes: transaction business data whose deviation does not meet the requirements; The first type of abnormal data includes at least: transaction data whose amount ratio does not meet the requirements and transaction data whose term difference does not meet the requirements; When the business category of the target transaction bill business curve is rediscount business, the first category of abnormal data also includes: internal transaction business data, back-to-back transaction business data, and bridge transaction business data.

5. The method according to claim 3, characterized in that: The method further comprises: According to the preset protocol rules, the kettle cluster is called to collect initial transaction business data from multiple data sources by executing the third batch of kettle tasks and using the corresponding secure data channel; and the initial transaction business data is stored in the cache layer; Call the kettle cluster to read the initial transaction business data from the cache layer by executing the third batch of kettle tasks; and preprocess the initial transaction business data to obtain preprocessed transaction business data; The kettle cluster is called to execute the third batch of kettle tasks to store the pre-processed transaction business data into the corresponding data source table in the operation data layer of the data warehouse.

6. The method according to claim 1, characterized in that Perform preset business curve generation calculations based on the target business wide table to generate the corresponding target transaction bill business curve, including: Call the kettle cluster to determine multiple key deadlines by executing the corresponding second batch of kettle tasks; and use the multiple key deadlines to divide the target sample business data in the target business wide table into multiple intervals; Call the kettle cluster to execute the corresponding second batch of kettle tasks, use the target sample business data in each interval, and determine the target business value of each key deadline through the first linear regression; Call the kettle cluster, execute the corresponding second batch of kettle tasks, use the data of key deadlines to perform curve fitting, and determine the target business values ​​of other deadlines; Call the kettle cluster and execute the corresponding second batch of kettle tasks to generate the corresponding target transaction bill business curve based on the target business value of the key deadline and the target business value of other deadlines.

7. The method according to claim 6, characterized in that In the case where the curve type of the target transaction bill business curve is a derivative curve, after calling the kettle cluster, executing the corresponding second batch of kettle tasks, using the target sample business data in each interval, and determining the target business value of each key deadline through the first linear regression, the method further includes: According to the target business value of the key deadline point of each interval and the target transaction business data in each interval, the interval regression line segment of each interval is determined through the second linear regression; According to the interval regression line segments of each interval, the slope parameters of each key deadline point are calculated and used to determine the first intermediate business value of each key deadline point through the first linear regression; According to the reference curve parameters and the reference threshold parameters, the target business value of each key deadline is obtained by performing result control on the first intermediate business value of each key deadline; Use the data of key deadlines to perform curve fitting to determine the target business values ​​of other deadlines; According to the target business value of the key term point and the target business value of other term points, the corresponding target transaction bill business curve is generated.

8. The method according to claim 1, characterized in that After storing the target transaction bill business curve in the application data layer of the data warehouse, the method further includes: Receiving a target transaction note valuation request initiated by a user terminal; wherein the target transaction note is a transaction note involved in a target transaction note business curve; In response to the target transaction bill valuation request, according to the fourth script file, the kettle cluster is called, and the target transaction bill business curve is used in combination with the relevant data source table to determine the valuation result of the target transaction bill; The valuation results of the target transaction notes are stored in the application data layer of the data warehouse.

9. A data processing device for transaction tickets, characterized in that: A server applied to a transaction business data processing system, the device comprising: A receiving module, configured to receive a request for generating a target transaction bill service curve initiated by a user terminal; wherein the request for generating a target transaction bill service curve carries at least identification information of a target transaction bill package; The first calling module is used to respond to the request for generating the target transaction bill business curve, determine the corresponding kettle task according to the first script file, and call the kettle cluster to obtain the target sample business data related to the target transaction bill business curve by executing the first batch of kettle tasks; The integration module is used to receive and integrate the target sample business data into the corresponding target business wide table according to the execution results of the first batch of kettle tasks; and store the target business wide table into the data detail layer of the data warehouse; The second calling module is used to call the kettle cluster according to the second script file to read the target business wide table from the data detail layer of the data warehouse by executing the second batch of kettle tasks; and perform a preset business curve generation operation according to the target business wide table to generate a corresponding target transaction bill business curve; The processing module is used to receive and store the target transaction bill business curve in the application data layer of the data warehouse based on the execution results of the second batch of kettle tasks when it is determined that a target transaction bill business curve that meets the requirements is generated; wherein the user terminal is connected to the application data layer of the data warehouse through a corresponding data interface, and obtains the target transaction bill business curve through the data interface.

10. A server, characterized in that: The method comprises a processor and a memory for storing processor-executable instructions, wherein the processor implements the steps of the method according to any one of claims 1 to 8 when executing the instructions.

Images