Data processing method and apparatus, device, and storage medium

Abstract

The application discloses a data processing method, device and equipment and a storage medium, and aims at the problems that the current prepayment system interface adopts an HTTP synchronous interface, the interface is completely open, the system pressure is large, and the interface is prone to timeout and resource occupation and collapse. Through real-time acquisition of transaction data, the transaction type of the transaction data is identified; the transaction types are classified according to real-time account and offline settlement; the synchronous data transmission mode is adopted for the transaction types of real-time account to perform data processing; the asynchronous data transmission mode or the file data transmission mode is adopted for the transaction types of offline settlement to perform data processing; and thus the interface pressure of the prepayment system is relieved, and the data processing efficiency of the prepayment system is improved.

Description

Technical Field

[0001] This invention belongs to the technical field of data processing, and particularly relates to a data processing method, apparatus, device, and storage medium for a prepayment system. Background Technology

[0002] Currently, the prepayment interface uses an HTTP synchronous interface, which is completely public and puts a heavy load on the system. This interface covers a variety of business scenarios, including top-ups, withdrawals, deductions, refunds, cancellations, and transfers. It also provides prepayment sending services to 68 upstream systems (various business systems), with a daily transaction volume of 210,000 transactions.

[0003] Based on the large and ever-growing business system, when the business is at its peak and there is high concurrency access to the prepayment interface, it is easy to cause interface timeouts, resource exhaustion and crashes, or even server downtime, resulting in problems such as incomplete data storage and abnormal data updates. Summary of the Invention

[0004] The purpose of this invention is to provide a data processing method, apparatus, device, and storage medium to establish a prepayment pre-service, reduce the pressure on the prepayment system, ensure the stability of the prepayment system, provide multi-dimensional whitelist configuration, reduce the pressure on the prepayment system during peak periods through flexible configuration, pre-verify data, reduce the logical complexity of the prepayment system, improve interface timeliness, reduce the pressure on the upstream system to resend due to the original prepayment interface being abnormal, and ensure the core functions of the prepayment system.

[0005] To solve the above problems, the technical solution of the present invention is as follows:

[0006] A data processing method for a prepayment system, comprising:

[0007] Real-time acquisition of transaction data; identification of transaction types within the transaction data.

[0008] Transaction types are categorized based on whether they are settled in real-time or offline.

[0009] For transactions that arrive in real time, synchronous data transmission is used for data processing.

[0010] For offline settlement transactions, asynchronous data transmission or file data transmission is used for data processing to alleviate the interface pressure on the prepayment system and improve its data processing efficiency.

[0011] According to an embodiment of the present invention, the real-time acquisition of transaction data and identification of the transaction type of the transaction data further includes:

[0012] The acquired transaction data is subjected to standardized character filtering to obtain the target transaction data;

[0013] Perform an object identification operation on the target transaction data to obtain the transaction object;

[0014] A preset transaction type identification model is invoked to process the transaction data of the transaction object in the target transaction data to obtain the transaction type of the transaction object.

[0015] According to an embodiment of the present invention, the classification of transaction types based on real-time settlement and offline settlement further includes:

[0016] Transaction data with transaction type of deposit, withdrawal or transfer is marked as real-time arrival and transmitted to the first data processing channel for real-time processing;

[0017] Transaction data with transaction types of deduction, refund, or reversal are marked as offline settlement and transmitted to the message queue in the second data processing channel for processing.

[0018] According to an embodiment of the present invention, the data processing for real-time settlement transactions using a synchronous data transmission method further includes:

[0019] The prepayment system will be used as a server to provide HTTP interface services and publish interface APIs.

[0020] After receiving data sent by the client according to the API, the server performs multiple data validity checks, generates a transaction record, and returns the prepayment result.

[0021] According to an embodiment of the present invention, the step of processing data for offline settlement transactions using an asynchronous data transmission method further includes:

[0022] Establish prepayment pre-service;

[0023] Set up a prepayment whitelist based on the transaction type;

[0024] The prepayment pre-service receives prepayment data sent by the client via Kafka and performs data validity verification.

[0025] The prepayment pre-processing service will send the verified data to the prepayment system.

[0026] According to an embodiment of the present invention, the data processing for offline settlement transactions using file data transfer further includes:

[0027] Establish prepayment pre-service;

[0028] Set up a whitelist for prepayments based on transaction type and flexibly configure the prepayment sending status to reduce pressure on the prepayment system during peak periods;

[0029] The database loads a CSV file and processes the data. The CSV file is a file generated in advance based on the prepayment data. The processed data is then used to generate another CSV file and uploaded to a specified path.

[0030] The prepayment pre-payment service loads the CSV file under the specified path and performs data validity verification;

[0031] The prepayment pre-payment service will aggregate the verified data according to the charging department, charging item, and prepayment sending time.

[0032] The prepayment pre-processing service sends the aggregated data to the prepayment system.

[0033] A data processing apparatus for a prepayment system, comprising:

[0034] The data recognition module is used to acquire transaction data in real time and identify the transaction type.

[0035] The data classification module is used to classify transactions by real-time settlement and offline settlement.

[0036] The first data processing module is used to process real-time transactions using synchronous data transmission.

[0037] The second data processing module is used to process offline settlement transactions using asynchronous or file data transmission methods to alleviate the interface pressure on the prepayment system and improve its data processing efficiency.

[0038] According to an embodiment of the present invention, the data identification module further includes:

[0039] The data filtering unit is used to perform standardized character filtering on the acquired transaction data to obtain the target transaction data;

[0040] An object identification unit is used to perform object identification operations on the target transaction data to obtain the transaction object;

[0041] The data processing unit is used to call a preset transaction type recognition model to process the transaction data of the transaction object in the target transaction data to obtain the transaction type of the transaction object.

[0042] A data processing apparatus includes a memory and a processor, wherein the memory stores computer-readable instructions, which, when executed by the processor, cause the processor to perform steps in a data processing method according to an embodiment of the present invention.

[0043] A storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform steps in a data processing method according to an embodiment of the present invention.

[0044] Because the present invention adopts the above technical solution, it has the following advantages and positive effects compared with the prior art:

[0045] The data processing method in one embodiment of the present invention addresses the problems of current prepayment systems using HTTP synchronous interfaces, which are fully public, leading to high system pressure and susceptibility to interface timeouts and resource exhaustion crashes. The method acquires transaction data in real time and identifies the transaction type; classifies the transaction type according to real-time arrival and offline settlement; processes real-time arrival transactions using synchronous data transmission; and processes offline settlement transactions using asynchronous data transmission or file data transmission. This alleviates the interface pressure on the prepayment system and improves its data processing efficiency. Attached Figure Description

[0046] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present invention;

[0047] Figure 2 This is a block diagram of a data processing device according to an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of a data processing device according to an embodiment of the present invention. Detailed Implementation

[0049] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a further detailed explanation of the data processing method, apparatus, device, and storage medium proposed in this invention. The advantages and features of this invention will become more apparent from the following description and claims.

[0050] Example 1

[0051] This embodiment addresses the current prepayment system's use of HTTP synchronous interfaces, which are fully public, leading to high system load, timeouts, resource exhaustion, and crashes. It provides a data processing method by establishing a prepayment front-end service to reduce system pressure, ensure system stability, provide multi-dimensional whitelist configuration, and flexibly configure the whitelist to alleviate pressure during peak periods. Pre-verification of data reduces the logical complexity of the prepayment system, improves interface timeliness, reduces the pressure on upstream systems to resend due to original prepayment interface anomalies, and safeguards core prepayment functions.

[0052] Please refer to Figure 1 The data processing method includes the following steps:

[0053] S1: Real-time acquisition of transaction data and identification of transaction types;

[0054] S2: Classify transaction types according to real-time settlement and offline settlement;

[0055] S3: For real-time transactions, synchronous data transmission is used for data processing;

[0056] S4: For offline settlement transactions, use asynchronous data transmission or file data transmission for data processing to alleviate the interface pressure on the prepayment system and improve its data processing efficiency.

[0057] In step S1, real-time acquisition of transaction data and identification of the transaction type can be achieved through the following steps:

[0058] The acquired transaction data is subjected to standardized character filtering to obtain the target transaction data;

[0059] Perform object identification on the target transaction data to obtain the transaction object;

[0060] The preset transaction type recognition model is invoked to process the transaction data of the transaction object in the target transaction data to obtain the transaction type of the transaction object.

[0061] In this embodiment, an extraction rule generator is first built in the application system. The extraction rule generator includes the log acquisition level, log acquisition program name, extraction time range, etc. When the log file content is added, the rules in the extraction rule generator are matched. If the match is successful, the transactions in the log are output to the data processor.

[0062] For example, if the extraction rule is set to retrieve transaction or interaction information at the Info level, the log retrieval program is named TransactionBusiness, and the extraction time is 0:00-24:00, then the extraction rule will extract the Info information of the specified program from the logs, ignoring other information by default. The output transactions will then be the transaction data to be analyzed, and this data will be stored.

[0063] After obtaining the transaction data to be analyzed, the transaction objects and transaction types are extracted from the transaction data.

[0064] In this process, standardized character filtering is performed on the transaction data to obtain the target transaction data.

[0065] In practical applications, transaction data may contain sensitive words, stop words, etc., so it is necessary to perform standardized character filtering on the transaction data.

[0066] Specifically, standardized character filtering operations refer to filtering sensitive words, replacing synonyms, and removing stop words from the transaction data to be analyzed, thereby obtaining multiple sentences with high semantic value, i.e., obtaining the target transaction data.

[0067] Perform object identification on the target transaction data to obtain the transaction object.

[0068] Specifically, the target transaction data is matched against preset templates in the knowledge base to identify the transaction object, i.e., which customer conducted the transaction. For example, the transaction objects in this case are A, B, C, D, E, F, and G.

[0069] The preset transaction type recognition model is invoked to process the transaction data of the transaction object in the target transaction data to obtain the transaction type of the transaction object.

[0070] In this embodiment, historical transaction data and transaction types can be collected to train the constructed neural network model until the model's loss function is less than a set value. The neural network model obtained at this point is called the preset transaction type recognition model.

[0071] During the model's use, the transaction data of the target transaction object is input into the preset transaction type recognition model to obtain the transaction type of the transaction object.

[0072] In general, there can be multiple types of transactions, such as deposits, withdrawals, transfers, and refunds. The same transaction object may involve multiple transaction types, or it may involve only one transaction type.

[0073] In step S2, the transaction types are classified according to real-time settlement and offline settlement.

[0074] In this embodiment, transaction data of the transaction type of recharge, withdrawal or transfer is marked as real-time arrival and transmitted to the first data processing channel for real-time processing.

[0075] Transaction data with transaction types of deduction, refund, or reversal are marked as offline settlement and transmitted to the message queue in the second data processing channel for processing.

[0076] Regarding the classification of real-time payments, in practical applications, based on the characteristics of the express delivery industry and from the customer's perspective, we ensure that the status of branch accounts is updated in a timely manner, and that the corresponding transaction types of top-ups, withdrawals, and transfers are settled in real time.

[0077] Real-time transaction data can be stored in an Oracle database transaction information table. This table can be monitored in real time to obtain incremental transaction information. OGG (Oracle GoldenGate) is used to replicate the incremental data from the transaction information table to a Kafka message queue in real time. Data is then read from the Kafka broker using a real-time ETL program or Spark program. The acquired data is then packaged and statistically analyzed according to the business scenario (e.g., deposit, withdrawal, or transfer). The statistically analyzed data is stored in MongoDB in JSON format, with a `last_up_time` field added to each record to store the time the data was stored in MongoDB.

[0078] A Spring Boot application performs scheduled scans of MongoDB, with the scan interval set to the second level. Incremental data is retrieved based on the `last_up_time` query and synchronized to the Aerospike database in real time.

[0079] The packaged transaction data is processed in real time through the first data processing channel (i.e., the synchronous data transmission channel). This first data processing channel calls JSON format data from the Aerospike database for processing, ensuring timely updates to the branch account status.

[0080] Regarding the classification of offline settlement, in practical applications, based on business scenarios such as ticket collection and signing, and assessment, prepayments are sent after data processing. The corresponding transaction types for such deductions, refunds, and cancellations are offline settlements.

[0081] Transaction data for offline settlement transactions can be synchronously aggregated into the operational data warehouse (ODS) offline. This reduces the need for direct manipulation of source data in the prepayment system.

[0082] Synchronize data from the data warehouse ODS to a wide table in the data warehouse DW. Create a wide table that summarizes all fields that need to be statistically analyzed in different scenarios (such as deductions, refunds, or reversals). This reduces table join operations when processing data in individual scenarios, maximizing the efficiency of data synchronization.

[0083] In Hive, data stored in wide tables is statistically summarized based on business requirements. The summarized results are then inserted into business tables in the data warehouse DM. These business tables store data for different business scenarios. Each partition of a business table represents the statistical results for a specific business scenario. Different partitions are used for different business scenarios, enabling precise and targeted processing of data for different business scenarios.

[0084] Data from each partition in the business table is synchronized to the Aerospike database with different tags and stored in JSON format. Transaction data from the business table is then processed through a second data processing channel (i.e., an asynchronous data transmission channel or a file data transmission channel). This second data processing channel calls the JSON format data in the Aerospike database for processing and sends the processed data to the prepayment system.

[0085] In step S3, for transactions that arrive in real time, data processing is performed using synchronous data transmission.

[0086] This step processes transaction data for real-time settlement transactions using a synchronous data transmission method to ensure timely updates of branch account status and enable real-time settlement services. The synchronous data transmission method is as follows:

[0087] 1) The prepayment system acts as a server, providing HTTP interface services and publishing API (Application Programming Interface);

[0088] 2) The client sends data according to the API interface;

[0089] 3) After receiving the data, the server performs multiple data validity checks and generates transaction records;

[0090] 4) The server returns the result of the prepayment being sent.

[0091] In the aforementioned synchronous data transmission methods, to achieve real-time communication, client-server message passing often employs polling technology, including traditional polling and long polling. Polling continuously sends HTTP requests to the server, which responds each time but may not necessarily carry the latest data. Long polling also continuously sends HTTP requests to the server, which returns data only when new data is available.

[0092] In step S4, for offline settlement transaction types, data processing is performed using asynchronous data transmission or file data transmission.

[0093] This step processes transaction data for offline settlement transactions using asynchronous or file data transfer methods. By establishing a prepayment pre-service, it reduces the burden on the prepayment system, ensures its stability, and provides multi-dimensional whitelist configuration. Flexible configuration alleviates pressure on the prepayment system during peak periods. Pre-verification of data reduces the logical complexity of the prepayment system, improves interface timeliness, reduces the pressure on upstream systems to resend due to original prepayment interface anomalies, and ensures the core functions of the prepayment system.

[0094] The asynchronous data transmission method is as follows:

[0095] 1) Establish a prepayment service;

[0096] 2) Based on transaction type, a prepayment whitelist can be set up, and the prepayment sending status can be flexibly configured to reduce the pressure on the prepayment system during peak periods;

[0097] 3) The client sends the prepayment data via Kafka;

[0098] 4) The prepayment pre-service receives data and verifies its validity;

[0099] 5) The prepayment pre-processing service will send the verified data to the prepayment system.

[0100] Traditional transaction business models concentrate all settlement power in the prepayment system, including functions such as top-up, withdrawal, deduction, refund, offsetting, and transfer. This prepayment system simultaneously provides prepayment services to 68 upstream systems, handling up to 210,000 transactions daily. Given this large and ever-growing business system, peak traffic and high concurrency access to the prepayment interface can easily lead to interface timeouts, resource exhaustion and crashes, and even server downtime, resulting in incomplete data storage and abnormal data updates.

[0101] This embodiment improves upon this by establishing a prepayment pre-service, which is responsible for the pre-identification of transaction types and the pre-verification of data validity, thereby reducing the pressure on the prepayment system and ensuring its stability.

[0102] The system allows for the creation of a prepayment whitelist based on transaction type, enabling flexible configuration of prepayment sending status. For example, setting deduction, refund, or cancellation transactions as whitelisted allows the prepayment system to process and send the deduction, refund, or cancellation transaction data separated in step S2 in real time. Similarly, setting recharge, withdrawal, and transfer transactions as whitelisted also allows for real-time processing and sending of the recharge, withdrawal, and transfer transaction data separated in step S2. Specific settings can be configured according to actual needs.

[0103] In the asynchronous data transmission method of this embodiment, when the client sends prepayment data via Kafka, the prepayment front-end service identifies the transaction type of the received prepayment data and performs data validity verification. This data validity verification includes verifying information such as the remitter's information, the payee's information, the transaction amount, and the transaction time.

[0104] Specifically, transaction data can be verified using digital tokens. For example, the client interacts with the authentication module beforehand to authenticate the user's identity for the transaction. After successful authentication, the authentication module runs a preset algorithm based on the transaction data uploaded by the client to generate a corresponding string (e.g., a token), i.e., a digital token.

[0105] The prepayment pre-service receives and verifies a digital token from the client. Based on the digital token, it runs a symmetric algorithm (the algorithm used by the authentication module to generate the new digital token) to obtain the corresponding transaction data, i.e., the data to be verified. The received transaction data is then compared with the transaction data obtained by parsing the digital token to determine if they match. If the transaction data matches the data to be verified, the digital token passes verification; otherwise, it fails verification.

[0106] The prepayment pre-processing service sends the verified data to the prepayment system, eliminating the need for the prepayment system to perform calculations related to transaction type identification and data validity verification. This reduces the logical complexity of the prepayment system, improves interface timeliness, and ensures the stability of the prepayment system.

[0107] The file data transfer method is as follows:

[0108] 1) Establish a prepayment service;

[0109] 2) Based on transaction type, a prepayment whitelist can be set up, and the prepayment sending status can be flexibly configured to reduce the pressure on the prepayment system during peak periods;

[0110] 3) The upstream system generates a CSV file from the prepayment data and uploads it to the specified path;

[0111] 4) Load CSV files into the GP (GreenPlum) database and perform further processing (such as mandatory settlement adjustment in the express delivery industry);

[0112] 5) The GP database generates a CSV file from the processed data and uploads it to the specified path;

[0113] 6) The prepayment pre-payment service loads the CSV file and performs data validity verification;

[0114] 7) The prepayment pre-payment service will summarize the verified data according to the charging department, charging item, and prepayment sending time.

[0115] 8) The prepayment pre-service will send the aggregated data to the prepayment system.

[0116] Among them, GP is the fastest and most cost-effective relational distributed database in the industry, with the following advantages:

[0117] Massive storage

[0118] GreepPlum supports the storage and management of massive amounts of data up to 50PB (1PB = 1024TB). GreenPlum integrates data from different source systems, departments, and platforms into a database for centralized storage, and stores detailed historical data traces. Business users no longer have to face one information silo after another, nor are they confused by the discrepancies caused by different versions of data. At the same time, it also reduces the complexity of management and maintenance work for IT personnel.

[0119] High concurrency

[0120] Greenplum provides workload management capabilities to manage database resources. Resource queue management allows for resource allocation by user group, such as the number of concurrent sessions and maximum resource values. Through workload management, resources can be allocated and prioritized at the user level, while also preventing low-quality SQL queries (such as multi-table joins without conditions) from consuming system resources.

[0121] High cost performance

[0122] Greenplum database software system nodes are based on various open hardware platforms in the industry, such as PC Server from manufacturers like SUN, HP, and DELL. They can achieve high performance on ordinary x86 Server, thus offering excellent cost-effectiveness. Compared to other closed data warehouse dedicated systems, Greenplum's investment per TB is 1 / 5 or even lower. Similarly, the maintenance cost of Greenplum products is much lower than that of similar vendors.

[0123] Easy-to-use system

[0124] Greenplum products are developed based on the popular PostgreSQL. Almost all PostgreSQL client tools and PostgreSQL applications can run on the Greenplum platform, and there are abundant PostgreSQL resources available for users on the Internet.

[0125] High availability

[0126] Greenplum is a highly available system, with existing case studies using clustered MPP environments of up to 96 machines. In addition to hardware-level RAID technology, Greenplum also provides a database-level mirroring mechanism for protection, meaning that data on each node is synchronously mirrored on other nodes, so a failure on a single node does not affect the use of the entire system.

[0127] For the master node, Greenplum provides a Master / Stand by mechanism for fault tolerance. When the master node fails, it can switch to a Stand by node to continue service.

[0128] Fast reaction speed

[0129] Greenplum enables real-time updates to the data warehouse through near real-time and real-time data loading methods, thereby realizing a dynamic data warehouse (ADW). Based on the dynamic data warehouse, business users can perform real-time BI analysis on current business data – "Just In Time BI" – enabling enterprises to keenly perceive market changes and accelerate decision support response speed.

[0130] This embodiment uses a GP database to load CSV files and perform in-depth processing (such as mandatory settlement adjustment in the express delivery industry). The processed data is then generated into a CSV file and uploaded to a specified path (such as a dedicated address for storing CSV files or an address for loading prepayment services). This allows for fast and efficient data processing of CSV files, improving data processing efficiency.

[0131] The file data transfer method and the asynchronous data transfer method mentioned above require the establishment of a prepayment service. The difference is that the file data transfer method requires in-depth processing of the upstream system data, which will not be elaborated here.

[0132] The data processing method in this embodiment reduces the pressure on the prepayment system and ensures its stability by building a prepayment pre-service; it also reduces the pressure on the prepayment system during peak periods by providing multi-dimensional whitelist configuration and flexibly configuring the sending status of the prepayment system; and it reduces the logical complexity of the prepayment system and improves interface timeliness by performing data verification in advance, thereby reducing the pressure on the upstream system to resend due to the original prepayment interface being abnormal and ensuring the core functions of the prepayment system.

[0133] Example 2

[0134] This embodiment provides a data processing device for a prepayment system. Please refer to [link / reference]. Figure 2 The data processing device includes:

[0135] Data recognition module 1 is used to acquire transaction data in real time and identify the transaction type of the transaction data;

[0136] Data classification module 2 is used to classify transaction types according to real-time settlement and offline settlement;

[0137] The first data processing module 3 is used to process real-time transactions using a synchronous data transmission method.

[0138] The second data processing module 4 is used to process offline settlement transactions using asynchronous or file data transmission methods to alleviate the interface pressure on the prepayment system and improve its data processing efficiency.

[0139] The data recognition module further includes:

[0140] The data filtering unit is used to perform standardized character filtering on the acquired transaction data to obtain the target transaction data;

[0141] An object identification unit is used to perform object identification operations on the target transaction data to obtain the transaction object;

[0142] The data processing unit is used to call a preset transaction type recognition model to process the transaction data of the transaction object in the target transaction data to obtain the transaction type of the transaction object.

[0143] The functions and implementation methods of the above-mentioned data identification module 1, data classification module 2, first data processing module 3 and second data processing module 4 are as described in Embodiment 1 above, and will not be repeated here.

[0144] Example 3

[0145] This embodiment provides a data processing device. Please refer to... Figure 3 The data processing device 500 can vary considerably depending on its configuration or performance, and may include one or more central processing units (CPUs) 510 (e.g., one or more processors) and memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing application programs 533 or data 532. The memory 520 and storage media 530 may be temporary or persistent storage. The program stored in the storage media 530 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the data processing device 500.

[0146] Furthermore, the processor 510 can be configured to communicate with the storage medium 530 and execute a series of instruction operations in the storage medium 530 on the data processing device 500.

[0147] The data processing device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input / output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Vista, etc.

[0148] Those skilled in the art will understand that Figure 3 The data processing device structure shown does not constitute a limitation on the data processing device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0149] Another embodiment of the present invention also provides a computer-readable storage medium.

[0150] The computer-readable storage medium can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the data processing method in Embodiment 1.

[0151] If a data processing method is implemented as program instructions and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in software. This computer software is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific identification content executed by the system and device described above can be referred to the corresponding process in the foregoing method embodiments.

[0153] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. A data processing method for a prepayment system, characterized by, include: Real-time acquisition of transaction data; identification of transaction types within the transaction data. Transaction types are categorized based on whether they are settled in real-time or offline. For transactions that arrive in real time, synchronous data transmission is used for data processing. For offline settlement transactions, asynchronous data transmission or file data transmission methods are used for data processing to alleviate the interface pressure on the prepayment system and improve its data processing efficiency. The method of processing data for offline settlement transactions using asynchronous data transmission further includes: Establish prepayment pre-service; Set up a prepayment whitelist based on the transaction type; The prepayment pre-service receives prepayment data sent by the client via Kafka and performs data validity verification. The prepayment pre-processing service will send the verified data to the prepayment system.

2. The data processing method of claim 1, wherein, The real-time acquisition of transaction data and identification of the transaction type further includes: The acquired transaction data is subjected to standardized character filtering to obtain the target transaction data; Perform an object identification operation on the target transaction data to obtain the transaction object; A preset transaction type identification model is invoked to process the transaction data of the transaction object in the target transaction data to obtain the transaction type of the transaction object.

3. The data processing method of claim 1, wherein, The classification of transaction types based on real-time settlement and offline settlement further includes: Transaction data with transaction type of deposit, withdrawal or transfer is marked as real-time arrival and transmitted to the first data processing channel for real-time processing; Transaction data with transaction types of deduction, refund, or reversal are marked as offline settlement and transmitted to the message queue in the second data processing channel for processing.

4. The data processing method of claim 1, wherein, For the real-time payment transaction type, the data processing using synchronous data transmission further includes: The prepayment system will be used as a server to provide HTTP interface services and publish interface APIs. After receiving data sent by the client according to the API, the server performs multiple data validity checks, generates a transaction record, and returns the prepayment result.

5. The data processing method of claim 1, wherein, The process of processing offline settlement transactions using file data transfer further includes: Establish prepayment pre-service; Set up a whitelist for prepayments based on transaction type and flexibly configure the prepayment sending status to reduce pressure on the prepayment system during peak periods; The database loads a CSV file and processes the data. The CSV file is a file generated in advance based on the prepayment data. The processed data is then used to generate another CSV file and uploaded to a specified path. The prepayment pre-payment service loads the CSV file under the specified path and performs data validity verification; The prepayment pre-payment service will aggregate the verified data according to the charging department, charging item, and prepayment sending time. The prepayment pre-processing service sends the aggregated data to the prepayment system.

6. A data processing apparatus for a prepayment system, which implements the data processing method according to any one of claims 1 to 5, characterized in that, include: The data recognition module is used to acquire transaction data in real time and identify the transaction type. The data classification module is used to classify transactions by real-time settlement and offline settlement. The first data processing module is used to process real-time transactions using synchronous data transmission. The second data processing module is used to process offline settlement transactions using asynchronous or file data transmission methods to alleviate the interface pressure on the prepayment system and improve its data processing efficiency.

7. The data processing apparatus as described in claim 6, characterized in that, The data recognition module further includes: The data filtering unit is used to perform standardized character filtering on the acquired transaction data to obtain the target transaction data; An object identification unit is used to perform object identification operations on the target transaction data to obtain the transaction object; The data processing unit is used to call a preset transaction type recognition model to process the transaction data of the transaction object in the target transaction data to obtain the transaction type of the transaction object.

8. A data processing device, characterized in that, include: A memory and a processor, wherein the memory stores computer-readable instructions that, when executed by the processor, cause the processor to perform the steps of the data processing method as described in any one of claims 1 to 5.

9. A storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, the one or more processors cause the one or more processors to perform the steps in the data processing method as described in any one of claims 1 to 5.

Images