Data processing method and device

Abstract

The embodiment of the invention provides a data processing method and device, and the method comprises the steps: obtaining transaction data from a database, and carrying out the grouping of the transaction data in a preset time window according to different time nodes, and obtaining a plurality of groups of first data; performing parallel processing on the multiple groups of first data, and performing total score balance operation and balance sharing operation on each group of first data to obtain multiple groups of second data; and writing the multiple groups of second data into a specified target data table by utilizing a multi-thread concurrent technology. Through the embodiment of the invention, the problems of low data processing speed, inaccurate cost calculation and the like of the existing data processing scheme in the related technology are solved, and the effect of improving the operation efficiency and risk management level of a financial institution is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information processing, in particular to a data processing method and device. BACKGROUND

[0002] With the development of financial markets and the advancement of information technology, financial institutions are facing increasingly complex trading environments and huge data processing demands. Traditional data processing methods, such as single-thread sequential processing, have been difficult to cope with the challenges of high-frequency and high-volume transaction data processing, especially in the sub-scene of bond trading, which can lead to slow data processing speed, affecting the generation of reports and the timeliness of decision-making. The existing "First-In-First-Out" (FIFO) or other static methods for cost calculation may not accurately reflect the actual cost and income of bond trading, especially in the case of frequent market interest rate fluctuations, which may lead to distorted cost and income accounting. In addition, in multiple transactions, if the par value balance of buying and selling does not completely match, there may be an unbalanced account situation, leading to incomplete and inaccurate financial records.

[0003] The above problems highlight the limitations of current data processing technology in financial applications, especially when processing bond trading data, which puts higher requirements on the speed, accuracy and flexibility of data processing. Therefore, developing a new type of data processing method that can accurately calculate and adjust the internal cost of bond trading in real time is crucial for financial institutions to improve operational efficiency, enhance competitiveness and ensure financial stability. SUMMARY

[0004] The embodiments of the present application provide a data processing method and device to at least solve the problems of slow data processing speed and inaccurate cost calculation in the existing data processing schemes in the related art.

[0005] According to an embodiment of the present application, a data processing method is provided, comprising: obtaining transaction data from a database, and grouping the transaction data in a preset time window according to different time nodes to obtain a plurality of groups of first data; processing the plurality of groups of first data in parallel, performing total and partial balance operations and balance allocation operations on each group of first data to obtain a plurality of groups of second data; and using multi-thread concurrent technology to write the plurality of groups of second data into a designated target data table.

[0006] In one embodiment, obtaining transaction data from a database comprises: extracting the transaction data from the database according to structured query language technology and a database adapter.

[0007] In one embodiment, the type of transaction data includes at least one of the following: sell transaction date, sell transaction amount, buy transaction date and buy transaction amount.

[0008] In one embodiment, the method further comprises adjusting the size of the preset time window, including one of the following: predicting a market activity interval according to real-time fluctuations of transaction data and time series analysis results; adjusting the size of the preset time window according to the market activity interval; or predicting market trends and transaction patterns according to macroeconomic situation analysis, historical market trend data and seasonal fluctuation rules, and adjusting the size of the preset time window according to the prediction results of the market trends and the transaction patterns.

[0009] In one embodiment, after grouping the transaction data in the preset time window according to different time nodes to obtain a plurality of groups of first data, the method further comprises: creating an independent data package for each group of first data; setting an intelligent tag and metadata for each data package, wherein the intelligent tag is used to identify the grouping basis, and the metadata is used to record at least one of the following: a timestamp of data package creation, a market interest rate state, a bond credit rating, aggregated face value balance information, and buy / sell transaction information.

[0010] In one embodiment, the plurality of groups of first data are processed in parallel, the bond transaction data and the bond position data of each group of first data are matched, and in the case that there is a face value balance difference between the bond transaction data and the bond position data, the face value balance difference is filled by simulating transaction records; a plurality of groups of third data are obtained according to the plurality of groups of first data with the face value balance difference.

[0011] In one embodiment, the plurality of groups of first data are processed in parallel, total balance operation and balance apportionment operation are performed on each group of first data to obtain a plurality of groups of second data, including: the plurality of groups of third data are processed in parallel, and the seller transaction detail data and the buyer transaction detail data in the third data are date-matched; in the case that the seller transaction date is later than or equal to the buyer transaction date, the face value balance and the opening amount of the seller transaction in the third data are adjusted according to the face value balance and the transaction amount proportion of the buyer transaction in the third data to obtain a plurality of groups of second data.

[0012] In one embodiment, the plurality of groups of second data are written into a designated target data table by using a multi-thread concurrent technology, including: a plurality of threads are set and managed by using a programming language technology, wherein each thread corresponds to a write operation of a group of data; the data write task of each thread is determined according to a load balancing strategy and the load status of the target database and the processing capacity of each thread; and the plurality of groups of second data are written into the designated target data table according to the data write task.

[0013] According to another embodiment of the present application, a data processing apparatus is provided, comprising: a grouping module configured to acquire transaction data from a database, and group the transaction data in a preset time window according to different time nodes to obtain a plurality of groups of first data; a parallel processing module configured to process the plurality of groups of first data in parallel, and perform total balance operation and balance allocation operation on each group of first data to obtain a plurality of groups of second data; and a writing module configured to write the plurality of groups of second data into a designated target data table by using multi-thread concurrent technology.

[0014] According to still another embodiment of the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps in any of the method embodiments.

[0015] According to still another embodiment of the present application, an electronic device is also provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in any of the method embodiments when executing the computer program.

[0016] According to still another embodiment of the present application, a computer program product is also provided, comprising a computer program, wherein the computer program is executed by a processor to implement the steps in any of the method embodiments.

[0017] According to the above embodiments of the present application, when acquiring transaction data from a database, the transaction data in a preset time window is grouped according to time nodes, which can make data processing more efficient and reduce waiting time; then each group of data is processed in parallel, and balance allocation operation is performed on each group of data, which can improve data processing speed and system response capability, avoid relying only on the earliest or latest transaction, improve the accuracy of cost calculation; at the same time, the processed data is quickly written into a target database table by using multi-thread concurrent technology, which can reduce database write delay, enhance data real-time performance and availability. Therefore, the problems of slow data processing speed and inaccurate cost calculation in the existing data processing scheme in the related art can be solved, and the effect of improving the operation efficiency and risk management level of a financial institution is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a computer terminal of a network attack source positioning method according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of a data processing method according to an embodiment of the present application;

[0021] Figure 3 is a flow chart of a data processing method according to an exemplary embodiment of the present application;

[0022] Figure 4 is a structural block diagram of a network attack source positioning device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] Hereinafter, the present application will be described in detail with reference to the accompanying drawings and embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0024] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence.

[0025] The method embodiment provided by the embodiment one of the present application can be executed in a computer terminal or similar computing device. Taking the case of running on a computer terminal, Figure 1 is a hardware structural block diagram of a computer terminal of a data processing method according to an embodiment of the present application. As shown in Figure 1 , the computer terminal 10 can include one or more (only one is shown in Figure 1 ) processor 102 (the processor 102 can include but not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, and optionally, the above computer terminal can also include a transmission device 106 for communication function and an input and output device 108. Those skilled in the art can understand that Figure 1 the structure shown is only schematic, which does not limit the structure of the above computer terminal. For example, the computer terminal 10 can also include more or less components than those shown in Figure 1 , or have a different configuration from Figure 1 .

[0026] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the data processing method in the embodiments of the present application. The processor 102 can execute various functions and data processing by running the computer programs stored in the memory 104, that is, implement the above method. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, and the remote memory can be connected to the computer terminal 10 through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0027] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module which is used to communicate with the Internet in a wireless manner.

[0028] In the present embodiment, a data processing method running on the above computer terminal is provided, Figure 2 The flowchart of the data processing method according to the embodiments of the present application is shown in FIG. 2, and the flow includes the following steps: Figure 2

[0029] Step S202, obtaining transaction data from a database, and grouping transaction data in a preset time window according to different time nodes to obtain a plurality of groups of first data;

[0030] Step S204, processing the plurality of groups of first data in parallel, performing total balance operation and balance apportionment operation on each group of first data to obtain a plurality of groups of second data;

[0031] Step S206, using a multi-thread concurrent technology to write the plurality of groups of second data into a designated target data table.

[0032] ​According to the above steps, when the transaction data is acquired from the database, the transaction data in the preset time window is grouped according to the time node, so that the data processing is more efficient, and the waiting time is reduced; and then each group of data is processed in parallel, and the balance of each group of data is apportioned, so that the data processing speed and the system response capability are improved, the dependence on the earliest or latest transaction is avoided, the accuracy of the cost calculation is improved, and the real-time performance and the availability of the data are improved. Therefore, the problems of slow data processing speed and inaccurate cost calculation in the related art can be solved, and the operation efficiency and the risk management level of the financial institution are improved.

[0033] In step S202 of the embodiment, the transaction data is acquired from the database, including: extracting the transaction data from the database according to the structured query language technology and the database adapter.

[0034] In one embodiment, the type of the transaction data at least includes one of the following: sell transaction date, sell transaction amount, buy transaction date and buy transaction amount.

[0035] In an exemplary embodiment, the transaction data is full-amount bond transaction detail data, and the transaction data is acquired from the database, including: acquiring full-amount bond transaction detail data according to a preset time period; or acquiring real-time full-amount bond transaction detail data by using a data stream processing platform such as Kafka, RabbitMQ or Amazon Kinesis.

[0036] For example, the preset time period can be one day; and the full-amount bond transaction detail data is acquired according to the preset time period, that is, the full-amount bond transaction detail data is introduced at the end of each day.

[0037] The full-amount bond transaction detail data includes traded detail data and pre-interest data. Specifically, it mainly includes the following types: business date, contract number, institution code, subject, currency, saleable product, initial balance, current balance, contract interest start date, contract expiration date, contract term, interest calculation category, execution interest rate, opening date, customer number, annual cumulative balance, quarterly cumulative balance, monthly cumulative balance, current re-pricing cycle cumulative balance, repayment method, interest rate adjustment method, last payment date, next payment date, payment frequency, payment frequency unit, fixed-floating flag, re-pricing flag, re-pricing frequency, re-pricing frequency unit, last re-pricing date, next re-pricing date, face value, face value, bond ID, investment type, bond buying and selling flag, issue date.

[0038] Among them, the main fields involved in the subsequent balance allocation step include but are not limited to one of the following: business date, bond ID, transaction ID, interest start date, currency, investment type, bond trading flag, face value, transaction amount, opening amount, regional category.

[0039] In one embodiment, the data processing method further comprises adjusting the size of the preset time window, including one of the following: predicting the market activity interval based on real-time fluctuations of transaction data and time series analysis results; adjusting the size of the preset time window based on the market activity interval; or predicting market trends and transaction patterns based on macroeconomic analysis, historical market trend data, and seasonal fluctuation rules; and adjusting the size of the preset time window based on the prediction results of the market trends and the transaction patterns. Specifically, the market activity interval is predicted based on real-time fluctuations of transaction data and time series analysis results; the size of the preset time window is adjusted based on the market activity interval, including:

[0040] Continuously monitor and collect high-frequency transaction data, including but not limited to bond buying and selling prices, transaction volumes, transaction times, and other information.

[0041] Use time series analysis techniques (such as ARIMA, LSTM, etc.) to deeply analyze transaction data, identify periodic patterns and trends in trading activities, and possible abnormal fluctuations.

[0042] Based on the above analysis results, use prediction models (such as Prophet, XGBoost, etc.) to predict the market activity interval in the next few hours to a day. If the prediction result shows that the market activity will significantly increase, the system will automatically shorten the preset time window to process data more frequently, ensuring that market changes can be captured in time, otherwise the window period can be appropriately extended.

[0043] Specifically, the market trends and transaction patterns are predicted based on macroeconomic analysis, historical market trend data, and seasonal fluctuation rules; and the size of the preset time window is adjusted based on the prediction results of the market trends and the transaction patterns, including:

[0044] Obtain macroeconomic indicators such as GDP growth rate, inflation rate, monetary policy direction, etc. from authoritative economic data sources, as well as seasonal fluctuation rules, to provide a macroeconomic background for medium and long-term market prediction;

[0045] By analyzing historical market trend data, including past transaction volumes, price trends, interest rate changes, etc., potential market cycles and long-term trends are identified;

[0046] Using prediction technologies such as Prophet and XGBoost, combined with macroeconomic analysis and historical data, to predict medium and long-term (e.g. days to months) market trends and bond trading patterns.

[0047] According to the prediction results, the size of the preset time window is intelligently adjusted. For example, when a trading off-season is predicted, the window period is extended to reduce unnecessary processing and save resources; and during the expected trading peak, the window is shortened to ensure that the system can quickly respond to market changes.

[0048] In step S202 of the embodiment, the transaction data within the preset time window is grouped according to different time nodes to obtain multiple groups of first data, including: according to the characteristics of bond trading and the business needs of financial institutions, and using a hybrid prediction model to determine the size of the preset time window.

[0049] In one embodiment, the transaction data within the preset time window is grouped according to different time nodes to obtain multiple groups of first data, and further including: performing secondary grouping on the first group of data according to at least one of the following information: bond identifier (Identifier, ID), currency, investment type, bond buying and selling flag, to obtain multiple groups of fourth data.

[0050] In one embodiment, after grouping the transaction data within the preset time window according to different time nodes to obtain multiple groups of first data, the method further includes: creating an independent data package for each group of first data; setting an intelligent tag and metadata for each data package: wherein the intelligent tag is used to identify the grouping basis, and the metadata is used to record at least one of the following: timestamp of data package creation, market interest rate state, bond credit rating, aggregated face value balance information, buying and selling transaction.

[0051] In this embodiment, correspondingly, step S204 includes: parallel processing each group of fourth data, and performing balance allocation operation on each group of fourth data to obtain multiple groups of second data.

[0052] Through secondary grouping, each group of fourth data contains homogeneous transaction information, laying a foundation for parallel processing in the next step.

[0053] In one embodiment, before grouping the transaction data within the preset time window according to different time nodes, the method further includes: data cleaning of the transaction data, wherein the data cleaning includes but is not limited to: removing duplicate records, filling missing values, correcting abnormal values, etc. Through data cleaning, the quality of each group of first data can be ensured, avoiding invalid or erroneous data interference with subsequent balance allocation operation.

[0054] In step S202 of the embodiment, the transaction data in the preset time window is grouped according to different time nodes to obtain a plurality of groups of first data, and further comprising: creating an independent record or data packet for each group of first data; wherein the independent record or data packet contains the basis (such as time node, bond ID, etc.) of each group, the aggregated par value balance information, the buying and selling transaction, etc. to facilitate subsequent parallel processing and result verification.

[0055] In one embodiment, the transaction data in the preset time window is grouped according to different time nodes to obtain a plurality of groups of first data, and further comprising: automatically optimizing the grouping of the transaction data under different time nodes in the preset time window according to the characteristics (such as transaction frequency, transaction amount distribution, etc.) of the transaction data according to machine learning techniques (such as clustering algorithms). With the assistance of machine learning techniques, the size and processing complexity of each group of data can be ensured to be suitable for parallel processing, while minimizing the processing delay caused by uneven grouping.

[0056] Through the above embodiments, when obtaining transaction data from the database, a dynamic method is used to group the transaction data in the preset time window according to the time node, which makes the data processing more efficient and reduces the waiting time.

[0057] After obtaining the transaction detail data and completing the preliminary data grouping and cleaning, further total and split balance operation processing is required for each group of first data, that is, when there is a par value balance difference between the bond transaction detail data and the bond position detail data, in order to ensure the accuracy of data pricing, maintain the balance of the account and the authenticity of the financial report, the account data needs to be reconciled.

[0058] In step S204 of the embodiment, the plurality of groups of first data are processed in parallel, and the total and split balance operation is performed on each group of first data, including: processing the plurality of groups of first data in parallel, and matching the bond transaction data and the bond position data of each group of first data; and in the case where there is a par value balance difference between the bond transaction data and the bond position data, filling the par value balance difference by simulating a transaction record; obtaining a plurality of groups of third data from the plurality of groups of first data with the par value balance difference.

[0059] In the embodiment, the bond transaction data and the bond position data of each group of first data are matched by processing the plurality of groups of first data in parallel, including: according to the bond ID and the investment type, the par value balance of the bond ID of the type is aggregated and matched with the position of the bond ID and the investment type, if the matching result is that there is no balance difference, the bond is not processed; if the matching result is that there is a balance difference, a simulated bond transaction record is generated for the difference part, and the corresponding attributes are referred to the corresponding difference, the bond ID, and the investment type of the latest record.

[0060] In one example embodiment, assume that in a certain bank internal fund management system, there are the following transaction data and position data for two different bond IDs (sec001 and sec002):

[0061] The bond transaction details data (first data) is shown in Table 1:

[0062] Table 1

[0063]

[0064] The bond position details data is shown in Table 2:

[0065] Table 2

[0066]

[0067] Figure 3 The flowchart of the data processing method according to the example embodiment of the present application is shown in FIG. 1, which includes the following steps: Figure 3

[0068] Step S301, parallel processing and matching the bond transaction details data and the bond position details data to determine whether there is a par value balance difference between the bond transaction data and the bond position data.

[0069] Step S302, if the matching result is no balance difference, no processing is performed on the bond;

[0070] Step S303, if the matching result is a balance difference, a bond transaction record is simulated for the difference part.

[0071] Specifically, for the matching result of a balance difference, as shown in Table 1 and Table 2:

[0072] According to the matching analysis of the sec001 bond, the following information is determined:

[0073] First group of first data (bond transaction data): par value balance = 2000 + 500 = 2500

[0074] Bond position data: par value balance = 3000

[0075] Conclusion: there is a par value balance difference, and a bond transaction record needs to be simulated for the difference part, as shown in the last two rows of data in Table 3:

[0076] Table 3

[0077]

[0078] ​By parallel processing and comparison of multiple sets of first data, the system can identify differences in face value balances between bond position data and timely generate simulated transaction records to adjust these differences, ultimately obtaining multiple sets of accurate and balanced third data to meet the requirements of internal funding transfer pricing for refined management. This mechanism not only improves the accuracy of cost calculation, but also helps banks make more effective financing decisions in complex and changing market environments, while maintaining the transparency of financial statements and the integrity of accounts.

[0079] In step S204 of the embodiment, the multiple sets of first data are processed in parallel, and total balance operation and balance allocation operation are performed on each set of first data to obtain multiple sets of second data, including: parallel processing of the multiple sets of third data, date matching of seller transaction detail data and buyer transaction detail data in the third data; in the case where the seller transaction date is later than or equal to the buyer transaction date, adjusting the face value balance and opening balance of the seller transaction in the third data according to the face value balance and transaction amount proportion of the buyer transaction in the third data to obtain multiple sets of second data.

[0080] In the embodiment, the corresponding bond ID buyer transaction details are found according to the bond buy-sell flag as sell transaction details, and the seller transaction date is less than or equal to the buyer transaction date. This part of the transaction is summarized according to bond ID, investment type, currency, institution information, and region category, and the seller transaction details are allocated according to the buyer transaction face value balance and transaction amount proportion.

[0081] In the embodiment, by processing each set of data in parallel, i.e., using multi-thread concurrent processing technology, the data processing speed and system response capability are greatly improved; at the same time, balance allocation operation is performed on each set of data, and the system can dynamically calculate the cost of the sell transaction according to the matching degree and time sequence of buy-sell transactions, rather than relying only on the earliest or latest transaction. This setting can improve the accuracy of cost calculation and ensure the authenticity of financial reports and the balance of accounts.

[0082] In step S206 of the embodiment, the multiple sets of second data are written into the specified target data table using multi-thread concurrent technology, including: setting and managing multiple threads through programming language technology, wherein each thread corresponds to the write operation of a set of data; determining the data write task of each thread according to the load balancing strategy and the load status of the target database and the processing capacity of each thread; and writing the multiple sets of second data into the specified target data table according to the data write task.

[0083] In the embodiment, the processed data is quickly and efficiently written into the target database table using multi-thread concurrent technology, which not only improves the overall efficiency of data processing, but also reduces the write delay of the database, enhances the real-time and availability of data.

[0084] Those skilled in the art can clearly understand that the method according to the above-mentioned embodiments can be realized by means of software and necessary general hardware platforms, and of course, can also be realized by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disc) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device) to execute the method described in the embodiments of the present application.

[0085] In the present embodiment, a network attack source positioning device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, implementation of hardware, or a combination of software and hardware, is also possible and is contemplated.

[0086] The present embodiment also provides a network attack source positioning device, which can be arranged on the network attack source positioning system. Figure 4 is a structural block diagram of the network attack source positioning device according to the present embodiment, as shown in Figure 4 The device includes:

[0087] The grouping module 41 is configured to obtain transaction data from a database and group transaction data in a preset time window according to different time nodes to obtain a plurality of groups of first data.

[0088] The parallel processing module 42 is configured to perform parallel processing on each group of first data, performs balance apportioning operation on each group of first data to obtain a plurality of groups of second data.

[0089] The writing module 43 is configured to write the plurality of groups of second data into a designated target data table by using multi-thread concurrent technology.

[0090] In one embodiment, the data processing device further includes an obtaining module configured to extract the transaction data from the database according to structured query language technology and a database adapter.

[0091] In one embodiment, the type of transaction data includes at least one of the following: sell transaction date, sell transaction amount, buy transaction date, and buy transaction amount.

[0092] In one embodiment, the parallel processing module is further configured to, according to a preset matching rule, process each group of the first data in parallel, match a sell transaction with a buy transaction in each group of the first data, and obtain a matching result; determine a corresponding relationship between the sell transaction date, the sell transaction amount, the buy transaction date and the buy transaction amount according to the matching result; and perform balance apportioning on each group of the first data according to the corresponding relationship, and obtain a plurality of groups of second data.

[0093] In one embodiment, the matching rule comprises at least one of the following rules: a time sequence matching rule, a bond identifier and investment type matching rule, a balance matching rule, and a priority matching rule.

[0094] In one embodiment, the writing module is further configured to set and manage a plurality of threads by using a programming language technology, wherein each thread corresponds to a write operation of a group of data; determine a data writing task of each thread according to a load balancing strategy and a load status of the target database and a processing capacity of each thread; and write the plurality of groups of second data into a designated target data table according to the data writing task.

[0095] The present application effectively solves the key problems of slow speed, low accuracy, high resource consumption and the like in financial transaction data processing, provides a more efficient and accurate data processing method, and is particularly suitable for high-frequency and large-amount bond transaction scenarios, and can significantly improve the efficiency and accuracy of internal fund management of financial institutions.

[0096] It should be noted that the above modules can be implemented by software or hardware, and for the latter, the following implementation manners can be used, but are not limited thereto: the above modules are located in the same processor; or the above modules are located in different processors in any combination.

[0097] Embodiments of the present application also provide a storage medium having a computer program stored therein, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0098] Optionally, in the present embodiment, the above storage medium can be configured to store a computer program for executing the following steps:

[0099] S1, obtaining transaction data from a database, and grouping transaction data in a preset time window according to different time nodes to obtain a plurality of groups of first data;

[0100] S2, processing each group of the first data in parallel, performing balance apportioning on each group of the first data to obtain a plurality of groups of second data;

[0101] S3, write the plurality of groups of second data into a designated target data table by using a multi-thread concurrent technology.

[0102] Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various storage media that can store computer programs.

[0103] The embodiment of the application further provides an electronic device including a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the method embodiments.

[0104] Optionally, the electronic device can further include a transmission device and an input / output device, wherein the transmission device is connected with the processor, and the input / output device is connected with the processor.

[0105] Optionally, in the embodiment, the processor can be configured to perform the following steps by the computer program:

[0106] S1, obtaining transaction data from a database, and grouping transaction data in a preset time window according to different time nodes to obtain a plurality of groups of first data;

[0107] S2, processing each group of first data in parallel, performing balance apportioning operation on each group of first data to obtain a plurality of groups of second data;

[0108] S3, writing the plurality of groups of second data into a designated target data table by using a multi-thread concurrent technology.

[0109] Optionally, specific examples in the embodiment can refer to examples described in the above embodiments and optional implementation manners, and the embodiment will not be described here.

[0110] In the technical solution of the application, the collection, storage, use, processing, transmission, provision and disclosure of the information such as financial data or user data comply with relevant laws and regulations and do not violate public order and good customs.

[0111] It should be noted that in the embodiments of the application, some industry existing schemes such as software, components and models can be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the technical solution of the application, but it does not mean that the applicant has or will necessarily use the scheme.

[0112] It should be apparent to those skilled in the art that the modules or steps of the application described above can be implemented with general computing devices, which can be centralized on a single computing device or distributed over a network of multiple computing devices, and optionally implemented with program code executable by a computing device, which can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be performed in a different order than shown, or made into individual integrated circuit modules, or multiple modules or steps made into a single integrated circuit module. Thus, the application is not limited to any particular combination of hardware and software.

[0113] The preferred embodiments of the application described above are intended to be merely illustrative, and not limiting. Having now described some embodiments of the application, it will be apparent to those of ordinary skill in the art that many modifications, substitutions, improvements, and other changes can be made to the application without deviating from the spirit and scope of the application.

Claims

1. A data processing method, characterized by, include: Transaction data is retrieved from the database, and the transaction data within a preset time window is grouped according to different time nodes to obtain multiple sets of first data; The multiple sets of first data are processed in parallel, and a total score balancing operation and a balance averaging operation are performed on each set of first data to obtain multiple sets of second data; Using multi-threaded concurrency technology, the multiple sets of second data are written into the specified target data table.

2. The method of claim 1, wherein, Retrieve transaction data from the database, including: The transaction data is extracted from the database using structured query language technology and a database adapter.

3. The method of claim 2, wherein, in, The types of transaction data include at least one of the following: sale transaction date, sale transaction amount, purchase transaction date, and purchase transaction amount.

4. The method of claim 1, wherein, The method further includes: Adjusting the size of the preset time window includes one of the following: predicting market activity ranges based on real-time fluctuations in trading data and time series analysis results; adjusting the size of the preset time window based on the market activity range; or... Based on macroeconomic analysis, historical market trend data, and seasonal fluctuation patterns, market trends and trading patterns are predicted; and the size of the preset time window is adjusted according to the predicted market trends and trading patterns.

5. The method of claim 1, wherein, After grouping transaction data within a preset time window according to different time points to obtain multiple groups of first data, the method further includes: Create a separate data packet for each group of first data; Each data packet is assigned a smart tag and metadata: the smart tag is used to identify the grouping basis, and the metadata is used to record at least one of the following: the timestamp of data packet creation, market interest rate status, bond credit rating, summary face value information, and buying and selling transaction details.

6. The method of claim 1, wherein, Parallel processing of the multiple sets of first data, performing a total score balancing operation on each set of first data, including: The multiple sets of first data are processed in parallel, and the bond transaction data and bond position data of each set of first data are matched. In the event of a discrepancy between the face value of bond transactions and bond positions, the discrepancy is filled by simulating transaction records. Based on the multiple sets of first data obtained from the differences in the ticket balance, multiple sets of third data are obtained.

7. The method of claim 6, wherein, The multiple sets of first data are processed in parallel. For each set of first data, a total score balancing operation and a balance averaging operation are performed to obtain multiple sets of second data, including: The multiple sets of third data are processed in parallel, and the seller transaction details data and buyer transaction details data in the third data are matched by date. If the seller's transaction date is later than or equal to the buyer's transaction date, the seller's transaction balance and account opening amount in the third data are adjusted according to the buyer's transaction balance and transaction amount ratio in the third data to obtain multiple sets of second data.

8. The method of claim 1, wherein, Using multi-threaded concurrency technology, the multiple sets of second data are written into a specified target data table, including: Multiple threads can be set up and managed using programming language techniques, with each thread corresponding to a set of data write operations; Based on the load balancing strategy, the load status of the target database, and the processing capacity of each thread, the data writing task for each thread is determined. According to the data writing task, the multiple groups of second data are written into a specified target data table.

9. A data processing apparatus, characterized by, The method comprises the following steps: The grouping module is configured to obtain transaction data from a database, and group the transaction data within a preset time window according to different time nodes to obtain multiple groups of first data; The parallel processing module is configured to perform parallel processing on the multiple groups of first data, and perform total balance operation and equal allocation operation on each group of first data to obtain multiple groups of second data; The writing module is configured to write the multiple groups of second data into a specified target data table by using a multi-thread concurrent technology.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method in any one of claims 1 to 8 are implemented. 11.An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 8.

12. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 8.

Images