Data processing method, system and equipment for generating portfolio adjustment information

Abstract

The invention provides a data processing method, system and equipment for generating portfolio adjustment information, and belongs to the technical field of data processing. The method comprises the following steps: a memory computing engine loads initial position data of each investment portfolio from a historical database to initialize a memory data set, a data capture module captures transaction events in real time to form a transaction event stream and inputs the transaction event stream into message middleware, and the message middleware performs routing distribution on the transaction events according to a portfolio identifier, the transaction events with the same combination identifier are distributed to the same processing unit, and the processing unit receives the transaction events in the memory computing environment, determines a processing sequence based on the sequence identifier, obtains to-be-updated position data according to the combination identifier, and updates the position data based on the transaction data to obtain real-time position data; and generating adjustment information of the investment portfolio according to the real-time position data. According to the invention, the timeliness and consistency of position data updating are improved, and the real-time performance of adjusting information generation is further improved.

Description

Data processing methods, systems, and devices for generating portfolio adjustment information Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data processing method, system, and device for generating portfolio adjustment information. Background Technology

[0002] With the increasing specialization of the asset management industry, portfolio management places higher demands on the real-time nature of holding data, the timeliness of decision-making, and the system's processing capabilities. The inventors' research revealed that existing portfolio management systems mostly generate holding data through batch calculations or periodic task scheduling. This involves first storing transaction data in the database and then summarizing and calculating the holding results through offline operations. Because the generation process from transaction data to holding data relies on database read / write and batch processing tasks, this method typically has a lag of minutes or even longer, making it difficult to meet the second-level response requirements of high-frequency trading.

[0003] Further research by the inventors revealed that, at the data processing level, traditional solutions often employ a separate architecture for database storage, offline computation, and result querying. Transaction data synchronization is typically based on periodic extraction or interface polling, making it difficult to capture full and continuous changes to the transaction data source. This leads to discrepancies in the temporal consistency of transaction and position data. Furthermore, position calculation results are primarily stored on disks, and the computational chain is limited by I / O bottlenecks. Even with the introduction of message queues to improve data transmission speed, the lack of a data access method that coordinates with the change data capture mechanism and the lack of an ordered incremental update mechanism based on portfolio dimensions often make it difficult to guarantee the event processing order and state consistency of the same portfolio in a concurrent environment. This, in turn, affects the accuracy and generation efficiency of real-time position data, and consequently, the real-time output of portfolio adjustment information.

[0004] In conclusion, improving the real-time generation of portfolio adjustment information to meet the timeliness requirements of high-frequency trading has become a pressing technical problem that needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a data processing method, system, and device for generating portfolio adjustment information, thereby solving the aforementioned technical problems in the prior art.

[0006] On the one hand, to achieve the above objectives, the present invention provides a data processing method, system and device for generating portfolio adjustment information.

[0007] The data processing method for generating portfolio adjustment information includes: an in-memory computing engine loading initial holding data of each portfolio from a historical database to initialize an in-memory dataset, wherein the in-memory dataset includes holding data indexed by portfolio identifiers; the in-memory computing engine includes multiple processing units, and the holding data corresponding to each portfolio identifier is processed by the same processing unit; a data capture module interfaces with the trading system database, captures trading events in real time to form a trading event stream, and inputs it into a message middleware, wherein the trading event includes trading data, a portfolio identifier, and a sequence identifier; the message middleware routes and distributes trading events according to the portfolio identifiers in the trading events, so that trading events with the same portfolio identifier are distributed to the same processing unit corresponding to the portfolio identifier; the processing unit receives trading events in the in-memory computing environment, determines the processing order of trading events based on the sequence identifiers in the trading events, obtains the holding data to be updated from the in-memory dataset based on the portfolio identifiers in the trading events, and updates the holding data to be updated according to the processing order based on the trading data in the trading events to obtain the real-time holding data corresponding to the portfolio identifier; and generates portfolio adjustment information based on the real-time holding data.

[0008] Furthermore, the data capture module connects to the trading system database, and the steps for capturing trading events in real time to form a trading event stream include: the data capture module performs table-level change capture on the trading system database by changing the data capture component, captures three types of trading data in real time: new, updated and deleted data, and constructs trading events based on the trading data, which includes order instructions, transaction records and fund change data.

[0009] Furthermore, the transaction event also includes a business type identifier. The message middleware includes message topics isolated by business type, which includes order type, transaction type, and fund type. The steps of the data capture module inputting the transaction event stream into the message middleware include: writing the transaction event into the corresponding message topic according to the business type identifier. The message middleware is also used to prioritize the routing and distribution of transaction events of the transaction type to the processing unit relative to transaction events of the order type and fund type through a traffic control mechanism.

[0010] Furthermore, the in-memory computing engine partitions and stores the holding data according to the portfolio identifier, and each portfolio maintains an independent holding computing context; the sequence identifier includes a timestamp, and the processing unit performs the following steps to update the holding data to be updated based on the transaction data in the transaction event according to the processing order: the processing unit adopts the event tracing mode according to the first preset frequency, sorts the transaction events by timestamp and superimposes them on the holding data to be updated to obtain real-time holding data.

[0011] Furthermore, the steps for generating portfolio adjustment information based on real-time holdings data include: connecting to an external data interface to obtain multi-dimensional impact data; processing the impact data of each dimension according to predetermined feature processing rules to obtain feature data; inputting the real-time holdings data and feature data into a preset multi-objective optimization model to generate candidate adjustment schemes, wherein the multi-objective optimization model includes multiple optimization objectives and several constraints; and determining adjustment information from the candidate adjustment schemes.

[0012] Furthermore, the steps for connecting to external data interfaces to obtain multi-dimensional impact data include: connecting to the data interface of the fund management system to obtain fund inflow and outflow data of the investment portfolio at a second preset frequency; connecting to the data interface of the market data gateway to obtain trend indicator data of investment products in the investment portfolio at a third preset frequency; and connecting to the data interface of a third-party risk warning system to receive risk event data within a preset time limit after a risk event is triggered.

[0013] Furthermore, the candidate adjustment schemes include the weight data of each component asset in the portfolio, and the adjustment information is repeatedly generated at a preset iteration period. After the step of determining the adjustment information from the candidate adjustment schemes, the method also includes: calculating the weight deviation based on the current weight data and the weight data of the previous period; and pushing the adjustment information to the trading system when the weight deviation reaches a preset deviation threshold.

[0014] On the other hand, to achieve the above objectives, the present invention provides a data processing system for generating portfolio adjustment information.

[0015] The data processing system for generating portfolio adjustment information includes a data processing layer, a decision layer, and an application service layer. The data processing layer comprises an in-memory computing engine, a data capture module, and a message middleware. The in-memory computing engine loads the initial holdings data of each portfolio from the historical database to initialize the in-memory dataset. The in-memory dataset includes holdings data indexed by portfolio identifiers. The in-memory computing engine includes multiple processing units, with the holdings data corresponding to each portfolio identifier processed by the same unit. The data capture module interfaces with the trading system database, captures trading events in real time to form a trading event stream, and inputs it into the message middleware. Trading events include trading data, portfolio... The system consists of a combination identifier and a sequence identifier. The message middleware routes and distributes transaction events based on their combined identifiers, ensuring that events with the same combined identifier are assigned to the same processing unit corresponding to that identifier. The processing unit receives transaction events in an in-memory computing environment, determines the processing order based on the sequence identifier, retrieves the position data to be updated from the in-memory dataset based on the combined identifier, and updates the position data according to the processing order, obtaining the real-time position data corresponding to the combined identifier. The decision layer generates portfolio adjustment information based on the real-time position data. The application service layer outputs the adjustment information.

[0016] On the other hand, to achieve the above objectives, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0017] On the other hand, to achieve the above objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method.

[0018] The present invention provides a data processing method, system, and device for generating portfolio adjustment information. First, an in-memory computing engine loads initial position data from a historical database and initializes the position data stored in an in-memory dataset. Then, a data capture module captures trading events in the trading system database in real time, forming a trading event stream containing trading data, portfolio identifiers, and sequence identifiers, and inputs it into a message middleware. The message middleware routes and distributes trading events based on portfolio identifiers, ensuring that trading events with the same portfolio identifier enter the same processing unit. The processing unit determines the processing order according to the sequence identifier in the in-memory computing environment and locates the position data to be updated from the in-memory dataset according to the portfolio identifier. It then updates the position data based on the trading data to obtain real-time position data. Finally, portfolio adjustment information is generated based on the real-time position data. Through this invention, by employing an in-memory initial position processing mechanism, real-time trading event capture, routing to a fixed processing unit according to portfolio identifiers, and sequential updating according to sequence identifiers, position updates no longer rely on periodic batch processing and disk I / O paths. Furthermore, the update process for the same portfolio maintains sequential consistency, thereby improving the timeliness and stability of trading changes in real-time position generation and providing a more timely position data foundation for subsequent adjustment information output. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0020] Figure 1 is a flowchart of a data processing method for generating portfolio adjustment information provided in Embodiment 1 of the present invention; Figure 2 is a block diagram of a data processing system for generating portfolio adjustment information provided in Embodiment 2 of the present invention; Figure 3 is a hardware structure diagram of a computer device provided in Embodiment 3 of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0022] Example 1: This embodiment of the invention provides a data processing method for generating portfolio adjustment information. This method enables real-time updates of portfolio holdings data based on in-memory computing, even when trading data is constantly changing, and generates portfolio adjustment information accordingly. Specifically, Figure 1 is a flowchart of the data processing method for generating portfolio adjustment information provided in Example 1 of the invention. As shown in Figure 1, the method includes the following steps S101 to S105.

[0023] Step S101: The in-memory computing engine loads the initial holdings data of each portfolio from the historical database to initialize the in-memory dataset.

[0024] The historical database stores the initial holdings data of each portfolio at a preset time point, which can be the start time of the trading day, the beginning time of the day, or the last settlement time. At the preset initialization time each day, the initial holdings data of at least one portfolio are loaded from the historical database into the in-memory dataset to form holdings data indexed by the portfolio identifier. Holdings data includes, but is not limited to, fields such as: portfolio identifier, asset identifier (e.g., security code), holding quantity / shares, cost, available balance, and frozen quantity.

[0025] The in-memory dataset is used to store position data in an in-memory computing environment. It uses the portfolio identifier as the index key and the corresponding portfolio details as the index value, thereby supporting the quick location of the position data to be updated by the portfolio identifier.

[0026] The in-memory computing engine comprises multiple processing units. The holding data corresponding to each combination identifier is processed by the same processing unit, and one processing unit can process holding data corresponding to several combination identifiers. The processing unit is a parallel processing entity within the in-memory computing engine, including threads, coroutines, Actor instances, or partition executors. In this embodiment, a binding relationship is established between the combination identifier and a specific processing unit; that is, holding data corresponding to the same combination identifier is processed by the same processing unit, while different combination identifiers are processed in parallel by different processing units. This ensures a consistent update order for holdings within the same combination and effectively avoids concurrent write conflicts.

[0027] Step S102: The data capture module connects to the transaction system database, captures transaction events in real time to form a transaction event stream, and inputs it into the message middleware.

[0028] The trading system database continuously records changes to trading-related data, such as order placement, transaction record updates, and fund changes. The data capture module interfaces with this database to capture data changes related to the investment portfolio in real time. Captured data changes are organized into trading events, and multiple trading events are arranged in chronological order to form a trading event stream. Each trading event includes trading data, a portfolio identifier, and a sequence identifier. The trading data describes the business content of the change and may include at least one data field from order placement, transaction, and fund changes. The portfolio identifier indicates the investment portfolio to which the trading event belongs. The sequence identifier represents the order of the trading event within the same portfolio, such as a monotonically increasing sequence number, timestamp, or log position number.

[0029] The data capture module writes the transaction event stream to the message middleware, making it the input data source for subsequent routing and distribution. The message middleware can be a message queue or a streaming messaging system, used to cache, forward, and distribute the transaction event stream.

[0030] Step S103: The message middleware routes and distributes transaction events based on the combination identifier in the transaction event, so that transaction events with the same combination identifier are distributed to the same processing unit corresponding to the combination identifier.

[0031] The message middleware parses each transaction event in the transaction event stream, reads the combination identifier, and performs routing accordingly. The goal of routing is to direct transaction events with the same combination identifier to the same processing unit, thereby ensuring that the position updates of that combination are processed serially by the same processing unit, avoiding lock contention, out-of-order updates, or data inconsistencies caused by multiple processing units updating the same combination concurrently.

[0032] Optionally, the message middleware and the processing unit can establish a connection through consumer groups, partition subscriptions, or message pull consumption mechanisms; however, regardless of whether push or pull methods are used, the distribution constraint that the same combination of identifiers corresponds to the same processing unit should be satisfied.

[0033] Step S104: The processing unit receives transaction events in the in-memory computing environment, determines the processing order according to the sequence identifier, and then updates the position data.

[0034] Specifically, the processing unit determines the processing order of transaction events based on the sequence identifier; retrieves the position data to be updated from the memory dataset based on the combined identifier; and performs the update on the position data to be updated based on the transaction data according to the processing order, thereby obtaining the real-time position data corresponding to the combined identifier.

[0035] The processing unit first receives the transaction events distributed to it from the message middleware, forming a queue of events to be processed for the processing unit. Then, it reads the sequence identifier of each transaction event and determines the processing order of transaction events within the same combination accordingly. For example, when the sequence identifier is a timestamp or an incrementing sequence number, they can be sorted in ascending order and processed sequentially to avoid position update errors caused by out-of-order arrival.

[0036] The processing unit reads the combination identifier from the transaction event and uses it as an index key to locate the corresponding position data to be updated in the in-memory dataset. Then, the processing unit performs an incremental update on the position data to be updated based on the transaction data in the transaction event, obtaining the real-time position data.

[0037] For example, when the transaction data is a transaction record, the holding quantity of the corresponding asset can be increased or decreased based on the transaction direction and quantity; when the transaction data is a fund change data, the available fund balance of the portfolio can be adjusted; when the transaction data is an order data, fields such as frozen quantity and available quantity can be changed accordingly. After the update is completed, the real-time holding data is still stored in the in-memory dataset and can be used as input data for subsequent generation of portfolio adjustment information.

[0038] Step S105: Generate portfolio adjustment information based on real-time holdings data.

[0039] Real-time portfolio data reflects the current holding status of the investment portfolio and serves as the foundation for generating adjustment information. Adjustment information includes, but is not limited to, fields such as the asset identifier to be adjusted, the adjustment direction (increase / decrease), the target weight or quantity, and the suggested execution priority. In one embodiment, real-time portfolio data is input into a preset adjustment strategy or model to output adjustment information. In another embodiment, the rules engine may trigger compliance constraint checks based on real-time portfolio data and then output adjustment information. Regardless of the generation method used, all fall within the scope of generating adjustment information based on real-time portfolio data.

[0040] In the data processing method for generating portfolio adjustment information provided in this embodiment, the in-memory computing engine first loads initial position data from the historical database and initializes the position data stored in the in-memory dataset; then, the data capture module captures trading events in the trading system database in real time, forming a trading event stream containing trading data, combination identifiers, and sequence identifiers, and inputs it into the message middleware; the message middleware routes and distributes trading events according to the combination identifiers, so that trading events with the same combination identifiers enter the same processing unit; the processing unit determines the processing order according to the sequence identifiers in the in-memory computing environment, and locates the position data to be updated from the in-memory dataset according to the combination identifiers, and performs updates on the position data based on the trading data to obtain real-time position data; finally, portfolio adjustment information is generated based on the real-time position data. The data processing method for generating portfolio adjustment information provided in this embodiment, through initial position in-memory processing, real-time capture of transaction events, routing to fixed processing units by portfolio identifier, and orderly updating by sequential identifier, enables position updates to no longer rely on periodic batch processing and disk I / O paths. Moreover, the update process of the same portfolio has sequential consistency, thereby improving the timeliness and stability of transaction changes to real-time position generation, and providing a more timely position data foundation for subsequent output of adjustment information.

[0041] Optionally, in one embodiment, the step of the data capture module connecting to the transaction system database and capturing transaction events in real time to form a transaction event stream includes: the data capture module performing table-level change capture on the transaction system database by changing the data capture component, capturing three types of transaction data in real time: new, updated and deleted, and constructing transaction events based on the transaction data, wherein the transaction data includes order instructions, transaction records and fund change data.

[0042] Specifically, the Flink CDC component is deployed, connecting to the trading system database. Capture rules are configured with database tables as objects to capture changes to the monitored tables, including additions, updates, and deletions. Real-time capture of these events, such as order instructions, transaction records, and fund changes, is generated, forming a structured record of transaction data changes. The capture latency is less than or equal to 100ms. When constructing a transaction event, the captured transaction data is encapsulated with its associated identifier and sequence identifier, outputting an event format that can be carried by a message broker.

[0043] The data processing method for generating portfolio adjustment information provided in the embodiments captures and covers three types of changes—addition, update, and deletion—at the table level in the trading system database, making the trading event stream reflect trading changes more completely and in real time. This reduces the risk of inconsistencies in real-time holdings caused by incomplete or delayed capture, and further improves the accuracy and timeliness of real-time holding data.

[0044] Optionally, in one embodiment, the transaction event further includes a business type identifier; the message middleware includes message topics isolated by business type, and the business types include order type, transaction type, and fund type; when the data capture module inputs the transaction event stream into the message middleware, it writes the transaction event into the corresponding message topic according to the business type identifier; the message middleware can also use a traffic control mechanism to prioritize the routing and distribution of transaction type events to the processing unit compared to order type and fund type events.

[0045] Specifically, the message middleware uses the Kafka message transmission module, creating three dedicated topics isolated by business type: delegation topic, transaction topic, and funds topic. These serve as logical channels to carry each type of message, and isolation by business type ensures that different types of events are handled and scheduled separately. The Kafka message transmission module receives transaction events output by the data capture module, using partitioned storage for physical isolation. A traffic control mechanism ensures priority transmission of transaction data, including priority queues, rate limiting and quotas, and scheduling weights. This ensures that transaction-related events are prioritized for consumption by processing units even during peak periods, with a peak throughput of ≥1000 messages / second.

[0046] The data processing method for generating portfolio adjustment information provided in the embodiments can prioritize the timely processing of events that have a more direct impact on holdings by writing transaction events into different message topics according to business type and prioritizing the distribution of transaction type events. This can further reduce the end-to-end latency of real-time holding generation and improve the timeliness of key holding updates during peak trading periods or event congestion.

[0047] Optionally, in one embodiment, the memory computing engine partitions and stores the holding data according to the combination identifier, and each portfolio maintains an independent holding computing context; the sequence identifier includes a timestamp, and the processing unit performs the step of updating the holding data to be updated based on the transaction data in the transaction event according to the processing order, including: the processing unit adopts the event tracing mode according to a first preset frequency, sorts the transaction events by timestamp and superimposes them on the holding data to be updated to obtain real-time holding data.

[0048] Specifically, the in-memory computing engine is implemented based on a distributed memory framework. It maintains a basic dataset of initial daily positions, which is loaded into memory from a historical database at 2:00 AM every day. After receiving Kafka messages, it performs real-time aggregation calculations. Data is stored in partitions according to portfolio identifiers, and each portfolio maintains an independent position calculation context. The position calculation context is a running data structure and processing state maintained by the processing unit for a specific portfolio, including the corresponding position data references of the portfolio in the in-memory dataset, pending transaction event caches and queues, and necessary intermediate states for updating calculations.

[0049] The processing unit, following a predetermined frequency and using an event tracing mode, sorts transaction events by timestamp and overlays them one by one onto the initial or current holding status to obtain the latest real-time holding data. Optionally, the holding data includes three types of data: accounting data, regulatory data, and valuation data. The core computing logic of the processing unit uses single-threaded pipeline technology, that is, it processes transaction events sequentially in a queue within a single execution thread, avoiding memory lock contention caused by multi-threaded concurrent writes to the same combination of holdings.

[0050] The data processing method for generating portfolio adjustment information provided in the embodiments can enhance the consistency and traceability of the update order within the same portfolio by partitioning the holding data by portfolio identifier, maintaining an independent holding calculation context for each portfolio, and introducing an event tracing update method that sorts and overlays by timestamps. This reduces the probability of holding deviations caused by disorder and concurrency, thereby improving the accuracy and stability of real-time holding data.

[0051] Optionally, the first preset frequency is 500ms / time, allowing the processing unit to batch sort, overlay, and update transaction events at a specific rhythm to achieve a balance between throughput and latency. By reducing lock contention and context switching overhead through a single-threaded pipeline approach and organizing the event processing rhythm at an appropriate frequency, computational jitter can be further reduced while ensuring the consistency of the update order for the same combination, thereby improving the determinism and predictability of real-time position updates.

[0052] Optionally, in one embodiment, the step of generating portfolio adjustment information based on real-time holdings data includes: connecting to an external data interface to obtain multi-dimensional impact data; processing the multi-dimensional impact data according to predetermined feature processing rules to obtain feature data; inputting the real-time holdings data and feature data into a preset multi-objective optimization model to generate candidate adjustment schemes, wherein the multi-objective optimization model includes multiple optimization objectives and several constraints; and determining adjustment information from the candidate adjustment schemes.

[0053] Specifically, multidimensional impact data can be used to characterize external information inputs affecting portfolio adjustment decisions, including quantitative or non-quantitative data. Each dimension of impact data is transformed into a unified feature representation usable by the model through preset feature processing rules. These rules may include standardization, encoding, or normalization. Optionally, the multidimensional impact data can be standardized according to these feature processing rules using a feature engineering module. For numerical data, such as volatility and cash balance, normalization is used; for categorical features, such as risk levels, one-hot encoding is used.

[0054] A pre-defined multi-objective optimization model is provided, in which the objectives to be optimized may include profit objectives, risk objectives, or cost objectives, and the constraints may include position constraints, amount constraints, and duration constraints. Users can set constraints, and through the multi-objective optimization model, with the aim of optimizing the objectives to be optimized, candidate adjustment schemes are calculated based on real-time position data and feature data, provided that the constraints are met. Candidate adjustment schemes may include multiple alternative schemes, and each scheme shall at least include a description of the target state of the assets in the portfolio after adjustment. The adjustment information is determined from the candidate adjustment schemes, including selecting the best one or selecting according to rules.

[0055] The data processing method for generating portfolio adjustment information provided in the embodiments further introduces external influence data on the basis of real-time holding data and outputs candidate solutions through featureization and multi-objective optimization. This makes the generation of adjustment information not only based on the latest holding status, but also takes into account external factors and constraints, thereby improving the adaptability and usability of adjustment suggestions and avoiding the deviation caused by generating adjustment suggestions based solely on static or lagging holdings.

[0056] Optionally, in one embodiment, the step of connecting to an external data interface to obtain multi-dimensional impact data includes: connecting to the data interface of a fund management system to obtain fund inflow and outflow data of the investment portfolio at a second preset frequency; connecting to the data interface of a market data gateway to obtain trend indicator data of investment products in the investment portfolio at a third preset frequency; and connecting to the data interface of a third-party risk warning system to receive risk event data within a preset time limit after a risk event is triggered.

[0057] Specifically, the multi-dimensional impact data includes funding data, market data, and risk event data. Specifically, through a data interface connected to the funding management system, it retrieves the portfolio's fund inflow and outflow data every 30 seconds, including available funds balance and redemption request amounts. Through a data interface connected to the market data gateway, it acquires real-time trend indicator data such as prices, volatility, and trading volume of stocks, bonds, and other products in the portfolio at an update frequency of ≤1 second. Through a data interface connected to a third-party risk warning system, it receives risk event data such as macroeconomic events, industry policies, and individual stock risks, categorized by impact level. This risk event data characterizes information triggered by macroeconomic, industry, or individual risks, and data access is completed within 10 seconds of the event being triggered.

[0058] The data processing method for generating portfolio adjustment information provided in the embodiments updates capital and market information at regular intervals and accesses risk event data within a limited time after a risk is triggered. This makes the feature data closer to real-time market and risk changes, thereby enabling the adjustment information generated based on real-time holdings to have a stronger timeliness response capability, especially in forming adjustment suggestions faster when risk events occur.

[0059] Optionally, in one embodiment, the candidate adjustment scheme includes weight data of each component asset in the portfolio; after the step of repeatedly generating adjustment information from the candidate adjustment schemes, the method of this embodiment further includes: calculating the weight deviation based on the current weight data and the weight data of the previous period; and pushing the adjustment information to the trading system when the weight deviation reaches a preset deviation threshold.

[0060] Specifically, the candidate adjustment scheme includes weight data for each asset component in the portfolio, describing the target proportion of each asset in the portfolio. Based on this, when generating adjustment information repeatedly at a preset iteration cycle, after determining the adjustment information in each cycle, the weight deviation from the previous cycle is calculated. This weight deviation measures the magnitude of adjustment change in adjacent iteration cycles. The weight deviation can be calculated as the ratio of the difference between two weights to the previous weight. Adjustment information is only pushed to the trading system when the weight deviation reaches a threshold, avoiding frequent pushes and frequent trading caused by minor fluctuations. For example, adjustment information is pushed to the trading system when the weight deviation is greater than or equal to 3%.

[0061] The data processing method for generating portfolio adjustment information provided in the embodiments continuously updates the adjustment information through an iterative cycle mechanism. At the same time, a weight deviation threshold is introduced to control the timing of the push. This can ensure that the adjustment suggestions are updated with changes in holdings and external factors, while reducing invalid pushes and execution noise caused by frequent small changes, and improving the stability and executability of the adjustment information output to the trading system.

[0062] Optionally, in one embodiment, before the step of generating portfolio adjustment information based on real-time holdings data, the method further includes: synchronizing the real-time holdings data to a cache system, wherein the cache system stores the real-time holdings data as the value and the portfolio identifier as the key; and querying the cache system to obtain the real-time holdings data.

[0063] Specifically, a caching system is set up to provide low-latency read capabilities for external queries or upper-layer services. Using a composite identifier as the key allows for quick location of real-time position data by combination. Querying the cache system to obtain real-time position data serves as the read method before generating adjustment information, enabling the adjustment module to directly read the required real-time position data from the cache. For example, multi-caliber position data can be synchronized to a Redis cache, setting the composite identifier as the key, the position data as the value, and the expiration time to 24 hours, supporting millisecond-level query response.

[0064] The data processing method for generating portfolio adjustment information provided in the embodiments can reduce the direct access pressure of upper-layer modules to the memory computing engine by synchronizing real-time holding data to the cache system and supporting query reading, thereby improving the access response speed of real-time holding data and making the overall link for generating adjustment information more stable and with lower latency.

[0065] Embodiment 2 corresponds to Embodiment 1 above. Embodiment 2 of the present invention provides a data processing system for generating portfolio adjustment information. The technical features and corresponding technical effects are detailed in Embodiment 1 above and will not be repeated here. Figure 2 is a block diagram of the data processing system for generating portfolio adjustment information provided in Embodiment 2 of the present invention. As shown in Figure 2, the system includes a data processing layer 10, a decision layer 20, and an application service layer 30. The data processing layer 10 includes a memory computing engine 11, a data capture module 12, and a message middleware 13.

[0066] The in-memory computing engine 11 is used to load the initial holding data of each portfolio from the historical database to initialize the in-memory dataset. The in-memory dataset includes holding data indexed by the portfolio identifier. The in-memory computing engine 11 includes multiple processing units, and the holding data corresponding to each portfolio identifier is processed by the same processing unit. The data capture module 12 is used to interface with the trading system database, capture trading events in real time to form a trading event stream, and input it into the message middleware 13. The trading event includes trading data, a portfolio identifier, and a sequence identifier. The message middleware 13 is used to process the trading event based on the portfolio identifier in the trading event. The system routes and distributes transaction events so that transaction events with the same combined identifier are distributed to the same processing unit corresponding to the combined identifier. The processing unit receives the transaction events in an in-memory computing environment, determines the processing order of the transaction events based on the sequence identifier in the transaction events, obtains the position data to be updated from the in-memory dataset based on the combined identifier in the transaction events, and updates the position data to be updated based on the transaction data in the transaction events according to the processing order to obtain the real-time position data corresponding to the combined identifier. The decision layer 20 is used to generate the adjustment information of the investment portfolio based on the real-time position data. The application service layer 30 is used to output the adjustment information.

[0067] Optionally, in one embodiment, the data capture module 12 is further configured to capture table-level changes in the transaction system database by changing the data capture component, capture three types of transaction data in real time: new, updated, and deleted data, and construct transaction events based on the transaction data, wherein the transaction data includes order instructions, transaction records, and fund change data.

[0068] Optionally, in one embodiment, the transaction event further includes a business type identifier, and the message middleware 13 includes message topics isolated by business type, wherein the business type includes order type, transaction type, and fund type; when the data capture module 12 inputs the transaction event stream into the message middleware 13, it writes the transaction event into the corresponding message topic according to the business type identifier; the message middleware 13 is also used to use a traffic control mechanism to prioritize the routing and distribution of transaction events of the transaction type to the processing unit relative to transaction events of the order type and the fund type.

[0069] Optionally, in one embodiment, the memory computing engine 11 is further configured to partition and store the holding data according to the combination identifier, and each portfolio maintains an independent holding computing context; the sequence identifier includes a timestamp, and the processing unit is further configured to use an event tracing mode at a first preset frequency to sort the transaction events according to the timestamp and superimpose them onto the holding data to be updated to obtain the real-time holding data.

[0070] Optionally, in one embodiment, the decision layer 20 includes: a multi-factor data access module for connecting to an external data interface to obtain multi-dimensional impact data; a feature engineering module for processing the impact data of each dimension according to a predetermined feature processing rule to obtain feature data; and a multi-objective optimization algorithm module for inputting the real-time position data and the feature data into a preset multi-objective optimization model to generate candidate adjustment schemes and determine the adjustment information from the candidate adjustment schemes, wherein the multi-objective optimization model includes multiple optimization objectives and several constraints.

[0071] Optionally, in one embodiment, when the multi-factor data access module connects to an external data interface to obtain multi-dimensional impact data, the specific steps performed include: connecting to the data interface of the fund management system to obtain the fund inflow and outflow data of the investment portfolio at a second preset frequency; connecting to the data interface of the market data gateway to obtain the trend indicator data of the investment products in the investment portfolio at a third preset frequency; and connecting to the data interface of a third-party risk warning system to receive risk event data within a preset time limit after a risk event is triggered.

[0072] Optionally, in one embodiment, the candidate adjustment scheme includes the weight data of each component asset in the portfolio, and the adjustment information is repeatedly generated at a preset iteration cycle. The decision layer 20 further includes a real-time iteration module, which is used to calculate the weight deviation based on the current weight data and the weight data of the previous cycle. The application service layer is also used to push the adjustment information to the trading system when the weight deviation reaches a preset deviation threshold.

[0073] The data processing layer 10 further includes a cache synchronization module for synchronizing the real-time holding data to a cache system. The cache system stores the data using the combined identifier as the key and the real-time holding data as the value. The processing unit is also used to query the cache system to obtain the real-time holding data.

[0074] Example 3 This example also provides a computer device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers), capable of executing programs. As shown in Figure 3, the computer device 01 of this embodiment includes, but is not limited to, a memory 012 and a processor 011 that can be interconnected via a system bus, as shown in Figure 3. It should be noted that Figure 3 only shows the computer device 01 with components memory 012 and processor 011; however, it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead.

[0075] In this embodiment, the memory 012 (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 012 may be an internal storage unit of the computer device 01, such as the hard disk or memory of the computer device 01. In other embodiments, the memory 012 may also be an external storage device of the computer device 01, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 01. Of course, the memory 012 may include both the internal storage unit and its external storage device of the computer device 01. In this embodiment, the memory 012 is typically used to store the operating system and various reference software installed on the computer device 01, such as the program code of the data processing system for generating portfolio adjustment information in Embodiment 2. In addition, memory 012 can also be used to temporarily store various types of data that have been output or will be output.

[0076] In some embodiments, processor 011 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 011 is typically used to control the overall operation of computer device 01. In this embodiment, processor 011 is used to run program code stored in memory 012 or process data, such as a data processing method for generating portfolio adjustment information.

[0077] Example 4 This example also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App referencing e-commerce platform, etc., which stores a computer program. When the program is executed by a processor, it performs corresponding functions. The computer-readable storage medium of this example is used to store a data processing system for generating portfolio adjustment information. When executed by a processor, it implements the data processing method for generating portfolio adjustment information of Example 1.

[0078] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0079] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0080] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0081] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A data processing method for generating portfolio adjustment information, characterized in that, include: The in-memory computing engine loads initial position data for each portfolio from a historical database to initialize an in-memory dataset. This dataset includes position data indexed by the portfolio's portfolio identifier. The in-memory computing engine comprises multiple processing units, with each portfolio identifier's corresponding position data processed by the same processing unit. A data capture module interfaces with the trading system database, capturing trading events in real-time to form a trading event stream, and inputting it into a message middleware. Each trading event includes trading data, a portfolio identifier, and a sequence identifier. The message middleware routes and distributes trading events based on their portfolio identifiers, ensuring that trading events with the same portfolio identifier are distributed to the same processing unit corresponding to that identifier. The processing unit receives the trading events in the in-memory computing environment, determines the processing order based on the sequence identifier, retrieves the position data to be updated from the in-memory dataset based on the portfolio identifier, updates the position data to be updated according to the processing order, and obtains the real-time position data corresponding to the portfolio identifier. Finally, it generates adjustment information for the portfolio based on the real-time position data.

2. The data processing method for generating portfolio adjustment information according to claim 1, characterized in that, The data capture module connects to the trading system database and captures trading events in real time to form a trading event stream. The steps include: the data capture module performs table-level change capture on the trading system database by changing the data capture component, capturing three types of trading data in real time: new, updated, and deleted data, and constructing trading events based on the trading data. The trading data includes order instructions, transaction records, and fund change data.

3. The data processing method for generating portfolio adjustment information according to claim 1, characterized in that, The transaction event also includes a business type identifier, and the message middleware includes message topics isolated by business type, which includes order type, transaction type and fund type. The step of the data capture module inputting the transaction event stream into the message middleware includes: writing the transaction event into the corresponding message topic according to the business type identifier; The message middleware is also used to prioritize the routing and distribution of transaction events of the transaction type to the processing unit relative to transaction events of the order type and the fund type through a traffic control mechanism.

4. The data processing method for generating portfolio adjustment information according to claim 1, characterized in that, The memory computing engine partitions and stores the holding data according to the combined identifier, and each investment portfolio maintains an independent holding computing context. The sequence identifier includes a timestamp. The step of the processing unit updating the position data to be updated based on the transaction data in the transaction event according to the processing order includes: the processing unit adopts an event tracing mode according to a first preset frequency, sorts the transaction events according to the timestamp and superimposes them on the position data to be updated to obtain the real-time position data.

5. The data processing method for generating portfolio adjustment information according to claim 1, characterized in that, The steps for generating adjustment information for the investment portfolio based on the real-time holdings data include: connecting to an external data interface to obtain multi-dimensional impact data; processing the impact data of each dimension according to predetermined feature processing rules to obtain feature data; inputting the real-time holdings data and the feature data into a preset multi-objective optimization model to generate candidate adjustment schemes, wherein the multi-objective optimization model includes multiple optimization objectives and several constraints; and determining the adjustment information from the candidate adjustment schemes.

6. The data processing method for generating portfolio adjustment information according to claim 5, characterized in that, The steps for connecting to external data interfaces to obtain multi-dimensional impact data include: connecting to the data interface of the fund management system to obtain the fund inflow and outflow data of the investment portfolio at a second preset frequency; connecting to the data interface of the market data gateway to obtain the trend indicator data of the investment products in the investment portfolio at a third preset frequency; and connecting to the data interface of a third-party risk warning system to receive risk event data within a preset time limit after a risk event is triggered.

7. The data processing method for generating portfolio adjustment information according to claim 5, characterized in that, The candidate adjustment scheme includes the weight data of each component asset in the portfolio. The adjustment information is repeatedly generated at a preset iteration period. After the step of determining the adjustment information from the candidate adjustment scheme, the method further includes: calculating the weight deviation based on the current weight data and the weight data of the previous period; and pushing the adjustment information to the trading system when the weight deviation reaches a preset deviation threshold.

8. A data processing system for generating portfolio adjustment information, characterized in that, The system comprises a data processing layer, a decision-making layer, and an application service layer. The data processing layer includes an in-memory computing engine, a data capture module, and a message middleware. The in-memory computing engine loads initial position data for each portfolio from a historical database to initialize an in-memory dataset. This in-memory dataset includes position data indexed by the portfolio's portfolio identifier. The in-memory computing engine includes multiple processing units, with each portfolio identifier's corresponding position data processed by the same processing unit. The data capture module interfaces with the trading system database, captures trading events in real-time to form a trading event stream, and inputs it into the message middleware. Each trading event includes trading data, a portfolio identifier, and a sequence identifier. The message middleware is used for... The transaction events are routed and distributed based on a combination identifier in the transaction event, so that transaction events with the same combination identifier are distributed to the same processing unit corresponding to the combination identifier. The processing unit is used to receive the transaction events in an in-memory computing environment, determine the processing order of the transaction events based on the sequence identifier in the transaction events, obtain the position data to be updated from the in-memory dataset based on the combination identifier in the transaction events, and update the position data to be updated based on the transaction data in the transaction events according to the processing order to obtain the real-time position data corresponding to the combination identifier. The decision layer is used to generate the portfolio adjustment information based on the real-time position data. The application service layer is used to output the adjustment information.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images