Financial product recommendation method and device and related equipment

Abstract

The invention discloses a financial product recommendation method and device and related equipment, and the method comprises the steps: firstly obtaining user operation sequence data and online operation behavior data, and carrying out the preprocessing; and then, inputting the preprocessed data into a pre-trained gating loop unit (GRU) model, and extracting user behavior characteristics. And finally, inputting the features into a pre-trained deep reinforcement learning model to generate a financial product recommendation result. In the training process of the deep reinforcement learning model, user behavior characteristics are used as an input state, financial product recommendation results are used as execution actions, and a reward function is constructed according to financial product risk degrees and / or user personal preferences, so that the model can learn an optimal recommendation strategy, and personalized requirements of users are met while risks are controlled.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and more specifically, to a financial product recommendation method, device and related equipment. Background Art

[0002] Financial product recommendation systems have become an important part of modern Internet services. Their goal is to filter out financial products that users may be interested in from massive amounts of information based on their personal preferences and needs. However, traditional financial product recommendation systems still face many challenges.

[0003] Many financial product recommendation systems rely on feedback information explicitly provided by users, such as ratings, comments, or questionnaires. This approach is inefficient, and due to the subjectivity of users filling out questionnaires, the collected data is often sparse, incomplete, and biased, making it difficult to accurately reflect the user's true preferences, resulting in inaccurate recommendation results; and recommendation systems usually fail to fully utilize the user's behavioral temporal information. The user's behavioral sequence contains rich information about user behavior patterns and long-term preferences. For example, several financial products that a user purchases in a short period of time may have some correlation, while a certain type of product that a user has been browsing for a long time can better reflect his or her stable interests. This temporal information is crucial for understanding user behavior, predicting user future needs, and achieving accurate personalized recommendations. Existing technical solutions, such as content-based recommendation and collaborative filtering, often ignore this valuable temporal information, or only treat it as a static feature, failing to fully tap its potential value, limiting the performance and effectiveness of financial product recommendation systems. Summary of the invention

[0004] In view of the above problems, this application is proposed to provide a financial product recommendation method, device and related equipment to recommend financial products that meet user needs. The specific solution is as follows:

[0005] In a first aspect, a financial product recommendation method is provided, comprising:

[0006] Obtain user operation sequence data and user online operation behavior data;

[0007] Preprocessing the user operation sequence data and the user online operation behavior data to obtain preprocessed user operation sequence data and user online operation behavior data;

[0008] Inputting the preprocessed user operation sequence data and the user online operation behavior data into a pre-trained gated recurrent unit (GRU) model to obtain user behavior features output by the GRU model;

[0009] The user behavior characteristics are input into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model. The deep reinforcement learning model is pre-trained using the user behavior characteristics as input states, the financial product recommendation results as execution actions, and the financial product risk level and / or user personal preferences to construct a reward function.

[0010] Preferably, the method further comprises: obtaining basic information of the user;

[0011] Performing data preprocessing on the basic information of the user to obtain preprocessed basic information of the user;

[0012] Accordingly, the inputting of the user behavior features into the pre-trained deep reinforcement learning model to obtain the financial product recommendation results output by the deep reinforcement learning model includes:

[0013] Inputting the user behavior characteristics and the preprocessed user basic information into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model;

[0014] Correspondingly, the input state of the deep reinforcement learning model training phase further includes the pre-processed user basic information in addition to the user behavior characteristics.

[0015] Preferably, the GRU model is trained in an unsupervised manner using preprocessed user operation sequence data samples and preprocessed user online operation behavior data samples.

[0016] Preferably, the risk level of the financial product includes: Sharpe ratio, Sortino ratio, maximum drawdown, and / or information ratio.

[0017] Preferably, the user's personal preferences include: user risk preference matching score, user investment period preference, user product feature preference, and / or user satisfaction feedback.

[0018] Preferably, the obtaining of user operation sequence data and user online operation behavior data includes:

[0019] When a user is detected entering a financial management channel, the user's operation sequence data and online operation behavior data are obtained.

[0020] Preferably, the deep reinforcement learning model is a dueling deep Q network Dueling DQN model.

[0021] In a second aspect, a financial product recommendation device is provided, comprising:

[0022] A data acquisition unit, used to acquire user operation sequence data and user online operation behavior data;

[0023] A data preprocessing unit, used to perform data preprocessing on the user operation sequence data and the user online operation behavior data to obtain preprocessed user operation sequence data and user online operation behavior data;

[0024] A feature extraction unit, used to input the preprocessed user operation sequence data and the user online operation behavior data into a pre-trained gated recurrent unit (GRU) model to obtain user behavior features output by the GRU model;

[0025] The recommendation unit is used to input the user behavior characteristics into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model. The deep reinforcement learning model is pre-trained by using the user behavior characteristics as input states, the financial product recommendation results as execution actions, and the financial product risk level and / or user personal preferences to construct a reward function.

[0026] In a third aspect, an electronic device is provided, comprising: a memory and a processor;

[0027] The memory is used to store programs;

[0028] The processor is used to execute the program to implement each step of the financial product recommendation method described in any one of the first aspects of the present application.

[0029] In a fourth aspect, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the financial product recommendation method described in any one of the first aspects of the present application are implemented.

[0030] Through the above technical solution, this application obtains user operation sequence data and user online operation behavior data. These data sources are wide-ranging and easy to obtain, providing a rich data foundation for subsequent model training, and combined with deep reinforcement learning models and GRU models, can effectively capture the temporal information of user behavior and understand the user's risk preferences. The GRU model can effectively capture the long-term dependencies and temporal patterns in user behavior sequences, and extract more representative and distinctive user behavior features. The deep reinforcement learning model can learn to select the best recommended products under different user states according to the user's behavioral characteristics through reinforcement learning, and use a reward function based on the risk level of financial products and / or user personal preferences. It can effectively guide the deep reinforcement learning model to learn user preferences and risk tolerance, so that the model can recommend financial products that are more in line with user needs. This combination effectively improves the accuracy and personalization of recommendations, and can better meet the real needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0032] Figure 1 A schematic diagram of an implementation system architecture of the financial product recommendation method provided in an embodiment of the present application;

[0033] Figure 2 A schematic diagram of a terminal structure provided in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a server structure provided in an embodiment of the present application;

[0035] Figure 4 A schematic diagram of a process of a financial product recommendation method provided in an embodiment of the present application;

[0036] Figure 5 Another schematic diagram of the process of recommending financial products provided in the embodiment of the present application;

[0037] Figure 6 A schematic diagram of the structure of a financial product recommendation device provided in an embodiment of the present application;

[0038] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] First of all, the method provided in this application involves concepts related to the gated recurrent unit (GRU) model and the deep reinforcement learning model. For ease of understanding, the related concepts of the GRU model and the deep reinforcement learning model are introduced below.

[0041] GRU model: The GRU model is a variant of the Recurrent Neural Network (RNN) model, which aims to solve the gradient vanishing and gradient exploding problems that are prone to occur in traditional RNN models when processing long sequence data.

[0042] The GRU model controls the flow of information by introducing a reset gate and an update gate to better capture long-distance dependencies. The reset gate determines how much past information should be discarded, while the update gate determines how much past information should be retained and passed to the next time step. Specifically, for each time step t, the GRU model receives the current input x_t and the hidden state h_{t-1} of the previous time step, and calculates the new hidden state h_t. The reset gate r_t acts on the linear combination of x_t and h_{t-1} through the sigmoid function, and its value is between 0 and 1. When r_t is close to 0, it means that most of the past information is discarded; when r_t is close to 1, it means that most of the past information is retained. The update gate z_t is calculated in a similar way to r_t, and also acts on the linear combination of x_t and h_{t-1} through the sigmoid function. Candidate hidden state The tanh function is applied to the linear combination of x_t and r_t and h_{t-1}, where r_t controls the influence of the previous hidden state h_{t-1} on the current candidate hidden state. The final hidden state h_t is controlled by the update gate z_t, combining the previous hidden state h_{t-1} and the candidate hidden state z_t determines how much past information h_{t-1} is retained in the new hidden state h_t and how much new information is introduced Through this gating mechanism, the GRU model can effectively control the flow of information, avoid gradient vanishing and gradient exploding problems, and learn long-term dependencies in long sequence data. The GRU model has been widely used in natural language processing, speech recognition, time series analysis and other fields. For example, in financial product recommendations, the GRU model can be used to model the user's historical operation sequence and extract user behavior characteristics, thereby providing a basis for personalized recommendations.

[0043] Deep reinforcement learning model: Deep reinforcement learning is a combination of deep learning and reinforcement learning. It uses the powerful function approximation ability of deep neural networks to solve complex problems in reinforcement learning. In deep reinforcement learning, the agent learns how to maximize the cumulative reward by interacting with the environment. The agent selects an action based on the current state of the environment, and the environment returns a reward and the next state based on the action. The goal of the agent is to learn a strategy that allows it to choose the best action in any state to obtain the maximum cumulative reward. The core of deep reinforcement learning is to use deep neural networks to represent policy functions, value functions or models. Common deep reinforcement learning algorithms include Q-learning, Deep Q-Network (DQN), Dueling Deep Q-Network (Dueling DQN), Policy Gradient (PG), etc. Deep reinforcement learning has achieved remarkable results in games, robot control, autonomous driving, financial transactions and other fields. In financial product recommendations, deep reinforcement learning can learn a recommendation strategy based on the user's historical behavior and characteristics to maximize the user's long-term benefits or satisfaction.

[0044] Dueling DQN model: The Dueling DQN model is an improved version of the DQN model that learns state value and advantage functions more effectively by modifying the network structure. In the traditional DQN model, the Q value directly represents the expected cumulative reward of taking an action in a certain state. The Dueling DQN model decomposes the Q value into the state value function V(s) and the advantage function A(s,a). The state value function V(s) represents the expected cumulative reward in state s, regardless of which specific action is taken; the advantage function A(s,a) represents the advantage of taking action a in state s relative to the average. The Q value can be calculated by combining V(s) and A(s,a): Q(s,a)=V(s)+A(s,a)-mean(A(s,a')), where a' represents all possible actions. Through this decomposition, the Dueling DQN model can learn state value and advantage functions more effectively, thereby improving learning efficiency and performance. Specifically, the network structure of the Dueling DQN model contains two branches: one branch outputs the state value V(s), and the other branch outputs the advantage function A(s,a) of all actions. The outputs of these two branches are finally combined to obtain the Q value. This structure enables the network to more clearly distinguish the value of the state and the advantages of different actions, so as to better learn the state value function, and can effectively learn even when the Q values ​​of all actions in certain states are similar. The Dueling DQN model has achieved better performance than the DQN model in tasks such as Atari games.

[0045] This application provides a financial product recommendation method that can be applied to Figure 1 The system architecture shown in FIG. 1 may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 A server is included as an example for explanation).

[0046] The terminal 100 or the server 200 can be used alone to execute the financial product recommendation method provided in the embodiment of the present application. In addition, the terminal 100 and the server 200 can also be used together to execute the financial product recommendation method provided in the embodiment of the present application.

[0047] Next describe Figure 1 The product form of the mid-terminal 100;

[0048] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any limitation on this.

[0049] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0050] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160, a speaker 161, a microphone 162, an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.

[0051] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0052] Among them, other input devices 132 can receive input data and so on.

[0053] The display unit 140 may be used to display information input by the user or provided to the user, various menus of the terminal 100, interactive interfaces, file display, and / or playback of any multimedia file. In the embodiment of the present application, the display unit 140 may be used to display various interactive interfaces, processing results, etc. in the financial product recommendation method.

[0054] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0055] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0056] Among them, the memory 120 can be used to store software codes related to the financial product recommendation method, the processor 170 can execute the steps of the financial product recommendation method, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to achieve corresponding functions.

[0057] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0058] In the embodiment of the present application, the RF unit 110 can send data to the server 200 and receive the processing result sent by the server 200. Exemplarily, the RF unit 110 sends the received user operation sequence data and user online operation behavior data to the server 200, and the server 200 obtains relevant financial products by analyzing the user data, and returns the financial products to the terminal 100 for recommendation.

[0059] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0060] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0061] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0062] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0063] Next describe Figure 1 The product form of the server 200;

[0064] Figure 3 A schematic diagram of the structure of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0065] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0066] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0067] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0068] The memory 204 may be used to store software codes related to the financial product recommendation method, and the processor 202 may execute the steps of the financial product recommendation method, and may also schedule other units to implement corresponding functions.

[0069] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0070] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0071] The present application embodiment provides a financial product recommendation method, taking the method applied to a computer device as an example, wherein the computer device may be Figure 1 The terminal 100 or the system consisting of the terminal 100 and the server 200. Figure 4 , the financial product recommendation method specifically includes the following steps:

[0072] Step S100: Acquire user operation sequence data and user online operation behavior data.

[0073] Specifically, user operation sequence data may be a record of a series of user operations, which may include user historical consumption habits, user income sources, user daily spending methods, user loans, overdraft limits, and related information on financial products held by users, etc.; user online operation behavior data may be various behavior data of users on the application platform, which may include the user's stay time on the page, the user's reading effect on fund research reports, the user's attention to different types of financial products, and the user's preference for financial products of different risk types, etc. These data can be extracted from the database of the financial platform.

[0074] In a possible implementation, this step may be to obtain user operation sequence data and user online operation behavior data after detecting that the user enters a financial management channel. The financial management channel is a function entrance provided in the application, and several different financial management products can be pre-configured in the financial management channel.

[0075] Step S110: preprocess the user operation sequence data and the user online operation behavior data to obtain preprocessed user operation sequence data and user online operation behavior data.

[0076] Specifically, data preprocessing can include: Data cleaning: remove invalid or erroneous data, such as missing value processing, outlier detection and processing, etc. Feature engineering: extract useful features from raw data. Data normalization: use techniques such as MinMaxScaler to normalize the data so that the data is distributed between [0, 1], which is conducive to model training. Data partitioning: divide the data into training sets and test sets to maintain the order of the sequence without random shuffling. Discretization: convert continuous features into discrete features. One-hot encoding: for categorical features, such as investment preferences, one-hot encoding can be used to convert them into numerical vectors.

[0077] Step S120: input the preprocessed user operation sequence data and user online operation behavior data into a pre-trained gated recurrent unit (GRU) model to obtain user behavior features output by the GRU model.

[0078] Specifically, the preprocessed data is input into a pre-trained gated recurrent unit (GRU) model. GRU is a recurrent neural network that can process sequence data and capture long-term dependencies in time series. Through training, the GRU model can learn the complex patterns and trends of user behavior and output user behavior features. The last hidden state of the GRU model can be used as a vector representation of user behavior features.

[0079] Step S130: input the user behavior characteristics into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model. The deep reinforcement learning model is pre-trained by using the user behavior characteristics as input state, the financial product recommendation result as execution action, and the financial product risk level and / or user personal preference to construct a reward function.

[0080] Specifically, the user behavior features extracted by the GRU model are input into the pre-trained deep reinforcement learning model for financial product recommendation. In this reinforcement learning framework, the user behavior features are used as the input state of the model, and the recommended financial products are used as the actions to be performed. The goal of the model is to learn a strategy to select the best financial product recommendation action based on the user's behavior features. In order to train the reinforcement learning model, a reward function needs to be defined to evaluate the quality of the recommended action. The design of the reward function is crucial and can comprehensively consider the risk level of the financial product and / or the user's personal preferences. By maximizing the cumulative reward, the deep reinforcement learning model can learn the best financial product recommendation strategy and output the final financial product recommendation results based on the user's behavior features.

[0081] In a possible implementation, the risk level of a financial product may specifically include the Sharpe ratio, the Sortino ratio, the maximum drawdown, and / or the information ratio.

[0082] User personal preferences may specifically include user risk preference matching score, user investment period preference, user product feature preference, and / or user satisfaction feedback.

[0083] In one possible implementation, the deep reinforcement learning model can be a Dueling DQN model. The Dueling DQN model separates the state value estimation from the advantage of taking a specific action. This can lead to more accurate state value estimation, which is crucial for long-term planning in reinforcement learning. In financial markets, rewards may be sparse, and the Dueling DQN model can better handle this situation by learning the general value of being in a specific state, even if the immediate reward is small, and by decoupling the state value and advantage, the model can learn more efficiently and may require fewer training samples to achieve good performance. The improved state value estimation can better generalize to unseen market situations or user profiles.

[0084] In addition, depending on the specific financial scenario, the following deep reinforcement learning models can also be used: Deep Deterministic Policy Gradient - DDPG model, which is a reinforcement learning algorithm based on the Actor-Critic architecture, suitable for continuous action space, and can handle complex portfolio construction problems in financial product recommendations. Proximal Policy Optimization (PPO) model, which is an improved policy gradient algorithm with better stability and robustness, can effectively avoid performance degradation caused by excessive policy updates, etc.

[0085] The combination of deep reinforcement learning and the GRU model can accurately capture the time series information of user behavior and understand the user's risk preference. The GRU model is good at identifying long-term dependencies and patterns in user behavior and extracting key features. The deep reinforcement learning model selects the best recommended products by learning user behavior characteristics, and uses a reward function based on product risk and user preference to optimize recommendations, improve the accuracy and personalization of the recommendation system, and better meet user needs.

[0086] Next, refer to Figure 5 In some embodiments of the present application, another implementation of the financial product recommendation method is introduced:

[0087] Step S200: Obtain user operation sequence data, user online operation behavior data and user basic information.

[0088] Specifically, the user's basic information, such as age, gender, region, income, occupation, education, marital status, number of family members and other demographic information, as well as the user's contact information, registration time, account status and other account information, may also include the user's risk tolerance assessment results, investment goals, investment period, liquidity requirements and other investment-related information. This information can be obtained through user registration forms, questionnaires, user portraits, third-party data providers and other channels.

[0089] Step S210: preprocess the user operation sequence data, the user online operation behavior data and the user basic information to obtain preprocessed user operation sequence data, the user online operation behavior data and the user basic information.

[0090] Specifically, after obtaining the user's basic information, the data preprocessing steps may include data cleaning (removing invalid data, duplicate data, and outliers), missing value processing (filling missing values ​​with mean, median, mode, or more advanced interpolation methods), discretization (converting continuous variables, such as age and income, into discrete variables or intervals), normalization (scaling numerical features, such as Min-Max scaling or Z-score standardization), and one-hot encoding (converting categorical variables, such as gender, occupation, and education level, into numerical vectors).

[0091] Step S220: input the preprocessed user operation sequence data and user online operation behavior data into a pre-trained gated recurrent unit (GRU) model to obtain user behavior features output by the GRU model.

[0092] Step S230: input the user behavior characteristics and the preprocessed user basic information into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model. The deep reinforcement learning model is pre-trained by using the user behavior characteristics and the preprocessed user basic information as input states, the financial product recommendation result as an execution action, and a reward function constructed based on the risk level of the financial product and / or the user's personal preferences.

[0093] Specifically, in this step, the preprocessed user basic information and the user behavior features extracted by the GRU model are input into the pre-trained deep reinforcement learning model. Compared with the previous embodiment, the input state of the deep reinforcement learning model in this embodiment includes not only the user behavior features, but also the preprocessed user basic information. The deep reinforcement learning model still uses the financial product recommendation results as the execution action, and constructs a reward function based on the risk level of the financial product and / or the user's personal preferences. During the training stage of the model, the input state may include not only the user behavior features, but also the preprocessed user basic information, so that the model learns how to combine the user basic information and behavior features to make more accurate financial product recommendations.

[0094] By introducing basic user information, the reinforcement learning model can better understand the user's overall situation, thereby improving the accuracy and personalization of financial product recommendations. For example, the model can recommend more suitable financial products based on the user's age, income, and risk tolerance, and avoid recommending overly conservative or overly aggressive products.

[0095] The following example introduces the training process of the GRU model in the above example:

[0096] The GRU model is trained in an unsupervised manner using preprocessed user operation sequence data samples and preprocessed user online operation behavior data samples.

[0097] Specifically, the preprocessed data samples will be used to train the GRU model to capture the temporal information and patterns in user behavior. In unsupervised training, the GRU model does not require external labels or feedback to guide the learning process. Instead, it relies on an internal state update mechanism to identify long-term dependencies and temporal patterns in the data. The GRU model is trained based on current and past data and predicts future data. An optional method is to divide the data into a series of time windows, each containing a certain number of time steps. The GRU model learns how to predict the data of the next time step based on the data in the current window. It can be optimized by minimizing the difference between the predicted value and the actual value. The mean square error (MSE) can be used as the loss function. During the training process, the parameters of the GRU model are continuously updated to reduce the prediction error. This enables the GRU model to gradually learn the complex patterns in the data and capture subtle changes in user behavior.

[0098] The GRU model can learn the complex characteristics of user behavior and provide strong support for subsequent financial product recommendations. This unsupervised training method enables the GRU model to autonomously learn and extract user behavior characteristics that are critical to the recommendation system without explicit labels.

[0099] The following is a description of a financial product recommendation device provided in an embodiment of the present application. The financial product recommendation device described below and the financial product recommendation method described above can refer to each other.

[0100] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of a financial product recommendation device disclosed in an embodiment of the present application.

[0101] like Figure 6 As shown, the device may include:

[0102] A data acquisition unit 11 is used to acquire user operation sequence data and user online operation behavior data;

[0103] A data preprocessing unit 12, configured to preprocess the user operation sequence data and the user online operation behavior data to obtain preprocessed user operation sequence data and user online operation behavior data;

[0104] A feature extraction unit 13 is used to input the pre-processed user operation sequence data and the user online operation behavior data into a pre-trained gated recurrent unit GRU model to obtain the user behavior features output by the GRU model;

[0105] The recommendation unit 14 is used to input the user behavior characteristics into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model. The deep reinforcement learning model is pre-trained by using the user behavior characteristics as input states, the financial product recommendation results as execution actions, and the financial product risk level and / or user personal preferences to construct a reward function.

[0106] Optionally, the device of the present application may further include:

[0107] A user information acquisition unit, used to acquire basic user information;

[0108] A user information preprocessing unit, used to perform data preprocessing on the basic user information to obtain preprocessed basic user information;

[0109] Accordingly, the process in which the recommendation unit inputs the user behavior features into the pre-trained deep reinforcement learning model to obtain the financial product recommendation results output by the deep reinforcement learning model includes:

[0110] Inputting the user behavior characteristics and the preprocessed user basic information into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model;

[0111] Correspondingly, the input state of the deep reinforcement learning model training phase further includes the pre-processed user basic information in addition to the user behavior characteristics.

[0112] Optionally, the GRU model is trained in an unsupervised manner using preprocessed user operation sequence data samples and preprocessed user online operation behavior data samples.

[0113] Optionally, the risk level of the financial product includes: Sharpe ratio, Sortino ratio, maximum drawdown, and / or information ratio.

[0114] Optionally, the user's personal preferences include: user risk preference matching score, user investment period preference, user product feature preference, and / or user satisfaction feedback.

[0115] Optionally, the process of the data acquisition unit acquiring the user operation sequence data and the user online operation behavior data includes:

[0116] When a user is detected entering a financial management channel, the user's operation sequence data and online operation behavior data are obtained.

[0117] Optionally, the deep reinforcement learning model is a dual deep Q network Dueling DQN model.

[0118] The present application also provides an electronic device in an embodiment. Figure 7 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, tablet computers, teaching large screens, wearable devices, etc. Figure 7 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0119] like Figure 7 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603, so as to implement the financial product recommendation method of the aforementioned embodiment of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0120] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0121] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any financial product recommendation method provided in the embodiment of the present application.

[0122] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0123] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0124] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0125] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0126] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0127] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A financial product recommendation method, characterized in that: include: Obtain user operation sequence data and user online operation behavior data; Preprocessing the user operation sequence data and the user online operation behavior data to obtain preprocessed user operation sequence data and user online operation behavior data; Inputting the preprocessed user operation sequence data and the user online operation behavior data into a pre-trained gated recurrent unit (GRU) model to obtain user behavior features output by the GRU model; The user behavior characteristics are input into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model. The deep reinforcement learning model is pre-trained using the user behavior characteristics as input states, the financial product recommendation results as execution actions, and the financial product risk level and / or user personal preferences to construct a reward function.

2. The method according to claim 1, characterized in that: Also includes: Get basic user information; Performing data preprocessing on the basic information of the user to obtain preprocessed basic information of the user; Accordingly, the step of inputting the user behavior features into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model includes: Inputting the user behavior characteristics and the preprocessed user basic information into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model; Correspondingly, the input state of the deep reinforcement learning model training phase further includes the pre-processed user basic information in addition to the user behavior characteristics.

3. The method according to claim 1, characterized in that The GRU model is trained in an unsupervised manner using preprocessed user operation sequence data samples and preprocessed user online operation behavior data samples.

4. The method according to claim 1, characterized in that: The risk level of the financial product includes: Sharpe ratio, Sortino ratio, maximum drawdown, and / or information ratio.

5. The method according to claim 1, characterized in that The user's personal preferences include: User risk preference matching score, user investment period preference, user product feature preference, and / or user satisfaction feedback.

6. The method according to claim 1, characterized in that The obtaining of user operation sequence data and user online operation behavior data includes: When a user is detected entering a financial management channel, the user's operation sequence data and online operation behavior data are obtained.

7. The method according to any one of claims 1 to 6, characterized in that The deep reinforcement learning model is a dual deep Q network Dueling DQN model.

8. A financial product recommendation device, characterized in that: include: A data acquisition unit, used to acquire user operation sequence data and user online operation behavior data; A data preprocessing unit, used to perform data preprocessing on the user operation sequence data and the user online operation behavior data to obtain preprocessed user operation sequence data and user online operation behavior data; A feature extraction unit, used to input the preprocessed user operation sequence data and the user online operation behavior data into a pre-trained gated recurrent unit (GRU) model to obtain user behavior features output by the GRU model; The recommendation unit is used to input the user behavior characteristics into a pre-trained deep reinforcement learning model to obtain a financial product recommendation result output by the deep reinforcement learning model. The deep reinforcement learning model is pre-trained by using the user behavior characteristics as input states, the financial product recommendation results as execution actions, and the financial product risk level and / or user personal preferences to construct a reward function.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the financial product recommendation method as described in any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the financial product recommendation method as described in any one of claims 1 to 7 is implemented.

Images