Data processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a data processing method and device, electronic equipment and a storage medium, and can be applied to the field of big data. The method comprises the steps that after it is monitored that a user terminal initiates transaction data to a transaction system, the transaction data are input into a pre-trained data processing model, and the data processing model is a multi-task model obtained based on historical transaction data training and comprises risk prevention and control tasks used for predicting transaction risks; the targeted delivery task is used for predicting user preferences; based on a risk prevention and control task and a directional delivery task in the data processing model, processing the transaction data at the same time to obtain risk prevention and control data and directional delivery data; and generating push data according to at least one of the risk prevention and control data and the directional delivery data, and transmitting the push data to the user terminal. According to the method, the transaction data initiated by the user can be dynamically responded, the transaction risk is reduced in real time, personalized service pushing is carried out for the user, and better experience is brought to the user.

Description

Technical Field

[0001] The present application relates to the field of big data, and in particular to a data processing method, device, electronic device and storage medium. Background Art

[0002] In the global financial environment, bank trading systems are facing increasingly complex trading scenarios and diverse customer needs. With the opening of international markets and the increase in cross-border financial activities, trading systems not only need to process a large amount of transaction data, but also need to cope with regulatory requirements in different regions. This complexity poses huge challenges to banks in terms of risk prevention and control and customer service during trading.

[0003] In the related technologies, when the banking system processes the risk prevention and control and personalized push services of transaction data, it performs risk prevention and control and personalized push services respectively by connecting to different external systems. For example, in the process of risk prevention and control, the transaction information is sent to a specific processing system through a specific interface for processing, and the processing system performs a simple list search on the transaction and then feeds back to the transaction system; or after the transaction is completed, the processing system analyzes and processes the transaction data and conducts a post-event risk analysis. If the data processing is not timely, the above risk processing methods often make it difficult to process and analyze massive data in real time, resulting in a lag in risk identification and response, and it is difficult to ensure transaction security. In addition, the current personalized push service has the following disadvantages. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, an apparatus, an electronic device, and a storage medium to solve at least one of the above-mentioned technical problems.

[0005] According to a first aspect of the present application, a data processing method is provided, comprising:

[0006] After monitoring that the user terminal initiates transaction data to the transaction system, the transaction data is input into a pre-trained data processing model, wherein the data processing model is a multi-task model trained based on historical transaction data, including a risk prevention and control task for predicting transaction risks, and a targeted delivery task for predicting user preferences;

[0007] Based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, the transaction data is processed simultaneously to obtain risk prevention and control data and targeted delivery data;

[0008] Generate push data according to at least one of the risk prevention and control data and the targeted delivery data, and transmit the push data to the user terminal.

[0009] In a possible implementation, the method further includes:

[0010] Acquire historical transaction data from the transaction system;

[0011] Preprocessing the historical transaction data to obtain a training data set;

[0012] Extracting shared features for risk prevention and control tasks and targeted delivery tasks in the initial data processing model based on the training data set, wherein the shared features include at least one of the following: transaction frequency, transaction amount, geographic location, device information, and historical behavior pattern;

[0013] An initial machine learning model is trained based on the shared features to obtain the data processing model; wherein the initial machine learning model includes at least one of the following: logistic regression, K nearest neighbor, support vector machine and decision tree.

[0014] In a possible implementation manner, preprocessing the historical transaction data to obtain a training data set includes:

[0015] Based on Python language, the historical transaction data is preprocessed to filter out invalid data in the historical transaction data to obtain cleaned data;

[0016] Performing data structuring processing on the cleaned data to obtain structured data;

[0017] Based on the structured data, determine each data type in the historical transaction data, and draw a corresponding box plot for each data type;

[0018] Determine an outlier range based on the box plot, search for corresponding outliers based on the outlier range, and smooth or eliminate the outliers to obtain outlier processing data;

[0019] The training data is obtained based on the exception processing data.

[0020] In a possible implementation manner, after preprocessing the historical transaction data, the method further includes:

[0021] Dividing the preprocessed historical transaction data into the training data set and the validation data set;

[0022] The step of training the initial data processing model based on the shared features to obtain the data processing model includes:

[0023] Training an initial data processing model based on the shared features to obtain a data processing intermediate model;

[0024] Based on the verification data set, it is verified whether the learning curvature of the data processing intermediate model meets the expected convergence ability. When it meets the expected convergence ability, the data processing model is obtained.

[0025] In a possible implementation manner, before simultaneously processing the transaction data based on the risk prevention and control task and the targeted delivery task in the data processing model, the method further includes:

[0026] Extracting shared features from the transaction data based on a feature extraction module in the data processing model;

[0027] The step of processing the transaction data based on the risk prevention and control task and the targeted delivery task in the data processing model includes:

[0028] Based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, the shared features are processed simultaneously.

[0029] In a possible implementation, generating push data according to at least one of the risk prevention and control data and the targeted delivery data includes at least one of the following:

[0030] In response to the risk prevention and control monitoring instruction message initiated by the user terminal, generating push data according to the risk prevention and control data;

[0031] In response to the directional push indication message initiated by the user terminal, generating push data according to the directional delivery data;

[0032] In response to the risk prevention and control monitoring instruction message and the targeted push instruction message initiated by the user terminal, push data is generated according to the risk prevention and control data and the targeted delivery data.

[0033] In a possible implementation, the method is applied to a targeted delivery tool, and the targeted delivery tool is integrated into the trading system via an application programming interface (API).

[0034] According to a second aspect of the present application, there is provided a data processing device, comprising:

[0035] A model input module is used to input the transaction data into a pre-trained data processing model after monitoring the transaction data initiated by the user terminal to the transaction system. The data processing model is a multi-task model trained based on historical transaction data, including a risk prevention and control task for predicting transaction risks and a targeted delivery task for predicting user preferences;

[0036] A model processing module, configured to process the transaction data based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, and obtain risk prevention and control data and targeted delivery data;

[0037] The data push module is used to generate push data according to the risk prevention and control data and the targeted delivery data, and transmit the push data to the user terminal.

[0038] In a possible implementation, the device further includes:

[0039] A data acquisition module, used to acquire historical transaction data from the transaction system;

[0040] A preprocessing module, used to preprocess the historical transaction data to obtain a training data set;

[0041] A first feature extraction module, configured to extract shared features for risk prevention and control tasks and targeted delivery tasks in the initial data processing model based on the training data set, wherein the shared features include at least one of the following: transaction frequency, transaction amount, geographic location, device information, and historical behavior pattern;

[0042] A training module is used to train an initial machine learning model based on the shared features to obtain the data processing model; wherein the initial machine learning model includes at least one of the following: logistic regression, K nearest neighbor, support vector machine and decision tree.

[0043] In a possible implementation, the preprocessing module includes:

[0044] A cleaning unit, used for preprocessing the historical transaction data based on Python language to filter out invalid data in the historical transaction data to obtain cleaned data;

[0045] A structuring unit, used for performing data structuring processing on the cleaned data to obtain structured data;

[0046] A box plot drawing unit, used to determine each data type in the historical transaction data based on the structured data, and draw a corresponding box plot for each data type;

[0047] An exception processing unit, used to determine an abnormal value range based on the box plot, search for corresponding abnormal values ​​based on the abnormal value range, and smooth or eliminate the abnormal values ​​to obtain abnormal processing data;

[0048] An acquisition unit is used to obtain the training data based on the exception processing data.

[0049] In a possible implementation, the device further includes:

[0050] A data set division module, used for dividing the pre-processed historical transaction data into the training data set and the verification data set;

[0051] The training module is specifically used to: train an initial data processing model based on the shared features to obtain a data processing intermediate model; verify whether the learning curvature of the data processing intermediate model meets the expected convergence ability based on the verification data set, and obtain the data processing model when it meets the expected convergence ability.

[0052] In a possible implementation, the device further includes:

[0053] A second feature extraction module, configured to extract shared features from the transaction data based on the feature extraction module in the data processing model;

[0054] The model processing module is specifically used to: process the shared features simultaneously based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model.

[0055] In a possible implementation, the data push module includes at least one of the following:

[0056] A first response generating unit, configured to generate push data according to the risk prevention and control data in response to the risk prevention and control monitoring instruction message initiated by the user terminal;

[0057] a second response generating unit, in response to the directional push indication message initiated by the user terminal, generating push data according to the directional delivery data;

[0058] The third response generating unit generates push data according to the risk prevention and control data and the targeted delivery data in response to the risk prevention and control monitoring instruction message and the targeted delivery instruction message initiated by the user terminal.

[0059] In a possible implementation, the device is applied to a targeted delivery tool, and the targeted delivery tool is integrated into the transaction system via an application programming interface API.

[0060] According to a third aspect of the present application, there is provided an electronic device, comprising: a memory, a processor;

[0061] The memory stores computer-executable instructions;

[0062] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the data processing method provided in any one of the first aspects above.

[0063] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the data processing method provided in any one of the first aspects above.

[0064] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the data processing method provided in any one of the first aspects above.

[0065] The data processing method, device, electronic device and storage medium provided in the embodiment of the present application, after monitoring that the user terminal initiates transaction data to the transaction system, the transaction data is input into the pre-trained data processing model, the data processing model is a multi-task model trained based on historical transaction data, including a risk prevention and control task for predicting transaction risks, and a targeted delivery task for predicting user preferences, based on the risk prevention and control task and the targeted delivery task in the data processing model, the transaction data is processed at the same time to obtain risk prevention and control data and targeted delivery data, and push data is generated according to at least one of the risk prevention and control data and the targeted delivery data, and the push data is transmitted to the user terminal. In this process, when the transaction data is monitored, the transaction data is input into the data processing model containing the risk prevention and control task and the targeted delivery task, and the transaction data is processed simultaneously by using these two tasks to obtain the risk prevention and control data and the targeted delivery data, and transmitted to the user terminal in the form of push data, without the need to be implemented through an external interface, so that the transaction data initiated by the user can be dynamically responded to, the transaction risk can be reduced in real time, and personalized service push can be performed for the user, bringing a better experience to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0067] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0068] Figure 2 A flowchart of another data processing method provided in an embodiment of the present application is provided;

[0069] Figure 3 It is a box plot of data type a in the embodiment of the present application;

[0070] Figure 4 It is a box plot of data type b in the embodiment of the present application;

[0071] Figure 5 It is a box plot of data type c in the embodiment of the present application;

[0072] Figure 6 This is an example diagram of risk distribution of data type a in the embodiment of the present application;

[0073] Figure 7This is an example diagram of risk distribution of data type b in the embodiment of the present application;

[0074] Figure 8 This is a validation curve diagram of the logistic regression model in the embodiment of the present application;

[0075] Fig. 9 It is a validation curve diagram of the decision tree model in the embodiment of the present application;

[0076] Fig.10 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0077] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0078] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0079] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0080] First, the terms involved in this application are explained:

[0081] Trading system: A computerized system used in financial markets that is designed to automate and optimize the buying and selling process of financial assets. In this embodiment, the trading system is an expanded concept that can be used to refer to payment systems and similar systems, where the payment system refers to a system that supports the process of clearing debts and liabilities through accounts within the same bank or between different banks.

[0082] Risk prevention and control: It is a means of identifying, assessing, monitoring and mitigating various risks associated with financial transactions to ensure the security, integrity and compliance of transactions.

[0083] Targeted delivery: is an identification service that identifies the user's transaction behavior, preferences, and habits, and accurately delivers personalized content, advertising, or product recommendations to target users. Preferences refer to the user's preference habits in daily behavior, which are not known to the user but have certain rules.

[0084] Batch tasks: In addition to online real-time transactions, it is necessary to unify or centralize certain business processing, or process data that depends on related systems to pave the way for online transactions, or issue some reconciliation texts, transaction reports and other action processes for business personnel / regulatory systems.

[0085] The business processing method, device, server and computer-readable storage medium provided in the present application can be used in the field of financial technology, and can also be used in any field outside the field of financial technology. The application field of the business processing method, device, server and computer-readable storage medium of the present application is not limited.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0087] In the related technology, the risk control in the bank transaction system takes the anti-money laundering process as an example. During the operation, the transaction system or payment system needs to send the transaction information to the specific system through a specific interface for processing. The interface between the processing system and the transaction system is a non-real-time system. The processing system performs a simple list search on the transaction and then feeds back to the transaction system; or performs post-management. After the transaction is completed, the risk processing system analyzes and processes the data and conducts corresponding risk analysis. The data processing is not timely and cannot meet the current multi-scenario risk control of large commercial banks. Similarly, in the targeted delivery process of the transaction system, it is difficult to meet the real-time personalized push to users, resulting in a poor user experience.

[0088] In view of this, the embodiment of the present application provides a data processing method, device, electronic device and storage medium. After monitoring that a user terminal initiates transaction data to a transaction system, the transaction data is input into a pre-trained data processing model. The data processing model is a multi-task model trained based on historical transaction data, including a risk prevention and control task for predicting transaction risks, and a targeted delivery task for predicting user preferences. Based on the risk prevention and control task and the targeted delivery task in the data processing model, the transaction data is processed at the same time to obtain risk prevention and control data and targeted delivery data. According to at least one of the risk prevention and control data and the targeted delivery data, push data is generated, and the push data is transmitted to the user terminal. In this process, when the transaction data is monitored, the transaction data is input into a data processing model containing risk prevention and control tasks and targeted delivery tasks, and the transaction data is processed simultaneously using these two tasks to obtain risk prevention and control data and targeted delivery data, and transmitted to the user terminal in the form of push data, without the need to be implemented through an external interface, so that the transaction data initiated by the user can be dynamically responded to, the transaction risk can be reduced in real time, and accurate service push can be performed for the user, bringing a better experience to the user.

[0089] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0090] Figure 1 A flow chart of the data processing method provided in this application, such as Figure 1 As shown, a data processing method is provided, which may include steps S101-S103:

[0091] Step S101: After monitoring that a user terminal initiates transaction data to a transaction system, the transaction data is input into a pre-trained data processing model. The data processing model is a multi-task model trained based on historical transaction data, including a risk prevention and control task for predicting transaction risks, and a targeted delivery task for predicting user preferences.

[0092] In this embodiment, by real-time monitoring of transaction requests initiated by user terminals, when users perform operations in the transaction system (such as purchases, transfers, etc.), transaction data is captured in real time. For example, real-time data capture technology, such as a stream processing framework, can be used to capture transaction requests from user terminals, or an API gateway or proxy server can be deployed to intercept and record transaction requests for subsequent processing.

[0093] Next, the captured transaction data is input into a pre-trained data processing model (which can be trained by the execution terminal of the method provided in this embodiment, or trained by other devices or servers, and this embodiment does not specifically limit this). The model has multi-task processing capabilities, including risk prevention and control tasks, which can predict potential risks in transactions. By analyzing transaction data, the model can identify abnormal behavior or possible fraudulent activities; and targeted delivery tasks, which can be used to predict user preferences and recommend personalized services to users by analyzing their transaction habits and preferences.

[0094] Exemplarily, the method provided in this embodiment may be applied to a targeted delivery tool, which is integrated into the trading system via an application programming interface (API).

[0095] In this example, the targeted delivery tool can transmit the data of the trading system to the targeted delivery tool in real time through the API interface. The API supports two-way communication to receive transaction data and return analysis results.

[0096] It can be understood that the targeted delivery tool is a software or platform for personalized push. Its main function is to display personalized push information to specific user groups based on the user's behavior, interests, preferences and other relevant data. In this embodiment, based on the data processing capability of the targeted delivery tool, in addition to pushing personalized services to users, it can also realize risk prevention and control functions to further optimize the user experience.

[0097] In some embodiments, the above method may also be applied to other devices or servers, and this embodiment does not specifically limit the application subject.

[0098] Step S102: Based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, the transaction data is processed simultaneously to obtain risk prevention and control data and targeted delivery data.

[0099] In the data processing model, transaction data is simultaneously input into the risk prevention and control task and the targeted delivery task for processing. The risk prevention and control task is used to analyze and process the transaction data, and the targeted delivery task is used to analyze and process the transaction data to simultaneously generate two types of output data, namely, risk prevention and control data, which is used to evaluate the security of the transaction and may include information such as whether it is a risk category, and targeted delivery data, which is used to identify user preferences and may include information such as recommended products or services.

[0100] Step S103: Generate push data according to at least one of the risk prevention and control data and the targeted delivery data, and transmit the push data to the user terminal.

[0101] Next, this embodiment further introduces the above steps of generating push data according to at least one of the risk prevention and control data and the targeted delivery data, which can be done in the following manner:

[0102] In one example, in response to a risk prevention and control monitoring instruction message initiated by the user terminal, push data is generated according to the risk prevention and control data.

[0103] In this example, when the user wants to understand the security status of the current transaction or perform risk control on certain transaction behaviors, he can initiate a risk control monitoring indication message to obtain the corresponding risk control data in the pushed data when initiating a transaction request. When the risk control data predicts that the current transaction is risky, the user can be prompted for the transaction risk, the transaction can be stopped, or other risk control methods can be used to ensure transaction security.

[0104] In another example, in response to a targeted push indication message initiated by the user terminal, push data is generated according to the targeted delivery data.

[0105] Accordingly, when a user is interested in the personalized recommendation service of the bank and hopes to receive relevant content, he or she can initiate a targeted push indication message to obtain the corresponding targeted delivery data in the push data when initiating a transaction request.

[0106] In yet another example, in response to the risk prevention and control monitoring indication message and the targeted push indication message initiated by the user terminal, push data is generated according to the risk prevention and control data and the targeted delivery data.

[0107] Combining the above two examples, when the user has both risk prevention and control needs and personalized push needs, push data is generated based on the user's risk prevention and control monitoring instruction message and targeted push instruction message, using risk prevention and control data and targeted delivery data, so that the user can understand the security of the transaction while obtaining relevant personalized content.

[0108] In some examples, in addition to the above examples, other methods can be used to generate push data. For example, in addition to risk prevention and control data and targeted delivery data, other analysis information of transaction data (such as time series analysis to optimize push time) can be combined to generate push data, etc. This embodiment does not go into details.

[0109] It can be understood that in response to the conditions or states on which the executed operations depend, when the dependent conditions or states are met, one or more operations executed can be in real time or have a set delay; unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0110] Through the above technical solutions, we can flexibly respond to different needs of users, provide customized push data, and enhance user experience and transaction security.

[0111] Figure 2 : is a flow chart of another data processing method provided by an embodiment of the present application. On the basis of the above embodiment, this embodiment further proposes a model training process, which can effectively improve the prediction accuracy of the model. In addition to the above steps S101-S103, the method provided by this embodiment can also include the following steps S201-S204:

[0112] Step S201: Acquire historical transaction data from the transaction system.

[0113] As the transaction system (or payment system) that supports various banking businesses, it is the convergence point of all transactions and can obtain the most complete and extensive transaction information. By obtaining historical transaction data from the transaction system and mining and analyzing the historical transaction data, the model training effect can be made more in line with the actual situation.

[0114] Step S202: pre-process the historical transaction data to obtain a training data set.

[0115] Exemplarily, in the above step S202, the historical transaction data is preprocessed to obtain a training data set in the following manner:

[0116] Based on Python language, the historical transaction data is preprocessed to filter out invalid data in the historical transaction data to obtain cleaned data;

[0117] Performing data structuring processing on the cleaned data to obtain structured data;

[0118] Based on the structured data, determine each data type in the historical transaction data, and draw a corresponding box plot for each data type;

[0119] Determine an outlier range based on the box plot, search for corresponding outliers based on the outlier range, and smooth or eliminate the outliers to obtain outlier processing data;

[0120] The training data is obtained based on the exception processing data.

[0121] In this example, Python has a rich data processing library, such as the Pandas library, which provides powerful data manipulation and analysis functions, and can easily clean, convert and analyze data. This embodiment uses Python to effectively improve the efficiency of data cleaning. In this example, invalid data may include duplicate data, missing data, log data, and formatted data that are not related to risk prevention and control and personalized push. In addition to cleaning the data, Python can also be used to classify and label all transaction data for model training.

[0122] Next, the cleaned data is structured (the Pandas library can be used for data structuring). For example, the structured data is shown in Table 1 below. The data used is the transaction data of a certain payment system within two days. It contains 284,807 transaction data (828,4807 rows×31 columns), each row represents a transaction, of which 492 are risky transaction data.

[0123] Table 1

[0124]

[0125] After structuring the data, determine the data type (determine the data type of each data column, such as transaction amount, time, etc.) and draw a box plot. You can use the box plot drawing tool to draw a box plot for each data type. The box plot is as follows: Figure 3-Figure 5 As shown, they are box plots of data types a, b, and c, respectively. Figure 3 and Figure 4 Outliers are identified in the data, and the data accuracy is improved by removing the identified outliers or smoothing them (the smoothed data can continue to be used as data for model training).

[0126] In some embodiments, considering that different data types have different importance for risk identification and targeted delivery in practical applications, especially for risk identification, risk identification is taken as an example below. In order to more accurately understand the behavior pattern of risky customers under specific indicators (i.e., data types), single factor analysis can use a sampling method to extract risk and non-risk samples in a one-to-one ratio because of sample bias. In the case of known risky transactions and non-risky transactions, the distribution of risks under different indicators is analyzed, such as Figure 6 and Figure 7 As shown in the figure, they are the distribution of data type a and data type b. Specifically, Figure 6 The distribution of (fraudulent transactions) of a is shown, and the horizontal axis shows the (normalized) value of the corresponding feature of a. Figure 7The distribution of b is shown. Furthermore, by drawing box plots of these indicators, we can observe the distribution of different indicators after they are classified into fraud and non-fraud categories. Generally speaking, the greater the difference (the distribution of medians, quartiles or outliers is obviously different), the stronger the indicator's ability to identify risks. When training the model, a higher weight can be assigned to the data type. In addition, when the amount of transaction data is large, there will be more data types. Through correlation analysis, we can analyze the correlation between observed indicators and divide the indicators with strong correlation into one data type to facilitate model training and prediction.

[0127] Step S203: extracting shared features for risk prevention and control tasks and targeted delivery tasks in the initial data processing model based on the training data set, wherein the shared features include at least one of the following: transaction frequency, transaction amount, geographic location, device information, and historical behavior patterns.

[0128] In this embodiment, by using shared features to train the model, the model can simultaneously identify whether it is a risky transaction and carry out targeted delivery for transaction data. These shared features may include transaction frequency, that is, the number of times a user conducts transactions within a specific time period. This can help identify abnormal transaction behavior (such as too frequent transactions) for risk prevention and control, and can also be used to identify active users for targeted delivery; transaction amount, the amount of a single transaction, abnormal large transactions may be a sign of risk, and high-consumption users may be the target of targeted delivery; geographic location, geographic location information when the user conducts transactions, abnormal changes in geographic location may indicate potential fraud, and can also be used for personalized service delivery related to geographic location; device information, the type and characteristics of the device used by the user when conducting transactions, changes in the device may be a signal of risk, and specific device users may be target customers for certain products; historical behavior patterns, users' past transactions and behavior patterns, these patterns can help identify abnormal behavior (such as transactions inconsistent with past behavior) and predict users' future personalized needs. In some embodiments, shared features may also include other information such as time features, which will not be elaborated here.

[0129] Step S204: training an initial machine learning model based on the shared features to obtain a data processing model; wherein the initial machine learning model includes at least one of the following: logistic regression, K nearest neighbor, support vector machine, and decision tree.

[0130] In practical applications, you can choose a suitable model for training according to your needs. Logistic regression is suitable for linearly separable classification tasks and is easy to explain. K-nearest neighbor (KNN) is a classification or regression method based on distance metrics and is suitable for small-scale data sets. Support vector machine (SVM) is suitable for classification tasks with high-dimensional data and complex boundaries. Decision trees are easy to understand and explain and are suitable for processing nonlinear relationships. In addition, neural network models can also be used for training. Compared with the above models, multiple tasks can be processed simultaneously by sharing network layers, and representations can be shared between tasks, thereby improving overall performance and generalization capabilities without the need for additional design and adjustment of the model. For example, logistic regression needs to expand multi-logistic regression to handle multi-class classification problems.

[0131] In some embodiments, the above-mentioned shared features can also be used to train the above-mentioned multiple models at the same time, so as to select a more accurate model, use the shared features for model training, and respectively construct machine learning models such as logistic regression, K nearest neighbor, support vector machine, and decision tree. Taking a certain experiment as an example, the development sample is 80% of the full sample randomly selected, and the accuracy of the four models is compared respectively, among which: the training accuracy of the logistic regression classifier is 94.0%; the training accuracy of the K nearest neighbor classifier is 94.0%; the training accuracy of the SVM classifier is 93.0%; and the training accuracy of the decision tree classifier is 90.0%. After adjusting the model hyperparameters using grid search, the receiver operating characteristic curves (ROC) of the four models are compared: logistic regression is: 0.9750525136535498, K nearest neighbor is 0.9345679876767959, support vector classifier is 0.9821033468701862, and decision tree classifier is 0.910075619661112. It can be seen that the accuracy of logistic regression and K nearest neighbor is better in the development samples. One of them can be selected as the initial model for training to improve the model accuracy.

[0132] In some embodiments, in order to further improve the accuracy of the model, the model is verified using a verification set, and when the model meets the expected convergence capability, a data processing model is obtained. Specifically, after the historical transaction data is preprocessed in the above steps, the following steps may also be included:

[0133] Dividing the preprocessed historical transaction data into the training data set and the validation data set;

[0134] The method of training the initial data processing model based on the shared features to obtain the data processing model includes: training the initial data processing model based on the shared features to obtain a data processing intermediate model; and verifying whether the learning curvature of the data processing intermediate model meets the expected convergence ability based on the verification data set, and obtaining the data processing model when it meets the expected convergence ability.

[0135] In this embodiment, the preprocessed historical transaction data is divided into two parts: a training data set and a validation data set. The training data set is used to train the model, while the validation data set is used to evaluate the performance of the model and verify its generalization ability. An initial data processing model is trained using shared features. This process will produce an intermediate data processing model (i.e., not the final model for prediction). The shared features may include transaction amount, timestamp, transaction type, etc. These features exist in both the training and validation data sets and are used for model training.

[0136] The performance of the data processing intermediate model is verified by using a validation data set. Specifically, the model can be optimized by checking whether the learning curvature of the model meets the expected convergence ability. The learning curvature refers to the change in the loss function of the model during the training process, which reflects the speed and effect of the model learning. The expected convergence ability refers to the loss function of the model reaching a certain threshold after a certain number of iterations (determined by technicians in this field in combination with actual applications), or the rate of change of the loss function is lower than a predetermined value. The learning curve usually uses the value of the loss function as the vertical axis to show its changes on the training set and the validation set, which meets the expected convergence ability, that is, the training loss and validation loss tend to be stable and the validation loss is low. For example, Figure 8 and Fig. 9 As shown, they are the validation curve diagrams of the logistic regression model of the example of this embodiment and the decision tree model. The model is validated using the validation set, and the learning rate curve is used to verify whether the model has overfitting. It can be seen that the decision tree has a better overfitting problem.

[0137] Furthermore, before the transaction data is input into the model for processing, feature extraction is first performed to improve the model processing efficiency. Specifically, before the transaction data is processed simultaneously based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, the following steps may also be included: based on the feature extraction module in the data processing model, the shared features in the transaction data are extracted; based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, the transaction data is processed simultaneously, including: based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, the shared features are processed simultaneously.

[0138] In the above-mentioned model training and model use process, this embodiment uses training shared features, and uses two tasks to analyze and process the shared features at the same time to output prediction results corresponding to the two tasks. In some embodiments, in addition to using shared features to train the data processing model, the model can also be trained in combination with (or alone using) proprietary features, for example, by extracting features from transaction data, extracting proprietary features for training risk prevention and control tasks (such as transaction failure rate), and proprietary features for training targeted delivery tasks (such as transaction preference categories).

[0139] In summary, when the transaction data is monitored, the present embodiment inputs the transaction data into a data processing model containing risk prevention and control tasks and targeted delivery tasks, and uses these two tasks to process the transaction data at the same time to obtain risk prevention and control data and targeted delivery data, and transmits them to the user terminal in the form of push data without going through an external interface. It can dynamically respond to transaction data initiated by users, reduce transaction risks in real time, and push accurate services to users, bringing users a better experience.

[0140] Fig.10 is a structural diagram of a data processing device provided in an embodiment of the present application, such as Fig.10 As shown, the device includes a model input module 101, a model processing module 102, and a data push module 103, wherein:

[0141] The model input module 101 is used to input the transaction data into the pre-trained data processing model after monitoring the transaction data initiated by the user terminal to the transaction system. The data processing model is a multi-task model trained based on historical transaction data, including a risk prevention and control task for predicting transaction risks and a targeted delivery task for predicting user preferences;

[0142] A model processing module 102, configured to process the transaction data based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, and obtain risk prevention and control data and targeted delivery data;

[0143] The data push module 103 is used to generate push data according to the risk prevention and control data and the targeted delivery data, and transmit the push data to the user terminal.

[0144] In a possible implementation, the device further includes:

[0145] A data acquisition module, used to acquire historical transaction data from the transaction system;

[0146] A preprocessing module, used for preprocessing the historical transaction data to obtain a training data set;

[0147] A first feature extraction module, configured to extract shared features for risk prevention and control tasks and targeted delivery tasks in the initial data processing model based on the training data set, wherein the shared features include at least one of the following: transaction frequency, transaction amount, geographic location, device information, and historical behavior pattern;

[0148] A training module is used to train an initial machine learning model based on the shared features to obtain the data processing model; wherein the initial machine learning model includes at least one of the following: logistic regression, K nearest neighbor, support vector machine and decision tree.

[0149] In a possible implementation, the preprocessing module includes:

[0150] A cleaning unit, used for preprocessing the historical transaction data based on Python language to filter out invalid data in the historical transaction data to obtain cleaned data;

[0151] A structuring unit, used for performing data structuring processing on the cleaned data to obtain structured data;

[0152] A box plot drawing unit, used to determine each data type in the historical transaction data based on the structured data, and draw a corresponding box plot for each data type;

[0153] An exception processing unit, used to determine an abnormal value range based on the box plot, search for corresponding abnormal values ​​based on the abnormal value range, and smooth or eliminate the abnormal values ​​to obtain abnormal processing data;

[0154] An acquisition unit is used to obtain the training data based on the exception processing data.

[0155] In a possible implementation, the device further includes:

[0156] A data set division module, used for dividing the pre-processed historical transaction data into the training data set and the verification data set;

[0157] The training module is specifically used to: train an initial data processing model based on the shared features to obtain a data processing intermediate model; verify whether the learning curvature of the data processing intermediate model meets the expected convergence ability based on the verification data set, and obtain the data processing model when it meets the expected convergence ability.

[0158] In a possible implementation, the device further includes:

[0159] A second feature extraction module, configured to extract shared features from the transaction data based on the feature extraction module in the data processing model;

[0160] The model processing module 102 is specifically used to: process the shared features simultaneously based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model.

[0161] In a possible implementation, the data push module 103 includes at least one of the following:

[0162] A first response generating unit, configured to generate push data according to the risk prevention and control data in response to the risk prevention and control monitoring instruction message initiated by the user terminal;

[0163] a second response generating unit, in response to the directional push indication message initiated by the user terminal, generating push data according to the directional delivery data;

[0164] The third response generating unit generates push data according to the risk prevention and control data and the targeted delivery data in response to the risk prevention and control monitoring instruction message and the targeted delivery instruction message initiated by the user terminal.

[0165] In a possible implementation, the device is applied to a targeted delivery tool, and the targeted delivery tool is integrated into the transaction system via an application programming interface API.

[0166] The data processing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be described in detail here.

[0167] Fig.11 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 110 provided in this embodiment includes: at least one processor 111 and a memory 112. Optionally, the device 110 also includes a communication component 113. The processor 111, the memory 112 and the communication component 113 are connected via a bus 114.

[0168] In a specific implementation process, at least one processor 111 executes the computer-executable instructions stored in the memory 112, so that at least one processor 111 executes the above method.

[0169] The specific implementation process of the processor 111 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0170] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0171] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0172] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0173] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0174] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0175] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0176] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0177] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0178] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0180] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0181] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0182] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: include: After monitoring that the user terminal initiates transaction data to the transaction system, the transaction data is input into a pre-trained data processing model, wherein the data processing model is a multi-task model trained based on historical transaction data, including a risk prevention and control task for predicting transaction risks, and a targeted delivery task for predicting user preferences; Based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, the transaction data is processed simultaneously to obtain risk prevention and control data and targeted delivery data; Generate push data according to at least one of the risk prevention and control data and the targeted delivery data, and transmit the push data to the user terminal.

2. The method according to claim 1, characterized in that Also includes: Acquire historical transaction data from the transaction system; Preprocessing the historical transaction data to obtain a training data set; Extracting shared features for risk prevention and control tasks and targeted delivery tasks in the initial data processing model based on the training data set, wherein the shared features include at least one of the following: transaction frequency, transaction amount, geographic location, device information, and historical behavior pattern; An initial machine learning model is trained based on the shared features to obtain the data processing model; wherein the initial machine learning model includes at least one of the following: logistic regression, K nearest neighbor, support vector machine and decision tree.

3. The method according to claim 2, characterized in that The preprocessing of the historical transaction data to obtain a training data set includes: Based on Python language, the historical transaction data is preprocessed to filter out invalid data in the historical transaction data to obtain cleaned data; Performing data structuring processing on the cleaned data to obtain structured data; Based on the structured data, determine each data type in the historical transaction data, and draw a corresponding box plot for each data type; Determine an outlier range based on the box plot, search for corresponding outliers based on the outlier range, and smooth or eliminate the outliers to obtain outlier processing data; The training data is obtained based on the exception processing data.

4. The method according to claim 2 or 3, characterized in that: After preprocessing the historical transaction data, the method further includes: Dividing the preprocessed historical transaction data into the training data set and the validation data set; The step of training the initial data processing model based on the shared features to obtain the data processing model includes: Training an initial data processing model based on the shared features to obtain a data processing intermediate model; Based on the verification data set, it is verified whether the learning curvature of the data processing intermediate model meets the expected convergence ability. When it meets the expected convergence ability, the data processing model is obtained.

5. The method according to any one of claims 2 to 4, characterized in that: Before simultaneously processing the transaction data based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, the method further includes: Extracting shared features from the transaction data based on a feature extraction module in the data processing model; The step of processing the transaction data based on the risk prevention and control task and the targeted delivery task in the data processing model includes: Based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, the shared features are processed simultaneously.

6. The method according to claim 1, characterized in that The generating of push data according to at least one of the risk prevention and control data and the targeted delivery data includes at least one of the following: In response to the risk prevention and control monitoring instruction message initiated by the user terminal, generating push data according to the risk prevention and control data; In response to the directional push indication message initiated by the user terminal, generating push data according to the directional delivery data; In response to the risk prevention and control monitoring instruction message and the targeted push instruction message initiated by the user terminal, push data is generated according to the risk prevention and control data and the targeted delivery data.

7. The method according to claim 1, characterized in that The method is applied to a targeted delivery tool, and the targeted delivery tool is integrated into the trading system via an application programming interface (API).

8. A data processing device, characterized in that: include: A model input module is used to input the transaction data into a pre-trained data processing model after monitoring the transaction data initiated by the user terminal to the transaction system. The data processing model is a multi-task model trained based on historical transaction data, including a risk prevention and control task for predicting transaction risks and a targeted delivery task for predicting user preferences; A model processing module, configured to process the transaction data based on the risk prevention and control tasks and the targeted delivery tasks in the data processing model, and obtain risk prevention and control data and targeted delivery data; The data push module is used to generate push data according to the risk prevention and control data and the targeted delivery data, and transmit the push data to the user terminal.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the data processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the data processing method according to any one of claims 1 to 7.

Images