Feature vector extraction method and device and related equipment

Abstract

The invention provides a feature vector extraction method and device and related equipment, and the method comprises the steps: obtaining first sample data; inputting the first sample data into a first target model obtained by pre-training for feature vector extraction to obtain a target feature vector, the target feature vector being used for financial prediction; wherein the first target model is a model obtained by training pre-acquired sample synthesis data, the first target model is used for feature extraction, the sample synthesis data comprises second sample data and third sample data, the second sample data is real sample data, and the third sample data is real sample data; the third sample data is data generated by performing camouflage transformation based on the second sample data. Thus, when the first sample data is input into the first target model for feature vector extraction, the target feature vector is obtained, and the accuracy of the target feature vector extracted by the first target model is improved.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and large model technology, and in particular to a feature vector extraction method, apparatus and related equipment. Background Technology

[0002] With the continuous development of model technology, various large models are being used more and more widely in people's lives. For example, large models can be used for credit default prediction or fraud identification. However, in actual use, there are often many disguised and transformed data in the data input to the large model. The large model usually cannot accurately distinguish between the above disguised and transformed data and the real sample data, which leads to the low accuracy of the feature vectors extracted by the current large model. Summary of the Invention

[0003] This application provides a feature vector extraction method, apparatus, and related equipment to address the problem of low accuracy in feature vectors extracted from current large models.

[0004] To solve the above problems, this application is implemented as follows:

[0005] In a first aspect, embodiments of this application provide a feature vector extraction method, including:

[0006] Obtain the first sample data;

[0007] The first sample data is input into a pre-trained first target model to extract feature vectors, thereby obtaining a target feature vector, which is used for financial prediction.

[0008] The first target model is a model trained using pre-acquired sample synthetic data. The first target model is used for feature extraction. The sample synthetic data includes second sample data and third sample data. The second sample data is real sample data, and the third sample data is data generated by masquerading based on the second sample data.

[0009] Secondly, embodiments of this application provide a feature vector extraction device, comprising:

[0010] The acquisition module is used to acquire the first sample data;

[0011] The extraction module is used to input the first sample data into a pre-trained first target model to extract feature vectors and obtain target feature vectors, which are used for financial prediction.

[0012] The first target model is a model trained using pre-acquired sample synthetic data. The first target model is used for feature extraction. The sample synthetic data includes second sample data and third sample data. The second sample data is real sample data, and the third sample data is data generated by masquerading based on the second sample data.

[0013] Thirdly, embodiments of this application also provide an electronic device, including: a memory, a processor, and a program stored in the memory and executable on the processor; the processor is configured to read the program in the memory to implement the steps in the method described in the first aspect above.

[0014] Fourthly, embodiments of this application also provide a readable storage medium for storing a program, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0015] Fifthly, embodiments of this application also provide a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the method described in the first aspect above.

[0016] In this embodiment, since the second sample data is real sample data and the third sample data is data generated by camouflage transformation based on the second sample data, the first target model can be trained using the sample composite data obtained by synthesizing the second sample data and the third sample data. This enables the first target model to accurately distinguish between camouflage transformation data and real sample data. Thus, when the first sample data is input into the first target model for feature vector extraction, the target feature vector is obtained, thereby improving the accuracy of the target feature vector extracted by the first target model. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the feature vector extraction method provided in the embodiments of this application;

[0019] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0020] Figure 3 This is a schematic diagram of the feature vector extraction device provided in the embodiments of this application;

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The terms "first," "second," etc., used in the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. Additionally, the use of "and / or" in this application indicates at least one of the connected objects, such as A and / or B and / or C, representing seven possibilities: including A alone, B alone, C alone, and the presence of both A and B, both B and C, both A and C, and the presence of A, B, and C.

[0024] Please see Figure 1 , Figure 1 This is a flowchart illustrating the feature vector extraction method provided in the embodiments of this application. Figure 1 The feature vector extraction method shown can be performed by an electronic device.

[0025] like Figure 1 As shown, the feature vector extraction method may include the following steps:

[0026] Step 101: Obtain the first sample data.

[0027] Step 102: Input the first sample data into the pre-trained first target model to extract feature vectors and obtain target feature vectors, which are used for financial prediction.

[0028] The first target model is a model trained using pre-acquired sample synthetic data. The first target model is used for feature extraction. The sample synthetic data includes second sample data and third sample data. The second sample data is real sample data, and the third sample data is data generated by masquerading based on the second sample data.

[0029] The first sample data can also be referred to as the data to be identified. The type of the first sample data is not specifically limited here. Optionally, the first sample data may include data in the field of financial risk control. Specifically, the first sample data may include credit-related data. The target feature vector extracted above can be used for financial prediction. It can be understood that the target feature vector can be applied to various predictions in the financial field. For example, the target feature vector can be applied to credit default risk prediction.

[0030] Optionally, the first sample data can also be data obtained after preprocessing the original feature data. The specific type of preprocessing is not limited here. For example, preprocessing may include at least one of the following: data cleaning (handling outliers and duplicate values), missing value imputation (using mean, median, mode or model-based prediction imputation), and feature type conversion (such as converting character-type categorical features into integer codes).

[0031] Optionally, the original feature data typically includes multi-dimensional data. For example, the original feature data may contain numerical feature data and categorical feature data, and the numerical feature data and categorical feature data can together constitute a user's behavioral profile.

[0032] Optionally, numerical feature data may include: average monthly call duration, number of outgoing calls, number of SMS messages sent, data traffic usage, network usage duration, average monthly consumption (average revenue per user (ARPU)), and estimated value of terminal devices, etc. These features can be standardized or normalized to eliminate the influence of unit of measurement.

[0033] Optionally, categorical feature data may include: package type, terminal brand and model, tier of resident city, preferred active time period, and application (App) usage category preference (such as the traffic share or usage time of social, gaming, and financial management apps). These features can be encoded using integers or one-hot encoding for model processing.

[0034] The specific type of the first target model is not limited here. Optionally, the type of the first target model may include a tree model. Since the tree model has fewer parameters, it can be called a lightweight tree model. In this way, because the tree model has fewer parameters, the first target model can reduce the consumption of computing resources when extracting feature vectors and improve the extraction rate of target feature vectors.

[0035] It should be noted that the specific structure of the tree model is not limited here. Optionally, the tree model can include an extreme gradient boosting (XGBoost) model or a random forest model. The goal of training the first objective model is to maximize the model's ability to distinguish between real sample data and synthetic user feature data samples (i.e., data generated by spoofing the second sample data) (such as accuracy or the area under the receiver operating characteristic (ROC) curve (AUC)). The model's hyperparameters (such as the number of trees, depth, learning rate, etc.) can be tuned on the validation set of the proxy task through cross-validation.

[0036] It should be noted that tree models exhibit excellent training and inference performance in both CPU and GPU environments, without relying on expensive CPU and GPU resources. This perfectly aligns with the application requirements of this application in a "computationally limited" scenario. Tree models are naturally adept at learning and capturing high-order interactions and complex patterns between features through a series of non-linear splitting rules. This is precisely the key information to be learned in the embodiments of this application—the feature dependencies inherent in real data that are disrupted in synthetic data. The core building block of the tree model is the decision tree. Each decision tree recursively divides the input feature space into multiple non-overlapping sub-regions through hierarchical splitting rules, with each final sub-region corresponding to a leaf node. When a user feature vector is input into an ensemble model composed of multiple trees, it traverses a unique path from the root node to a specific leaf node in each tree. Therefore, the combination of leaf node indices where the sample ultimately falls across all trees constitutes a high-dimensional, discrete, and structured description of the sample. This structured representation can be efficiently extracted directly from the trained model without the need for complex backpropagation or decoder reconstruction required by deep learning models (such as autoencoders). This method of directly utilizing the tree's inherent structure for encoding is key to achieving efficient embedding extraction (i.e., extracting target feature vectors) in this application.

[0037] The inventive concept of this application is as follows: By constructing a proxy task to distinguish between real sample data and data generated by disguising and transforming the real sample data, the proxy task can be understood as a task for identifying synthetic sample data. By inputting the proxy task and the synthetic sample data into a first target model, the first target model learns the inherent structure and pattern of the data, thereby accurately distinguishing between real sample data and data generated by disguising and transforming the real sample data. Furthermore, the target feature vector can be efficiently extracted from the first target model, and the target feature vector can be a low-dimensional dense feature embedding vector. The first target model can include a lightweight tree ensemble model.

[0038] It should be noted that there are complex and meaningful intrinsic relationships and statistical regularities among the features in real sample data. By designing a perturbation strategy that can systematically disrupt these relationships to generate synthetic data, and then training the first target model to distinguish them, the first target model can be forced to learn and understand these valuable intrinsic structures. This allows the first target model to accurately distinguish between real sample data and data generated by spoofing and transforming real sample data.

[0039] It should be noted that the target feature vector used for financial prediction can be understood as follows: Optionally, the target feature vector can be an embedding vector, which can serve as a high-quality enhancement feature and be input into the downstream financial risk prediction model (i.e., the second target model mentioned later). This significantly improves the model's predictive performance without relying on or with minimal reliance on the true risk labels. Thus, the embodiments of this application can be successfully applied to real-world business scenarios where labels are scarce and resources are limited, resulting in a second target model with better predictive performance.

[0040] It should be noted that, in training the first target model and the second target model, the embodiments of this application can reduce the dependence on real risk labels, that is, only a small amount of real sample data (i.e., second sample data) is needed, and a tree model with higher computational efficiency can be selected as the first target model. This allows the embodiments of this application to be successfully applied to actual business scenarios where labels are scarce and resources are limited. In the embodiments of this application, by constructing the above-mentioned proxy task, the unsupervised problem can be cleverly transformed into a supervised classification problem, forcing the first target model to learn the essential structure of the data. This allows the first target model to more accurately distinguish between real sample data and data generated by disguising and transforming real sample data. In addition, when the target feature vector is an embedding vector, the discrete decision path of the tree model can be efficiently transformed into a continuous and information-condensed vector representation, so that the target feature vector can be trained to obtain the second target model more efficiently.

[0041] In the process of training the first target model using sample synthetic data, the aforementioned sample synthetic data can be understood as: enabling the first target model to learn the inherent structure and pattern of the second and third sample data, thereby accurately distinguishing the differences between the second and third sample data, and thus accurately extracting the target feature vector.

[0042] In this embodiment of the application, through steps 101 to 102, since the second sample data is real sample data and the third sample data is data generated by camouflage transformation based on the second sample data, the first target model can be trained using the sample synthesis data obtained by synthesizing the second sample data and the third sample data. This enables the first target model to accurately distinguish between camouflage transformation data and real sample data. Thus, when the first sample data is input into the first target model for feature vector extraction, the target feature vector is obtained, thereby improving the accuracy of the target feature vector extracted by the first target model.

[0043] As an optional implementation, the step of inputting the first sample data into a pre-trained first target model for feature vector extraction to obtain a target feature vector includes:

[0044] The first sample data is input into a pre-trained first target model to extract feature vectors, thereby obtaining the first feature vector.

[0045] The first feature vector is subjected to dimensionality reduction mapping to obtain the target feature vector.

[0046] The target feature vector has a lower dimension than the first feature vector, so the target feature vector can be called a low-dimensional dense feature embedding vector.

[0047] In this embodiment, the target feature vector is obtained by performing a dimensionality reduction mapping process on the first feature vector. This reduces the dimensionality of the target feature vector, making it more suitable for financial prediction. This reduces the consumption of computing resources and further improves the efficiency of financial prediction.

[0048] As an optional implementation, the first target model is a tree model, the first feature vector is a leaf node index sequence vector, and the step of performing dimensionality reduction mapping on the first feature vector to obtain the target feature vector includes:

[0049] Construct the string identifier corresponding to the leaf node index sequence vector;

[0050] The string identifier is mapped to a hash bucket index using a hash function;

[0051] The target feature vector is determined in a preset embedding matrix based on the hash bucket index.

[0052] This implementation can be understood as follows: a lightweight tree ensemble model is used for training to learn the intrinsic structure of the data, and then the leaf node index sequence is extracted. The target feature vector is determined through feature hash mapping. This solves the problem of unsupervised feature representation learning that can generate low-dimensional dense general feature embeddings without relying on any external financial labels, using only the user's original feature data.

[0053] It should be noted that the hash function, hash bucket index, and preset embedding matrix mentioned above are all pre-set. The hash function, hash bucket index, and preset embedding matrix mentioned above can correspond to the tree model. That is, if the tree model is different, the hash function, hash bucket index, and preset embedding matrix mentioned above will also be different.

[0054] It should be noted that the aforementioned embedding matrix may refer to the matrix embedded in the tree model.

[0055] In this embodiment, the target feature vector is determined in the preset embedding matrix based on the hash bucket index, which can further improve the accuracy and efficiency of the determined target feature vector.

[0056] As an optional implementation, determining the target feature vector in a preset embedding matrix based on the hash bucket index includes:

[0057] Multiple second feature vectors are determined based on the hash bucket index in a preset embedding matrix, wherein the dimension of the second feature vector is less than the dimension of the first feature vector;

[0058] The multiple second feature vectors are aggregated to generate the target feature vector.

[0059] Here, multiple second feature vectors can refer to feature vectors corresponding to different nodes in the tree model, or they can refer to multiple feature vectors corresponding to the same node in the tree model. No specific limitations are imposed here.

[0060] In this embodiment, multiple second feature vectors can be determined first, and these multiple second feature vectors can be processed in parallel. Then, the multiple second feature vectors are aggregated to generate the target feature vector, which can further improve the generation efficiency of the target feature vector.

[0061] It should be noted that since the dimension of the second feature vector is smaller than that of the first feature vector, the processing efficiency of the second feature vector is higher than that of the first feature vector. In addition, multiple second feature vectors can be processed simultaneously, which can further improve the processing efficiency of feature vectors.

[0062] For example: the second sample data can be used This indicates that the third sample data can be used This indicates that the synthesized sample data can be used It indicates that, for Each sample data x in i This data is then fed into a pre-trained ensemble model (i.e., the first objective model) containing T decision trees. Sample data x i In the t-th tree, a specific leaf node will be encountered, and the index of this leaf node will be recorded as L. it After traversing all T trees, each sample data x i We obtain a leaf node index sequence (i.e., a leaf node index sequence vector) V of length T. i =L i1 ,L i2 ,…,L iT This sequence is a highly condensed representation of the decision path of the samples in the model; it is itself a high-dimensional and sparse feature.

[0063] Directly using leaf node index sequences as features (e.g., through one-hot encoding) leads to the curse of dimensionality and information sparsity. To address this issue, this application employs an efficient optimization method combining feature hashing and random vector aggregation to map high-dimensional sparse index sequences to a low-dimensional dense vector space.

[0064] 1. Set the hash space and embedding dimension: Pre-set two key parameters: hash space size d hash (A large integer, such as 4096 or 8192, used to control the probability of hash collisions) and the final target embedding dimension d embed (A smaller integer, such as 32, 64, or 128, represents the dimension of the final feature.)

[0065] 2. Initialize the embedded lookup table: Create a table of size d hash ×d embed Embedding matrix (lookup table) M embed Each row of this matrix is ​​a d embed These are 3D vectors. They require no training and are generated solely through a one-time random initialization, sampled from a standard normal distribution N(0, 1). The specific initialization parameters have little impact on the final embedding quality; the key is to provide a unique, fixed, random direction for each hash bucket.

[0066] 3. Hash mapping: For sample data x i Leaf node index sequence V i Each index L in itFirst, a unique string identifier is constructed to distinguish the same index on different trees. Then, a non-cryptographic hash function (MurmurHash3 to ensure computational speed and good distribution) is used to map this string identifier to a [0, d] map. hash Hash bucket index h within the range of -1 it .

[0067] 4. Vector search and aggregation: Based on the calculated T hash bucket indices From the embedding matrix M embed Find the corresponding T d in the middle embed T-dimensional vectors. Finally, aggregate these T vectors to generate sample data x. i The final embedding vector E i Alternatively, the aggregation method may be average pooling.

[0068] .

[0069] It's important to note that the principle behind the above implementation lies in its clever utilization of the concept of random projection. A single random vector is meaningless, but the selective aggregation of a large number of random vectors can reveal structured information. Samples with similar behavioral patterns are highly likely to fall into the same leaf node combinations, thus finding a highly overlapping set of random vectors. According to the law of large numbers, their average values ​​will also be very close in the vector space. Conversely, samples with vastly different behaviors will have average vectors that are far apart. This method efficiently transforms the discrete structural similarity learned by the tree model into distance similarity in a continuous vector space.

[0070] It should be noted that the specific method for generating the third sample data based on the spoofing transformation of the second sample data is not limited here.

[0071] As an optional implementation, before inputting the first sample data into a pre-trained first target model for feature vector extraction to obtain the target feature vector, the method further includes at least one of the following:

[0072] At least a portion of the data in the second sample data is randomly shuffled to obtain the third sample data;

[0073] At least a portion of the data in the second sample data is replaced with the target data to obtain the third sample data;

[0074] The target data corresponds one-to-one with the at least part of the data, and the content of the target data is different from that of the at least part of the data.

[0075] It should be noted that the purpose of generating the third sample data by performing a disguise transformation based on the second sample data is to destroy the inherent correlation between the original features included in the second sample data, rather than to significantly change the marginal distribution of individual features.

[0076] In this application, the diversity and flexibility of the methods for generating third sample data can be increased.

[0077] It should be noted that randomly shuffling at least a portion of the data in the second sample to obtain the third sample can be called the Independent Feature Permutation (IPP) strategy, which is an efficient and effective strategy. Specifically, each column (i.e., each feature) in the second sample is independently and randomly rearranged. For example, the values ​​of the "call duration" column for all users in the second sample are randomly shuffled and reassigned to all users. After this operation, the distribution of each feature itself (such as mean, variance, and histogram) remains unchanged, but within a specific user sample, the original relationship between its "call duration" and other features such as "number of SMS messages" and "data usage" is completely disrupted.

[0078] Replacing at least a portion of the data in the second sample data with target data to obtain the third sample data can be termed a feature subset masking strategy, which simulates a scenario where some information is lost. For each sample, a feature subset is randomly selected (e.g., randomly selected features at a preset ratio, such as 20%), and its values ​​are replaced with predefined mask values ​​(e.g., 0, -1) or the global statistics of that feature (e.g., global mean or mode). In other words, the above samples can be replaced with target data, which can be understood as the randomly selected feature subset, and the content of the feature subset is different from the content of the above samples.

[0079] For example: the second sample data can be used This means to make a copy. Then, one or more data perturbation strategies are applied to each sample in the replication set to perform camouflage transformation, generating a synthetic feature dataset of the same size but different content. This refers to the third sample data. All sample data are assigned the label Y=1 (representing "true"), which is... All samples in the dataset are assigned the label Y=0 (representing "synthesis"), and then the labeled samples are... and The datasets are merged to form the final dataset for the agent task used for training. (i.e., synthetic sample data). The construction of this dataset is entirely self-supervised and does not depend on any external labels.

[0080] As an optional implementation, after inputting the first sample data into a pre-trained first target model for feature vector extraction to obtain the target feature vector, the method further includes:

[0081] The target feature vector is input into the model to be trained to obtain the second target model, or...

[0082] The target feature vector is concatenated with the sample vector corresponding to the fourth sample data obtained by screening to obtain the enhanced sample vector, and the enhanced sample vector is input into the model to be trained for training to obtain the second target model;

[0083] The second target model is a model used for financial forecasting, and the fourth sample data and the first sample data are from the same source.

[0084] The fourth sample data and the first sample data are from the same source, which can be understood as: the fourth sample data and the first sample data are sample data from the same application scenario. For example: the fourth sample data and the first sample data both belong to the financial field, or the fourth sample data and the first sample data belong to the same sample set, or the fourth sample data may belong to a part of the first sample data, that is, the fourth sample data can be called data in the feature subset of the first sample data.

[0085] In this embodiment, the target feature vector can be directly used to train the second target model, or it can be concatenated with the fourth sample data to form an enhanced sample vector, which can then be used to train the second target model. This can significantly improve the predictive performance of the second target model for financial forecasting, that is, improve the accuracy of the financial forecasting results of the second target model.

[0086] It should be noted that the above method can be understood as follows: the embedding vector E (i.e. the target feature vector) can be used as a feature set on its own, or it can be concatenated with some of the selected original features (i.e. the fourth sample data) to form a more comprehensive enhanced feature set for use by the downstream model (i.e. the model to be trained) to train the second target model.

[0087] Downstream model training: Using a small amount of available real financial label data (such as credit default records), train a lightweight downstream prediction model (i.e., a second objective model, which may include a Lightweight Gradient Boosting Machine (LightGBM), logistic regression, or other models).

[0088] It should be noted that after obtaining the second objective model, its performance can be evaluated and verified. Specific steps are described below:

[0089] The second objective is to improve the performance of downstream tasks: compare the performance difference between the model using the embedded features in this application embodiment and the baseline model using only the original features on independent test sets. Key risk control indicators include the area under the ROC curve (AUC) and the Kolmogorov-Smirnov (KS) test value. The significant performance improvement directly proves the effectiveness of the embedding.

[0090] Visualization Analysis: Nonlinear dimensionality reduction techniques such as t-Distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) can be used to project the generated high-dimensional embeddings onto a two-dimensional or three-dimensional space for observation. If the embedding quality is good, users with similar behavioral profiles should form relatively concentrated clusters in the embedding space.

[0091] Feature importance analysis: In the trained downstream risk prediction model, the relative importance of newly generated unsupervised embedded features is evaluated. If these embedded features have high weights in the model's decisions, it indicates that they contribute significantly to the prediction results.

[0092] Embedding stability test: The consistency of the embedding results can be evaluated by observing whether the generated embedding remains relatively stable by making small perturbations to the input user feature data, or by repeating the agent task training and embedding generation process multiple times.

[0093] It should be noted that the application scenarios of this application embodiment may include: the unsupervised feature embedding (i.e., target feature vector) generated in this application embodiment, which originates from user feature data (i.e., first sample data), has broad application potential in the field of financial risk control after being combined with financial tag data to train a downstream model (i.e., second target model).

[0094] Core application: Credit default risk assessment: Directly used as input features to enhance the accuracy and robustness of risk prediction models in credit approval, loan monitoring and other processes.

[0095] Customer segmentation and precise profiling: Unsupervised embedding can serve as the basis for customer clustering, helping financial institutions to conduct more refined risk stratification, behavioral pattern analysis, or marketing target customer identification from new dimensions.

[0096] Fraud detection and anomaly warning: The embedded vector may contain signals that deviate from normal behavior patterns, which can help identify potential fraudulent behavior or provide early warning of signs of credit deterioration.

[0097] Risk mitigation in cold start scenarios: For new customers and new products that lack historical transaction or behavioral data, this method can generate feature embeddings based solely on their user characteristic data, providing a basis for preliminary risk assessment.

[0098] It is important to note that one of the core goals of unsupervised learning is to learn a general data representation that is valuable for various downstream tasks. The design concept of the proxy task (distinguishing between real and synthetic user feature data) in this application embodiment is to guide the model to learn and master the "real structure" and "intrinsic patterns" of user feature data. In the financial field, the occurrence of credit default risk is not entirely random, but rather an external manifestation of a complex pattern jointly determined by various characteristics of the applicant (including stability and social activity indirectly reflected in their communication behavior patterns) and their interactions. In other words, certain specific combinations of user behavior features or data structures inherently contain high-risk signals. If the proxy task can successfully force the model to capture these risk-related real user feature data structures (even if the model does not know what "financial risk" is when training the proxy task), then the generated feature embedding will naturally have a positive effect on the downstream risk prediction task trained using financial labels. Therefore, the way the data is synthesized—specifically, which user feature data structures are disrupted and how these structures are disrupted—is crucial to whether the final generated embedding aligns with the financial risk assessment objective at the "data essence" level. For example, if synthetic data disrupts the normal correlations between user communication behavior characteristics (e.g., users with regular sleep schedules typically correspond to stable call patterns, while synthetic data might match regular sleep schedules with chaotic call patterns), then the model must learn to identify these normal behavioral correlation patterns in order to distinguish between real and synthetic data. These correlation patterns often indirectly reflect key information about a user's credit risk. Therefore, carefully designing unsupervised proxy tasks (operating on user feature data) that align with the "data essence" of downstream financial risk assessment tasks is crucial for improving the effectiveness of unsupervised representation learning.

[0099] It should be noted that the embodiments of this application can be applied to electronic devices, which may include a data preparation module, an unsupervised feature embedding module, and a downstream task application module. For details, please refer to... Figure 2 .

[0100] 1. Data Preparation Module: As the entry point, this module receives and processes two types of data: one is raw user feature data used to generate feature embeddings (i.e., raw feature data, such as carrier data), and the other is financial tag data (e.g., historical default records), which may be very limited in quantity, used to train and evaluate downstream models. This module performs standard data preprocessing procedures (i.e., preprocessing) on ​​the raw user feature data, including but not limited to data cleaning (handling outliers and duplicates), missing value imputation (using mean, median, mode, or model-based prediction imputation), and feature type conversion (e.g., converting character-based categorical features to integer encoding).

[0101] 2. Unsupervised Feature Embedding Module: This is the core functional unit of the electronic device. Its input is only preprocessed user feature data (i.e., second sample data), completely independent of any financial labels. The module's mission is to generate high-quality feature embeddings. It contains three closely cooperating sub-modules:

[0102] The proxy task construction submodule is responsible for converting unlabeled data into supervised learning tasks. It receives real user feature data (i.e., the second sample data) and generates corresponding "fake" synthetic feature data (i.e., the third sample data) through a specific data perturbation strategy. Subsequently, it assigns proxy labels of "true" (label=1) and "false" (label=0) to the real data (i.e., the second sample data) and the synthetic data (i.e., the third sample data), respectively, thereby constructing a binary classification proxy task dataset (i.e., sample synthetic data).

[0103] Tree Model Training Submodule: The core of this submodule is a tree ensemble model (i.e., the primary objective model, preferably XGBoost or Random Forest). It is trained using a proxy dataset generated by the proxy task construction submodule, and its optimization objective is to accurately distinguish between real and synthetic data.

[0104] Feature Embedding Extraction Submodule: After the tree model is trained, this submodule is responsible for inputting the original, real user feature data (i.e., the first sample data) into the trained model. It does not care about the model's prediction results, but extracts the decision path information of each data sample within the model (specifically, the leaf node index sequence), and compresses and transforms this high-dimensional discrete information into the final low-dimensional dense feature embedding vector (i.e., the target feature vector) through a series of efficient operations such as feature hashing and vector aggregation.

[0105] 3. Downstream Task Application Module: This module is responsible for applying feature embeddings to actual business operations. It uses the embedding vectors generated by the unsupervised feature embedding module as a powerful new set of features, and fuses them with some of the original features (i.e., the fourth sample data). Then, using a small amount of financial label data provided by the data preparation module, it trains a downstream risk prediction model (i.e., the second objective model, such as logistic regression, LightGBM, etc.). Finally, this module includes an evaluation component to verify the performance improvement of the downstream risk prediction model on key business metrics (such as AUC, KS value) after introducing the new embedding features on an independent test set, and ultimately outputs the risk assessment results for new users.

[0106] See Figure 3 , Figure 3 This is a structural diagram of the feature vector extraction device provided in the embodiments of this application. The feature vector extraction device 300 includes:

[0107] Module 301 is used to acquire the first sample data;

[0108] Extraction module 302 is used to input the first sample data into a pre-trained first target model to extract feature vectors and obtain target feature vectors, which are used for financial prediction.

[0109] The first target model is a model trained using pre-acquired sample synthetic data. The first target model is used for feature extraction. The sample synthetic data includes second sample data and third sample data. The second sample data is real sample data, and the third sample data is data generated by masquerading based on the second sample data.

[0110] As an optional implementation, the extraction module 302 includes:

[0111] The extraction submodule is used to input the first sample data into a pre-trained first target model to extract feature vectors and obtain the first feature vector.

[0112] The dimension reduction mapping processing submodule is used to perform dimension reduction mapping processing on the first feature vector to obtain the target feature vector.

[0113] As an optional implementation, the first target model is a tree model, the first feature vector is a leaf node index sequence vector, and the dimensionality reduction mapping processing submodule includes:

[0114] The construction unit is used to construct the string identifier corresponding to the leaf node index sequence vector;

[0115] A mapping unit is used to map the string identifier to a hash bucket index using a hash function;

[0116] The determining unit is used to determine the target feature vector in a preset embedding matrix based on the hash bucket index.

[0117] As an optional implementation, the determining unit includes:

[0118] A subunit is defined for determining multiple second feature vectors in a preset embedding matrix based on the hash bucket index, wherein the dimension of the second feature vector is less than the dimension of the first feature vector.

[0119] The aggregation processing subunit is used to aggregate the plurality of second feature vectors to generate the target feature vector.

[0120] As an optional implementation, the feature vector extraction device 300 further includes at least one of the following:

[0121] A random shuffling module is used to randomly shuffle at least a portion of the data in the second sample data to obtain the third sample data;

[0122] A replacement module is used to replace at least a portion of the data in the second sample data with target data to obtain the third sample data;

[0123] The target data corresponds one-to-one with the at least part of the data, and the content of the target data is different from that of the at least part of the data.

[0124] As an optional implementation, the feature vector extraction device 300 further includes:

[0125] The first training module is used to input the target feature vector into the model to be trained for training, so as to obtain the second target model, or...

[0126] The second training module is used to concatenate the target feature vector with the sample vector corresponding to the fourth sample data obtained by screening to obtain an enhanced sample vector, and input the enhanced sample vector into the model to be trained for training to obtain the second target model;

[0127] The second target model is a model used for financial forecasting, and the fourth sample data and the first sample data are from the same source.

[0128] The feature vector extraction device 300 can achieve the functions described in the embodiments of this application. Figure 1 The various processes in the method embodiments, and the ways to achieve the same beneficial effects, will not be repeated here to avoid repetition.

[0129] This application also provides an electronic device. Please refer to [link to relevant documentation]. Figure 4 The electronic device may include a processor 401, a memory 402, and a program 4021 stored in the memory 402 and executable on the processor 401. When the program 4021 is executed by the processor 401, it can achieve... Figure 1 Any steps in the corresponding method embodiments and the achievement of the same beneficial effects will not be repeated here.

[0130] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware related to program instructions, and the program can be stored in a readable medium. This application also provides a readable storage medium storing a computer program, which, when executed by a processor, can implement the above-described methods. Figure 1 Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0131] The storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0132] This application also provides a computer program product, including computer instructions, which, when executed by a processor, can perform the above-described functions. Figure 1 Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0133] The above description represents the preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A feature vector extraction method, characterized in that, include: Obtain the first sample data; The first sample data is input into a pre-trained first target model to extract feature vectors, thereby obtaining a target feature vector, which is used for financial prediction. The first target model is a model trained using pre-acquired sample synthetic data. The first target model is used for feature extraction. The sample synthetic data includes second sample data and third sample data. The second sample data is real sample data, and the third sample data is data generated by masquerading based on the second sample data.

2. The method according to claim 1, characterized in that, The step of inputting the first sample data into a pre-trained first target model for feature vector extraction to obtain the target feature vector includes: The first sample data is input into a pre-trained first target model to extract feature vectors, thereby obtaining the first feature vector. The first feature vector is subjected to dimensionality reduction mapping to obtain the target feature vector.

3. The method according to claim 2, characterized in that, The first target model is a tree model, and the first feature vector is a leaf node index sequence vector. The step of performing dimensionality reduction mapping on the first feature vector to obtain the target feature vector includes: Construct the string identifier corresponding to the leaf node index sequence vector; The string identifier is mapped to a hash bucket index using a hash function; The target feature vector is determined in a preset embedding matrix based on the hash bucket index.

4. The method according to claim 3, characterized in that, Determining the target feature vector in a preset embedding matrix based on the hash bucket index includes: Multiple second feature vectors are determined based on the hash bucket index in a preset embedding matrix, wherein the dimension of the second feature vector is less than the dimension of the first feature vector; The multiple second feature vectors are aggregated to generate the target feature vector.

5. The method according to claim 1, characterized in that, Before inputting the first sample data into a pre-trained first target model for feature vector extraction to obtain the target feature vector, the method further includes at least one of the following: At least a portion of the data in the second sample data is randomly shuffled to obtain the third sample data; At least a portion of the data in the second sample data is replaced with the target data to obtain the third sample data; The target data corresponds one-to-one with the at least part of the data, and the content of the target data is different from that of the at least part of the data.

6. The method according to any one of claims 1 to 5, characterized in that, After inputting the first sample data into a pre-trained first target model for feature vector extraction to obtain the target feature vector, the method further includes: The target feature vector is input into the model to be trained to obtain the second target model, or... The target feature vector is concatenated with the sample vector corresponding to the fourth sample data obtained by screening to obtain the enhanced sample vector, and the enhanced sample vector is input into the model to be trained for training to obtain the second target model; The second target model is a model used for financial forecasting, and the fourth sample data and the first sample data are from the same source.

7. A feature vector extraction device, characterized in that, include: The acquisition module is used to acquire the first sample data; The extraction module is used to input the first sample data into a pre-trained first target model to extract feature vectors and obtain target feature vectors, which are used for financial prediction. The first target model is a model trained using pre-acquired sample synthetic data. The first target model is used for feature extraction. The sample synthetic data includes second sample data and third sample data. The second sample data is real sample data, and the third sample data is data generated by masquerading based on the second sample data.

8. An electronic device, comprising: A memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is configured to read the program from the memory to implement the steps of the feature vector extraction method as described in any one of claims 1 to 6.

9. A readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps in the feature vector extraction method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps in the feature vector extraction method as described in any one of claims 1 to 6.

Images