Financial risk control feature on-demand computing method and device

Abstract

The application provides a financial risk control feature on-demand calculation method and device. The method comprises the following steps: obtaining a feature configuration file and original financial data, analyzing the feature configuration file to obtain a feature variable set to be calculated; based on the feature variable set, matching and identifying the feature types of each target feature, and recursively processing the target features relied on according to the feature types to construct a feature dependency graph; topologically sorting the feature dependency graph to obtain a feature calculation sequence, and combining the mapping relationship of the operators in a pre-defined operator library and the feature calculation sequence to generate a minimized operator execution plan; calling corresponding operators according to the operator execution plan through a distributed computing framework, using the operators to perform feature calculation on the target features to be calculated and the original financial data, obtaining feature calculation results, and finally obtaining feature calculation output results. The application solves the problems of waste of computing resources, low computing efficiency and inflexible iteration in the traditional feature calculation method.

Description

Technical Field

[0001] This invention relates to the field of risk control management technology, and in particular to a method and apparatus for on-demand calculation of financial risk control characteristics. Background Technology

[0002] Financial risk control feature calculation is a core technical aspect of financial risk management, playing a crucial role in assessing user credit risk, identifying fraudulent behavior, and optimizing credit granting decisions. With the development of big data technology, financial institutions typically need to extract hundreds of thousands or even millions of feature variables from massive amounts of user behavior and transaction data to build accurate risk control models.

[0003] Currently, common technical solutions in the industry are mainly based on distributed computing frameworks (such as Spark and Hadoop) for large-scale feature computation. A typical practice involves writing batch scripts or programs to define the computation logic for all possible feature variables, and then calculating and updating all features within a preset time window (such as daily or weekly). For example, using the Spark engine to process a user's transaction records for the past six months, a pre-coded algorithm can be used to calculate risk control indicators such as "number of overdue payments in the past three months" and "income-to-debt ratio".

[0004] The common approach in existing technologies is to hard-code code containing all feature calculation logic within the data processing flow. The core characteristic of this method is that it calculates all predefined feature variables at once, regardless of how many features the final online risk control model actually needs. For example, even if the risk control model only uses 500 feature variables, it will still calculate all 100,000 predefined features, resulting in a significant waste of computational resources on generating unnecessary features.

[0005] However, this full-scale calculation method has obvious technical drawbacks: First, it wastes a lot of computing resources because most of the calculated features are not used in the actual model; second, it is inefficient, as full-scale calculation results in excessively long feature generation time, which cannot meet the business needs of real-time risk control; finally, it is inflexible in iteration, as each addition or modification of features requires rewriting and deploying the code, which seriously affects the iteration speed of model optimization. Summary of the Invention

[0006] The purpose of this invention is to overcome the technical defects existing in the prior art and provide a method and device for on-demand calculation of financial risk control features. By constructing a predefined operator library, designing feature configuration file format and syntax, developing a configuration parsing engine, designing an on-demand calculation execution framework, and realizing result output and performance optimization, the invention solves the problems of wasted computing resources, low computing efficiency, and inflexible iteration in traditional feature calculation methods.

[0007] To solve the above-mentioned technical problems, the technical solution provided by this invention is: a method for on-demand calculation of financial risk control features, comprising:

[0008] Obtain feature configuration files and raw financial data, parse the feature configuration files to obtain a set of feature variables to be calculated, the set of feature variables includes a list of target features with at least one target feature and feature dependency mapping relationships;

[0009] Based on the set of feature variables, the feature types of each target feature are matched and identified, and the dependent target features are recursively processed according to the feature types to construct a feature dependency graph, wherein the feature dependency graph is used to indicate the dependency relationship between the target features;

[0010] The feature dependency graph is topologically sorted to obtain a feature computation sequence. Combined with the mapping relationship of operators in the predefined operator library and the feature computation sequence, a minimized operator execution plan is generated. The feature computation sequence is used to indicate the order of computation of the target features to be computed in the feature dependency graph, and the operator execution plan is used to indicate the calling sequence of operators corresponding to the target features to be computed, arranged in dependency order.

[0011] The corresponding operators are called according to the operator execution plan through a preset distributed computing framework. The called operators are used to perform feature calculations on the target features to be calculated and the original financial data to obtain the feature calculation results.

[0012] The feature calculation results are filtered and formatted to output the feature calculation results of the target features specified in the feature configuration file.

[0013] Preferably, parsing the feature configuration file to obtain the set of feature variables to be calculated includes:

[0014] Based on the feature configuration file, the target feature fields are read through a preset JSON format parser, and the corresponding target feature names are extracted to obtain a list of target features;

[0015] Regular expressions are used to identify the format pattern of each target feature in the target feature list, and the configuration items that the feature depends on are matched based on the format pattern of the target feature to obtain the type identifier of each target feature;

[0016] Based on the type identifier, the corresponding dependency configuration is found in the feature configuration file to obtain the feature dependency mapping relationship for each target feature.

[0017] The target feature list and the feature dependency mapping relationship are combined to obtain the set of feature variables to be calculated.

[0018] Preferably, the step of matching and identifying the feature types of each target feature based on the set of feature variables, and recursively processing the dependent target features according to the feature types to construct a feature dependency graph includes:

[0019] The feature type of the target feature is obtained by performing pattern matching on each target feature in the set of feature variables using a preset regular expression;

[0020] Based on the feature type of the target feature, find the basic field of the target feature in the dependency configuration, identify the required basic features based on the basic field, and construct a direct dependency list from the basic features.

[0021] For each basic feature in the list of direct dependencies, a dependency lookup is recursively performed to construct a feature dependency tree with multi-level dependency chains;

[0022] The feature dependency tree is transformed into a feature dependency graph represented as a directed acyclic graph structure, wherein the nodes of the feature dependency graph represent the target features, and the edges of the feature dependency graph represent the dependencies between the target features.

[0023] Preferably, the step of performing topological sorting on the feature dependency graph to obtain a feature computation sequence, and combining the mapping relationships of operators in a predefined operator library with the feature computation sequence to generate a minimized operator execution plan, includes:

[0024] The nodes in the feature dependency graph are topologically sorted to determine the feature calculation order without dependency conflicts, thus obtaining the feature calculation sequence.

[0025] Based on the feature type of each target feature in the feature calculation sequence, a corresponding operator is matched in the predefined operator library as the target operator for feature calculation, so as to generate operator calling information. The operator calling information is used to indicate the target operator and the corresponding operator calling relationship.

[0026] The operator call information is arranged according to the preset feature calculation priority, the call requests of the same target operator are merged and redundant calculation steps are eliminated to obtain the operator execution sequence;

[0027] Based on the computational complexity and data dependencies of the target operator, the execution sequence of the operator is assigned execution priority and resource allocation to obtain the operator execution plan.

[0028] Preferably, the step of calling the corresponding operator according to the operator execution plan through a preset distributed computing framework, and using the called operator to perform feature calculation on the target feature to be calculated and the original financial data to obtain the feature calculation result includes:

[0029] Based on the operator execution plan, the original data fields and data ranges required by each target operator are analyzed to generate a data loading requirement list, and the minimum dataset definition is obtained based on the data loading requirement list;

[0030] The corresponding data partitions are read from the original financial data according to the definition of the minimum dataset using a preset distributed computing framework, and the data partitions are cached to obtain the dataset required for operator execution.

[0031] Based on the dataset, the target operators in the predefined operator library are instantiated and called sequentially according to the order indicated by the operator execution plan. The intermediate calculation results of the target operators are obtained according to the operator execution plan and the called target operators.

[0032] The intermediate calculation results of each target operator are managed and transmitted, and the intermediate calculation results of all target operators are summarized to obtain the feature calculation results of the target feature.

[0033] Preferably, the step of matching and identifying the feature types of each target feature based on the set of feature variables, and recursively processing the dependent target features according to the feature types to construct a feature dependency graph, further includes:

[0034] Acquire historical feature calculation data, and extract the calculation time, resource consumption, and business impact data of each target feature from the historical feature calculation data to generate a feature performance history record;

[0035] Based on the historical performance records of the features, the heteroscedasticity characteristics of different target features are calculated using a preset White test algorithm. Based on the heteroscedasticity characteristics, a variance function model of the target features is constructed. The variance function model is used to indicate the heteroscedasticity distribution characteristics of the target features.

[0036] Based on the heteroscedasticity distribution characteristics indicated by the variance function model, the business value score, computational cost value, and stability coefficient of the target feature are determined. Based on the business value score, computational cost value, and stability coefficient of the target feature, the weight of the target feature is calculated. Based on the weight of the target feature and the feature dependency tree, a weighted feature map is constructed.

[0037] The weights in the weighted feature map are optimized using a preset generalized least squares algorithm, and the priority order of feature calculation in the weighted feature map is adjusted to obtain the optimized weighted feature map.

[0038] Anomalies in the optimized weighted feature map are identified using a pre-defined Breusch-Pagan test algorithm to generate a first test result. The autocorrelation of the dependencies between the target features is evaluated using a pre-defined Durbin-Watson test algorithm to generate a second test result. The structure of the optimized weighted feature map is adjusted based on the first and second test results to obtain a feature dependency map.

[0039] Preferably, the step of performing topological sorting on the feature dependency graph to obtain a feature computation sequence, and generating a minimized operator execution plan by combining the mapping relationships of operators in a predefined operator library with the feature computation sequence, further includes:

[0040] The operator execution plan is transformed into an operator execution graph, wherein the vertices in the operator execution graph represent the target operators indicated in the operator execution plan, and the edges in the operator execution graph represent the data dependencies and calling relationships between the target operators;

[0041] Based on the operator execution graph, the operator attributes of the target operators are extracted using a preset graph attention network. The intrinsic relationships between the target operators are determined based on the operator attributes. The low-dimensional embedding representation of the target operators is obtained from the intrinsic relationships. The operator attributes include computational complexity features, data dependency features, and resource consumption features.

[0042] Based on the low-dimensional embedding representation of the target operators, a similarity matrix between the target operators is calculated. Based on the operator computation patterns and clustering relationships identified by the similarity matrix, an operator knowledge graph is constructed, wherein the operator knowledge graph is used to indicate the semantic association information between the target operators.

[0043] Based on the operator knowledge graph, a multi-objective optimization model is constructed, and a constrained optimization model for operator scheduling is obtained based on the multi-objective optimization model. The multi-objective optimization model includes an optimization objective and constraints. The optimization objective is used to indicate minimizing the total computation time and resource consumption, and the constraints include dependency constraints, resource constraints, and time constraints.

[0044] The constrained optimization model is solved using a pre-defined particle swarm optimization-simulated annealing hybrid algorithm to obtain an optimized operator execution plan. The particle swarm optimization-simulated annealing hybrid algorithm is used to indicate the search for the optimal operator scheduling scheme by adaptively adjusting the particle position and velocity update parameters and the simulated annealing acceptance probability mechanism.

[0045] Preferably, the step of using the invoked operator to perform feature calculations on the target feature to be calculated and the original financial data to obtain feature calculation results, and performing feature calculations on demand, further includes heterogeneous feature calculations based on multi-physics coupling, including:

[0046] The temporal features in the target features are mapped to a temporal feature field, the statistical features in the target features are mapped to a statistical feature field, the correlation features in the target features are mapped to a correlation feature field, and the temporal feature field, the statistical feature field, and the correlation feature field are combined to form a feature physical field;

[0047] Based on the aforementioned characteristic physical fields, a set of coupled equations is constructed, which includes time-series characteristic field equations, statistical characteristic field equations, and correlation characteristic field equations.

[0048] Based on the coupling terms and source terms between the characteristic fields determined by the coupling equations, a multi-characteristic field coupling mathematical model is obtained.

[0049] The multi-feature field coupled mathematical model is solved by using a preset operator splitting algorithm. In the separation solution, a preset alternating direction implicit algorithm is used to calculate alternately in different feature dimensions and iteratively process the coupling terms to obtain the separation calculation results of each feature field.

[0050] A multi-level computational grid is constructed, and a preset V-loop strategy is used to solve the multi-level computational grid to obtain multi-resolution feature calculation results. The multi-level computational grid includes a coarse grid level and a fine grid level. The coarse grid level is used for fast approximate calculation, and the fine grid level is used for high-precision region calculation.

[0051] Dynamic load balancing is performed on the multi-resolution feature calculation results, so that a preset work stealing strategy and task migration strategy are used to calculate resource allocation data. Based on the resource allocation data, elastic resource scheduling and dynamic scaling of computing nodes are performed to obtain heterogeneous feature calculation results.

[0052] Based on the heterogeneous feature calculation results, the feature calculation results are obtained.

[0053] Preferably, the method further includes constructing the predefined operator library, specifically including:

[0054] Based on the feature calculation requirements of financial risk control scenarios, basic operators are determined, and specific basic calculation logic and standard interfaces are encapsulated for each basic operator to obtain a basic operator library. The basic operators include statistical operators, time series operators, correlation operators and user behavior basic operators.

[0055] Based on the aforementioned basic operator library, a combined operator library is constructed, which includes arithmetic combined operators, logical combined operators, conditional combined operators, and composite combined operators. The combined operators in the combined operator library are used to indicate the combined calculations between target features.

[0056] Based on the basic operator library and the combined operator library, an operator registry is established. The operator registry is used to indicate the metadata information of the basic operators and the combined operators. The metadata information includes operator name, type, input parameter definition, output format and dependency relationship.

[0057] Based on the operator registry, dynamic operator loading information is determined, and a dynamic operator invocation framework is obtained according to the dynamic operator loading information, wherein the dynamic operator loading information is used to indicate the dynamic instantiation and invocation of the basic operator and the combined operator;

[0058] The basic operator library, the combined operator library, the operator registry, the operator dynamic loading information, and the dynamic operator calling framework are integrated, and a unified operator access interface and calling specification are established to obtain a predefined operator library.

[0059] Preferably, the present invention also provides a financial risk control feature on-demand calculation device, comprising:

[0060] The configuration parsing module is used to obtain the feature configuration file and the original financial data, parse the feature configuration file to obtain the set of feature variables to be calculated, the set of feature variables includes a list of target features with at least one target feature and feature dependency mapping relationships;

[0061] The dependency graph construction module is used to match and identify the feature types of each target feature based on the set of feature variables, and recursively process the dependent target features according to the feature types to construct a feature dependency graph, wherein the feature dependency graph is used to indicate the dependency relationship between the target features;

[0062] An execution plan generation module is used to perform topological sorting on the feature dependency graph to obtain a feature calculation sequence, and combine the mapping relationship of operators in a predefined operator library with the feature calculation sequence to generate a minimized operator execution plan. The feature calculation sequence is used to indicate the order of calculation of the target features to be calculated in the feature dependency graph, and the operator execution plan is used to indicate the calling sequence of operators corresponding to the target features to be calculated, arranged in dependency order.

[0063] The distributed computing module is used to call the corresponding operators according to the operator execution plan through a preset distributed computing framework, and use the called operators to perform feature calculations on the target features to be calculated and the original financial data to obtain feature calculation results.

[0064] The result output module is used to filter and format the feature calculation results and output the feature calculation results of the target features specified in the feature configuration file.

[0065] The beneficial effects of this invention include: modularizing and reusable feature calculation logic by constructing a predefined operator library; declaratively defining feature requirements by designing feature configuration file formats and syntax; dynamically identifying feature dependencies and constructing operator chains by developing a configuration parsing engine; efficiently generating features by designing an on-demand computation execution framework; and meeting online risk control requirements by implementing result output and performance optimization. Therefore, this application solves the problems of wasted computational resources, low computational efficiency, and inflexible iteration in traditional feature calculation methods, and has significant technical and practical value. Attached Figure Description

[0066] Figure 1 A flowchart illustrating an on-demand calculation method for financial risk control features provided in an embodiment of the present invention;

[0067] Figure 2 This is a schematic diagram of a financial risk control feature on-demand calculation device provided in an embodiment of the present invention. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] like Figure 1 As shown, this embodiment provides a method for on-demand calculation of financial risk control features, including:

[0070] S101. Obtain the feature configuration file and the original financial data, parse the feature configuration file to obtain the set of feature variables to be calculated, the set of feature variables includes a list of target features with at least one target feature and feature dependency mapping relationship.

[0071] In step S101, two key inputs are first obtained from the user or data source: feature configuration files and raw financial data. A feature configuration file is a structured document, typically in JSON format, defining the list of target features to be calculated and the dependencies between these features. Raw financial data consists of user transaction records, account information, behavior logs, etc., stored in a distributed file system or database. Specifically, the feature configuration file is read by a configuration file parser, extracting the list of target features and parsing the dependency configurations between features. It should be noted that target features refer to the feature variables that the user needs to use in the risk control model, such as "number of overdue payments in the past 6 months" or "income-to-debt ratio." The feature dependency mapping describes the other features or raw data fields that each feature calculation depends on, forming the logical chain of feature calculation. Therefore, step S101 converts the unstructured configuration file into a structured set of feature variables to facilitate subsequent feature calculations.

[0072] S102. Based on the set of feature variables, match and identify the feature types of each target feature, and recursively process the dependent target features according to the feature types to construct a feature dependency graph, wherein the feature dependency graph is used to indicate the dependency relationship between the target features.

[0073] In step S102, to further analyze the set of feature variables, the specific type of each target feature is identified, and a complete dependency graph is constructed. It should be noted that feature type refers to the mathematical attributes and calculation methods of the feature. Common types include statistical features (such as summation, mean, standard deviation, etc.), time-series features (such as sliding window, time decay, etc.), correlation features (such as correlation count, correlation ratio, etc.), and user behavior features (such as overdue count, transaction frequency, etc.). A predefined regular expression pattern matching method is used to analyze the feature variable names and identify their respective feature types. For example, for the feature "overdue count in the last 6 months," it is identified as a count feature with a time window of 6 months. After identifying the feature types, based on the dependency configuration in the configuration file, other features that each feature depends on are identified. For features with multi-level dependencies, a recursive processing method is used, continuously tracing downwards until all basic features or original data fields are found. Finally, these dependencies are organized into a directed acyclic graph (DAG), i.e., a feature dependency graph. In the feature dependency graph, each node represents a feature, and directed edges represent the dependencies between features.

[0074] S103. Perform topological sorting on the feature dependency graph to obtain a feature calculation sequence, and combine the mapping relationship of operators in the predefined operator library with the feature calculation sequence to generate a minimized operator execution plan. The feature calculation sequence is used to indicate the order of calculation of the target features to be calculated in the feature dependency graph, and the operator execution plan is used to indicate the calling sequence of operators corresponding to the target features to be calculated, arranged in dependency order.

[0075] In this application, the feature dependency graph is first topologically sorted to ensure that the order of feature computation satisfies the dependency requirements. It should be noted that topological sorting is an algorithm for sorting directed acyclic graphs, ensuring that for each directed edge (u, v), node u always appears before node v in the sort. Specifically, depth-first search or the Kahn algorithm is used for topological sorting to obtain a feature computation sequence that guarantees dependencies. After obtaining the feature computation sequence, a corresponding computation operator is found for each feature. An operator is a function component that encapsulates specific computational logic and is stored in a predefined operator library. Specifically, based on the feature type and name, a matching operator is searched in the operator library to establish a mapping relationship between features and operators. This process requires consideration not only of operator functional matching but also of operator efficiency and resource consumption. For complex derived features, multiple basic operators may need to be combined. Finally, based on the feature computation sequence and the feature-operator mapping relationship, a minimized operator execution plan is generated. Minimization refers to attempting to merge multiple calls to the same operator, eliminate redundant computation steps, and optimize the operator call order to maximize computational efficiency and resource utilization. The generated operator execution plan includes information such as operator call sequence, parameter configuration, and intermediate result management.

[0076] S104. The corresponding operators are called according to the operator execution plan through the preset distributed computing framework. The called operators are used to perform feature calculation on the target feature to be calculated and the original financial data to obtain the feature calculation result.

[0077] In this application, the operator execution plan is submitted to a distributed computing framework (such as Spark, Hadoop, etc.) for execution. The distributed computing framework is a software system capable of processing large-scale data in parallel across multiple servers, providing core functions such as data partitioning, task scheduling, and fault tolerance. First, the required raw data fields and data ranges are analyzed based on the operator execution plan to generate minimal data loading requirements. This on-demand loading strategy significantly reduces data I / O and memory consumption, improving computational efficiency. The necessary data partitions are read from the raw financial data through the distributed computing framework, and appropriate caching optimizations are performed. Then, operators are instantiated and called sequentially according to the operator execution plan. Each operator receives input data (which may be raw data or the output of other operators), executes specific computational logic, and produces output results. For each link in the computation chain, the input and output of operators are managed to ensure correct data transfer between operators. Simultaneously, the performance and resource consumption of operator execution are monitored, and optimizations are made as necessary. Finally, the computation results of all operators are aggregated to obtain the feature calculation results of the target feature.

[0078] S105. Filter and format the feature calculation results, and output the feature calculation output results of the target features specified in the feature configuration file.

[0079] In this application, the calculated feature results undergo post-processing to ensure the final output meets user requirements. First, the calculation results are filtered according to the target feature list specified in the feature configuration file, retaining only the feature variables actually needed by the user. This filtering process significantly reduces the amount of output data, lowering storage and transmission overhead. Next, the retained feature results are formatted and converted into the user-required output format, such as DataFrame, JSON, or CSV. During formatting, it is ensured that feature names, data types, and missing value handling conform to predetermined standards. Furthermore, feature quality checks are performed, detecting key indicators such as outliers and the proportion of missing values ​​to ensure the quality of the output features. Finally, the formatted feature results are output to a specified target location, such as a database, file system, or API interface, for use by downstream risk control models or business systems. Therefore, through precise filtering and formatting, the usability and effectiveness of the feature calculation results are ensured.

[0080] In some embodiments, based on the above embodiments, the step S101 of parsing the feature configuration file to obtain the set of feature variables to be calculated includes steps S201 to S204.

[0081] S201. Based on the feature configuration file, the target feature fields are read through a preset JSON format parser, and the corresponding target feature names are extracted to obtain a target feature list.

[0082] In this embodiment, a dedicated JSON parser is used to process the feature configuration file. It should be noted that JSON (JavaScript Object Notation) is a lightweight data exchange format, widely used for configuration files and data transmission due to its simplicity and readability. Specifically, the parser first verifies the JSON syntax of the configuration file, ensuring its structural integrity and absence of syntax errors. Then, the parser locates the "target_features" field in the configuration file, which contains the names of all user-specified feature variables to be calculated. The parser extracts the name of each feature from this field and performs normalization processing, such as removing extra spaces and standardizing case. During extraction, the validity of the feature names can be checked to ensure they conform to naming conventions. For cases where feature grouping exists, the "feature_groups" field is processed, parsing the grouping information and associating it with the feature list. Finally, a structured list of target features is generated, containing the names of all feature variables to be calculated and their grouping information. Therefore, this list of target features serves as the foundational data for subsequent feature dependency analysis.

[0083] S202. Use regular expressions to identify the format pattern of each target feature in the target feature list, and match the configuration items that the feature depends on based on the format pattern of the target feature to obtain the type identifier of each target feature.

[0084] It should be noted that regular expressions are a powerful tool for matching string patterns and play a crucial role in feature recognition. In this sub-step, a series of regular expression patterns are predefined, each corresponding to a feature type or format. Each feature name in the target feature list is iterated through, attempting to match it using various predefined patterns. When a matching pattern is found, key parameters, such as time window size and aggregation method, can be extracted from the feature name. Simultaneously, the matching pattern also identifies the feature type, such as statistical, time-series, or correlation-based. Based on the matching results, the corresponding configuration items are located in the "feature_dependencies" section of the feature configuration file. These configuration items define the feature's dependencies, computational parameters, and other metadata. Through this process, a type identifier is assigned to each target feature; this identifier will be used for subsequent dependency analysis and operator matching. The type identifier not only contains the feature's mathematical properties (such as mean and standard deviation) but also its business meaning (such as overdue behavior or transaction behavior), providing comprehensive contextual information for feature computation.

[0085] S203. Based on the type identifier, find the corresponding dependency configuration in the feature configuration file to obtain the feature dependency mapping relationship for each target feature.

[0086] In this embodiment, feature dependency mapping relationships are key information describing the other features or raw data required for feature computation. Using the previously determined feature type identifier, the dependency configuration for each feature is located in the "feature_dependencies" section of the configuration file. Dependency configurations typically include a "base" field, which lists the basic features or data fields required for feature computation. For example, for a composite feature "standard deviation of overdue payment frequency in the last 6 months," its dependencies might include basic features such as "number of overdue payments in the last 6 months." These dependency configurations are then parsed to construct a complete feature dependency mapping table. For some complex features, their dependencies may contain parameterized templates, which need to be instantiated based on the parameters extracted from the feature name. When processing dependencies, anomalies such as circular dependencies and missing dependencies are checked to ensure the integrity and correctness of the dependencies. The feature dependency mapping relationship not only records direct dependencies but also includes important information such as computation parameters and data ranges, providing detailed guidance for subsequent feature computation.

[0087] S204. Combine the target feature list and the feature dependency mapping relationship to obtain the set of feature variables to be calculated.

[0088] In this embodiment, the target feature list and feature dependency mapping relationships obtained from the previous analysis are integrated to form a complete feature variable set. This feature variable set is a complex data structure containing all the feature variables that need to be calculated and their complete metadata information. Specifically, for each feature in the target feature list, a corresponding entry is created in the feature variable set. This entry includes information such as the feature name, feature type, dependent feature list, and calculation parameters. Intermediate features found in the dependency chain but not in the original target feature list are also added to the feature variable set to ensure the integrity of the dependency chain. During the combination process, the feature variable set is optimized, such as removing duplicate features, merging similar features, and identifying shared dependencies, to improve the efficiency of subsequent calculations. Therefore, the final generated feature variable set is a hierarchical data structure that clearly describes all the feature variables that need to be calculated and their relationships, providing a comprehensive information foundation for subsequent dependency graph construction and calculation optimization.

[0089] In some embodiments, based on the above embodiments, the step of matching and identifying the feature types of each target feature based on the feature variable set, and recursively processing the dependent target features according to the feature types to construct a feature dependency graph, i.e., step S102 includes steps S301 to S304.

[0090] S301. Use a preset regular expression to perform pattern matching on each target feature in the set of feature variables to obtain the feature type of the target feature.

[0091] In this embodiment, a more in-depth type identification is performed on each feature in the feature variable set. It should be noted that although preliminary type identification was performed in the previous steps, this type identification is more refined and accurate. A more comprehensive regular expression library is used, capable of identifying various complex feature patterns. Regular expressions are a string processing technique based on pattern matching, composed of a series of characters and metacharacters, capable of describing and matching strings that conform to specific rules. In feature type identification, a corresponding regular expression pattern is defined for each feature type. These regular expressions are applied to each feature name; when a match is found, not only is the feature type determined, but type-related parameters are also extracted. Feature type is the core attribute of feature computation, determining the feature calculation method, required data, and operator selection. Accurate type identification lays the foundation for subsequent dependency analysis and operator matching.

[0092] S302. Based on the feature type of the target feature, find the basic fields of the target feature in the dependency configuration, identify the required basic features based on the basic fields, and construct a direct dependency list from the basic features.

[0093] In this embodiment, based on the previously identified feature types, the direct dependencies of the features are further analyzed. Base fields refer to the basic data items described in the dependency configuration of the feature profile, which describe the basic data required for feature computation. First, configuration items matching the feature type are searched in the dependency configuration, and the content of the "base" field is extracted. These base fields are usually template expressions, which need to be instantiated based on the parameters extracted from the feature names to obtain specific dependent feature names. Then, these instantiated dependent feature names are collected to form a direct dependency list. The direct dependency list records all other features or raw data fields that a feature directly depends on, forming the basis for constructing a complete dependency graph. For some complex features, their direct dependencies may include multiple features of different types, requiring proper identification and handling of these heterogeneous dependencies. The direct dependency list not only contains the names of the dependent features but also metadata such as dependency types and parameters, providing comprehensive information for subsequent recursive processing.

[0094] S303. For each basic feature in the list of direct dependencies, recursively perform dependency lookup to construct a feature dependency tree with multi-level dependency chains.

[0095] In this embodiment, a recursive method is used to deeply analyze the dependencies of each feature and construct a complete dependency tree. Recursion is an algorithmic strategy that solves problems by calling itself, and it is particularly suitable for handling hierarchical problems. For each basic feature in the list of direct dependencies, it is treated as a new target feature, and the previous type identification and direct dependency lookup process is repeated. This recursive process extends downwards until one of the following termination conditions is encountered: the original data field is found (basic data that does not require further calculation); a feature that has already been processed is found (avoiding circular dependencies); or a preset maximum recursion depth is reached (preventing excessively deep dependency chains). Through this recursive approach, multi-level dependencies between features can be discovered and organized into a tree structure, namely, a feature dependency tree. In the dependency tree, each node represents a feature, and the edges between nodes represent dependencies, with the direction of the edges pointing from the dependent feature to the dependent feature. The dependency tree clearly shows the hierarchical structure of feature computation, but may contain duplicate nodes because different features may depend on the same basic feature. This tree structure provides the foundation for subsequent directed acyclic graph transformations.

[0096] S304. Transform the feature dependency tree into a feature dependency graph represented as a directed acyclic graph structure, wherein the nodes of the feature dependency graph represent the target features, and the edges of the feature dependency graph represent the dependencies between the target features.

[0097] In this embodiment, the previously constructed feature dependency tree is transformed into a more efficient directed acyclic graph (DAG) structure. A DAG is a directed graph without cycles, ideally suited for representing systems with dependencies but no circular dependencies. Compared to a dependency tree, a dependency graph can more effectively represent shared dependencies, avoiding duplicate nodes and computations. First, an empty directed graph is created, and then all nodes and edges in the dependency tree are traversed. For each node (feature), if it is not yet in the graph, it is added; for each edge (dependency), a corresponding directed edge is created in the graph. During this process, circular dependencies are checked; if a cycle is found, a warning is issued and the cycle is broken, ensuring the graph's acyclicity. Simultaneously, multiple instances of the same feature are merged, ensuring that each feature has only one node in the graph. The resulting feature dependency graph is an optimized data structure where nodes represent target features, and directed edges represent dependencies between features, with the direction of the edges pointing from the dependent to the dependent. Therefore, this graph structure not only clearly expresses the dependencies in feature computation but also provides an ideal foundation for subsequent topological sorting and computational optimization. Feature dependency graphs are the core data structure of on-demand computing systems, ensuring the correctness of the computation order and the efficient use of computing resources.

[0098] In some embodiments, based on the above embodiments, the step of performing topological sorting on the feature dependency graph to obtain a feature computation sequence, and combining the mapping relationship of operators in a predefined operator library with the feature computation sequence to generate a minimized operator execution plan, includes:

[0099] The nodes in the feature dependency graph are topologically sorted to determine the feature calculation order without dependency conflicts, thus obtaining the feature calculation sequence.

[0100] Based on the feature type of each target feature in the feature calculation sequence, a corresponding operator is matched in the predefined operator library as the target operator for feature calculation, so as to generate operator calling information. The operator calling information is used to indicate the target operator and the corresponding operator calling relationship.

[0101] The operator call information is arranged according to the preset feature calculation priority, the call requests of the same target operator are merged and redundant calculation steps are eliminated to obtain the operator execution sequence;

[0102] Based on the computational complexity and data dependencies of the target operator, the execution sequence of the operator is assigned execution priority and resource allocation to obtain the operator execution plan.

[0103] In this embodiment, the constructed feature dependency graph is topologically sorted to determine the optimal execution order of feature computation. Topological sorting is a sorting algorithm for directed acyclic graphs (DAGs). Its principle is to arrange all nodes into a linear sequence such that for every directed edge (u, v) in the graph, node u always appears before node v in the sequence. This sorting method ensures that all dependent features have been computed before any feature is computed, thus avoiding dependency conflicts. It should be noted that topological sorting typically employs two classic algorithms: the Kahn algorithm or the Depth-First Search (DFS) algorithm. The basic idea of ​​the Kahn algorithm is to continuously remove nodes with an in-degree of 0 (i.e., no dependent features) and update the in-degree of their adjacent nodes until there are no nodes left in the graph. The DFS algorithm, on the other hand, traverses the graph in a depth-first manner, adding nodes to the result sequence during the backtracking phase, and finally reverses the sequence to obtain the topological sort. Regardless of the algorithm used, the graph is checked for cycles. If a cycle exists, it indicates a circular dependency between features, which is a logical error. A warning is issued, and an attempt is made to break the cycle. The resulting feature computation sequence is an ordered list representing the order in which features are computed, ensuring that no dependencies are left unmet during the computation process. Therefore, this topological sorting strategy not only guarantees the correctness of the computation but also provides a foundation for parallel computing and resource optimization.

[0104] Next, each feature in the feature calculation sequence is matched with operators in the predefined operator library to establish a correspondence between features and operators. An operator is a function component that encapsulates specific computational logic and serves as the basic execution unit for feature calculation. The predefined operator library is a collection of various operators, organized by function and type, including basic operators (such as summation, average, standard deviation, etc.) and combined operators (such as composite calculations, conditional calculations, etc.). First, based on the feature type identifier, the corresponding operator name is searched in the operator mapping configuration. These mapping relationships are usually defined in the "operator_mappings" section of the configuration file, containing the correspondence between feature types and operators, as well as the parameters required by the operator. After finding a matching operator, the parameter information in the feature name, such as time window size and aggregation method, is extracted and converted into the operator's input parameters. For complex derived features, multiple operators may need to be combined for calculation; the order of operator calls and parameter passing relationships are determined based on the feature's computational logic. Finally, operator call information is generated for each feature, including the operator name, input parameters, and output format. This information will be used in the subsequent generation of the operator execution plan. Therefore, the operator call information serves as a bridge connecting feature definition and actual computation, ensuring consistency between computational logic and feature requirements.

[0105] Furthermore, the generated operator call information is optimized to form an efficient operator execution sequence. It should be noted that feature computation priority refers to the importance and urgency of a feature in the computation process, typically determined based on its business value, computational complexity, and dependencies. First, the operator call information is sorted according to a preset priority rule to ensure that high-priority features are computed first. Then, the sorted call information is optimized, mainly in two aspects: merging call requests for identical operators and eliminating redundant computation steps. For merging call requests for identical operators, requests that call the same operator but with different parameters are identified and merged into a single batch call, reducing the overhead of operator instantiation and invocation. For example, if multiple features need to calculate the average value over a certain time period, these requests can be merged into a single call to the average value operator, processing different data columns. For eliminating redundant computation steps, the dependencies between operators and data flow are analyzed to identify duplicate computations. By caching intermediate results or adjusting the computation order, duplicate computations are avoided, significantly improving computational efficiency and reducing resource consumption. Finally, an optimized operator execution sequence is generated, which satisfies dependency requirements while maintaining good computational efficiency. Therefore, configuring the operator execution sequence directly affects the performance and resource consumption of feature computation.

[0106] Furthermore, to further optimize the operator execution sequence, considering the limitations of computing resources and the execution environment, a final operator execution plan is generated. Computational complexity refers to the computing resources and time required for operator execution, typically related to algorithm complexity, data size, and processing logic. Data dependencies refer to the data flow relationships between operators, determining the order of operator execution and the data transfer method. First, the computational complexity of each operator is evaluated, usually based on algorithm analysis and historical execution data, categorizing operators into different complexity levels, such as linear complexity, quadratic complexity, etc. Then, the data dependencies between operators are analyzed, a data flow graph is constructed, and the data transfer paths and methods are determined. Based on this information, execution priority and resource allocation are assigned to each operator. Execution priority determines the priority of operators during resource contention; generally, operators with higher complexity and lower dependencies receive higher priority. Resource allocation includes parameters such as memory allocation, number of CPU cores, and parallelism, which are allocated reasonably according to the complexity and importance of the operators. For operators on the critical path, more resources are allocated to accelerate execution; for operators that can be executed in parallel, parallel computing strategies are implemented. Finally, a complete operator execution plan is generated, including information such as the operator call sequence, parameter configuration, resource allocation, and parallelization strategy. Therefore, this execution plan in this embodiment provides detailed guidance for the distributed computing framework to perform feature computations, directly affecting the efficiency and reliability of the computation.

[0107] In some embodiments, the step of calling corresponding operators according to the operator execution plan through a preset distributed computing framework, and using the called operators to perform feature calculations on the target features to be calculated and the original financial data to obtain feature calculation results includes:

[0108] Based on the operator execution plan, the original data fields and data ranges required by each target operator are analyzed to generate a data loading requirement list, and the minimum dataset definition is obtained based on the data loading requirement list;

[0109] The corresponding data partitions are read from the original financial data according to the definition of the minimum dataset using a preset distributed computing framework, and the data partitions are cached to obtain the dataset required for operator execution.

[0110] Based on the dataset, the target operators in the predefined operator library are instantiated and called sequentially according to the order indicated by the operator execution plan. The intermediate calculation results of the target operators are obtained according to the operator execution plan and the called target operators.

[0111] The intermediate calculation results of each target operator are managed and transmitted, and the intermediate calculation results of all target operators are summarized to obtain the feature calculation results of the target feature.

[0112] In this embodiment, an in-depth analysis of the operator execution plan is conducted to determine the minimum dataset required for feature computation, thereby optimizing data loading and processing efficiency. The original data fields refer to basic data columns stored in a distributed file system or database, such as user ID, transaction amount, and timestamp. The data range refers to the time span, user scope, and other limiting conditions of the data to be processed. First, all operators in the operator execution plan are traversed, extracting the input data fields and filtering conditions required for each operator. This information is typically defined in the operator's parameter configuration, including field names, data types, and time ranges. Then, these requirements are merged and optimized, removing duplicate fields and unifying the time range to form a complete list of data loading requirements. This list details all data items and conditions required for feature computation. Based on this list of requirements, a minimal dataset definition is generated. This is a streamlined data query and loading scheme that includes only the necessary data fields and ranges. Minimization means reducing the amount of data loaded, lowering IO overhead, and reducing memory usage while meeting computational requirements. The relationships between fields are analyzed, similar data requests are merged, query conditions are optimized, and an efficient data loading strategy is generated. Minimizing dataset definitions is typically expressed using SQL queries, Spark DataFrame operations, or other data processing languages, providing clear guidance for subsequent data loading. Therefore, this embodiment significantly improves data processing efficiency and resource utilization through an on-demand loading strategy.

[0113] Furthermore, efficient data loading and preprocessing are achieved using distributed computing frameworks (such as Spark and Hadoop). A distributed computing framework is a software system capable of processing large-scale data in parallel across multiple servers, providing core functionalities such as data partitioning, task scheduling, and fault tolerance. Data partitioning refers to dividing large-scale data into multiple smaller parts, distributed across different computing nodes, facilitating parallel processing. First, the minimal dataset definition is converted into data loading operations supported by the distributed computing framework, such as SparkSQL queries or DataFrame operations in Spark. Then, the corresponding data partitions are read from the original financial data through these operations. During the reading process, filtering conditions from the minimal dataset definition, such as time range and user conditions, are applied to further reduce the amount of data loaded. Appropriate data caching optimization is performed on the loaded data partitions. Data caching refers to storing frequently used data in memory to reduce the overhead of repeated loading and computation. Data usage patterns in the operator execution plan are analyzed to identify frequently accessed data partitions, which are then cached in memory to improve access speed. For extremely large-scale data, partial caching or sliding caching strategies are adopted to achieve optimal caching performance within memory constraints. Through these optimization measures, an efficient dataset was obtained that is required for operator execution. This dataset contains the necessary data fields and has good access performance, providing a solid foundation for subsequent operator execution.

[0114] Next, the actual feature computation begins, invoking operators from the predefined operator library according to the operator execution plan. Operator instantiation refers to creating a runtime instance of the operator based on its definition and parameter configuration, preparing it for computation. First, each operator is processed sequentially according to the order in the operator execution plan. For each operator, its definition and parameter configuration are loaded, and a corresponding operator instance is created. During instantiation, the validity of parameters is checked, the operator's internal state is initialized, and a connection to the dataset is established. Then, the operator's execution method is called, passing in the necessary input data and parameters. The feature computation logic is executed by the operator to process the input data and generate computation results. The feature computation logic is the core function of the operator and may include various operations such as data filtering, aggregation, and transformation to meet specific feature computation requirements. During execution, the operator's running status is monitored, including execution time, resource consumption, and abnormal situations, and optimizations are made as necessary. For computationally intensive operators, parallel computing mode may be enabled, dividing the data into multiple parts, processing them in parallel on multiple computing nodes, and then merging the results. For operators that depend on the results of other operators, ensure that the dependent operators have completed execution and that intermediate results are correctly passed. After each operator completes execution, save its calculation result as an intermediate result for use by subsequent operators. These intermediate results may be stored in memory, a distributed cache, or a temporary file, depending on the size and frequency of use. Through this sequential invocation and execution, efficient feature computation is achieved, generating intermediate calculation results for all target operators.

[0115] Furthermore, intermediate results generated during operator execution are systematically managed to ensure correct data transmission and final result aggregation. Intermediate calculation results refer to temporary data generated by each operator during feature calculation that has not yet formed the final feature output. First, an intermediate result management mechanism is established, including result identification, storage location, and lifecycle control. For each intermediate result generated by an operator, a unique identifier is assigned, and metadata information such as its data structure, source operator, and target operator is recorded. Then, the transmission of intermediate results is arranged according to the data flow relationships defined in the operator execution plan. For intermediate results used as input to other operators, their availability and correctness are ensured, and format conversion or data reorganization is performed when necessary to meet the input requirements of the target operator. During transmission, the data flow path is optimized to reduce unnecessary data movement and copying, and improve transmission efficiency. For intermediate results that are no longer needed, the occupied storage resources are released in a timely manner to avoid memory leaks and resource waste. After all operators have been executed, the final calculation results are aggregated to form the feature calculation results of the target feature. This aggregation process may include operations such as result merging, format unification, and missing value handling to ensure the integrity and consistency of the final result. In a distributed computing environment, it is also necessary to handle the collection and aggregation of results, merging the partial results scattered across various computing nodes into a complete feature result set. Therefore, this embodiment, through this systematic result management and transmission mechanism, ensures the accuracy and efficiency of feature computation, generating feature computation results that meet the requirements.

[0116] In some embodiments, the step of matching and identifying the feature types of each target feature based on the set of feature variables, and recursively processing the dependent target features according to the feature types to construct a feature dependency graph, further includes:

[0117] Acquire historical feature calculation data, and extract the calculation time, resource consumption, and business impact data of each target feature from the historical feature calculation data to generate a feature performance history record;

[0118] Based on the historical performance records of the features, the heteroscedasticity characteristics of different target features are calculated using a preset White test algorithm. Based on the heteroscedasticity characteristics, a variance function model of the target features is constructed. The variance function model is used to indicate the heteroscedasticity distribution characteristics of the target features.

[0119] Based on the heteroscedasticity distribution characteristics indicated by the variance function model, the business value score, computational cost value, and stability coefficient of the target feature are determined. Based on the business value score, computational cost value, and stability coefficient of the target feature, the weight of the target feature is calculated. Based on the weight of the target feature and the feature dependency tree, a weighted feature map is constructed.

[0120] The weights in the weighted feature map are optimized using a preset generalized least squares algorithm, and the priority order of feature calculation in the weighted feature map is adjusted to obtain the optimized weighted feature map.

[0121] Anomalies in the optimized weighted feature map are identified using a pre-defined Breusch-Pagan test algorithm to generate a first test result. The autocorrelation of the dependencies between the target features is evaluated using a pre-defined Durbin-Watson test algorithm to generate a second test result. The structure of the optimized weighted feature map is adjusted based on the first and second test results to obtain a feature dependency map.

[0122] In this embodiment, historical feature calculation performance data is collected and analyzed to provide a basis for subsequent feature optimization. Historical feature calculation data refers to various performance metrics and business data recorded during past feature calculations. First, raw data from historical feature calculations is obtained from performance monitoring, log files, or databases. For example, this data typically includes detailed records of each feature calculation, such as performance metrics like calculation time, memory usage, and CPU utilization, as well as business data such as feature usage and importance scores. Then, this raw data is processed and analyzed to extract key performance metrics for each target feature. Calculation time refers to the time required for feature calculation, reflecting the feature's computational complexity and efficiency; resource consumption includes memory usage, CPU utilization, and I / O operations, reflecting the feature's resource requirements; business impact data includes the feature's importance in the risk control model, usage frequency, and business value, reflecting the feature's business value. Statistical analysis is performed on this extracted data, calculating the average, variance, and distribution characteristics of each metric to form a performance profile for each feature. Finally, these analysis results are organized into a structured feature performance history record, containing detailed performance and business metrics for each feature. These historical records provide important evidence for subsequent feature heteroscedasticity analysis and optimization, enabling the identification of features with high computational costs but low business value, and allowing for targeted optimization.

[0123] Subsequently, in-depth statistical analysis of the historical performance records of features is conducted to identify heteroscedasticity in feature computation performance, providing a scientific basis for feature optimization. Heteroscedasticity refers to the different variances of performance indicators (such as computation time and resource consumption) among different features, manifesting as uneven data distribution and uncertainty. The White test is a statistical method used to detect heteroscedasticity in data. By analyzing the relationship between residuals and independent variables, it determines whether heteroscedasticity exists. Specifically, the historical performance records of features are first compiled into a dataset suitable for statistical analysis, including various performance indicators of the features and their influencing factors. Then, the White test algorithm is applied to analyze the performance data of each feature to detect whether significant heteroscedasticity exists. The basic idea of ​​the White test is to construct a regression equation between the squared residuals and the independent variable, and to determine whether heteroscedasticity exists by testing whether the regression coefficient is significantly non-zero. For features found to have heteroscedasticity, the specific patterns and influencing factors of heteroscedasticity are further analyzed, such as whether they are related to data size, time window size, etc. Based on the above analysis, a variance function model is constructed for each feature, which is a mathematical model describing the relationship between the variance of feature performance indicators and influencing factors. The variance function model can be a linear model, an exponential model, or other suitable mathematical form. It is used to predict the performance fluctuation of features under different conditions, providing an important basis for subsequent feature weight calculation and optimization, and enabling more accurate assessment and management of the risks and costs of feature calculation.

[0124] Furthermore, based on the preceding heteroscedasticity analysis results, features are comprehensively evaluated, feature weights are calculated, and a weighted feature map is constructed. It should be noted that the business value score refers to the importance and contribution of a feature in a business scenario, typically assessed based on factors such as the feature's importance in the model and its usage frequency; the computational cost value refers to the time and resource consumption required for feature computation, reflecting the feature's computational efficiency; and the stability coefficient refers to the stability of the feature's computational performance, determined by the volatility of the variance function model. Specifically, firstly, based on the analysis results of the variance function model, a business value score, computational cost value, and stability coefficient are assigned to each feature. The business value score is usually assessed by business experts or derived through model analysis, reflecting the feature's business importance; the computational cost value is calculated based on the feature's average computation time and resource consumption, reflecting the feature's computational cost; and the stability coefficient is assessed based on the feature's heteroscedasticity characteristics—the more significant the heteroscedasticity, the lower the stability. Then, the comprehensive weight of the feature is calculated based on these three indicators. The weight calculation formula is typically: Weight = (Business Value Score / Computational Cost Value) * Stability Coefficient, which balances the feature's business value and computational cost while considering performance stability, resulting in a more reasonable feature weight. Next, this weight information is combined with the feature dependency tree to construct a weighted feature graph. In this graph, nodes represent features, edges represent dependencies, and the weights of nodes represent the importance and priority of features. Therefore, this embodiment, through a weighted mechanism, can more scientifically manage the priority and resource allocation of feature computation, improving overall computational efficiency and business value.

[0125] Furthermore, the Generalized Least Squares (GLS) algorithm is applied to further optimize the weighted feature map, improving the scientific rigor and rationality of the weight allocation. GLS is a statistical method used to handle regression problems with heteroscedasticity or autocorrelation. By introducing a weight matrix, observations are assigned different weights to obtain more accurate parameter estimates. Specifically, first, an objective function for feature weight optimization is constructed, typically aiming to maximize overall business value or minimize computational cost, while considering the dependencies between features and computational constraints. Then, the GLS algorithm is applied to solve this objective function, yielding the optimized feature weights. It should be noted that the core of the GLS algorithm is constructing an appropriate weight matrix that reflects the variance and covariance relationships of each feature's performance indicators, and then solving for the parameters using the weighted least squares method. For features with significant heteroscedasticity, GLS assigns different weights to reduce the impact of heteroscedasticity on the optimization results. During the optimization process, the dependencies between features are considered to ensure that the feature weights on the dependency chain are reasonably allocated, avoiding unreasonable situations where dependent features have low weights but dependent features have high weights. After optimization, the priority ranking of feature calculations is adjusted according to the new weight values, and the node weights in the weighted feature map are updated. This weight optimization based on statistical methods makes the resource allocation for feature calculations more scientific and reasonable, enabling the achievement of maximum business value with limited resources.

[0126] In this embodiment, anomaly relationships in the optimized weighted feature map are identified using a preset Breusch-Pagan test algorithm, generating a first test result. A second test result is generated by evaluating the autocorrelation of the dependencies between the target features using a preset Durbin-Watson test algorithm. The structure of the optimized weighted feature map is then adjusted based on the first and second test results to obtain a feature dependency map. It should be noted that two statistical testing methods are applied to further verify and optimize the weighted feature map, ensuring the rationality and reliability of the feature dependencies. The Breusch-Pagan test is a statistical method for detecting heteroscedasticity. By analyzing the relationship between the squared residuals and the independent variables, it determines whether heteroscedasticity exists in the data. The Breusch-Pagan algorithm is applied to test the relationships in the weighted feature map, identifying abnormal association patterns, such as erroneous dependencies and unreasonable weight allocation. The basic idea of ​​this test is to construct a regression equation between the squared residuals and the independent variables, and determine whether anomaly relationships exist by testing whether the regression coefficients are significantly non-zero. The test results (first test results) include information such as the statistical significance and degree of anomaly of each relationship. The Durbin-Watson test is a statistical method for detecting autocorrelation in sequences. It analyzes the relationships between adjacent terms in the residual sequence to determine if autocorrelation exists. The Durbin-Watson algorithm is applied to assess the autocorrelation of dependencies between features, identifying redundant or cyclic dependencies. The core of the test is calculating the Durbin-Watson statistic, which is based on the ratio of the sum of squared differences between adjacent terms in the residual sequence to the sum of squared residuals, reflecting the degree of autocorrelation. The test results (second test results) include information such as the autocorrelation coefficient and significance of the dependencies. Based on these two test results, the weighted feature map is adjusted and optimized. For detected abnormal associations, their weights may be reduced, their necessity reassessed, or alternative dependency paths sought; for detected highly autocorrelated dependencies, dependency chains may be simplified, redundant dependencies removed, or the calculation order optimized. Therefore, this embodiment improves the scientific rationality of the feature dependency map, providing reliable dependency guidance for subsequent feature calculations.

[0127] Example 7:

[0128] In some embodiments, the step of performing topological sorting on the feature dependency graph to obtain a feature computation sequence, and combining the mapping relationships of operators in a predefined operator library with the feature computation sequence to generate a minimized operator execution plan, further includes:

[0129] The operator execution plan is transformed into an operator execution graph, wherein the vertices in the operator execution graph represent the target operators indicated in the operator execution plan, and the edges in the operator execution graph represent the data dependencies and calling relationships between the target operators;

[0130] Based on the operator execution graph, the operator attributes of the target operators are extracted using a preset graph attention network. The intrinsic relationships between the target operators are determined based on the operator attributes. The low-dimensional embedding representation of the target operators is obtained from the intrinsic relationships. The operator attributes include computational complexity features, data dependency features, and resource consumption features.

[0131] Based on the low-dimensional embedding representation of the target operators, a similarity matrix between the target operators is calculated. Based on the operator computation patterns and clustering relationships identified by the similarity matrix, an operator knowledge graph is constructed, wherein the operator knowledge graph is used to indicate the semantic association information between the target operators.

[0132] Based on the operator knowledge graph, a multi-objective optimization model is constructed, and a constrained optimization model for operator scheduling is obtained based on the multi-objective optimization model. The multi-objective optimization model includes an optimization objective and constraints. The optimization objective is used to indicate minimizing the total computation time and resource consumption, and the constraints include dependency constraints, resource constraints, and time constraints.

[0133] The constrained optimization model is solved using a pre-defined particle swarm optimization-simulated annealing hybrid algorithm to obtain an optimized operator execution plan. The particle swarm optimization-simulated annealing hybrid algorithm is used to indicate the search for the optimal operator scheduling scheme by adaptively adjusting the particle position and velocity update parameters and the simulated annealing acceptance probability mechanism.

[0134] In this embodiment, the initially generated operator execution plan is transformed from a linear sequence into a structured graphical representation for more advanced optimization analysis. The operator execution graph is a data structure that uses a graph structure to represent the operator computation process, more intuitively showing the dependencies between operators and the data flow. Specifically, an empty directed graph is first created, with each target operator as a vertex (node). Then, each operator and its dependencies in the operator execution plan are traversed, and directed edges are established for each pair of dependent operators. It should be noted that the direction of these edges is typically from the data provider to the data user, representing the direction of data flow. For each vertex (operator), its detailed information is stored, including operator type, parameter configuration, resource requirements, etc.; for each edge, information such as the type, format, and size of data transmission is recorded. During the construction process, the integrity and consistency of the graph are checked to ensure that all operators are included in the graph and that the dependencies are correct. After the graph is constructed, preliminary analysis is performed to calculate the in-degree and out-degree of each node and identify critical paths and bottleneck nodes. In this embodiment, the operator execution graph not only provides a visual representation of operator computation but also serves as a foundational data structure for subsequent graph algorithm analysis and optimization. Therefore, this embodiment, through this graph structure, can more effectively analyze the relationships between operators and identify optimization opportunities, such as parallel execution and data reuse, thereby improving overall computational efficiency.

[0135] Subsequently, deep learning methods are applied to perform advanced feature extraction and relationship modeling on the operator execution graph, providing deeper algorithmic support for subsequent optimization. Graph Attention Networks (GAT) are a neural network model specifically designed for graph-structured data. They effectively capture relationships between nodes by assigning different weights to each node through a self-attention mechanism. Specifically, the operator execution graph is first converted into a format that the GAT model can process. Each operator node is represented as an initial feature vector containing the operator's basic attributes. These initial features mainly include three categories: computational complexity features, such as time complexity, space complexity, and algorithm type, reflecting the computational difficulty of the operator; data dependency features, such as the number of dependencies, dependency type, and data flow, reflecting the degree of correlation between operators; and resource consumption features, such as memory usage, CPU utilization, and parallelism, reflecting the operator's demand for system resources. Then, the GAT model is trained to learn the attention weights between operator nodes. During training, the model automatically adjusts the attention weights, focusing on more important neighbor nodes to capture important relationships between operators. After training, the GAT model is used to process the operator execution graph, generating updated feature representations for each operator node. It should be noted that the updated feature representations integrate the node's own features and information from neighboring nodes, providing a more comprehensive reflection of the operator's role and characteristics throughout the computation process. Finally, dimensionality reduction techniques (such as t-SNE and PCA) are used to convert the high-dimensional feature representations into low-dimensional embedding representations, facilitating visualization and subsequent processing. The low-dimensional embedding representations preserve the intrinsic relationships and structural information between operators, providing a foundation for subsequent similarity calculations and knowledge graph construction. Through the application of the GAT model, deep learning and representation of operator relationships are achieved, providing more intelligent decision support for operator scheduling optimization.

[0136] Furthermore, based on this low-dimensional embedding representation, the similarity and correlation between operators are further analyzed to construct a more advanced knowledge representation model. It should be noted that the similarity matrix is ​​a square matrix, where each element represents the degree of similarity between two operators, typically calculated using metrics such as cosine similarity and Euclidean distance. Specifically, firstly, based on the low-dimensional embedding representation of the operators, the similarity between all operator pairs is calculated to form a similarity matrix. Therefore, by constructing the similarity matrix, the similarity relationships between operators can be intuitively displayed; operators with high similarity may have similarities in function, number, or performance characteristics. Then, the similarity matrix is ​​analyzed to identify specific operator computation patterns and clustering relationships. Operator computation patterns refer to operator combinations and call sequences that frequently occur in feature computation, such as common patterns like "filtering-aggregation-sorting"; clustering relationships refer to groups of operators with similar functions or characteristics, such as all statistical operators, all time-series operators, etc. Optionally, clustering algorithms (such as K-means, DBSCAN, etc.) or pattern mining algorithms can be used to discover these patterns and relationships from the similarity matrix. Based on the identified patterns and relationships, an operator knowledge graph is constructed. This operator knowledge graph is a semantic network where nodes represent operators or groups of operators, and edges represent semantic relationships between operators. Therefore, the operator knowledge graph not only contains direct dependencies between operators but also richer semantic association information, such as functional similarity, performance characteristics, and alternative options. During construction, similarity data, domain expert knowledge, and historical execution records are comprehensively considered to ensure the accuracy and completeness of the knowledge graph. The final generated operator knowledge graph provides rich semantic knowledge for subsequent operator scheduling optimization, enabling more intelligent scheduling decisions, such as identifying substitutable operators, optimizing execution order, and merging similar operations.

[0137] Furthermore, the operator scheduling problem is formalized into a mathematical optimization problem, establishing a rigorous optimization model to provide a theoretical foundation for subsequent solutions. Multi-objective optimization refers to a mathematical problem that simultaneously optimizes multiple objective functions, often with conflicting objectives requiring a balanced solution. Specifically, the optimization objectives are first determined based on the operator knowledge graph and the preceding analysis. In feature computation scenarios, typical optimization objectives include minimizing total computation time and minimizing resource consumption. It should be noted that total computation time refers to the time from start to finish of all feature computations, influenced by the operator execution order, parallelism, and scheduling strategy; resource consumption includes the total consumption of system resources such as memory usage, CPU usage, and network bandwidth. Then, constraints are determined, limiting the range of feasible solutions. Dependency constraints refer to the dependencies that operator execution must follow to ensure the correctness of data flow; resource constraints refer to the upper limit of available system computing resources, such as memory capacity and the number of CPU cores; time constraints refer to the requirement that feature computation must be completed within a specified time. These objectives and constraints are formalized into mathematical expressions to construct a multi-objective optimization model. Because conflicts may exist between multiple objectives (e.g., accelerating computation may increase resource consumption), it is usually necessary to define appropriate weights or Pareto optimal strategies to balance different objectives. Finally, the multi-objective optimization model is transformed into a constrained optimization model, which is a more specific mathematical model that defines the precise mathematical expression of decision variables (such as operator execution order, resource allocation, etc.), objective function, and constraints. This constrained optimization model is the direct input for subsequent algorithm solutions; by solving it, the optimal or near-optimal operator scheduling scheme that satisfies all constraints can be found.

[0138] Furthermore, advanced optimization algorithms are applied to solve the previously established constrained optimization model, yielding optimal or near-optimal operator execution plans. The Particle Swarm Optimization-Simulated Annealing hybrid algorithm is a hybrid metaheuristic algorithm combining the advantages of Particle Swarm Optimization (PSO) and Simulated Annealing (SA), particularly suitable for solving complex combinatorial optimization problems. PSO is a swarm intelligence-based optimization method that simulates bird flock foraging behavior, where multiple particles move and share information in the solution space to jointly find the optimal solution. Simulated Annealing, inspired by the metallurgical annealing process, is a stochastic search algorithm that introduces randomness into the search process, enabling it to escape local optima and find the global optimum with a higher probability. Specifically, a swarm of particles is first initialized, with each particle representing a possible operator scheduling scheme. During initialization, based on the previous analysis results, reasonable initial positions are assigned to the particles to improve search efficiency. Then, the iterative search process begins. In each iteration, the position and velocity of each particle are updated. Position updates are based on the particle's current velocity, representing the particle's movement in the solution space; velocity updates consider both the particle's own optimal position and the swarm's optimal position, guiding the particle towards a better solution. Unlike traditional PSO, this embodiment employs an adaptive parameter adjustment strategy, dynamically adjusting the speed update parameters based on the search stage and particle performance to balance global exploration and local exploitation. Simultaneously, this embodiment introduces a simulated annealing mechanism to add randomness to particle position updates. By defining a temperature parameter and an acceptance probability function, particles are allowed to accept poor solutions with a certain probability, thus escaping local optima. The temperature parameter gradually decreases with iteration, shifting the search process from large-scale exploration to refined local search. During iteration, the scheduling scheme represented by each particle is continuously evaluated, updating the individual optimal and global optimal solutions. The evaluation criteria are based on the previously defined objective function, while also considering the degree to which constraints are satisfied. When the termination condition (such as the maximum number of iterations or convergence criterion) is met, the global optimal solution is output as the final operator execution plan. This optimized execution plan, while satisfying all constraints, achieves an optimal balance between computation time and resource consumption, significantly improving the efficiency and reliability of feature computation.

[0139] In some embodiments, the feature calculation using the invoked operator and the original financial data to obtain the feature calculation result, and the on-demand feature calculation further includes heterogeneous feature calculation based on multi-physics coupling, including:

[0140] The temporal features in the target features are mapped to a temporal feature field, the statistical features in the target features are mapped to a statistical feature field, the correlation features in the target features are mapped to a correlation feature field, and the temporal feature field, the statistical feature field, and the correlation feature field are combined to form a feature physical field;

[0141] Based on the aforementioned characteristic physical fields, a set of coupled equations is constructed, which includes time-series characteristic field equations, statistical characteristic field equations, and correlation characteristic field equations.

[0142] Based on the coupling terms and source terms between the characteristic fields determined by the coupling equations, a multi-characteristic field coupling mathematical model is obtained.

[0143] The multi-feature field coupled mathematical model is solved by using a preset operator splitting algorithm. In the separation solution, a preset alternating direction implicit algorithm is used to calculate alternately in different feature dimensions and iteratively process the coupling terms to obtain the separation calculation results of each feature field.

[0144] A multi-level computational grid is constructed, and a preset V-loop strategy is used to solve the multi-level computational grid to obtain multi-resolution feature calculation results. The multi-level computational grid includes a coarse grid level and a fine grid level. The coarse grid level is used for fast approximate calculation, and the fine grid level is used for high-precision region calculation.

[0145] Dynamic load balancing is performed on the multi-resolution feature calculation results, so that a preset work stealing strategy and task migration strategy are used to calculate resource allocation data. Based on the resource allocation data, elastic resource scheduling and dynamic scaling of computing nodes are performed to obtain heterogeneous feature calculation results.

[0146] Based on the heterogeneous feature calculation results, the feature calculation results are obtained.

[0147] In this embodiment, the concept of a physical field model is introduced, abstracting different types of features into physical fields to lay the foundation for subsequent multiphysics coupling calculations. A physical field is a concept in physics that describes the distribution of physical quantities in space, such as electric fields and magnetic fields. This concept is borrowed here to abstract feature calculation into the interaction and evolution process of multiple physical fields. Specifically, the target features are first classified to identify different types of features. It should be noted that time-series features are those closely related to the time dimension, such as the frequency of recent user transactions and the trend of changes in consumption amounts; statistical features are those extracted from data using statistical methods, such as the mean, standard deviation, and quantiles; and correlation features are those calculated through relationships between entities, such as social network connectivity and co-occurrence transaction frequency. Then, each type of feature is mapped to a corresponding feature field. For example, a time-series feature field describes the distribution and evolution of features over time, typically with time as the independent variable and feature values ​​as the dependent variable, represented as a function f(t) or a time series; a statistical feature field describes the distribution characteristics of features in a statistical space, such as probability density functions and cumulative distribution functions; and a correlation feature field describes the distribution and propagation characteristics of features in a relational network, typically represented as a graph structure G(V,E). This mapping process involves not only conceptual abstraction but also the transformation of mathematical representation, reconstructing the feature computation problem into a physical field problem. In this embodiment, appropriate mathematical representations, boundary conditions, and initial conditions are defined for each feature field to construct a complete field description. Finally, these different types of feature fields are combined into a feature physical field, which is a multi-field coupled composite system where each subfield is both relatively independent and mutually influential. The feature physical field provides the conceptual and mathematical foundation for the subsequent construction and solution of coupling equations, enabling the handling of complex feature computation problems using methods from physics and numerical computation.

[0148] Next, mathematical descriptions are established for the previously defined characteristic physical fields, constructing a set of equations expressing the internal laws of each field and the interactions between fields. The coupling equation set is a collection of mathematical equations describing multiple mutually influential systems. Each equation describes the behavior of a subsystem, while coupling terms between equations represent the interactions between subsystems. Specifically, basic equations are first established for each characteristic field. It should be noted that time-series characteristic field equations typically adopt the form of time series models or differential equations, such as autoregressive models (AR), moving average models (MA), or combinations thereof (ARMA, ARIMA), describing the changes in characteristics over time; statistical characteristic field equations use probabilistic statistical models, such as probability density functions like normal distribution and Poisson distribution, or mathematical expressions describing the calculation process of statistics; correlation characteristic field equations are usually based on graph theory or network science models, such as the PageRank algorithm and community detection algorithms, describing the distribution and propagation of characteristics in the network structure. Then, the mutual influence between fields is analyzed, and coupling terms are defined. Coupling terms are mathematical expressions connecting different field equations, representing the influence of one field on another. For example, changes in the temporal field may affect the distribution parameters of the statistical field, and the structure of the correlated field may affect the evolution rate of the temporal field. Based on the actual dependencies and interaction patterns between features, the form and strength of the coupling terms are determined. Finally, the fundamental equations of each field and the coupling terms are combined into a complete set of coupled equations. This set of equations is a complex mathematical system, potentially containing various types of equations, such as differential equations, integral equations, and algebraic equations, and their interrelationships. Therefore, this embodiment ensures the mathematical completeness and physical consistency of the equation set, laying the foundation for subsequent numerical solutions. This embodiment, by constructing a set of equations, formalizes the feature calculation problem into a multiphysics coupling problem, enabling efficient solutions using mature numerical methods.

[0149] In this embodiment, the coupled equations are further refined by adding source terms and boundary conditions to form a complete mathematical model. Source terms refer to terms in the equations that do not depend on field variables, typically representing external inputs or independent excitations. In feature calculation, source terms may be original data inputs, user-defined parameters, or the influence of external systems. Specifically, the input sources of each feature field are first analyzed to identify the influence of external factors such as original data fields and configuration parameters on the feature fields, and these influences are represented as source terms in the equations. The form of source terms may be constants, functions, or time variables, depending on the nature and variation of the external factors. Then, combined with the previously defined coupling terms, a complete multi-feature field coupled mathematical model is constructed. This multi-feature field coupled mathematical model is a complex system containing multiple equations, multiple variables, multiple coupling terms, and source terms. Each equation in the multi-feature field coupled mathematical model describes the evolution of a feature field, the coupling terms between equations describe the interactions between fields, and the source terms describe the influence of external inputs. Therefore, this embodiment ensures the mathematical completeness of the model, including the equality of the number of equations and unknowns, and the complete definition of initial and boundary conditions. Meanwhile, this embodiment also focuses on the physical rationality of the model, ensuring that the model reflects the actual process and laws of feature calculation. The multi-feature field coupled mathematical model is the theoretical foundation of feature calculation, abstracting the complex feature calculation problem into a rigorous mathematical problem that can be solved using advanced numerical methods.

[0150] Subsequently, advanced numerical algorithms are applied to solve the previously established multi-feature field coupled mathematical model, yielding the feature calculation results. Operator splitting is a numerical method for handling complex differential equations or systems of equations. Its core idea is to decompose a complex problem into multiple simpler sub-problems, solve them separately, and then combine the results. In feature calculation, operator splitting is used to separate the coupled feature field equations into independent sub-equations, simplifying the solution process. Specifically, the operator splitting method is first applied to the multi-feature field coupled model, decomposing the original system of equations into multiple sub-equations, each corresponding to a feature field. During the splitting process, it is necessary to ensure the numerical stability and convergence of the splitting scheme, typically employing mature techniques such as the fractional step method or directional splitting. After splitting, each sub-equation system may still be complex, further solved using the Alternating Directional Implicit (ADI) algorithm. It should be noted that the ADI algorithm is an efficient implicit numerical method suitable for solving multidimensional problems. Its core idea is to alternately apply implicit difference schemes across different dimensions, solving implicitly in only one dimension at a time, while using known values ​​for the other dimensions. This method significantly reduces computational complexity and improves solution efficiency. In feature computation, "dimension" may refer to time, space (such as user feature space), or feature type, etc., and computation is then performed alternately across these dimensions to gradually approximate the final solution. For handling coupling terms, an iterative strategy is employed. In each iteration, the values ​​of other fields are assumed to be known, and the equation for the current field is solved; then, the updated field values ​​are used to solve the equation for the next field, and so on. Therefore, this strategy effectively handles the interdependencies between fields, ensuring the accuracy of the solution. The iterative process continues until a preset convergence condition is met (e.g., residual less than a threshold) or the specified number of iterations is completed. This embodiment efficiently handles complex feature calculation problems through this separate solution method, obtaining separate calculation results for each feature field, reflecting the distribution and variation patterns of various features, and providing a foundation for subsequent multi-resolution calculations and result integration.

[0151] Furthermore, this embodiment introduces a multi-grid method to perform multi-resolution processing on the feature computation problem, improving computational efficiency and accuracy. A multi-level computational grid is a computational structure that uses grids of different resolutions, improving computational efficiency while maintaining accuracy. Specifically, a multi-level computational grid is first constructed, including grids at multiple resolution levels. The coarse grid level uses larger grid cells to cover the entire computational domain, used for fast approximation and capturing large-scale features; the fine grid level uses smaller grid cells, typically covering only key areas (such as high-value user groups, areas of abnormal behavior, etc.), used for high-precision computation and capturing detailed features. The hierarchical structure and distribution strategy of the grid are determined based on the importance of the features, the gradient of change, and computational requirements. Then, a V-loop strategy is used to solve the multi-level grid. It should be noted that the V-loop is an iterative strategy in the multigrid method, and its execution flow is as follows: First, several iterations are performed on the finest grid; then the problem is restricted to a coarse grid and solved on the coarse grid; then, the restriction continues to an even coarser grid until the coarsest grid; after solving on the coarsest grid, the result is interpolated back to the finest grid and smoothed; this process is repeated, forming a V-shaped processing flow. This strategy can effectively handle problems of different scales and quickly eliminate errors of different frequencies. In feature computation, the V-loop strategy can simultaneously handle macroscopic trends and microscopic details, balancing computational efficiency and accuracy. Through multi-level grids and the V-loop strategy, multi-resolution feature computation results are obtained, which can describe the distribution and variation patterns of features at different resolution levels. High-resolution results provide local detail information, suitable for fine analysis and special scene processing; low-resolution results provide global trend information, suitable for macroscopic decision-making and rapid screening. Therefore, this embodiment significantly improves the adaptability and efficiency of feature computation through a multi-resolution computation strategy, and can flexibly adjust computational accuracy and resource allocation according to actual needs.

[0152] Furthermore, dynamic management and optimization of the distributed computing environment ensure efficient utilization of computing resources and rapid task completion. Dynamic load balancing refers to adjusting task allocation and resource configuration based on real-time load conditions during distributed computing to maintain a balanced load across computing nodes, preventing some nodes from being overloaded while others are idle. Specifically, the load status of each computing node during multi-resolution feature computation is first monitored, including metrics such as CPU utilization, memory usage, and task queue length. Then, based on this monitoring data, load balancing is achieved using work-stealing and task migration strategies. Work-stealing is a dynamic load balancing technique where a processor "steals" tasks from other processors' queues after completing its own task, reducing its burden. Therefore, this strategy is particularly suitable for handling uneven workloads, automatically balancing the computing tasks across nodes. Task migration, on the other hand, is a technique that transfers executing or pending tasks from one node to another, typically used when node load is severely unbalanced or when a node fails. Therefore, both strategies are used in combination to dynamically adjust task allocation based on real-time load conditions, ensuring maximum utilization of computing resources. Furthermore, this embodiment also calculates resource allocation data based on load balancing results to guide elastic resource scheduling and node scaling. Elastic resource scheduling refers to dynamically adjusting resource configuration according to computing needs, such as increasing or decreasing node memory, CPU cores, etc.; dynamic node scaling refers to increasing or decreasing the number of computing nodes according to the overall load situation to adapt to changes in computing scale. Based on predefined strategies and rules, combined with real-time load data and resource quotas, resource scheduling and node management operations are executed to ensure that the computing environment can efficiently support feature computing tasks. Through these dynamic optimization measures, heterogeneous feature computing is achieved, that is, efficient feature computing under different hardware conditions and different load states. Therefore, the heterogeneous feature computing results in this embodiment are multi-level, multi-dimensional feature data sets, meeting the feature computing requirements of different business scenarios and application needs.

[0153] It should be noted that the feature calculation results are the final output of the feature engineering process and are directly used by downstream risk control models or business systems. In this embodiment, the calculation results of each node in the heterogeneous feature calculation environment are first collected. These results may be distributed across different calculation nodes, different storage locations, and even different data formats. These scattered result data are obtained through predefined data collection mechanisms, such as result push, periodic retrieval, or event triggering. Then, the collected results are verified and cleaned to ensure data integrity and consistency. The verification process includes checking data format, range, and completeness, identifying and correcting potential errors or anomalies; the cleaning process includes deduplication, filling in missing values, and standardizing formats to make the data conform to predetermined standards. Next, the cleaned data is merged and integrated, combining results from different sources and in different formats into a unified data structure. The integration process needs to handle potential data conflicts and version differences to ensure the consistency and accuracy of the final results. Finally, the integrated data undergoes final processing, including format conversion, index creation, and compressed storage, to generate feature calculation results that meet the requirements of downstream systems. The results of these feature calculations are typically stored in the form of structured data, such as data tables, DataFrames, or files in a specific format, making them easy for downstream systems to access and use. Through this series of processing steps, the results of complex heterogeneous calculations are transformed into unified, standardized feature data, providing a solid data foundation for risk control decisions and business analysis.

[0154] In some embodiments, the method further includes constructing the predefined operator library, specifically including:

[0155] Based on the feature calculation requirements of financial risk control scenarios, basic operators are determined, and specific basic calculation logic and standard interfaces are encapsulated for each basic operator to obtain a basic operator library. The basic operators include statistical operators, time series operators, correlation operators and user behavior basic operators.

[0156] Based on the aforementioned basic operator library, a combined operator library is constructed, which includes arithmetic combined operators, logical combined operators, conditional combined operators, and composite combined operators. The combined operators in the combined operator library are used to indicate the combined calculations between target features.

[0157] Based on the basic operator library and the combined operator library, an operator registry is established. The operator registry is used to indicate the metadata information of the basic operators and the combined operators. The metadata information includes operator name, type, input parameter definition, output format and dependency relationship.

[0158] Based on the operator registry, dynamic operator loading information is determined, and a dynamic operator invocation framework is obtained according to the dynamic operator loading information, wherein the dynamic operator loading information is used to indicate the dynamic instantiation and invocation of the basic operator and the combined operator;

[0159] The basic operator library, the combined operator library, the operator registry, the operator dynamic loading information, and the dynamic operator calling framework are integrated, and a unified operator access interface and calling specification are established to obtain a predefined operator library.

[0160] In this embodiment, a basic operator library is designed and implemented based on the actual needs of the financial risk control field, providing basic computational units for feature calculation. The basic operator is the smallest functional unit for feature calculation, encapsulating specific computational logic and interacting with the system through a standard interface. Specifically, a requirements analysis is first conducted, comprehensively investigating common feature calculation needs in financial risk control scenarios. These requirements originate from multiple channels, including the experience of business experts, analysis of existing risk control models, and research on best practices in the market. Then, based on the results of the requirements analysis, the categories and functions of the basic operators are determined. It should be noted that statistical operators implement various statistical calculation functions, such as summation, average, standard deviation, quantiles, and frequency statistics, suitable for processing statistical features of numerical data; time-series operators handle time-series related calculations, such as sliding window statistics, time decay calculation, and periodicity analysis, suitable for capturing the characteristics of data changing over time; correlation operators handle the calculation of relationships between entities, such as network centrality, path analysis, and community detection, suitable for mining features in user relationship networks; and user behavior basic operators focus on user behavior analysis, such as behavior sequence pattern recognition, abnormal behavior detection, and user profile features, suitable for characterizing user behavior features. For each type of operator, specific functions are further subdivided, such as the mean operator, median operator, and mode operator in statistical operators. Then, computational logic and standard interfaces are designed for each basic operator. The computational logic is the core functional implementation of the operator, including specific data processing and calculation methods; the standard interface defines the interaction method between the operator and external systems, including input parameter definitions, output formats, and error handling. In this embodiment, a modular and configurable design principle is adopted, enabling operators to be used independently or combined into more complex computational flows. Finally, these basic operators are implemented and tested to ensure their correct functionality and excellent performance, and they are organized into a structured basic operator library. This basic operator library serves as the infrastructure for subsequent feature computation, providing rich and standard computational components, significantly simplifying the difficulty of feature development and implementation.

[0161] Furthermore, based on the basic operator library, more advanced combinatorial operators are designed and implemented to enhance the expressiveness and flexibility of feature computation. Combinatorial operators are advanced computational units built by combining basic operators or other combinatorial operators, enabling more complex feature computation logic. Specifically, common combination patterns and complex logics in feature computation are first analyzed to determine the types of combinations that need to be supported. For example, arithmetic combinatorial operators implement various mathematical operations, such as addition, subtraction, multiplication, division, exponentiation, and logarithmic operations, for mathematical transformations and combinations of features; logical combinatorial operators implement logical operations, such as AND, OR, NOT, and XOR, for combining multiple conditions or judgment results; conditional combinatorial operators implement conditional judgments and branching processes, such as if-then-else structures and switch-case structures, for selecting different computational paths based on conditions; and composite combinatorial operators integrate multiple basic and combinatorial operators to form complex computational flows for implementing advanced feature computation in specific domains. Then, implementation schemes are designed for each type of combinatorial operator. Optionally, there are two main implementation methods: one is to encode the combinational logic into new operators to directly implement the combinational function; the other is to build an operator combination framework and dynamically construct combinational operators by configuring combinational relationships. The appropriate implementation method is chosen based on performance requirements and flexibility needs. For simple combinations such as arithmetic and logical combinations, the corresponding combinational operators are usually implemented directly; for complex combinations such as conditional and composite combinations, a dynamic combination framework is preferred, supporting flexible configuration and customization. Next, the standard interface and internal logic of the combinational operators are implemented. The standard interface ensures that the combinational operators have a consistent usage method with the basic operators, facilitating integration and invocation; the internal logic implements the specific functions of the combinational operators, processing input data, invoking relevant operators, and generating calculation results. Finally, the implemented combinational operators are organized into a combinational operator library, forming a complete operator system together with the basic operator library. Therefore, this embodiment provides multi-level computing capabilities from simple to complex by configuring the operator system, meeting the needs of various feature calculations in financial risk control and greatly improving the efficiency and flexibility of feature development.

[0162] Next, a central registration mechanism is created to manage the metadata information of all operators, providing support for operator lookup, selection, and invocation. The operator registry is a system component that centrally stores and manages operator metadata, similar to an index in a service registry or component library. Specifically, the data structure and storage format of the operator registry are designed first. The registry typically uses structured data formats, such as JSON, XML, or relational database tables, to store various metadata information for operators. Each operator has a unique identifier in the registry, associated with multiple attribute fields. Then, the metadata information of all operators is collected and organized. It should be noted that metadata is data that describes data, specifically referring to information describing operator characteristics and usage methods. The operator name is the unique identifier of the operator, used for referencing and finding operators; the operator type identifies the functional category of the operator, such as statistical, time-series, etc.; input parameters define the input data format, type, and constraints required by the operator; the output format specifies the data structure and content of the results produced by the operator; and dependencies record the operator's dependencies on other operators or components. This metadata information is obtained by analyzing operator code, reading configuration files, or receiving explicit registration requests. Next, the registry management functions are implemented, including operator registration, querying, updating, and deletion. Registration adds the metadata of a new operator to the registry; querying retrieves operator information based on conditions (such as name, type, etc.); updating modifies the metadata of an existing operator; and deletion removes unused operators from the registry. These operations are typically provided to other components and users through an API or management interface. Finally, the registry is integrated into the feature computation framework, making it the core component for operator management and invocation. When a specific operator is needed, the registry is first queried to obtain the operator's metadata information, and then the operator is correctly initialized and invoked based on this information. This centralized management approach improves maintainability and scalability, making adding new operators and updating existing ones simple and secure.

[0163] Subsequently, this embodiment designs and implements a dynamic operator invocation framework to support dynamic loading and flexible invocation of operators, enhancing the system's adaptability and scalability. The dynamic operator invocation framework is a mechanism that enables dynamic loading and invocation of operators at runtime, similar to a plugin system or dynamic library loader. Specifically, firstly, based on the operator registry, the dynamic loading information of the operator is determined. This dynamic loading information includes the operator's physical location (e.g., file path, package name), loading method (e.g., reflection loading, dynamic linking), initialization parameters, etc., which guide the system to correctly load and initialize operators at runtime. This embodiment can extract this information from the registry or obtain supplementary information through configuration files, environment variables, etc. Next, the architecture and interface of the dynamic operator invocation framework are designed. This framework needs to support the following core functions: dynamically loading operator code or libraries, initializing operator instances according to configuration parameters, providing a unified invocation interface, managing the operator lifecycle, and handling loading and invocation exceptions. The framework typically adopts design patterns such as the factory pattern, strategy pattern, or command pattern to provide a flexible and consistent user experience. Finally, the specific mechanism for dynamic loading and invocation is implemented. Different loading strategies may be required for different types of operators. For languages ​​that support reflection, such as Java or .NET, classes are dynamically loaded and instances are created using reflection. For compiled languages ​​like C / C++, operators may need to be loaded via dynamic link libraries (DLLs, SOs). For scripting languages ​​like Python, operators can be loaded by dynamically importing modules. Regardless of the method used, the security and robustness of the loading process must be ensured, handling potential loading failures, version incompatibility, and other exceptions. Finally, the dynamic operator invocation framework is integrated into the feature computation system, serving as a bridge between operator definitions and actual computation. When feature computation is required, the necessary operators are dynamically loaded through this framework, the corresponding parameters are passed in, the computation logic is executed, and the results are obtained. This dynamic invocation mechanism can adapt to constantly changing computational needs, supports plug-and-play for new operators, and reduces the complexity of updates and maintenance.

[0164] Furthermore, the aforementioned components are integrated to establish a complete predefined operator library, providing comprehensive support for feature computation. The predefined operator library is the core component of the on-demand feature computation system, providing algorithms and functional modules required for various feature computations. Specifically, firstly, all operators in the basic operator library and combined operator library are incorporated into unified management. These operators are the core resources of the operator library, providing various computational capabilities from basic statistics to complex analysis. This embodiment ensures the code quality, performance, and reliability of these operators through measures such as code review, unit testing, and performance testing. Then, an operator registry and dynamic loading mechanism are integrated to establish metadata management and dynamic invocation capabilities for operators. The registry provides metadata information for operators, while the dynamic loading mechanism supports flexible operator invocation; the combination of the two forms the management and access layer of the operator library. This layer handles operator registration, discovery, and version management functions, ensuring that the required operators can be correctly found and used. Next, a unified operator access interface and invocation specification are established. The access interface serves as the unified entry point for other components to interact with the operator library. It is typically provided in the form of an API, supporting operations such as operator querying, parameter validation, and call execution. The calling specification defines the standard procedures and conventions for operator usage, including parameter passing methods, error handling mechanisms, and result return formats. These interfaces and specifications simplify and unify the use of the operator library, reducing the complexity of integration and invocation. Finally, a management and maintenance mechanism for the operator library is implemented, supporting functions such as operator updates, version control, and dependency management. Furthermore, this embodiment may provide management tools or interfaces, allowing administrators to monitor operator usage, add new operators, update existing operators, and handle version conflicts. Therefore, these management functions ensure that the operator library can continuously evolve and improve, adapting to ever-changing business needs. Through these integration and consolidation efforts, a complete predefined operator library is obtained. This library not only contains rich operator resources but also provides a robust management and invocation mechanism, serving as a core component and technological foundation of the feature-on-demand computing system.

[0165] like Figure 2 As shown, this embodiment provides a financial risk control feature on-demand calculation device 200, including:

[0166] The configuration parsing module 201 is used to obtain the feature configuration file and the original financial data, parse the feature configuration file to obtain the set of feature variables to be calculated, the set of feature variables includes a list of target features with at least one target feature and feature dependency mapping relationship;

[0167] The dependency graph construction module 202 is used to match and identify the feature types of each target feature based on the set of feature variables, and recursively process the dependent target features according to the feature types to construct a feature dependency graph, wherein the feature dependency graph is used to indicate the dependency relationship between the target features;

[0168] The execution plan generation module 203 is used to perform topological sorting on the feature dependency graph to obtain a feature calculation sequence, and combine the mapping relationship of operators in the predefined operator library with the feature calculation sequence to generate a minimized operator execution plan. The feature calculation sequence is used to indicate the order of calculation of the target features to be calculated in the feature dependency graph, and the operator execution plan is used to indicate the calling sequence of operators corresponding to the target features to be calculated arranged in the dependency order.

[0169] The distributed computing module 204 is used to call the corresponding operators according to the operator execution plan through the preset distributed computing framework, and use the called operators to perform feature calculation on the target feature to be calculated and the original financial data to obtain the feature calculation result;

[0170] The result output module 205 is used to filter and format the feature calculation results and output the feature calculation output results of the target features specified in the feature configuration file.

[0171] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for financial risk control feature on-demand computing, characterized in that, Comprise: Obtaining a feature configuration file and original financial data, parsing the feature configuration file to obtain a set of feature variables to be calculated, the set of feature variables including a target feature list having at least one target feature and a feature dependency mapping relationship; Based on the set of feature variables, matching and identifying the feature types of each target feature, and recursively processing the target features depending on the feature types to construct a feature dependency graph, wherein the feature dependency graph is used to indicate the dependency relationship between the target features; The feature dependency graph is topologically sorted to obtain a feature calculation sequence, and a minimum operator execution plan is generated by combining the mapping relationship of the operators in the predefined operator library and the feature calculation sequence, including: topologically sorting the nodes in the feature dependency graph to determine the feature calculation order without dependency conflicts, and obtaining the feature calculation sequence; according to the feature type of each target feature in the feature calculation sequence, matching the corresponding operator in the predefined operator library as the target operator for feature calculation, to generate operator call information, the operator call information is used to indicate the target operator and the corresponding operator call relationship; the operator call information is arranged according to the preset feature calculation priority, the calling request of the same target operator is merged and the redundant calculation steps are eliminated to obtain the operator execution sequence; according to the calculation complexity and data dependency relationship of the target operator, the operator execution sequence is assigned with execution priority and resource configuration to obtain the operator execution plan; the operator execution plan is converted into an operator execution graph, wherein the vertices in the operator execution graph represent the target operators indicated in the operator execution plan, and the edges in the operator execution graph represent the data dependency and call relationship between the target operators; based on the operator execution graph, the operator properties of the target operators are extracted using a preset graph attention network, the internal relationship between the target operators is determined according to the operator properties, the low-dimensional embedding representation of the target operators is obtained from the internal relationship, and the operator properties include calculation complexity features, data dependency features and resource consumption features; according to the low-dimensional embedding representation of the target operators, a similarity matrix between the target operators is calculated, and an operator knowledge graph is constructed based on the operator calculation mode and clustering relationship identified based on the similarity matrix, wherein the operator knowledge graph is used to indicate the semantic association information between the target operators; based on the operator knowledge graph, a multi-objective optimization model is constructed, and a constraint optimization model of operator scheduling is obtained based on the multi-objective optimization model, wherein the multi-objective optimization model includes optimization objectives and constraint conditions, the optimization objectives are used to indicate the minimization of total calculation time and resource consumption, and the constraint conditions include dependency constraints, resource constraints and time constraints; a preset particle swarm-simulated annealing hybrid algorithm is used to solve the constraint optimization model to obtain an optimized operator execution plan, wherein the particle swarm-simulated annealing hybrid algorithm is used to indicate that the optimal operator scheduling scheme is searched by adaptively adjusting the particle position and velocity update parameters and the simulated annealing acceptance probability mechanism; wherein the feature calculation sequence is used to indicate the calculation sequence of the target features to be calculated in the feature dependency graph, and the operator execution plan is used to indicate the calling sequence of the operators corresponding to the target features to be calculated arranged in the dependency order; The corresponding operators are called through a preset distributed computing framework according to the operator execution plan, and the called operators are used to perform feature calculation on the target features to be calculated and the original financial data to obtain feature calculation results. Filter and format the feature calculation result, and output the feature calculation output result of the target feature specified in the feature configuration file.

2. The method of claim 1, wherein, The feature configuration file is parsed to obtain a set of feature variables to be calculated, including: Based on the feature configuration file, read the target feature field through a preset JSON format parser, and extract the corresponding target feature name to obtain a target feature list; Use regular expressions to identify the format pattern of each target feature in the target feature list, and match the feature-dependent configuration items based on the format pattern of the target feature to obtain the type identifier of each target feature; Based on the type identifier, find the corresponding dependency configuration in the feature configuration file to obtain the feature dependency mapping relationship of each target feature; Combine the target feature list and the feature dependency mapping relationship to obtain the set of feature variables to be calculated.

3. The method of claim 2, wherein, Based on the set of feature variables, match and identify the feature type of each target feature, and recursively process the dependent target features according to the feature type to construct a feature dependency graph, including: Use a preset regular expression to perform pattern matching on each target feature in the feature variable set to obtain the feature type of the target feature; Based on the feature type of the target feature, find the base field of the target feature in the dependency configuration, and identify the required base feature according to the base field, and form a direct dependency relationship list from the base feature; For each base feature in the direct dependency relationship list, recursively perform dependency relationship search to construct a feature dependency tree with multiple dependency chains; Convert the feature dependency tree into a feature dependency graph represented as a directed acyclic graph structure, wherein the nodes of the feature dependency graph represent the target features, and the edges of the feature dependency graph represent the dependency relationship between the target features.

4. The method of claim 3, wherein, Based on the operator execution plan, analyze the required original data field and data range of each target operator to generate a data loading requirement list, and obtain a minimized data set definition based on the data loading requirement list; Read the corresponding data partition from the original financial data according to the minimized data set definition through a preset distributed computing framework, cache the data partition, and obtain the data set required for operator execution; Based on the data set, sequentially instantiate and call the target operators in the predefined operator library according to the order indicated by the operator execution plan, and execute the feature calculation logic according to the operator execution plan and the called target operators to obtain the intermediate calculation result of the target operator; Manage and transfer the intermediate calculation results of each target operator, and aggregate the intermediate calculation results of all target operators to obtain the feature calculation result of the target feature. ​ 5. The method of claim 4, wherein, The step of matching and identifying the feature types of each target feature based on the set of feature variables, and recursively processing the dependent target features according to the feature types to construct a feature dependency graph, further includes: Acquire historical feature calculation data, and extract the calculation time, resource consumption, and business impact data of each target feature from the historical feature calculation data to generate a feature performance history record; Based on the historical performance records of the features, the heteroscedasticity characteristics of different target features are calculated using a preset White test algorithm. Based on the heteroscedasticity characteristics, a variance function model of the target features is constructed. The variance function model is used to indicate the heteroscedasticity distribution characteristics of the target features. Based on the heteroscedasticity distribution characteristics indicated by the variance function model, the business value score, computational cost value, and stability coefficient of the target feature are determined. Based on the business value score, computational cost value, and stability coefficient of the target feature, the weight of the target feature is calculated. Based on the weight of the target feature and the feature dependency tree, a weighted feature map is constructed. The weights in the weighted feature map are optimized using a preset generalized least squares algorithm, and the priority order of feature calculation in the weighted feature map is adjusted to obtain the optimized weighted feature map. Anomalies in the optimized weighted feature map are identified using a pre-defined Breusch-Pagan test algorithm to generate a first test result. The autocorrelation of the dependencies between the target features is evaluated using a pre-defined Durbin-Watson test algorithm to generate a second test result. The structure of the optimized weighted feature map is adjusted based on the first and second test results to obtain a feature dependency map.

6. The method of claim 4, wherein, The process of using the invoked operator to perform feature calculations on the target feature to be calculated and the original financial data to obtain feature calculation results, and performing feature calculations on demand, also includes heterogeneous feature calculations based on multi-physics coupling, including: The temporal features in the target features are mapped to a temporal feature field, the statistical features in the target features are mapped to a statistical feature field, the correlation features in the target features are mapped to a correlation feature field, and the temporal feature field, the statistical feature field, and the correlation feature field are combined to form a feature physical field; Based on the aforementioned characteristic physical fields, a set of coupled equations is constructed, which includes time-series characteristic field equations, statistical characteristic field equations, and correlation characteristic field equations. Based on the coupling terms and source terms between the characteristic fields determined by the coupling equations, a multi-characteristic field coupling mathematical model is obtained. The multi-feature field coupled mathematical model is solved by using a preset operator splitting algorithm. In the separation solution, a preset alternating direction implicit algorithm is used to calculate alternately in different feature dimensions and iteratively process the coupling terms to obtain the separation calculation results of each feature field. A multi-level computing grid is constructed, a preset V-cycle strategy is used to solve the multi-level computing grid, and a multi-resolution feature calculation result is obtained, wherein the multi-level computing grid includes a coarse grid level and a fine grid level, the coarse grid level is used for fast approximate calculation, and the fine grid level is used for high-precision regional calculation; Dynamic load balancing is performed on the multi-resolution feature calculation result, so that resource allocation data is calculated by using a preset work stealing strategy and a task migration strategy, elastic resource scheduling and dynamic expansion and contraction of computing nodes are performed based on the resource allocation data, and a heterogeneous feature calculation result is obtained; Based on the heterogeneous feature calculation result, a feature calculation result is obtained.

7. The method of claim 6, wherein, The method further includes constructing the predefined operator library, specifically including: Based on the feature calculation demand of the financial risk control scene, determining the basic operator, encapsulating the specific basic calculation logic and standard interface for each basic operator, obtaining the basic operator library, and the basic operator includes statistical operator, time series operator, correlation operator and user behavior basic operator; Based on the basic operator library, a combination operator library is constructed, the combination operator library includes arithmetic combination operator, logical combination operator, conditional combination operator and composite combination operator, and the combination operator in the combination operator library is used to indicate the combination calculation between target features; Based on the basic operator library and the combination operator library, an operator registration table is established, the operator registration table is used to indicate the metadata information of the basic operator and the combination operator, and the metadata information includes operator name, type, input parameter definition, output format and dependency relationship; Based on the operator registration table, determine the operator dynamic loading information, and obtain the dynamic operator calling framework according to the operator dynamic loading information, wherein the operator dynamic loading information is used to indicate the dynamic instantiation and calling of the basic operator and the combination operator; Integrate the basic operator library, the combination operator library, the operator registration table, the operator dynamic loading information and the dynamic operator calling framework, and establish a unified operator access interface and calling specification to obtain a predefined operator library.

8. A financial risk control feature on-demand computing apparatus characterized by, Including: The configuration analysis module is configured to obtain a feature configuration file and original financial data, analyze the feature configuration file to obtain a set of feature variables to be calculated, and the set of feature variables includes a target feature list having at least one target feature and a feature dependency mapping relationship; The dependency graph construction module is configured to match and identify the feature types of each target feature based on the set of feature variables, and recursively process the dependent target features according to the feature types to construct a feature dependency graph, wherein the feature dependency graph is used to indicate the dependency relationship between the target features; The execution plan generation module is configured to topologically sort the feature dependency graph to obtain a feature calculation sequence, and generate a minimized operator execution plan by combining a mapping relationship of operators in a predefined operator library and the feature calculation sequence. The execution plan generation module includes: topologically sorting nodes in the feature dependency graph to determine a feature calculation order without dependency conflicts, and obtaining a feature calculation sequence; matching corresponding operators in the predefined operator library as target operators for feature calculation according to a feature type of each target feature in the feature calculation sequence, to generate operator call information, the operator call information being used to indicate the target operators and corresponding operator call relationships; arranging the operator call information according to a preset feature calculation priority, merging call requests of the same target operators, and eliminating redundant calculation steps, to obtain an operator execution sequence; assigning an execution priority and resource configuration to the operator execution sequence according to a calculation complexity of the target operators and a data dependency relationship, to obtain the operator execution plan; converting the operator execution plan into an operator execution graph, wherein vertices in the operator execution graph represent the target operators indicated in the operator execution plan, and edges in the operator execution graph represent data dependencies and call relationships between the target operators; extracting operator attributes of the target operators by using a preset graph attention network based on the operator execution graph, determining an intrinsic relationship between the target operators according to the operator attributes, obtaining a low-dimensional embedding representation of the target operators from the intrinsic relationship, and the operator attributes including a calculation complexity feature, a data dependency feature, and a resource consumption feature; calculating a similarity matrix between the target operators according to the low-dimensional embedding representation of the target operators, identifying operator calculation modes and clustering relationships based on the similarity matrix, and constructing an operator knowledge graph, wherein the operator knowledge graph is used to indicate semantic association information between the target operators; constructing a multi-objective optimization model based on the operator knowledge graph, and obtaining a constraint optimization model of operator scheduling based on the multi-objective optimization model, wherein the multi-objective optimization model includes an optimization objective and a constraint condition, the optimization objective is used to indicate minimized total calculation time and resource consumption, and the constraint condition includes a dependency constraint, a resource constraint, and a time constraint; solving the constraint optimization model by using a preset particle swarm-simulated annealing hybrid algorithm to obtain an optimized operator execution plan, wherein the particle swarm-simulated annealing hybrid algorithm is used to indicate that an optimal operator scheduling scheme is searched by adaptively adjusting particle position and velocity update parameters and a simulated annealing acceptance probability mechanism; the feature calculation sequence is used to indicate a calculation order of target features to be calculated in the feature dependency graph, and the operator execution plan is used to indicate a call sequence of operators corresponding to the target features to be calculated arranged in a dependency order. The distributed computing module is configured to invoke corresponding operators according to the operator execution plan through a preset distributed computing framework, perform feature calculation on the target feature to be calculated and the original financial data by using the invoked operators, and obtain a feature calculation result. The result output module is configured to filter and format the feature calculation result, and output a feature calculation output result of the target feature specified in the feature configuration file.

Images