Feature backtracking method, apparatus, device, and storage medium

Abstract

This application discloses a feature backtracking method, apparatus, device, and storage medium, relating to the field of data query technology. The method includes: receiving a historical feature query request sent by a client; performing backtracking matching on a feature-level storage system based on a target time point in the historical feature query request using a hierarchical query strategy to obtain first feature data; performing a broadcast query based on the historical feature query request to obtain second feature data corresponding to the historical feature query request; and integrating the first and second feature data to obtain target feature data corresponding to the historical feature query request. This application improves the efficiency of historical feature backtracking queries by combining a hierarchical query strategy with a broadcast query to obtain target feature data through backtracking matching on a feature-level storage system.

Description

Technical Field

[0001] This application relates to the field of data query technology, and in particular to feature backtracking methods, apparatus, devices and storage media. Background Technology

[0002] With the development of big data technology, building feature systems based on user behavior and business data has become a crucial foundation for data analysis, risk control, and intelligent decision-making. In practical applications, it is often necessary to reconstruct customer characteristics based on a specific historical point in time. Existing technologies commonly employ sample-driven backtracking mechanisms for acquiring historical features. Specifically, when features at a specific historical point in time are needed, the system takes sample data as input, backtracks to the original data within the corresponding time window based on the time field in the sample, and re-executes the feature calculation logic to obtain the target feature value. However, these existing technologies suffer from significant drawbacks in practical applications, including high computational overhead and low efficiency. Each historical feature query requires re-executing the entire feature calculation logic from the original data. With large datasets and complex feature logic, this leads to lengthy computation times, making it difficult to meet the demands for efficient queries.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a feature backtracking method, apparatus, device and storage medium, which aims to solve the technical problem of low efficiency in backtracking and querying historical features.

[0005] To achieve the above objectives, this application proposes a feature backtracking method, the method comprising: Receive a historical feature query request sent by a customer, and based on the target time point in the historical feature query request, perform backtracking matching on the feature hierarchical storage system through a hierarchical query strategy to obtain the first feature data; A broadcast query is performed based on the historical feature query request to obtain the second feature data corresponding to the historical feature query request; Based on the first feature data and the second feature data, feature integration is performed to obtain the target feature data corresponding to the historical feature query request.

[0006] In one embodiment, the step of receiving a historical feature query request sent by a customer further includes: Obtain raw business data; Based on the preset feature calculation logic, the original business data is pre-calculated to generate daily incremental feature data and feature snapshot data of the original business data; Based on the daily incremental feature data and the feature snapshot data, a feature-based hierarchical storage system for the original business data is constructed.

[0007] In one embodiment, the step of pre-calculating the original business data according to a preset feature calculation logic to generate daily incremental feature data and feature snapshot data of the original business data includes: Based on the preset feature calculation logic, incremental calculation is performed on the original business data to obtain the daily incremental feature data; At the end of the preset target period, the daily incremental feature data is summarized and solidified to generate the feature snapshot data.

[0008] In one embodiment, the step of performing incremental calculations on the original business data based on the preset feature calculation logic to obtain the daily incremental feature data includes: A hierarchical computing system is constructed based on data lineage, and the hierarchical computing system includes basic features, intermediate features, and advanced features; Based on the hierarchical calculation system, incremental calculations are performed on the original business data level by level to obtain incremental feature data for the corresponding date.

[0009] In one embodiment, the feature hierarchical storage system includes a daily incremental feature storage area and a feature snapshot storage area. The step of obtaining the first feature data by backtracking and matching the feature hierarchical storage system based on the target time point in the historical feature query request using a hierarchical query strategy includes: Based on the target time point in the historical feature query request, determine the target period to which the target time point belongs; Based on the target time point, query the latest feature data of the target customer identifier within the target period in the daily incremental feature storage area; If the latest feature data is found within the target period, then the latest feature data is used as the first feature data; If the latest feature data is not found within the target period, the feature snapshot data of the target customer identifier in the previous period of the target period is retrieved from the feature snapshot storage area to obtain the first feature data.

[0010] In one embodiment, the step of performing a broadcast query based on the historical feature query request to obtain the second feature data corresponding to the historical feature query request includes: Parse the query sample data from the historical feature query request, and distribute the query sample data as a broadcast variable to each computing node in the distributed computing environment; On each of the computing nodes, the broadcast variables are associated and matched with the feature hierarchical storage system to generate local feature results for each computing node; The local feature results of each computing node are merged to obtain the second feature data.

[0011] In one embodiment, the step of integrating features based on the first feature data and the second feature data to obtain the target feature data corresponding to the historical feature query request includes: The first feature data and the second feature data are aligned and matched according to the feature identifier, and duplicate feature data are removed to obtain intermediate feature data. The intermediate feature data is confirmed and corrected according to the preset feature priority rules; The corrected intermediate feature data is standardized and encapsulated to obtain the target feature data corresponding to the historical feature query request.

[0012] Furthermore, to achieve the above objectives, this application also proposes a feature backtracking device, which includes: The matching backtracking module is used to receive historical feature query requests sent by customers, and based on the target time point in the historical feature query request, backtracking and matching the feature hierarchical storage system through a hierarchical query strategy to obtain the first feature data. The broadcast query module is used to perform a broadcast query based on the historical feature query request to obtain the second feature data corresponding to the historical feature query request. The feature integration module is used to integrate features based on the first feature data and the second feature data to obtain the target feature data corresponding to the historical feature query request.

[0013] In addition, to achieve the above objectives, this application also proposes a feature backtracking device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the feature backtracking method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the feature backtracking method described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the feature backtracking method described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: This application proposes a feature backtracking method, apparatus, device, and storage medium. It receives a historical feature query request sent by a client. Based on the target time point in the historical feature query request, it performs backtracking matching on a hierarchical feature storage system using a hierarchical query strategy to obtain first feature data. It then performs a broadcast query based on the historical feature query request to obtain second feature data corresponding to the historical feature query request. Finally, it integrates the first and second feature data to obtain the target feature data corresponding to the historical feature query request. This application improves the efficiency of historical feature backtracking queries by combining backtracking matching on a hierarchical feature storage system using a hierarchical query strategy with broadcast queries to obtain target feature data. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an embodiment of the feature backtracking method of this application. Figure 2 A simplified flowchart illustrating the feature backtracking method provided in this application embodiment; Figure 3 This is a schematic diagram of the module structure of the feature backtracking device according to an embodiment of this application; Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the feature backtracking method in this application embodiment.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of this application embodiment is: receiving a historical feature query request sent by a client; based on the target time point in the historical feature query request, performing backtracking matching on the feature hierarchical storage system through a hierarchical query strategy to obtain first feature data; performing a broadcast query based on the historical feature query request to obtain second feature data corresponding to the historical feature query request; and performing feature integration based on the first feature data and the second feature data to obtain target feature data corresponding to the historical feature query request.

[0024] In this embodiment, for ease of description, the feature backtracking system will be used as the execution subject for the following description.

[0025] With the development of big data technology, building feature systems based on user behavior and business data has become a crucial foundation for data analysis, risk control, and intelligent decision-making. In practical applications, it is often necessary to reconstruct customer characteristics based on a specific historical point in time. Existing technologies commonly employ sample-driven backtracking mechanisms for acquiring historical features. Specifically, when features at a specific historical point in time are needed, the system takes sample data as input, backtracks to the original data within the corresponding time window based on the time field in the sample, and re-executes the feature calculation logic to obtain the target feature value. However, these existing technologies suffer from significant drawbacks in practical applications, including high computational overhead and low efficiency. Each historical feature query requires re-executing the entire feature calculation logic from the original data. With large datasets and complex feature logic, this leads to lengthy computation times, making it difficult to meet the demands for efficient queries.

[0026] This application provides a solution that improves the efficiency of historical feature backtracking queries by combining a hierarchical query strategy with a feature hierarchical storage system for backtracking matching and broadcast queries.

[0027] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or feature backtracking system capable of performing the above functions. The following description uses a feature backtracking system as an example to illustrate this embodiment and the subsequent embodiments.

[0028] Based on this, embodiments of this application provide a feature backtracking method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the feature backtracking method of this application.

[0029] In this embodiment, the feature backtracking method includes steps S11 to S13: Step S11: Receive a historical feature query request sent by the customer. Based on the target time point in the historical feature query request, perform backtracking matching on the feature hierarchical storage system through a hierarchical query strategy to obtain the first feature data.

[0030] It should be noted that a historical feature query request refers to a data request initiated by a client or business system to obtain the feature status of a specific customer at a certain historical moment. This request must include at least two necessary parameters: the target customer identifier and the target time point. The hierarchical query strategy is a retrieval method based on a storage hierarchy, matching data level by level from fine to coarse granularity over time. The feature-based hierarchical storage system is a pre-constructed storage structure that organizes feature data hierarchically by time and type. Daily incremental data stores fine-grained changes, while snapshot data stores the full state periodically.

[0031] Understandably, the core objective of this step is to quickly locate and retrieve the feature data closest to the target time point from the stored pre-calculated results without triggering recalculation. This utilizes the idea of ​​trading storage for computation: the offline pre-calculated results are stored hierarchically by time dimension. During queries, fine-grained daily incremental data is prioritized for matching based on the target time point; only when the fine-grained data is unavailable does it revert to coarse-grained snapshot data. This avoids recalculating from the original data for every query, significantly reducing query latency.

[0032] Specifically, upon receiving a query request, the system first parses out the target customer identifier (such as user ID or device fingerprint) and the target time point (such as May 15, 2024). Based on the target time point, the system locates the corresponding storage partition and prioritizes searching the daily incremental storage area within that partition to find if the customer has any characteristic storage records on the target time point or the most recent date. If the query finds a match, the daily incremental data is used as the first characteristic data; if not, the system automatically reverts to the characteristic snapshot storage area of ​​the period to which the time point belongs (such as the current month) and obtains the full characteristic snapshot of the customer at the beginning of that period as the first characteristic data.

[0033] For example, in a risk control review scenario, the reviewer needs to query the credit score characteristics of customer Zhang San when he applied for a loan on May 15, 2024. Upon receiving this request, the system first searches the daily incremental characteristic table for Zhang San's most recent characteristic change record on or before May 15. If there is a characteristic update record on May 10, the system returns that daily incremental data as the first characteristic data. If Zhang San's characteristics remain unchanged throughout May, the system reverts to the month-end characteristic snapshot table of April 30 to obtain Zhang San's full characteristic data at the end of April as the first characteristic data.

[0034] Step S12: Perform a broadcast query based on the historical feature query request to obtain the second feature data corresponding to the historical feature query request.

[0035] It should be noted that broadcast query is an optimization strategy in distributed computing. It replicates and distributes a small dataset (in this case, the sample data to be queried) to all computing nodes in a distributed environment, enabling each node to independently perform the association calculation between its local data and the broadcast data, avoiding the transmission of large-scale data across nodes. The second feature data refers to the feature result set obtained through broadcast query that is directly associated with the sample data in the query request.

[0036] Understandably, the purpose of this step is to efficiently complete the association matching between sample data and the feature storage system. By using a broadcast mechanism, a small sample dataset (usually a few hundred or a few thousand records) is sent to each computing node that stores the feature data, allowing each node to complete the association operation locally. This avoids the network overhead and computational bottlenecks caused by transmitting large-scale feature data to a central node over the network, as is required in traditional distributed association queries.

[0037] Specifically, the process begins by extracting the sample set to be queried from historical feature query requests. This set may contain identification information for multiple customers and their corresponding query timestamps. The size of the sample set is then checked against a preset broadcast threshold (e.g., 100MB). If the threshold is met, the sample set is encapsulated as a broadcast variable. This broadcast variable is then distributed to all computing nodes in the cluster via the broadcast mechanism of a distributed computing framework (e.g., Spark, Flink). Upon receiving the broadcast variable, each computing node retrieves matching feature data from its local feature hierarchy storage system and generates its local feature result.

[0038] For example, a risk control system needs to perform batch feature backtracking on 1000 historical case samples, each sample containing a customer ID and the case occurrence time. The system distributes these 1000 sample data as broadcast variables to 10 computing nodes. Each node stores a portion of the customer's feature data. Nodes locally associate the samples in the broadcast variables with the customer ID and directly calculate the customer features they are responsible for, eliminating the need to aggregate feature data to a single node for centralized association, significantly reducing network transmission volume and association calculation time.

[0039] Step S13: Based on the first feature data and the second feature data, feature integration is performed to obtain the target feature data corresponding to the historical feature query request.

[0040] It should be noted that feature integration refers to the process of merging, deduplicating, validating, and formatting feature data obtained from different data sources or different query paths. The target feature data is the complete feature result set that is finally output to the client and meets the requirements of the query request. It contains all the feature fields required by the query and their corresponding feature values.

[0041] Understandably, since the first feature data comes from a hierarchical query strategy (biased towards acquiring existing benchmark features), and the second feature data comes from a broadcast query association (biased towards acquiring supplementary features strongly correlated with the sample), there may be overlap or conflict between the two. The feature integration step ensures the uniqueness and accuracy of the final result through deduplication and priority rules, while ensuring the consistency and usability of the output format through standardized encapsulation.

[0042] Specifically, the first and second feature data are aligned according to feature name or feature ID. For identical feature fields contained in both, the system prioritizes them according to a preset priority rule: the preset rule can be configured to either have the second feature data overwrite the first feature data, or have the first feature data take precedence and the second feature data only supplements missing fields. A consistency check is performed on the merged feature data to check for logical contradictions (such as different feature values ​​from two different sources that are not marked as valid). After the check passes, the system serializes and encapsulates the feature data according to the output format specified in the request (such as JSON, Protobuf, or CSV).

[0043] For example, the first feature data includes customer Zhang San's login count in the past 30 days = 15 and credit score = 720; the second feature data is obtained through a broadcast query, showing Zhang San's login count in the past 30 days = 15 and overdue mark = 0. During system integration, duplicate login counts in the past 30 days are identified, and the two values ​​from the same source are retained according to the rule of prioritizing the second feature data. Simultaneously, the credit score from the first feature data and the overdue mark from the second feature data are merged to form a complete feature set. Finally, the integration result is encapsulated in JSON format and returned to the review system.

[0044] This embodiment, through the above-described scheme, breaks down the historical feature backtracking process into three independent yet coordinated steps: hierarchical matching to obtain baseline data, broadcast query to obtain associated data, and feature integration. This solves the inefficiency problem caused by the need to recalculate from the original data for each query in existing technologies. The hierarchical query strategy utilizes pre-stored hierarchical data for rapid location, while the broadcast query mechanism leverages the advantages of distributed computing for efficient association. The combination of these two approaches enables the system to accurately and quickly output target historical features without repeatedly executing complex computational logic, significantly improving query response speed and system throughput.

[0045] Based on the above implementation scheme, in one feasible implementation, the step of receiving the historical feature query request sent by the customer further includes steps S21 to S23: Step S21: Obtain raw business data.

[0046] It should be noted that raw business data refers to unprocessed, underlying data collected from business systems, log systems, or data warehouses, including but not limited to customer behavior logs, transaction records, account information, and device information. Acquisition refers to pulling or receiving real-time data from data sources through data access tools (such as data acquisition engines, message queues, and ETL pipelines).

[0047] Understandably, the purpose of this step is to provide a data foundation for subsequent pre-computation. The principle behind this is to establish a unified data acquisition channel, aggregating heterogeneous data scattered across different business systems onto the feature computing platform, ensuring data integrity, timeliness, and consistency.

[0048] Specifically, the feature backtracking system periodically extracts customer information tables and transaction logs from the business database through the data access module, collects user behavior tracking logs from the log system, and consumes real-time event streams from the message queue. The collected data first undergoes format validation and integrity checks, filtering out data records with missing fields or abnormal formats.

[0049] For example, in a financial risk control scenario, the system retrieves the transaction flow data, account change data, and login behavior data newly added the previous day from the ODS (Operational Data Storage) layer every morning at midnight. The data volume is approximately 5 million records, covering fields such as customer ID, transaction amount, transaction time, and login IP.

[0050] Step S22: Based on the preset feature calculation logic, pre-calculate the original business data to generate daily incremental feature data and feature snapshot data of the original business data.

[0051] It should be noted that the preset feature calculation logic refers to a predefined set of calculation rules that transform raw business data into feature indicators, with each feature corresponding to a calculation expression or function. Pre-calculation refers to the process of actively performing feature calculations and storing the results before the actual query occurs. Daily incremental feature data refers to feature values ​​calculated at the natural day level, reflecting the addition or changes on that day. Feature snapshot data refers to a data snapshot that fully solidifies the feature status at a specific point in time (such as the end of the month).

[0052] Understandably, the core purpose of this step is to move the computation task from the query stage to the offline stage, enabling computation to be performed once and used multiple times. The complex computation logic that was originally executed during the query is broken down into offline tasks that can be executed in batches, running according to a fixed scheduling cycle (such as daily or monthly), and the computation results are persistently stored for direct use in subsequent queries.

[0053] Specifically, based on preset feature calculation logic, the acquired raw business data is processed in batches. Incremental calculation tasks are executed daily, processing only newly added business data for that day, and generating feature change records for each customer for that day, forming daily incremental feature data. Simultaneously, on the last day of each month, the system executes a full-scale solidification task, generating a snapshot of the complete feature status of all customers at the end of the month, which serves as a rollback baseline for subsequent queries.

[0054] For example, the calculation logic for the preset feature of the cumulative transaction amount in the past 30 days is as follows: sum the transaction amounts of the past 30 days. At midnight every day, incremental calculations are performed on the transaction data added the previous day to update the feature value; at the same time, on the last day of each month, the complete values ​​of 100 feature fields, including the cumulative transaction amount of all customers in the past month, are stored as a month-end snapshot for use in the following month's query.

[0055] Step S23: Based on the daily incremental feature data and the feature snapshot data, construct a feature-level storage system for the original business data.

[0056] It should be noted that building a feature-level storage system refers to organizing and persisting the pre-calculated daily incremental feature data and feature snapshot data according to the time dimension and storage level to form a data storage structure that supports efficient querying.

[0057] Understandably, the purpose of this step is to establish a physical storage structure that matches the hierarchical query strategy. Feature data of different granularities are stored in different tables or partitions. The daily incremental feature table stores fine-grained change records, while the feature snapshot table stores periodic full status data. The two are logically linked through a time field and a customer identifier field.

[0058] Specifically, two core storage tables are created: a daily incremental feature table with date and customer ID as primary keys, storing customer records with feature changes each day, where each record includes the customer ID, date, and the current values ​​of each feature field; and a feature snapshot table with period identifier (e.g., year or month) and customer ID as primary keys, storing all full features for all customers at the end of each period. Both tables can be partitioned by date or period and optimized for compression using a columnar storage format to reduce storage space.

[0059] For example, the system creates a `dim_feature_daily` table in Hive to store daily incremental data, partitioned by `dt` date; and creates a `dim_feature_monthly_snapshot` table to store month-end snapshots, partitioned by `month` month. Both tables use the customer ID as the bucket key to accelerate subsequent join queries.

[0060] This embodiment, through the above-described solution, solves the problems of redundant calculations and resource waste caused by simultaneous querying and calculation in the prior art by adding data pre-calculation and hierarchical storage construction steps before the query step. By advancing the complex feature calculation logic to the offline stage and storing the calculation results hierarchically according to two granularities—daily increment and periodic snapshot—it provides the data foundation for the hierarchical query strategy in claim 1, enabling the query stage to directly read the pre-calculated results without recalculation, thus achieving an optimized balance between computing resources and query performance.

[0061] Based on the above implementation scheme, in one feasible implementation, the step of pre-calculating the original business data according to the preset feature calculation logic to generate daily incremental feature data and feature snapshot data of the original business data includes S31~S32: Step S31: Based on the preset feature calculation logic, perform incremental calculation on the original business data to obtain the daily incremental feature data.

[0062] It should be noted that incremental computation is a computational optimization strategy that refers to updating the data only for newly added or changed data based on existing historical calculation results, rather than recalculating the entire dataset each time. Daily incremental feature data is the daily output set of incremental computation, containing only the feature records that changed that day.

[0063] Understandably, the purpose of this step is to significantly reduce the computational load of daily feature calculations. By identifying newly added or modified business data for the day, only the features affected by this data are recalculated, while the features of unaffected customers remain unchanged, thus avoiding the huge overhead of performing a full recalculation for all customers every day.

[0064] Specifically, a set of affected customers is maintained, recording all customer IDs whose values ​​changed in the daily business data. The incremental calculation task first obtains the feature status of all customers from yesterday as a baseline, and then recalculates the feature values ​​of only the customers in the affected customer set based on the newly added business data of today, generating feature records for these customers for today. For unaffected customers, no daily incremental records are generated, indicating that their feature values ​​have not changed compared to yesterday.

[0065] For example, on May 16th, the system processed 100,000 new transaction records added the previous day. Analysis revealed that these transactions involved only 5,000 customers. The incremental calculation task only recalculated the cumulative transaction amount and other characteristics of the past 30 days for these 5,000 customers, generating 5,000 daily incremental records and storing them in the May 16th partition of the dim_feature_daily table. No records were generated for the remaining 9 million customers on that day, implying that their characteristic values ​​were the same as those on May 15th.

[0066] Step S32: At the end of the preset target period, the daily incremental feature data is summarized and solidified to generate the feature snapshot data.

[0067] It should be noted that the preset target end-of-cycle time point refers to the pre-set time node used to generate a full state snapshot. This solution preferably selects the last day of each calendar month. The aggregation and consolidation process refers to the process of overlaying all daily incremental feature data within the cycle onto the snapshot of the previous cycle, generating a new full snapshot, and persistently storing it.

[0068] Understandably, the purpose of this step is to generate a reliable historical rollback baseline, addressing the issue of decreased query efficiency caused by excessively long daily incremental chains. By folding the incremental changes scattered across each day within a period into a deterministic full state point, when the query time falls within that period and there are no daily incremental records, this snapshot can be directly used as the baseline, avoiding the complex calculations of backtracking day by day along the time chain.

[0069] Specifically, on the first day of the second month, the system initiates a snapshot generation task. This task first obtains the feature snapshot from the end of the previous month as a base, then iterates through the daily incremental feature data for each day of the month in chronological order, applying the incremental changes sequentially to the base snapshot to obtain the latest full feature status for all customers at the end of the month. The system writes this status to the feature snapshot storage area and marks it with the corresponding period identifier.

[0070] For example, the system generates a snapshot of the end of May on June 1: it reads the snapshot of May 1 as a base, then applies the daily incremental records from May 1 to May 31 (about 5,000 changes per day), and finally merges them to generate a snapshot of May 31 containing the complete feature status of 10 million customers, which is then stored in a partition of the dim_feature_monthly_snapshot table.

[0071] This embodiment addresses the problem of excessive computational resource consumption in the full-scale computation mode by decomposing pre-computation into two independent stages: daily incremental computation and periodic snapshot solidification. Incremental computation ensures efficiency in processing only changed data daily, while periodic snapshot solidification controls the length of the historical backtracking path. The collaboration between the two allows feature data to track changes in a fine-grained manner while providing fast access points in a coarse-grained manner, providing an optimal data organization structure for hierarchical query strategies.

[0072] Based on the above implementation scheme, in one feasible implementation, the step of performing incremental calculation on the original business data based on the preset feature calculation logic to obtain the daily incremental feature data includes S41~S42: Step S41: Construct a hierarchical calculation system based on data lineage, wherein the hierarchical calculation system includes basic features, intermediate features, and advanced features.

[0073] It should be noted that data lineage refers to the dependency and transitive relationships between features and between features and original data fields. It describes the feature generation path and computation chain. The hierarchical computation system organizes features into a hierarchical computation framework based on lineage. Basic features are atomic features calculated directly from the original data fields. Intermediate features are derived features calculated based on one or more basic features. High-level features are composite features calculated based on intermediate features or across multiple feature dimensions.

[0074] Understandably, the purpose of this step is to establish a directed acyclic graph for feature computation, enabling reasonable layering and reuse of computational tasks. By analyzing the computational expressions of all features, the dependencies between features are identified. Features without dependencies are designated as the base layer, those that depend only on base features are designated as intermediate layers, and those that depend on intermediate features are designated as high-level layers, forming a clear hierarchical structure and avoiding circular dependencies.

[0075] Specifically, the system performs static analysis on all pre-defined feature calculation logic to extract the input fields or other features that each feature depends on. A topological sorting algorithm is used to determine the calculation order of the features, identifying features with zero in-degree (not dependent on other features) as basic features, features that depend on basic features as intermediate features, and features that depend on intermediate features as high-level features. The system stores the hierarchical results in a metadata center for subsequent calculation scheduling.

[0076] For example, the system defines feature A as the daily transaction amount (based on the original transaction amount field), feature B as the cumulative transaction amount over the past 30 days (dependent on daily feature A), and feature C as the average daily transaction amount over the past 30 days (dependent on feature B divided by 30). Through lineage analysis, the system derives the following: feature A is the basic feature, feature B is the intermediate feature, and feature C is the advanced feature. During calculation, feature A must be calculated first, then feature B, and finally feature C.

[0077] Step S42: Based on the hierarchical calculation system, perform incremental calculations on the original business data level by level to obtain incremental feature data for the corresponding date.

[0078] It should be noted that incremental computation refers to performing feature computation tasks layer by layer upwards, starting from the lowest level (basic features) according to the hierarchical order of the hierarchical computation system, with the computation results of each layer serving as the input for the next layer.

[0079] Understandably, the purpose of this step is to ensure the correctness of the feature calculation process and maximize the reuse of computing resources. Calculations are performed bottom-up, following data dependencies. Once a dependent feature has been calculated in the previous layer, the next layer can directly reference its result without repeatedly calculating the same data.

[0080] Specifically, the system first identifies the basic features affected that day (i.e., features directly related to the newly added raw data that day), and calculates the daily incremental values ​​of these basic features in parallel. After the calculation is complete, the scheduler identifies intermediate features that depend on these calculated basic features and initiates the incremental calculation of these intermediate features. This process continues layer by layer upwards until all affected features have been calculated. The calculation results of each layer are cached or stored for reuse by features in higher layers.

[0081] For example, 50,000 new transaction records were added on that day. The system first calculates the basic characteristic of the affected customers' transaction amount for that day, resulting in 50,000 records. Next, the system automatically triggers the calculation of the intermediate characteristic's cumulative transaction amount over the past 30 days: for the customers corresponding to these 50,000 records, the system reads their historical cumulative value and adds it to the current day's value to generate a new cumulative value. Finally, the system triggers the calculation of the advanced characteristic's average daily transaction amount over the past 30 days; the result is obtained by directly dividing the previously calculated cumulative value by 30, without needing to repeatedly read the original transaction records.

[0082] This embodiment, through the above-described scheme, constructs a hierarchical computing system based on data lineage and performs incremental computation at each level, thus resolving the computational errors and redundant calculations caused by chaotic dependencies during feature computation. The hierarchical system ensures the correctness and manageability of the computation order, while the incremental mechanism allows the computation results of each layer to be fully reused by the features of the upper layer, avoiding the repeated reading and processing of the same original data, and significantly improving the overall efficiency and resource utilization of the pre-computation stage.

[0083] Based on the above implementation scheme, in one feasible implementation, the feature hierarchical storage system includes a daily incremental feature storage area and a feature snapshot storage area. The step of obtaining the first feature data by backtracking and matching the feature hierarchical storage system based on the target time point in the historical feature query request through a hierarchical query strategy includes S51~S54: Step S51: Based on the target time point in the historical feature query request, determine the target period to which the target time point belongs.

[0084] It should be noted that the target period refers to the preset time segment unit where the target time point is located, and in this scheme, the month is preferred as the period unit. The determination operation refers to extracting the year and month information (e.g., May 2024) from the specific date value of the target time point (e.g., May 15, 2024).

[0085] Understandably, the purpose of this step is to map consecutive time points to discrete storage partition identifiers, so that the correct data partition can be quickly located for subsequent queries. Based on the partitioning rules in the feature-based hierarchical storage system (partitioned by year / month), the query time point is converted into the corresponding partition key value.

[0086] Specifically, the target time point field is parsed from the historical feature query request. This field is usually a date type value (in YYYY-MM-DD format). The year and month portions of this date are extracted and concatenated to generate a period identifier string (in YYYYMM format). This identifier is subsequently used to locate the specific partition of the feature snapshot table.

[0087] For example, if the target time in the query request is "May 15, 2024", the system extracts the year "2024" and the month "05" to generate a period identifier "202405". This identifier is used to locate the daily incremental data partition and the month-end snapshot partition of May 2024 in the monthly partitioned storage structure.

[0088] Step S52: Based on the target time point, query the latest feature data of the target customer identifier within the target period in the daily incremental feature storage area.

[0089] It should be noted that the daily incremental feature storage area is a dedicated storage area in the feature hierarchical storage system for storing daily change records. The latest feature data refers to the feature status corresponding to the last daily incremental record generated for this customer within the target period, from the start date of the period to the target time point.

[0090] Understandably, the purpose of this step is to locate the latest status of customer characteristics at the target time point within fine-grained data. Following the logic that daily incremental records reflect the characteristic status of the day, the daily incremental records for the customer within the target period are sorted in reverse chronological order by date. The first record, no later than the target time point, represents the customer's characteristic status at the target time point or after the most recent change.

[0091] Specifically, a query is executed in the daily incremental feature storage area: the filtering criteria are that the period identifier equals the defined target period, the customer identifier equals the target customer identifier in the request, and the date field is less than or equal to the date value at the target time point. The query results are sorted in descending order by the date field, and the first record is taken as the latest feature data. If the query result is empty, it means that the customer has not had any feature changes up to the target time point within the target period.

[0092] For example, query the characteristics of customer Zhang San on May 15, 2024. In the daily incremental table, there are records for Zhang San in the May 2024 partition: one on May 3rd and one on May 10th. After sorting, the system takes the record from May 10th as the latest characteristic data, because this is the most recent change record before May 15th, meaning that Zhang San's characteristic status after the update on May 10th continued until May 15th.

[0093] Step S53: If the latest feature data is found within the target period, then the latest feature data is used as the first feature data.

[0094] It should be noted that the first feature data refers to assigning the latest queried feature data to the first feature data variable for use in subsequent steps.

[0095] Understandably, the purpose of this step is to complete the logical branch of returning the result upon a match, ensuring maximum query efficiency. When matching data is found in the daily incremental storage area, the result is used directly without further access to the snapshot storage area.

[0096] Specifically, when the query in step S12 returns a valid record, the system directly assigns that record object to the first feature data. At this point, the hierarchical strategy of the query process is complete, and the system can skip step S14 and directly proceed to the broadcast query step.

[0097] For example, continuing from the previous example, we successfully retrieved Zhang San's daily incremental record for May 10th, containing the complete values ​​of his feature fields. This record is marked as the first feature data, and we no longer need to query the month-end snapshot table.

[0098] Step S54: If the latest feature data is not found within the target period, then query the feature snapshot data of the target customer identifier in the previous period from the feature snapshot storage area to obtain the first feature data.

[0099] It should be noted that the feature snapshot data of the previous period refers to the full feature snapshot that was fixed at the end of the previous period of the period in which the target time point is located. The feature snapshot storage area is a storage area in the feature hierarchical storage system that is dedicated to storing periodic full snapshots.

[0100] Understandably, the purpose of this step is to provide a fallback solution when fine-grained data is missing. When a customer has no feature change records within the target period, it means that their feature status has not changed since the end of the previous period. Therefore, the snapshot status at the end of the previous period can accurately represent their feature status at any point in time within the current period.

[0101] Specifically, when the latest feature data is not found, the target period identifier is decremented by 1 to obtain the previous period identifier. A query is executed in the feature snapshot storage area: the filtering conditions are that the period identifier equals the previous period identifier and the customer identifier equals the target customer identifier. Since the snapshot storage area stores all the full features of all customers at the end of each period, this query will always return a record (provided the customer existed before that period). The system uses this snapshot record as the first feature data.

[0102] For example, query the characteristics of customer Li Si on May 15, 2024. There are no records for Li Si in the May 2024 partition of the daily incremental table, indicating that Li Si's characteristics remained unchanged throughout May. The system queries the month-end snapshot table of April 2024 to obtain Li Si's full characteristic data on April 30 as the first characteristic data, which accurately represents Li Si's characteristic status on May 15.

[0103] This embodiment, through the above-described scheme, refines the hierarchical query strategy into a four-step logic: first determine the period, then query the daily incremental data, return the result if a match is found, and check the snapshot if no match is found. This solves the problem of query logic confusion caused by unclear storage path selection during the query process. This strategy ensures that each query can locate the target feature data with the fewest data accesses: most active customers can directly and quickly obtain data from the daily incremental storage area, while inactive customers can obtain the complete baseline from the snapshot storage area in one go. Both avoid the inefficient query method of scanning across multiple tables and partitions.

[0104] Based on the above implementation scheme, in one feasible implementation, the step of performing a broadcast query based on the historical feature query request to obtain the second feature data corresponding to the historical feature query request includes S61~S63: Step S61: Parse the sample data to be queried from the historical feature query request, and distribute the sample data to be queried as a broadcast variable to each computing node in the distributed computing environment.

[0105] It should be noted that the sample data to be queried refers to the set of samples carried in historical feature query requests that require feature backtracking. It typically contains multiple sample records, each containing at least a customer identifier and a target time point. Broadcast variables are an optimization mechanism provided by distributed computing frameworks (such as Apache Spark). They replicate a read-only variable to every compute node in the cluster, allowing each task to access the variable and avoiding frequent cross-node transfers. A distributed computing environment refers to a cluster composed of multiple compute nodes that collaborate to complete large-scale data processing tasks.

[0106] Understandably, the purpose of this step is to efficiently distribute small-scale sample data to all computing nodes, preparing for subsequent localized correlation computations. By using a broadcast mechanism to push sample data to the node where the data resides, rather than pulling the data to the node where the samples reside, the amount of network data transmission is significantly reduced.

[0107] Specifically, the system first parses the complete content of the sample data to be queried from the historical feature query request message body. This data is typically encapsulated in a structured format (such as a JSON array or CSV rows). The system calculates the estimated memory size of the sample data. If it is less than a preset broadcast threshold (such as 100MB), it encapsulates it as a broadcast variable. The system then calls the broadcast API of the distributed computing framework to send the broadcast variable to all computing nodes in the cluster. Each node caches the content of the broadcast variable in its local memory.

[0108] For example, the risk control system needs to perform feature backtracking on 5,000 historical cases, and the size of these 5,000 sample data is approximately 8MB. The system encapsulates these 5,000 sample data as broadcast variables and distributes them to the memory of 20 computing nodes in the cluster through Spark's broadcast() method. Each node can directly access the sample set without network communication.

[0109] Step S62: On each computing node, the broadcast variable is associated and matched with the feature hierarchical storage system to generate local feature results for each computing node.

[0110] It should be noted that association matching refers to the process of joining sample data in the broadcast variables with feature data in the feature hierarchy storage system stored locally on each computing node, according to the client identifier. Local feature results refer to the feature dataset generated after a single computing node independently completes association matching, containing only the portion of data processed by that node.

[0111] Understandably, the purpose of this step is to achieve data-localized computation and avoid large-scale data transfer across nodes. Utilizing the principle of data locality: feature data in a feature-hierarchical storage system is typically partitioned and stored according to customer identifiers, with each computing node responsible for storing a portion of the customer's feature data. When sample data carried by broadcast variables arrives at a node, the node only needs to complete the feature data lookup and association within its own storage space; no data needs to be retrieved from other nodes throughout the entire process.

[0112] Specifically, after the broadcast variables are distributed, the system assigns a computing task to each computing node. This task iterates through each sample record in the broadcast variables, extracting the customer identifier. The task then uses this customer identifier to query the feature hierarchy storage partition under the node's jurisdiction, retrieving the first feature data corresponding to that customer. The task concatenates the sample record with the retrieved feature data to generate a result record. All result records are then aggregated to form the local feature result for that node.

[0113] For example, compute node A locally stores feature data for customer IDs starting with 0-1999. After receiving 5000 samples from the broadcast variable, node A filters out 1500 samples with customer IDs starting with 0-1999, searches for the feature data of these 1500 customers in its local storage, and generates 1500 local feature results. Other nodes perform similar operations in parallel.

[0114] Step S63: Merge the local feature results of each computing node to obtain the second feature data.

[0115] It should be noted that the merging process refers to collecting the local feature results generated by each computing node to the driver node, performing deduplication, sorting, and combination to form a complete global result set. The second feature data is the total result set obtained after merging all local feature results.

[0116] Understandably, the purpose of this step is to aggregate the computation results distributed across various nodes into a unified output. Through the collection mechanism of the distributed computing framework, the local results returned by the tasks on each node are pulled to the driver node, where the final result assembly operation is then performed.

[0117] Specifically, when initiating distributed computing, a result collector is set up. After each computing node completes local correlation matching, it sends the local feature results to the driver node through the framework's network communication mechanism. After receiving the results from all nodes, the driver node reorders the results according to the original order of the samples or the specified sorting rules, and checks for any missing sample records. After confirming completeness, the merged result set is marked as the second feature data.

[0118] For example, the system merges the 20 local feature result files returned by 20 computing nodes. The driver node verification finds that all 5000 samples have corresponding results (the sum of the number of results returned by each node is 5000). After rearranging the samples according to their original ID order, a complete feature result set containing 5000 records is generated as the second feature data.

[0119] This embodiment addresses the performance bottleneck of large-scale feature association queries in a distributed environment by refining the broadcast query process into three stages: broadcast distribution, local association, and result merging. The broadcast mechanism proactively distributes small-scale sample data to each data storage node, enabling association calculations to be completed locally. This fundamentally avoids the huge network overhead caused by the cross-node transmission of feature data in traditional solutions, significantly improving the execution efficiency of batch sample feature backtracking.

[0120] Based on the above implementation scheme, in a feasible implementation, the step of integrating the first feature data and the second feature data to obtain the target feature data corresponding to the historical feature query request includes S71~S73: Step S71: Align and match the first feature data and the second feature data according to the feature identifier, remove duplicate feature data, and obtain intermediate feature data.

[0121] It should be noted that a feature identifier is a name or ID used to uniquely identify a feature field. Alignment matching refers to matching and comparing feature fields in the first and second feature data according to their feature identifiers. Removing duplicate feature data means that when fields with the same feature identifier exist in two data sources, only one copy is retained to avoid redundancy.

[0122] Understandably, the purpose of this step is to address the redundancy issue between multi-source feature data. The union of all feature identifiers from the two data sources is extracted. For feature identifiers appearing in both data sources, a pre-defined deduplication strategy (such as retaining values ​​from one source) is applied to generate a redundant intermediate dataset.

[0123] Specifically, the system iterates through all feature key-value pairs in the first and second feature data to construct a feature identifier set. For each feature identifier, the system checks its occurrence in the first and second feature data. If it appears only in one data source, the feature value is directly copied to the intermediate feature data. If it appears in both data sources, the system selects to retain one of the values ​​according to a preset strategy (e.g., using the second feature data as the standard, using the first feature data as the standard, or reporting an error if the values ​​are inconsistent).

[0124] Step S72: Confirm and correct the intermediate feature data according to the preset feature priority rules.

[0125] It should be noted that the preset feature priority rule refers to a predefined feature value conflict resolution strategy, which can be prioritized based on the reliability of the feature's data source, update timestamp, or importance in a specific business scenario. Confirmation and correction refers to verifying and adjusting the feature values ​​in the intermediate feature data to ensure the correctness and consistency of the final feature values.

[0126] Understandably, the purpose of this step is to handle feature value conflicts and anomalies, ensuring the quality of the output feature data. After deduplication, an independent validation layer is introduced to detect and correct any potential outliers based on business rules and logical constraints between features.

[0127] Specifically, a pre-configured feature priority rule table is loaded, which defines the priority order for feature fields that may have conflicts or calculation differences. Each feature field in the intermediate feature data is traversed; if a priority rule exists for that field, the feature value is checked to see if it meets the rule requirements (such as value range or logical consistency). For feature values ​​that do not meet the rules, the system attempts to recalculate or correct them from other sources (such as parent features in the feature lineage chain), or marks them as invalid and fills them with default values.

[0128] Step S73: Standardize and encapsulate the corrected intermediate feature data to obtain the target feature data corresponding to the historical feature query request.

[0129] It should be noted that standardized encapsulation refers to serializing the corrected intermediate feature data according to a preset output format to generate a standard data format that can be used for network transmission or program parsing. The target feature data is the final complete response result returned to the requester.

[0130] Understandably, the purpose of this step is to ensure that the output of feature data has a unified format specification, making it easy for downstream systems to parse and use. A standard data exchange protocol is defined to losslessly convert the internal feature data structure into a standard format conforming to this protocol.

[0131] Specifically, the system retrieves a preset output format definition, which specifies the structure of the feature data (e.g., object structure, array structure), field naming conventions (e.g., snake-case, camelCase), data type (e.g., string, numeric, boolean), and optional timestamp and version information. The system then iterates through each field in the intermediate feature data according to this definition, performing data type conversion and field renaming, and finally uses the serialized byte stream or string as the target feature data.

[0132] This embodiment, through the above-described scheme, refines feature integration into a three-step process of alignment and deduplication, priority correction, and standardized encapsulation. This resolves the redundancy, conflicts, and format inconsistencies that may arise during the merging of multi-source feature data. This process ensures that regardless of the specific source and quality of the first and second feature data, the final output target feature data possesses complete and non-redundant feature fields, verified and corrected feature values, and a unified output format, providing downstream business systems with high-quality, readily usable, and reliable data input.

[0133] For example, to help understand the implementation process of the feature backtracking method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 2 , Figure 2 A simplified flowchart of a feature backtracking method is provided, specifically: like Figure 2 As shown, the feature backtracking method proposed in this application is generally divided into three collaborative layers: a computation layer, a storage layer, and a query layer.

[0134] At the computation layer, the system first acquires raw business data, including customer behavior data, transaction records, and business data. Based on data lineage, the system constructs a hierarchical computation system, dividing features into three levels: basic features, intermediate features, and advanced features. Basic features are calculated directly from raw data fields (e.g., "daily transaction amount"), intermediate features depend on basic features (e.g., "cumulative transaction amount in the past 30 days"), and advanced features are further derived from intermediate features (e.g., "average daily transaction amount in the past 30 days"). The system performs incremental calculations in a bottom-up order: first, it calculates the basic features of affected customers; then, it uses the results of the basic features to calculate intermediate features; and finally, it calculates advanced features based on the intermediate features. The daily incremental calculation results are output to the storage layer as daily incremental feature data. Simultaneously, at the end of each period (e.g., the end of the month), the system summarizes and solidifies all incremental data within the period, generating a full feature snapshot data.

[0135] At the storage layer, the system constructs a hierarchical storage system based on the daily incremental feature data and feature snapshot data output from the computing layer. Specifically, the daily incremental feature storage area stores newly added or changed feature records daily, organized by date partitions. Each record contains the customer identifier, date, and the current values ​​of each feature field, reflecting the incremental changes of features daily. The feature snapshot storage area stores periodic full feature snapshots, organized by period (e.g., month), storing the complete feature status of all customers at the end of each period. The two types of storage areas are logically linked through the time field and the customer identifier field, forming a hierarchical storage structure of "daily incremental records for fine-grained tracking of changes, and periodic snapshots providing a full baseline."

[0136] At the query layer, when the system receives a historical feature query request from a customer, it executes two query paths in parallel. The first path is a hierarchical query strategy: the system determines the target period based on the target time point in the request, and prioritizes querying the customer's latest feature data within the target period from the daily incremental feature storage area. If the query matches, this data is used as the first feature data; if it doesn't match (i.e., the customer has no feature changes within the period), it falls back to the feature snapshot storage area and obtains the customer's full feature snapshot at the end of the previous period as the first feature data. The second path is a broadcast query: the system parses the sample data to be queried from the request and distributes it as a broadcast variable to each computing node in the distributed computing environment. Each node locally associates and matches the broadcast variable with the feature hierarchical storage system to generate local feature results. Subsequently, the system merges the local feature results from each node to obtain the second feature data. Finally, the system enters the feature integration stage: the first and second feature data are aligned and matched according to feature identifiers, duplicate feature data is removed, intermediate feature data is confirmed and corrected according to preset feature priority rules, and then the target feature data is output after standardization and encapsulation.

[0137] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the feature backtracking method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0138] This application also provides a feature tracing device, please refer to... Figure 3 The feature backtracking device includes: The matching backtracking module 301 is used to receive historical feature query requests sent by customers, and based on the target time point in the historical feature query request, backtracking and matching the feature hierarchical storage system through a hierarchical query strategy to obtain the first feature data. Broadcast query module 302 is used to perform a broadcast query based on the historical feature query request to obtain the second feature data corresponding to the historical feature query request; The feature integration module 303 is used to perform feature integration based on the first feature data and the second feature data to obtain the target feature data corresponding to the historical feature query request.

[0139] The feature backtracking device provided in this application, employing the feature backtracking method described in the above embodiments, can solve the technical problem of low efficiency in backtracking queries of historical features. Compared with the prior art, the beneficial effects of the feature backtracking device provided in this application are the same as those of the feature backtracking method described in the above embodiments, and other technical features in the feature backtracking device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0140] This application provides a feature backtracking device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the feature backtracking method in Embodiment 1 above.

[0141] The following is for reference. Figure 4 The diagram illustrates a structural schematic of a feature backtracking device suitable for implementing embodiments of this application. The feature backtracking device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The feature backtracking device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0142] like Figure 4As shown, the feature tracing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in the read-only memory 1002 or a program loaded from the storage device 1003 into the random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the feature tracing device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the feature tracing device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows feature tracing devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0143] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0144] The feature backtracking device provided in this application, employing the feature backtracking method described in the above embodiments, can solve the technical problem of low efficiency in backtracking queries of historical features. Compared with the prior art, the beneficial effects of the feature backtracking device provided in this application are the same as those of the feature backtracking method described in the above embodiments, and other technical features in this feature backtracking device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0145] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0146] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0147] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the feature backtracking method in the above embodiments.

[0148] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0149] The aforementioned computer-readable storage medium may be included in the feature tracing device; or it may exist independently and not assembled into the feature tracing device.

[0150] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the feature backtracking device, the feature backtracking device: receives a historical feature query request sent by a client; performs backtracking matching on the feature hierarchical storage system based on the target time point in the historical feature query request using a hierarchical query strategy to obtain first feature data; performs a broadcast query based on the historical feature query request to obtain second feature data corresponding to the historical feature query request; and performs feature integration based on the first feature data and the second feature data to obtain target feature data corresponding to the historical feature query request.

[0151] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0153] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0154] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described feature backtracking method, thereby solving the technical problem of low efficiency in backtracking queries of historical features. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the feature backtracking method provided in the above embodiments, and will not be repeated here.

[0155] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the feature backtracking method described above.

[0156] The computer program product provided in this application can solve the technical problem of low efficiency in backtracking queries of historical features. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the feature backtracking method provided in the above embodiments, and will not be repeated here.

[0157] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A feature backtracking method, characterized in that, The feature backtracking method includes: Receive a historical feature query request sent by a customer, and based on the target time point in the historical feature query request, perform backtracking matching on the feature hierarchical storage system through a hierarchical query strategy to obtain the first feature data; A broadcast query is performed based on the historical feature query request to obtain the second feature data corresponding to the historical feature query request; Based on the first feature data and the second feature data, feature integration is performed to obtain the target feature data corresponding to the historical feature query request.

2. The feature backtracking method as described in claim 1, characterized in that, The step of receiving a historical feature query request sent by a customer also includes: Obtain raw business data; Based on the preset feature calculation logic, the original business data is pre-calculated to generate daily incremental feature data and feature snapshot data of the original business data; Based on the daily incremental feature data and the feature snapshot data, a feature-based hierarchical storage system for the original business data is constructed.

3. The feature backtracking method as described in claim 2, characterized in that, The step of pre-calculating the original business data according to the preset feature calculation logic to generate daily incremental feature data and feature snapshot data of the original business data includes: Based on the preset feature calculation logic, incremental calculation is performed on the original business data to obtain the daily incremental feature data; At the end of the preset target period, the daily incremental feature data is summarized and solidified to generate the feature snapshot data.

4. The feature backtracking method as described in claim 3, characterized in that, The step of performing incremental calculations on the original business data based on the preset feature calculation logic to obtain the daily incremental feature data includes: A hierarchical computing system is constructed based on data lineage, and the hierarchical computing system includes basic features, intermediate features, and advanced features; Based on the hierarchical calculation system, incremental calculations are performed on the original business data level by level to obtain incremental feature data for the corresponding date.

5. The feature backtracking method as described in claim 1, characterized in that, The feature hierarchical storage system includes a daily incremental feature storage area and a feature snapshot storage area. The step of obtaining the first feature data by backtracking and matching the feature hierarchical storage system based on the target time point in the historical feature query request using a hierarchical query strategy includes: Based on the target time point in the historical feature query request, determine the target period to which the target time point belongs; Based on the target time point, query the latest feature data of the target customer identifier within the target period in the daily incremental feature storage area; If the latest feature data is found within the target period, then the latest feature data is used as the first feature data; If the latest feature data is not found within the target period, the feature snapshot data of the target customer identifier in the previous period of the target period is retrieved from the feature snapshot storage area to obtain the first feature data.

6. The feature backtracking method as described in claim 1, characterized in that, The step of performing a broadcast query based on the historical feature query request to obtain the second feature data corresponding to the historical feature query request includes: Parse the query sample data from the historical feature query request, and distribute the query sample data as a broadcast variable to each computing node in the distributed computing environment; On each of the computing nodes, the broadcast variables are associated and matched with the feature hierarchical storage system to generate local feature results for each computing node; The local feature results of each computing node are merged to obtain the second feature data.

7. The feature backtracking method as described in claim 1, characterized in that, The step of integrating features based on the first feature data and the second feature data to obtain the target feature data corresponding to the historical feature query request includes: The first feature data and the second feature data are aligned and matched according to the feature identifier, and duplicate feature data are removed to obtain intermediate feature data. The intermediate feature data is confirmed and corrected according to the preset feature priority rules; The corrected intermediate feature data is standardized and encapsulated to obtain the target feature data corresponding to the historical feature query request.

8. A feature backtracking device, characterized in that, The feature backtracking device includes: The matching backtracking module is used to receive historical feature query requests sent by customers, and based on the target time point in the historical feature query request, backtracking and matching the feature hierarchical storage system through a hierarchical query strategy to obtain the first feature data. The broadcast query module is used to perform a broadcast query based on the historical feature query request to obtain the second feature data corresponding to the historical feature query request. The feature integration module is used to integrate features based on the first feature data and the second feature data to obtain the target feature data corresponding to the historical feature query request.

9. A feature backtracking device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the feature backtracking method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the feature backtracking method as described in any one of claims 1 to 7.

Images