Model full life cycle management method and system for recommendation engine

Abstract

The application provides a model full life cycle management method and system for a recommendation engine, comprising the following steps: S1, managing the meta information, versions and running states of various models in a recommendation system through a model management module; S2, managing the feature assets used by various models in the recommendation system through a feature management module; and S3, configuring a target model and corresponding feature values based on the recommendation system according to user requirements. Through the bidirectional linkage of the feature management module and the model management module, the application forms a positive cycle of 'feature definition -> model training -> effect feedback -> feature optimization', fundamentally solves the problem of disconnection between features and models, and improves the iteration quality and long-term stability of the entire recommendation system.

Description

Technical Field

[0001] This invention relates to the field of computer software technology, specifically to a method and system for full lifecycle management of models for recommendation engines, and more specifically to a parameterized feature management and scoring model configuration platform for recommendation engines. Background Technology

[0002] Traditional intelligent recommendation systems suffer from the following technical shortcomings: Inadequate model management: In traditional systems, model files are scattered, lacking unified release management standards, and the management of the entire model lifecycle is incomplete.

[0003] Hard-coded sorting logic: Traditional systems use hard-coded scoring and sorting during the sorting stage, which lacks flexibility and maintainability.

[0004] Disorganized feature management: Traditional systems have a large number of features and lack effective classification, search and management mechanisms, which leads to difficulties in feature finding, high maintenance costs and low feature iteration efficiency.

[0005] Lack of AI capabilities: Traditional systems lack the integration of AI capabilities and cannot leverage AI technology to improve the efficiency of feature engineering and model development.

[0006] Patent document CN119884483A (application number: 202510022893.8) discloses an artificial intelligence intelligent recommendation system and method, including: a data acquisition module that acquires user behavior logs, including multimodal behavioral data and item feature data, from a shared account; a behavior pattern recognition module that uses a clustering algorithm to identify the unique behavior patterns of different users and generates an initial behavior feature vector for each user; a time series analysis module that combines these feature vectors with time factors to generate a time series preference matrix and a time series behavior feature vector, and generates a user preference matrix through matrix factorization; an item time series feature matrix generation unit that generates an item time series feature matrix based on item feature data and time factors; and a prediction model generation module that integrates the user time series preference matrix and the item time series feature matrix to generate a multilayer perceptron prediction model, which significantly improves the personalized recommendation effect in a multi-user shared account scenario by optimizing hyperparameters and minimizing prediction error. Summary of the Invention

[0007] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a method and system for full lifecycle management of models for recommendation engines.

[0008] A method for managing the entire lifecycle of a model for a recommendation engine, provided by the present invention, includes: Step S1: Manage the metadata, version, and running status of various models in the recommendation system through the model management module; Step S2: Manage the feature assets used by various models in the recommendation system through the feature management module; Step S3: Configure the target model and corresponding feature values ​​based on the recommendation system according to user needs.

[0009] Preferably, step S1 includes: Step S1.1: Classify and manage various types of models through the model classification submodule, including: machine learning models, deep learning models, and AI models; Step S1.2: Implement full lifecycle management of the model in the recommendation system through the model lifecycle management submodule, including: creation, testing, deployment, canary release and decommissioning.

[0010] Preferably, the AI ​​model includes: acquiring user natural language query information, parsing the user natural language query information to obtain user intent; configuring dynamic expressions based on user intent to construct a dynamic expression model; and performing full lifecycle management of the constructed dynamic expression model and the features used by the dynamic expression model.

[0011] Preferably, step S2 includes: Step S2.1: Classify the features used by various models according to the type dimension and the source dimension through the feature classification submodule, and construct a feature directory tree based on the classified features for structured classification management; persistently store the feature metadata, including: data source, data type, update frequency, business domain, and definition description through the feature directory tree; Step S2.2: The feature search submodule searches for features based on the feature directory tree according to any one or more dimensions, including feature identifier, name, and business tag, thereby locating the target feature; Step S2.3: The real-time feature value calculation submodule dynamically calculates the target feature value according to user needs or model requirements.

[0012] Preferably, step S3 includes: configuring candidate models according to user needs, ranking the candidate models by scores using an offline model evaluation and selection method based on A / B testing and / or user feedback, and recommending the candidate model with the highest score to the user.

[0013] According to the present invention, a model lifecycle management system for recommendation engines includes: Module M1: Manages the metadata, version, and running status of various models in the recommendation system through the model management module; Module M2: Manages the feature assets used by various models in the recommendation system through the feature management module; Module M3: Configures the target model and corresponding feature values ​​based on user needs and the recommendation system.

[0014] Preferably, the module M1 includes: Module M1.1: Classifies and manages various models through the model classification submodule, including: machine learning models, deep learning models, and AI models; Module M1.2: Implements full lifecycle management of models in the recommendation system through the model lifecycle management submodule, including: creation, testing, deployment, canary release, and decommissioning.

[0015] Preferably, the AI ​​model includes: acquiring user natural language query information, parsing the user natural language query information to obtain user intent; configuring dynamic expressions based on user intent to construct a dynamic expression model; and performing full lifecycle management of the constructed dynamic expression model and the features used by the dynamic expression model.

[0016] Preferably, the module M2 includes: Module M2.1: The feature classification submodule categorizes the features used by various models according to the type and source dimensions, and constructs a feature directory tree based on the categorized features for structured classification management; the feature directory tree is used to persistently store feature metadata, including: data source, data type, update frequency, business domain, and definition description. Module M2.2: The feature search submodule searches for features based on the feature directory tree according to any one or more dimensions, including feature identifier, name, and business tag, thereby locating the target feature; Module M2.3: The real-time feature value calculation submodule dynamically calculates the target feature values ​​according to user needs or model requirements.

[0017] Preferably, module M3 includes: configuring candidate models according to user needs, ranking the candidate models by scores using an offline model evaluation and selection method based on A / B testing and / or user feedback, and recommending the candidate model with the highest score to the user.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention solves the problems of "scattered, disordered, and untraceable" features in traditional recommendation systems by constructing a structured feature classification system, supporting multi-field fuzzy search and feature association analysis, and improving the discoverability and governance capabilities of feature assets; 2. This invention integrates machine learning models, deep learning models, and dynamic expression models into the same management framework, providing a standardized process from creation, training, testing, release to decommissioning. It solves the pain points of existing systems such as chaotic model versions, reliance on manual scripts for deployment, and lack of rollback mechanisms, significantly improving the reliability and compliance of model deployment and achieving unified lifecycle management of multiple types of models. 3. Through the dynamic expression model submodule, this invention allows business personnel to directly write scoring expressions based on registered features, enabling real-time adjustment of sorting logic and A / B testing without developer intervention. This reduces the strategy iteration cycle from "days" to "minutes", significantly improving business response speed and supporting zero-code, highly flexible online sorting strategy configuration. 4. On the one hand, this invention automatically parses feature definitions, usage scenarios, and associated models through natural language dialogue, reducing the understanding threshold for non-technical personnel; on the other hand, based on feature metadata and historical model library, it automatically generates candidate model structures and training code suggestions, improving the efficiency of new model development and reducing human design bias, thus realizing an AI-driven closed loop of feature understanding and model generation. 5. This invention forms a positive cycle of "feature definition → model training → effect feedback → feature optimization" through the bidirectional linkage between the feature management module and the model management module, fundamentally solving the problem of feature and model disconnection and improving the iteration quality and long-term stability of the entire recommendation system. Attached Figure Description

[0019] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the feature engineering and model management system for an intelligent recommendation system.

[0020] Figure 2 This is a flowchart illustrating the method for automatically creating and publishing models based on AI capability modules. Detailed Implementation

[0021] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0022] Example 1 A method for managing the entire lifecycle of a model for a recommendation engine, provided by the present invention, includes: Step S1: Manage the metadata, version, and running status of various models in the recommendation system through the model management module; Specifically, step S1 includes: Step S1.1: Classify and manage various types of models through the model classification submodule, including: machine learning models, deep learning models, and AI models; at the same time, store the metadata of each type of model, including: input feature requirements, output format, and version information; Step S1.2: Implement full lifecycle management of the model in the recommendation system through the model lifecycle management submodule, including: creation, testing, deployment, canary release and decommissioning; at the same time, feed back the feature sequences that the model depends on to the feature management module.

[0023] The AI ​​model includes: acquiring user natural language query information, parsing the user natural language query information to obtain user intent; configuring dynamic expressions based on user intent to construct a dynamic expression model; and performing full lifecycle management of the constructed dynamic expression model and the features used by the dynamic expression model.

[0024] Step S2: Manage the feature assets used by various models in the recommendation system through the feature management module; Specifically, step S2 includes: Step S2.1: Classify the features used by various models according to the type dimension and the source dimension through the feature classification submodule, and construct a feature directory tree based on the classified features for structured classification management; persistently store the feature metadata, including: data source, data type, update frequency, business domain, and definition description through the feature directory tree; Step S2.2: The feature search submodule searches for features based on the feature directory tree according to any one or more dimensions, including feature identifier, name, and business tag, thereby locating the target feature; Step S2.3: The real-time feature value calculation submodule dynamically calculates the target feature value according to user needs or model requirements.

[0025] Step S3: Configure the target model and corresponding feature values ​​based on the recommendation system according to user needs.

[0026] Specifically, step S3 includes: configuring candidate models according to user needs, ranking the candidate models by scores using an offline model evaluation and selection method based on A / B testing and / or user feedback, and recommending the candidate model with the highest score to the user.

[0027] Example 2 Example 2 is a preferred example of Example 1. According to the present invention, a model lifecycle management system for recommendation engines is provided, such as... Figures 1 to 2 As shown, it includes: The model management module is used to uniformly manage the metadata, version, and running status of various models in the recommendation system, and provides model calling and update interfaces for the feature management module and the AI ​​capability module.

[0028] The model management module includes: The model classification submodule is used to classify and manage the model types supported by the system, including machine learning models such as LightGBM and ItemCF; deep learning models such as DNN and Wide&Deep; and dynamic expression models, such as lightweight scoring models based on expressions and features. This submodule stores metadata for each type of model, such as input feature requirements, output format, and version information, and is called by the model lifecycle management submodule to perform subsequent operations.

[0029] The Model Lifecycle Management submodule manages the entire lifecycle of a model, from creation, testing, deployment, canary releases, to decommissioning. This submodule receives model definitions or code generated by the AI / human model creation submodule; writes model metadata and version information to the model registry; and notifies the Dynamic Expression Model submodule or the online service module to load the new model. Simultaneously, this submodule provides the Feature Management module with a list of features the model depends on for feature lineage tracking.

[0030] The model includes full lifecycle management from creation, testing, deployment, canary release to decommissioning, including: First, the model is pre-trained offline. During the model's runtime, the model's performance on user-side data is analyzed, and the model is iteratively upgraded to meet the model's objectives. Once the model pre-training is complete and meets the deployment criteria, a new model and the features required by the model can be created in the current recommendation system. After the model is created, the input and output validation and testing can be completed in the current recommendation system through the model trial function. After the model passes the test, it can be released. The current release only means that it is available and not immediately applicable. After the model goes live, experiments can be created in specified scenarios. The model is bound to the scoring strategy of the experiment, and a certain proportion of user traffic is allocated to the current experiment for A / B testing. The model's performance is compared with the control group model within the data statistics period. If the current model's performance is significantly improved, the current experiment traffic can be expanded to 100%, and the model can be used in the entire scenario.

[0031] This embodiment includes two parts: the full lifecycle management of the model in the current recommendation system and model iteration. Model iteration involves continuously optimizing features based on the actual business objectives in each real recommendation scenario and improving model performance through continuous A / B testing.

[0032] The Dynamic Expression Model submodule supports business users in implementing real-time scoring and ranking without code development by configuring expressions, such as: score=w1*user_click_rate+w2*item_ctr. This submodule retrieves registered feature values ​​from the feature management module, executes the expression calculation, and outputs the ranking score, which is then used by the recommendation engine. The expression configuration information is suggested and generated by the AI ​​model creation submodule or manually configured, and stored in the model library after manual review.

[0033] The feature management module is used to uniformly register, retrieve, analyze, and govern the feature assets used by various models in the recommendation system, providing high-quality and traceable feature data services for the model management module and the AI ​​capability module.

[0034] The feature management module includes: The feature classification submodule is used for structured classification and management of features according to type and source dimensions. The type dimension includes user features, content features, and global features; the source dimension includes original features, cross-features, and derived features. This submodule maintains a feature directory tree and persistently stores feature metadata for use by the feature search and feature association analysis submodules. The feature metadata includes data source, data type, update frequency, business domain, and definition.

[0035] The feature search submodule supports users in performing fuzzy or precise searches based on the feature directory tree using multiple dimensions such as feature identifier (feature_id), name, and business tags, enabling them to quickly locate available features. Search results include feature definitions, example values, their category and a list of associated models, and feature dependencies. The data originates from the metadata storage of the feature classification submodule and can be accessed by the AI ​​dialogue submodule to respond to user queries.

[0036] The feature association analysis submodule is used to analyze the association relationships between features and models, and between features themselves. The real-time feature value calculation submodule supports real-time feature value calculation. It achieves dynamic feature combination through features and combination functions (such as addition, subtraction, multiplication, division, inclusion / exclusion, Jaccard similarity, etc.). This submodule receives feature metadata from the feature classification submodule and dynamically calculates new feature values ​​based on user needs or model requirements. For example, given the original feature: the click-through rate of news articles in the last 3 days, and considering the significant differences in click-through rates among different news articles due to factors such as popularity and timeliness, the feature value needs to be scaled. Model A might use Min-Max, while Model B might use Z-score. In this case, new features can be created based on this original feature using different functions. The feature values ​​read by the model are the results of real-time calculation based on the source feature and the function. Input: Registered feature values, combination functions, such as Plus, Contains, MinMaxScale, Jaccard, etc.; Processing: Extract the required features, call the online feature service to obtain the latest feature values, and apply the combination function to calculate the new feature values; the results are cached in memory (such as Redis) for use in subsequent requests.

[0037] Output: The calculated feature values, which can be used by the recommendation engine or other downstream modules.

[0038] The AI ​​capability module is used to improve the automation and interaction efficiency of feature engineering and model development through large models or intelligent agent technology, and to lower the usage threshold for algorithm engineers and business personnel.

[0039] The AI ​​capability module includes: The AI ​​dialogue submodule receives user natural language queries, parses the intent, retrieves relevant information from the metadata and document library of the feature management module, and generates a structured answer. This submodule relies on the retrieval capabilities of the feature search submodule and can call the results of the feature association analysis submodule to interpret feature validity and improve question-answering accuracy.

[0040] The AI ​​model creation submodule automatically generates candidate model structure suggestions, feature combination schemes, and executable expression templates based on user-specified objectives and available feature sets. This submodule retrieves feature metadata and correlation analysis results from the feature management module, performs inference using a library of historically successful models, and submits the output to the model lifecycle management submodule to initiate the creation process. It can also generate dynamic expressions for deployment by the dynamic expression model submodule.

[0041] Example 3 Example 3 is a preferred example of Example 1. According to the present invention, a model lifecycle management system for recommendation engines includes: Module M1: Manages the metadata, version, and running status of various models in the recommendation system through the model management module; Module M2: Manages the feature assets used by various models in the recommendation system through the feature management module; Module M3: Configures the target model and corresponding feature values ​​based on user needs and the recommendation system.

[0042] In this embodiment, the model management module includes: Model classification management: The recommendation system has two built-in model templates: Machine learning model: Based on LightGBM / XGBoost, with configuration of training parameter templates, and support for uploading model structure files (.txt); Dynamic expression model: based on a lightweight expression engine (Expr4j) Model lifecycle management involves users initiating a "Create New Model" operation through the platform interface. After selecting the model type, filling in relevant metadata, and choosing the required features, users click "Save and Publish." The recommendation system then pushes the model file to the online model repository (such as S3 or OSS) and notifies all service nodes, enabling the recommendation engine to hot-load the new model. All operation records (operator, time, status) are stored in the audit log, supporting version rollback.

[0043] The dynamic expression model includes: a front-end integrated FormulaEditor component, allowing users to drag and drop registered features (from the feature management module) and combine operators (such as +, *, log()) to generate expressions like: news_hour_difference * 1.5 + news_hot_score_7d * 1 + author_news_daily_click_cnt * 1 + news_daily_views_cnt * 1 + news_like_rate_3d_normalize * 0.8. After syntax validation, the expression is saved, published, and the service node is notified to hot-update the model. The online service parses this expression in real-time during the scoring phase, retrieves the corresponding feature values ​​from the feature service, and calculates the score, all without requiring service restarts or code writing.

[0044] In this embodiment, the feature management module includes: Feature classification management: The recommendation system predefines three types of metadata dimensions: Feature types: USER (user features), ITEM (content features), GLOBAL (global statistical features); Feature generation method (sourceType): ORIGINAL (original feature), CROSS (cross feature), DERIVED (derived feature); Feature category tags: such as user category, news category, video category, live streaming category, advertising category, etc.; When registering features, feature producers (such as business services, Flink jobs) submit feature metadata through the platform API. The system persists it to the feature metadata database (such as MySQL) and displays it on the front end in the form of multi-level tags, supporting filtering by any combination of dimensions.

[0045] Feature search functionality: The front-end provides a search function. After the user enters keywords, the system calls the API interface. The back-end supports fuzzy matching of the featureId, name, and description fields, and can add filtering conditions such as featureType and sourceType. The returned results include feature definitions, example values, update frequency, a list of associated models, and a list of referenced features, allowing users to quickly assess usability.

[0046] Real-time feature value calculation: To support the dynamic combination configuration of derived and cross features and the real-time calculation of their feature values, the recommendation system has designed a dedicated real-time feature value calculation submodule. The following is its implementation: Original Feature Registration and Metadata Management: During the feature registration phase, feature producers (such as business services or Flink jobs) submit original feature metadata (such as feature ID, data type, and update frequency) through the platform or API and persist it to a feature metadata database (such as MySQL). Feature values ​​are stored in an online feature service (such as Redis) to support low-latency access.

[0047] Derived and cross-feature registration: Business personnel select original features such as news_click_rate_7d (click rate of news in the past 7 days) according to business needs through the platform, select feature processing functions such as min_max_scale (range standardization function), complete feature metadata configuration, and submit them to the feature metadata database (such as MySQL).

[0048] Feature value acquisition and real-time calculation: The recommendation system retrieves the latest version information of features from the feature metadata database, parses feature dependencies to extract the required features, and calls an online feature service (such as Redis) to obtain the latest feature values. It then calls specific functions to perform the actual calculations. The calculation results are cached in memory for quick return on subsequent identical requests, avoiding duplicate calculations.

[0049] This also includes: an AI capability module; AI dialogue functionality: The recommendation system integrates large language models (such as Tongyi Qianwen and DeepSeek) to build a feature knowledge base: metadata (name, definition, calculation logic, examples) in the feature management module is vectorized and stored in a vector database (such as Milvus). When a user asks, "How to calculate a user's 7-day click-through rate?", the system first retrieves the most relevant feature entries, and then the LLM generates a natural language answer: "The feature name is user_click_rate_7d, based on the user's click behavior statistics over the past 7 days, updated hourly." The dialogue record can be used for subsequent optimization of feature documents.

[0050] AI model creation function: Users can ask the platform's AI assistant to "generate a dynamic expression model for 'information' with the goal of increasing user click-through rates"; the specific implementation steps include: Large model identifies user intent The large model calls the feature management submodule to obtain features and select the most suitable feature set. Generate candidate solutions based on user goals The results are displayed to the user, who can click the "Create" link to be automatically redirected to the model creation page. The features and expressions generated from the large model will be automatically populated and a model draft will be created, thus entering the model lifecycle management process.

[0051] This process reduces the traditional 2-3 day manual modeling time to within 10 minutes.

[0052] All modules are deployed using a microservice architecture and developed based on Spring Boot. Feature metadata is stored in MySQL, and feature values ​​are stored in Redis+HBase; Model files are stored in object storage (such as OSS), and the model service provides a low-latency scoring interface via gRPC. The AI ​​capability module is decoupled from the core platform through the API gateway, supporting independent scaling up and down.

[0053] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0054] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for managing the entire lifecycle of a model for recommendation engines, characterized in that, include: Step S1: Manage the metadata, version, and running status of various models in the recommendation system through the model management module; Step S2: Manage the feature assets used by various models in the recommendation system through the feature management module; Step S3: Configure the target model and corresponding feature values ​​based on the recommendation system according to user needs.

2. The model lifecycle management method for recommendation engines according to claim 1, characterized in that, Step S1 includes: Step S1.1: Classify and manage various types of models through the model classification submodule, including: machine learning models, deep learning models, and AI models; at the same time, store the metadata of each type of model, including: input feature requirements, output format, and version information; Step S1.2: Implement full lifecycle management of the model in the recommendation system through the model lifecycle management submodule, including: creation, testing, deployment, canary release and decommissioning; at the same time, feed back the feature sequences that the model depends on to the feature management module.

3. The model lifecycle management method for recommendation engines according to claim 2, characterized in that, The AI ​​model includes: acquiring user natural language query information, parsing the user natural language query information to obtain user intent; constructing a dynamic expression model based on user intent by configuring dynamic expressions; and performing full lifecycle management of the constructed dynamic expression model and the features used by the dynamic expression model.

4. The model lifecycle management method for recommendation engines according to claim 1, characterized in that, Step S2 includes: Step S2.1: Classify the features used by various models according to the type dimension and the source dimension through the feature classification submodule, and construct a feature directory tree based on the classified features for structured classification management; persistently store the feature metadata, including: data source, data type, update frequency, business domain, and definition description through the feature directory tree; Step S2.2: The feature search submodule searches for features based on the feature directory tree according to any one or more dimensions, including feature identifier, name, and business tag, thereby locating the target feature; Step S2.3: The real-time feature value calculation submodule dynamically calculates the target feature value according to user needs or model requirements.

5. The method for full lifecycle management of a model for a recommendation engine according to claim 1, characterized in that, Step S3 includes: configuring candidate models according to user needs, ranking the candidate models by scores using an offline model evaluation and optimization method based on A / B testing and / or user feedback, and recommending the candidate model with the highest score to the user.

6. A model lifecycle management system for recommendation engines, characterized in that, include: Module M1: Manages the metadata, version, and running status of various models in the recommendation system through the model management module; Module M2: Manages the feature assets used by various models in the recommendation system through the feature management module; Module M3: Configures the target model and corresponding feature values ​​based on user needs and the recommendation system.

7. The model lifecycle management system for recommendation engines according to claim 6, characterized in that, The module M1 includes: Module M1.1: Classifies and manages various models through the model classification submodule, including: machine learning models, deep learning models, and AI models; Module M1.2: Implements full lifecycle management of models in the recommendation system through the model lifecycle management submodule, including: creation, testing, deployment, canary release, and decommissioning.

8. The model lifecycle management system for recommendation engines according to claim 7, characterized in that, The AI ​​model includes: acquiring user natural language query information, parsing the user natural language query information to obtain user intent; constructing a dynamic expression model based on user intent by configuring dynamic expressions; and performing full lifecycle management of the constructed dynamic expression model and the features used by the dynamic expression model.

9. The model lifecycle management system for recommendation engines according to claim 6, characterized in that, The module M2 includes: Module M2.1: The feature classification submodule categorizes the features used by various models according to the type and source dimensions, and constructs a feature directory tree based on the categorized features for structured classification management; the feature directory tree is used to persistently store feature metadata, including: data source, data type, update frequency, business domain, and definition description. Module M2.2: The feature search submodule searches for features based on the feature directory tree according to any one or more dimensions, including feature identifier, name, and business tag, thereby locating the target feature; Module M2.3: The real-time feature value calculation submodule dynamically calculates the target feature values ​​according to user needs or model requirements.

10. The model lifecycle management system for recommendation engines according to claim 6, characterized in that, The module M3 includes: configuring candidate models according to user needs, ranking the candidate models by scores using an offline model evaluation and optimization method based on A / B testing and / or user feedback, and recommending the candidate model with the highest score to the user.

Images