Feature mining method, recommendation method and related device and system

Abstract

One or more embodiments of the invention provide a feature mining method, a recommendation method and a related device and system. By calling a preset large model, exploratory feature mining is performed on the original data based on a first mining prompt word under the condition that a target feature is not set by utilizing the understanding and reasoning capability of the preset large model for knowledge in each domain, and exploratory features are obtained; and performing validity analysis on the exploratory features to determine effective features. When exploratory feature mining is executed, conventional feature mining can be carried out based on the preset feature name list; and recording the effective features into a preset feature name list to realize online learning. The effectiveness analysis on the exploratory features comprises the steps of carrying out formal effectiveness analysis on feature formats, feature values and the like, and carrying out substantial effectiveness analysis on the basis of an offline AB experiment and an online AB experiment. The feature mining method provided by the embodiment of the invention can be realized based on an Agent architecture.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of information processing technology, and in particular, to a feature mining method, a recommendation method, and related devices and systems. Background Art

[0002] The popularity of the Internet has brought a large amount of information to users, satisfying their demand for information. However, the increase in the amount of information also requires users to spend more time to find the information they really need. In response to this, personalized recommendation systems have emerged. They can analyze the interests and preferences of each user based on feature mining technology, and predict the information and products that users are interested in through personalized calculations. They are widely used in news recommendations, business recommendations, entertainment recommendations, decision support and other scenarios.

[0003] In the recommendation system, feature mining is a crucial link, which has a great impact on the prediction ability of the recommendation system and user experience. However, in related technologies, feature mining usually requires a lot of manual participation. Designing and selecting meaningful features for target application scenarios is not only time-consuming and labor-intensive, but also difficult to respond to data changes and difficult to be compatible with different fields or scenarios. Summary of the invention

[0004] In order to improve the efficiency and compatibility of feature mining technology, one or more embodiments of this specification provide a feature mining method, a recommendation method, and related devices and systems.

[0005] In a first aspect, one or more embodiments of this specification provide a feature mining method, including:

[0006] Obtaining a first mining prompt word;

[0007] Calling a preset large model, performing exploratory feature mining on the original data based on the first mining prompt word, and obtaining at least one exploratory feature;

[0008] The validity analysis is performed on the exploratory features to determine the effective features.

[0009] In a possible implementation, the feature mining method further includes:

[0010] Obtaining a second mining prompt word and a preset feature name list;

[0011] The preset large model is called, and conventional feature mining is performed on the original data based on the second mining prompt word and the preset feature name list to obtain at least one conventional feature.

[0012] In a possible implementation, the feature mining method further includes:

[0013] The feature name of the valid feature is recorded in the preset feature name list.

[0014] In a possible implementation, the performing validity analysis on the exploratory feature includes:

[0015] Conduct formal validity analysis on the exploratory features described;

[0016] The formal validity analysis includes at least one of the following: determining whether the format of the exploratory feature complies with a preset format, and determining whether the feature value of the exploratory feature is not empty.

[0017] In a possible implementation, the performing validity analysis on the exploratory feature includes:

[0018] Based on the exploratory features, construct and execute offline AB experiments and / or online AB experiments;

[0019] Obtaining experimental results of the offline AB experiment and / or the online AB experiment, and performing substantial effectiveness analysis on the exploratory features according to the experimental results;

[0020] The substantive validity analysis includes determining the contribution of the exploratory features to the experimental results.

[0021] In a possible implementation, the step of constructing and executing an offline AB experiment includes:

[0022] Obtain historical operation data of the target service system;

[0023] Constructing and executing the offline AB experiment according to the historical operation data and the exploratory features;

[0024] Wherein, the target service system includes a recommendation system for making project recommendations based on the preset large model.

[0025] In a possible implementation, the step of constructing and executing an online AB experiment includes:

[0026] Based on the exploratory features, the preset large model is subjected to fusion training;

[0027] The preset large model before the fusion training is used as a control group, and the preset large model after the fusion training is used as an experimental group, to construct and execute the online AB experiment.

[0028] In a second aspect, one or more embodiments of this specification provide a recommendation method, including:

[0029] According to the feature mining method described in the first aspect, feature mining is performed on user data and / or project data to obtain user features and / or project features;

[0030] Recommend items to target users based on the user characteristics and / or item characteristics.

[0031] In a third aspect, one or more embodiments of this specification further provide a feature mining device, including:

[0032] A first acquisition unit is used to acquire a first mining prompt word

[0033] A first mining unit is used to call a preset large model to perform exploratory feature mining on the original data based on the first mining prompt word to obtain at least one exploratory feature;

[0034] The feature analysis unit is used to perform validity analysis on the exploratory features to determine valid features.

[0035] In a possible implementation, the feature mining device further includes:

[0036] A second acquisition unit, used to acquire a second mining prompt word and a preset feature name list;

[0037] The second mining unit is used to call the preset large model, perform conventional feature mining on the original data based on the second mining prompt word and the preset feature name list, and obtain at least one conventional feature.

[0038] In a possible implementation, the feature mining device further includes:

[0039] The feedback unit is used to record the feature name of the effective feature into the preset feature name list.

[0040] In a possible implementation, the feature analysis unit includes:

[0041] A formal analysis unit, used for performing formal validity analysis on the exploratory features;

[0042] The formal validity analysis includes at least one of the following: determining whether the format of the exploratory feature complies with a preset format, and determining whether the feature value of the exploratory feature is not empty.

[0043] In a possible implementation, the feature analysis unit includes:

[0044] A substantial analysis unit, configured to construct and execute an offline AB experiment and / or an online AB experiment based on the exploratory feature, obtain experimental results of the offline AB experiment and / or the online AB experiment, and perform substantial effectiveness analysis on the exploratory feature according to the experimental results;

[0045] The substantive validity analysis includes determining the contribution of the exploratory features to the experimental results.

[0046] In a possible implementation, the substantial analysis unit includes:

[0047] An offline analysis subunit, configured to obtain historical operation data of a target service system, and to construct and execute the offline AB experiment according to the historical operation data and the exploratory features;

[0048] Wherein, the target service system includes a recommendation system for making project recommendations based on the preset large model.

[0049] In a possible implementation, the substantial analysis unit includes:

[0050] The online analysis subunit is used to perform fusion training on the preset large model based on the exploratory features, and to construct and execute the online AB experiment by taking the preset large model before the fusion training as a control group and the preset large model after the fusion training as an experimental group.

[0051] In a fourth aspect, one or more embodiments of this specification provide a feature mining device, including:

[0052] A planning module, used to obtain and store an execution plan formed based on the feature mining method described in the first aspect;

[0053] The execution module is used to execute relevant operations according to the execution plan to realize feature mining of the original data.

[0054] In a possible implementation, the feature mining device further includes at least one of the following:

[0055] A memory module, used to store the raw data to be mined and / or a list of preset feature names;

[0056] A configuration module, used to obtain and store attribute configuration information to influence the behavior of the preset large model during the feature mining process performed by the execution module; the attribute configuration information includes information describing the role, goal, and capability of the feature mining device;

[0057] The observation module is used to collect the experimental results of the offline AB experiment and / or the online AB experiment, and trigger the execution module to perform the following operations: perform substantive effectiveness analysis on the exploratory features according to the experimental results.

[0058] In a possible implementation, the execution module is configured with at least one of the following application interfaces:

[0059] A large model interface, which is used to trigger a preset large model to perform exploratory feature mining and / or conventional feature mining to obtain exploratory features and / or conventional features when called by the execution module;

[0060] A memory read-write interface, used for reading the original data and the preset feature name list from the memory module when called by the execution module, or recording the feature names of the valid features in the exploratory features into the preset feature name list;

[0061] A training model interface, used to trigger the model training platform to perform fusion training on the preset large model based on the exploratory features when called by the execution module;

[0062] An AB experiment interface, used for triggering the AB experiment platform to construct and execute an offline AB experiment or an online AB experiment based on the exploratory features when called by the execution module;

[0063] The data analysis interface is used to trigger the data analysis system to analyze the exploratory features and / or experimental results when called by the execution module.

[0064] In a fifth aspect, one or more embodiments of this specification provide a recommendation system, including:

[0065] A feature mining unit, used to perform feature mining on the original data according to the feature mining method described in the first aspect;

[0066] The item recommendation unit is used to recommend items to users based on the features mined by the feature mining unit.

[0067] In a sixth aspect, one or more embodiments of this specification provide a recommendation system, including:

[0068] A feature mining device, used for performing feature mining on raw data;

[0069] An item recommendation unit, used to recommend items to users based on the features mined by the feature mining device;

[0070] Wherein, the feature mining device includes the feature mining device described in the third aspect or the fourth aspect.

[0071] In the seventh aspect, one or more embodiments of the present specification also provide an electronic device, comprising a memory and a processor; the memory is used to store a computer program product; the processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, the method of the first aspect or the second aspect mentioned above is implemented.

[0072] In an eighth aspect, one or more embodiments of the present specification further provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed, the method of the first aspect or the second aspect described above is implemented.

[0073] To sum up, first of all, this manual, without pre-setting the target features, calls the preset large model, utilizes its understanding and reasoning ability of knowledge in various fields, and performs exploratory feature mining on the original data based on the first mining prompt word, so as to discover as many valuable features in the original data as possible; such feature mining method can be applied to any scenario or field, will not limit the feature mining ability due to different domain expertise, does not require manual design for different fields, realizes automated feature mining that is universal in all fields, improves the efficiency and compatibility of feature mining technology, and can respond to data changes in corresponding scenarios or fields in a timely manner, automatically mine new features that may exist in the changed data, and ensure the service effect and user experience of related service systems.

[0074] Secondly, due to the preset large model's ability to understand natural language, it can ensure that the exploratory features mined are meaningful features, and there will be no meaningless features that are difficult to understand; at the same time, this embodiment also performs validity analysis on the mined exploratory features, retains the valid features, avoids a substantial increase in feature dimensions, and thus avoids the degradation of model performance caused by dimensionality explosion.

[0075] Thirdly, while performing exploratory feature mining, this embodiment can also perform conventional feature mining based on a preset feature name list through a preset large model, to ensure that the common and foreseeable target features listed in the preset feature name list are accurately mined, and provide usable features to the relevant service system in a timely manner; the exploratory features that have been analyzed and found to be effective can also be recorded in the above-mentioned preset feature name list for conventional feature mining of the next mining task, thereby realizing automatic feedback and learning of the feature mining process, eliminating the need for manual summary and accumulation of feature mining experience, reducing the dependence of the feature mining process on manual experience, improving the degree of automation of the feature mining process, and enhancing the efficiency and versatility of feature mining.

[0076] This embodiment performs validity analysis on exploratory features through offline AB experiments and online AB experiments, thereby ensuring the credibility of the analysis results.

[0077] In addition, this embodiment can realize full-process automated feature mining based on a multimodal large model and an intelligent agent, and can continuously learn and accumulate effective features. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions of one or more embodiments of the present specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of one or more embodiments of the present specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0079] Figure 1 A flowchart of a feature mining method provided for one or more embodiments of this specification;

[0080] Figure 2 A flowchart of a recommended method provided for one or more embodiments of this specification;

[0081] Figure 3 A structural block diagram of a feature mining device provided for one or more embodiments of this specification;

[0082] Figure 4 A structural block diagram of another feature mining device provided for one or more embodiments of this specification;

[0083] Figure 5 A structural block diagram of another feature mining device provided for one or more embodiments of this specification;

[0084] Figure 6 A workflow diagram of a feature mining device provided for one or more embodiments of this specification;

[0085] Figure 7 A structural block diagram of a recommendation system provided for one or more embodiments of this specification;

[0086] Figure 8 A structural block diagram of an electronic device provided for one or more embodiments of this specification. DETAILED DESCRIPTION

[0087] One or more embodiments of the present invention are further described in detail below through the accompanying drawings and examples. Through these descriptions, the features and advantages of one or more embodiments of the present invention will become clearer and more specific.

[0088] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise noted.

[0089] In addition, the technical features involved in different implementations of one or more embodiments of this specification described below can be combined with each other as long as they do not conflict with each other.

[0090] To facilitate understanding, the application scenarios of the technical solutions provided by one or more embodiments of this specification are first described below.

[0091] Informatization is a major trend in the development of today's era. People generally obtain various types of information through various applications, such as news clients, video clients, shopping clients, etc. These applications can be regarded as recommendation systems, which recommend products, content, services and other items that users may be interested in. They collect relevant data, such as search keywords entered by users, users' historical behaviors, users' registration information, and information about recommended items, and then perform feature mining on these data to obtain features used to describe user characteristics and item characteristics. Based on these features, through collaborative filtering, deep school and other algorithms, the relationship between users and items is calculated, and then personalized recommendation results are generated for each user based on the relationship. It can be seen that feature mining is an important part of the recommendation system. Accurate and comprehensive feature mining can better discover the characteristics of users and items, so that the subsequent recommendation algorithm can more accurately analyze the relationship between users and items, thereby obtaining recommendation results that are more in line with user interests and preferences, and improving user experience.

[0092] In related technologies, in order to accurately mine user and project features, relevant technical personnel are usually required to pre-evaluate and decide which types of features to mine. Related large models or feature mining methods mine features based on manual evaluation results, including determining whether the user / project has features of a preset type, and what the specific feature value is. In different application scenarios, users and recommended projects may have some personalized features in the corresponding scenarios, so relevant personnel are required to evaluate and configure the types of features to be mined for different scenarios, which is time-consuming and labor-intensive.

[0093] In view of this, in order to improve the efficiency and compatibility of feature mining technology, one or more embodiments of this specification provide a feature mining method, a recommendation method and related devices and systems, which are described in detail below.

[0094] Figure 1 A flow chart of a feature mining method provided by an embodiment of this specification. Figure 1 , the feature mining method includes the following steps.

[0095] Step 102, obtaining a first mining prompt word;

[0096] The first mining prompt word, ie, the exploratory mining prompt word, may be stored in advance in a related storage unit in the form of a configuration file, etc., so that when the feature mining method is executed, the first mining prompt word is directly read from the related storage unit.

[0097] Step 104, calling a preset large model, performing exploratory feature mining on the original data based on the first mining prompt word, and obtaining at least one exploratory feature;

[0098] The first mining prompt word is used as input data of the preset large model to prompt the preset large model to perform exploratory mining tasks, that is, without giving the target feature, the preset large model automatically discovers valuable features in the original data using the world knowledge it has mastered. The first mining prompt word is irrelevant to the specific application results of feature mining. Therefore, through the above exploratory mining method, relevant technicians no longer need to predetermine which types of features (i.e., target features) to be mined for different application scenarios, and the mining task is completely completed automatically by the large model.

[0099] The above-mentioned preset large model can adopt a multimodal large model, that is, a large neural network model with multiple types of data processing capabilities integrated based on a unified framework. For example, on the basis of the large language model (LLM), an image encoder, a video encoder, an audio encoder, an image generator, a video generator, an audio generator, etc. are integrated to form a multimodal large model, which can convert the input data of any modality (text, image, video, audio, etc.) into a feature representation that can be understood by the machine, and convert the output of the LLM backbone module into output data of any modality according to application requirements. In this embodiment, the preset large model adopts a multimodal large model, and the original data to be mined can be different data modalities such as images, texts, and audios, and the preset large model can effectively interact and integrate information between these modalities, better and more comprehensively perceive various information in the application scenario, so as to mine more valuable features.

[0100] Optionally, the first mining prompt word may specifically include information such as the role of the large model, task objectives, output data format, etc. For example, the first mining prompt word may be as follows:

[0101] "You are a feature mining expert. You need to give full play to your professional knowledge and language or text exploration ability to mine as many features as possible. I will provide you with relevant raw data (text, pictures). You need to follow the steps below:

[0102] 1. Extract all the features you can understand from the text

[0103] 2. Extract all the features you can understand from the image

[0104] 3. Combine text and pictures to extract features

[0105] Output format: "[feature name|feature value|reason, feature name|feature value|reason]"

[0106] Original text: Text XXX

[0107] Original image: image"

[0108] After the above prompt words are input into the preset big model, the preset big model can analyze and derive one's role as "feature mining expert", the task goal is "to use all the knowledge and abilities you have mastered to mine as many features as possible from raw data in the form of text, pictures, etc.", and the mined features are output in the format of "[feature name|feature value|reason, feature name|feature value|reason]".

[0109] Step 106: Perform validity analysis on the exploratory features to determine valid features.

[0110] In this embodiment, the exploratory features mined by the preset large model in step 104 are analyzed for effectiveness, and effective features that can be applied to the application scenarios corresponding to the original data (such as project recommendation scenarios) are determined, and the remaining invalid features can be discarded. For example, through step 104, the preset large model may mine 10 exploratory features, and after analysis in step 106, 4 of them are valid features, then only these 4 valid features need to be retained and recorded; while the remaining 6 features are invalid and can be ignored and not recorded.

[0111] That is to say, in this embodiment, effective exploratory features are finally retained, which avoids a substantial increase in feature dimensions, thus avoiding the degradation of model performance caused by dimensionality explosion; at the same time, since the preset large model has reasoning and understanding capabilities, it can ensure that the mined features are interpretable. Therefore, this embodiment realizes automated feature mining and improves feature mining efficiency through an exploratory mining method based on the preset first mining prompt word; at the same time, by utilizing the knowledge of various fields possessed by the preset large model itself, it can be compatible with different application scenarios.

[0112] In a possible implementation, the feature mining method may further include the following steps:

[0113] Step 108, obtaining a second mining prompt word and a preset feature name list;

[0114] Step 110, calling the preset large model, performing conventional feature mining on the original data based on the second mining prompt word and the preset feature name list, and obtaining at least one conventional feature.

[0115] The second mining prompt, i.e., the conventional mining prompt, is used to prompt the preset large model to mine the original data with the features recorded in the preset feature name list as the target features. The second mining prompt may specifically include information such as the role, task objectives, and output data format of the large model. For example, the second mining prompt may be as follows:

[0116] "You are an expert in feature mining in the e-commerce field. You need to give full play to your expertise in the e-commerce field and perform feature mining according to the feature name list I give you. I will provide you with relevant raw data (text, pictures). You need to follow the steps below.

[0117] 1. Fully understand the meaning of the feature names I provide in the e-commerce field

[0118] 2. Combine the original text and pictures to extract all the features given the feature names

[0119] Output format: "[feature name|feature value|reason, feature name|feature value|reason]"

[0120] Original text: Original text XXX

[0121] Original image: image"

[0122] In different application scenarios, the field type specified in the above second mining prompt word can be modified according to needs, such as "e-commerce", "social", etc., to prompt the preset large model to more accurately understand the meaning of the feature name in the current application scenario and improve the accuracy of feature mining.

[0123] The target features recorded in the above-mentioned preset feature name list may include common features of different application scenarios, such as user behavior characteristics (clicks, favorites, sharing, browsing time, ratings of products, etc.), user relationship characteristics (following each other, liking each other, watching the same movie, being in the same interest group, etc.), user and project attributes / labels (user's address, occupation, etc., project type, brand, color, etc.), etc.

[0124] In step 110, the second mining prompt word and the preset feature name list obtained in step 108 can be integrated and input into the preset large model together, so that the preset large model can mine from the original data according to the target features listed in the preset feature name list, determine the feature values ​​corresponding to each user or project on each target feature, and realize conventional feature mining.

[0125] From the above description, it can be seen that the feature mining method provided in this embodiment can, on the one hand, use the preset feature name list to perform routine feature mining, to ensure that the common and foreseeable target features listed in the preset feature name list are accurately mined, and to provide usable features to the relevant service system in a timely manner; on the other hand, it can also use the knowledge of various fields and the ability to understand and reason mastered by the preset large model to perform exploratory mining without limiting the feature names, so as to discover as many effective features as possible in the original data, so that only the mining prompt words and the preset feature name list need to be set to realize automatic feature mining that is universal in all fields, without the need to manually design and select meaningful features for different fields or application scenarios, thereby improving the efficiency and compatibility of feature mining technology.

[0126] In a possible implementation, the feature mining method may further include the following steps:

[0127] Step 112: Record the feature name of the valid feature into the preset feature name list.

[0128] In this embodiment, the valid features among the exploratory features determined by the analysis in step 106 are added to the preset feature name list, so that when feature mining is performed next time, these valid features can be directly obtained by conventional feature mining, that is, automatic feedback learning of the target features is realized, and there is no need to manually summarize and accumulate feature mining experience, which reduces the dependence of the feature mining process on manual experience, improves the degree of automation of the feature mining process, and improves the efficiency and versatility of feature mining.

[0129] In a possible implementation, the validity analysis of the exploratory features in step 106 may be performed by at least one of the formal validity analysis described in step 1062 and the substantive validity analysis described in step 1064. The two analysis methods are described in detail below.

[0130] Step 1062, performing formal validity analysis on the exploratory features;

[0131] The formal validity analysis includes at least one of the following: determining whether the format of the exploratory feature complies with a preset format, and determining whether the feature value of the exploratory feature is not empty.

[0132] The above-mentioned formal validity analysis is to analyze whether the exploratory features are reasonable and valid in form. In one possible implementation, the large model's own ability to understand and analyze language can be used to determine whether each exploratory feature is formally valid. In another possible implementation, when the amount of data is large and the number of mined exploratory features is large, the mined exploratory features can be stored in a preset database in a structured form to facilitate subsequent processing, reduce cache pressure, etc.; in this way, in step 1062, the database can be queried based on the Structured Query Language (SQL) to determine whether each exploratory feature temporarily stored in the preset database is formally valid.

[0133] Step 1064, constructing and executing an offline AB experiment and / or an online AB experiment based on the exploratory features;

[0134] Step 1066, obtaining the experimental results of the offline AB experiment and / or the online AB experiment, and performing substantial effectiveness analysis on the exploratory features according to the experimental results;

[0135] The substantive validity analysis includes determining the contribution rate of the exploratory features to the experimental results.

[0136] AB experiment, also known as control experiment. For this embodiment, AB experiment is used to determine whether each exploratory feature is an effective feature. From a macro perspective, it is to determine whether each exploratory feature can improve the performance of the relevant large model or system, or improve the user experience.

[0137] Taking the application scenario of feature mining in the recommendation system as an example, the purpose of mining features is to use these features to estimate CTR (Click-Through-Rate) and other indicators that can be used to characterize the user's satisfaction with the recommendation results. Specifically, CTR is the probability that each item is clicked by each user after it is recommended to each user; the higher the CTR, the higher the probability that the user clicks on the item, that is, the higher the user's interest in the item. Correspondingly, this embodiment conducts a substantive effectiveness analysis on the above-mentioned exploratory features through AB experiments, that is, the recommendation system that integrates the above-mentioned exploratory features is used as the experimental group, and the original recommendation system that does not integrate the above-mentioned exploratory features is used as the control group, to determine whether each exploratory feature can improve the CTR and other indicators of the experimental group relative to the control group, and to what extent, that is, to obtain the contribution rate of each exploratory feature to the CTR improvement of the experimental group; exploratory features whose contribution rate exceeds a certain threshold can be regarded as effective features.

[0138] In the actual implementation process, it takes a long time to conduct substantive effectiveness analysis through AB experiments, especially online AB experiments, which require a certain amount of time to collect user feedback results, and the more exploratory features to be analyzed, the more complex the experiments are, and the more resources are occupied. In view of this, in a possible implementation method, in order to improve the analysis efficiency, for each exploratory feature mined in step 104, a formal effectiveness analysis can be first performed through step 1062, and then for the formally valid exploratory features, a substantive effectiveness analysis can be performed through step 1064 to obtain valid features that are both formally and substantially qualified; and features that are unqualified in form and substantively unqualified are all invalid features, which can be discarded to avoid wasting storage resources. At the same time, only valid features are recorded and used, and invalid features will not be integrated into application scenarios such as related large models or recommendation systems, ensuring that the performance of related models or systems is positively improved.

[0139] In a possible implementation, the construction and execution of the offline AB experiment in step 1064 may specifically include the following steps:

[0140] Step 10642, obtaining historical operation data of the target service system;

[0141] Step 10644, constructing and executing the offline AB experiment according to the historical operation data and the exploratory features;

[0142] The target service system, i.e., the scenario in which the feature mining method of this embodiment is applied, may specifically include a variety of different types, such as a recommendation system that recommends items based on the preset large model, or a search system based on the preset large model. In this embodiment, the historical operation data of the target service system is used to conduct an offline AB experiment, which may specifically be to divide the historical operation data into an experimental group and a control group according to whether the historical operation data has various exploratory features, and then calculate and compare the CTR and other indicators of the experimental group and the control group respectively to determine whether each exploratory feature brings about an improvement in the corresponding indicator.

[0143] For example, in an e-commerce recommendation system, its historical operation data may include the product recommendation results displayed to users each time, and the user's feedback information on each display result (such as clicking on a product in the recommendation results, returning to the homepage, or entering a new search keyword, etc.). Assume that one of the exploratory features of the above-mentioned potential recommendation system mined by this embodiment is "brand awareness". For each product, the size of the feature value of the exploratory feature "brand awareness" is the brand awareness of the product; for users, through offline AB experiments, it is determined whether the user is more interested in products with higher brand awareness. Then, multiple operation data L containing products with higher brand awareness in the recommendation results can be selected from the historical operation data of the e-commerce recommendation system. A As the experimental group, select multiple running data L in which the recommended results do not include products with high brand awareness B As the control group, according to the running data L in the two groups A and L B Calculate the user's actual CTR in the corresponding recommendation results (CTR = (number of clicks / number of impressions) * 100%). If the CTR of the experimental group is significantly higher than that of the control group, it can be said that brand awareness is highly correlated with the user's interest in the product, and can be used as an effective feature in the product recommendation process.

[0144] The aforementioned offline AB experiment uses the historical running data that has been generated to analyze the effectiveness of the features. To ensure the accuracy of the analysis results, this embodiment can also collect users' immediate feedback on the recommendation results based on exploratory features through online AB experiments. In view of this, in another possible implementation, the construction and execution of the online AB experiment described in step 1064 may specifically include the following steps:

[0145] Step 10646, performing fusion training on the preset large model based on the exploratory features;

[0146] Step 10648, using the preset large model before the fusion training as a control group and the preset large model after the fusion training as an experimental group, constructing and executing the online AB experiment.

[0147] Through the fusion training of step 10646, the preset large model can mine the above exploratory features more accurately, and can accurately capture the characteristic values ​​of each user and project on the relevant exploratory features under real-time traffic, so that the recommendation system can output recommendation results under the influence of relevant exploratory features.

[0148] Assume that the original preset large model before fusion training is recorded as M 0 , the new preset large model obtained after fusion training is recorded as M 1 , then based on the original preset large model M0 The recommendation system of 1 The recommendation systems of the two groups were used as the experimental group. The recommendation systems in the two groups were put online and run separately. The access traffic was randomly assigned to them, and the feedback results were collected. For example, for the recommendation results output by the two groups of recommendation systems each time, the user's browsing time, how many items were clicked, which items were clicked, the number of collections / shares / orders for the clicked items, etc.

[0149] In step 1066, based on the experimental results of the online AB experiment, a new preset large model M that incorporates exploratory features can be analyzed. 1 Whether it will increase users' click-through rate, collection rate and other indicators for recommended items, that is, whether it will increase users' interest and satisfaction with the recommended results, thereby determining whether the corresponding exploratory features are effective features.

[0150] In a possible implementation, the feature mining method may further include:

[0151] Based on the analysis results of step 1066, experimental decisions are made to optimize the traffic distribution of related service systems.

[0152] For example, when applied to a recommendation system, if offline and online AB experiments and analysis in step 1066 reveal that project recommendations based on a certain exploratory feature X can increase the user's click-through rate (CTR) of the recommendation results by 80%, this means that project recommendations based on the feature X can greatly increase the user's interest in the recommendation results, and the feature can be applied to all traffic; accordingly, the following experimental decision can be made: all recommendation requests of the current recommendation system are responded to based on the feature X (i.e., correlation calculation is performed based on the feature X to generate recommendation results).

[0153] For another example, if the analysis in step 1066 shows that item recommendation based on a certain exploratory feature X can increase the user's click-through rate (CTR) of the recommended results by 5%, it means that the user's interest in the recommended results is increased based on the item recommendation based on the feature X, but the increase is not large; accordingly, the following experimental decision can be made: further analyze the application scope of the feature X in combination with other features or indicators.

[0154] It should be noted that in actual application scenarios, for the same exploratory feature, multiple groups of offline or online AB experiments can be constructed for effectiveness analysis, or two or more exploratory features can be taken as a whole and analyzed for effectiveness through the same group of AB experiments, which is not limited in this embodiment. In addition, in different implementations, the above-mentioned offline AB experiment process and online AB experiment process can be executed one by one, or in parallel, or the offline AB experiment process can be executed first and then the online AB experiment process, which can be determined according to the number of exploratory features to be analyzed, the requirements of related systems, etc., which is not limited in this embodiment.

[0155] Based on the above-mentioned feature mining method, one or more embodiments of this specification also provide a recommendation method for recommending products, text or video content, services, etc. to users. Figure 2 A flowchart of a recommended method for one or more embodiments of this specification is provided, referring to Figure 2 , the recommended method may include the following steps:

[0156] Step 202, obtaining original data such as user data and data of recommended items;

[0157] The user data and project data can be obtained from the current application scenario as raw data, and feature mining is performed on them in the subsequent step 204. For example, when the recommendation method is applied to an e-commerce system, the user data is the data of each user registered in the current e-commerce system, including the user's registration information, operation records, browsing records, etc.; the project data is the data of each commodity on sale in the current e-commerce system, such as commodity attribute information, sales records, order evaluations, etc.

[0158] Step 204, based on the feature mining method described in any of the above embodiments, feature mining is performed on the original data such as the user data and project data to obtain corresponding user features and project features;

[0159] Step 206: Recommend items to the target user based on the user characteristics and item characteristics and a preset recommendation algorithm.

[0160] In actual application scenarios, users can interact with the recommendation system through the corresponding customer degree, and can execute the preset recommendation algorithm at any time according to the user's recommendation request, obtain the recommendation result and output it to the user's client. For example, after the user successfully logs in to an e-commerce client, the e-commerce client can automatically send a homepage recommendation request to the e-commerce server; or, the user enters the product keyword in the search bar of the e-commerce client, and the e-commerce client can send a product recommendation request with the keyword to the e-commerce server. At this time, the e-commerce server, as the executor of the recommendation method described in this embodiment, can execute step 206 when receiving the above-mentioned homepage recommendation request and the product recommendation request with keywords, generate and output the product recommendation result for the current user (i.e., the target user), and output the product recommendation result to the e-commerce client to display it to the current user.

[0161] In different application scenarios, the preset recommendation algorithm can adopt collaborative filtering algorithm, deep learning algorithm, etc.; project recommendations are made to target users through the preset recommendation algorithm, that is, the correlation between the user characteristics of the target user and the project characteristics of each project is calculated through the preset recommendation algorithm. The higher the correlation, the higher the target user's interest in the corresponding project; and then multiple projects with a correlation higher than a certain range can be screened out, arranged in a certain order (such as arranged in order of correlation from high to low) to form a recommendation list or a recommendation display page, and displayed to the target user.

[0162] In one possible implementation, the above steps 202 to 204 can be periodically executed at preset time intervals, that is, periodic feature mining is performed on the original data of the scenario where the recommendation system is located; in another possible implementation, the original data can also be monitored, and when changes in the original data are detected, the above steps 202 to 204 are started to be executed, and feature mining is performed on the changed original data.

[0163] Whether feature mining is performed periodically or after the original data is changed, its purpose is to timely discover the changes in the feature values ​​of the newly added effective features or the original conventional features in the original data generated by the current scene without wasting computing resources. For example, when new users or projects are added to the current scene, it is necessary to discover the feature values ​​of these new users and new projects on the relevant features through feature mining, so that personalized project recommendations can be made to the new users in the subsequent step 206, or the new projects can be accurately recommended to the appropriate users; for example, as time goes by, the projects of interest to the same user may change, and feature mining is needed to determine the current user features to ensure that the subsequent step 206 can obtain more accurate recommendation results. Therefore, in actual application scenarios, the trigger conditions or loop methods of the feature mining process can be set according to application requirements.

[0164] The user features and item features used for correlation calculation in step 206 of the above recommendation method may include conventional features obtained based on conventional feature mining as described above, or may include exploratory features obtained based on exploratory feature mining.

[0165] Among them, in the case of item recommendation based on exploratory features, the recommendation results obtained and the user's feedback on the recommendation results can be used as the experimental results of the online AB experiment in the aforementioned feature mining method to analyze the effectiveness of the exploratory features.

[0166] It can be seen that the feature mining method and recommendation method provided in this embodiment complement each other, that is, feature mining based on the feature mining method provided in this embodiment can improve the calculation accuracy of the correlation between users and items in the recommendation method, and the recommendation results obtained by the recommendation method and the user's operation records on the recommendation results can be used as feedback information of the feature mining method, which is used to analyze and determine the effective features in the mined exploratory features.

[0167] To sum up, the feature mining method and recommendation method provided in the embodiments of this specification can discover different effective features in various fields based on automated exploratory mining and analysis, without much manual intervention, and can realize full-process automation including feature mining, feature analysis, feedback learning and other steps. It has full-field versatility and can respond to data changes in actual application scenarios in a timely manner, ensuring high efficiency, high accuracy and personalization of feature mining and project recommendations.

[0168] It can be understood that the above embodiments are only examples and can be modified in actual implementation. Those skilled in the art can understand that the modification methods of the above embodiments without creative labor fall within the protection scope of one or more embodiments of this specification and will not be repeated in the embodiments.

[0169] Based on the same inventive concept, one or more embodiments of the present specification also provide a feature mining device. Since the principle of solving the problem by the device is similar to that of the aforementioned feature mining method, the implementation of the device can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0170] Figure 3 This is a structural block diagram of a feature mining device provided by one or more embodiments of this specification. Figure 3 As shown, the feature mining device 300 may include:

[0171] The first acquisition unit 301 is used to acquire a first mining prompt word

[0172] A first mining unit 302 is used to call a preset large model to perform exploratory feature mining on the original data based on the first mining prompt word to obtain at least one exploratory feature;

[0173] The feature analysis unit 303 is used to perform validity analysis on the exploratory features to determine valid features.

[0174] A second acquisition unit 304 is used to acquire a second mining prompt word and a preset feature name list;

[0175] The second mining unit 305 is used to call the preset large model, and perform conventional feature mining on the original data based on the second mining prompt word and the preset feature name list to obtain at least one conventional feature.

[0176] In one possible implementation, Figure 4 As shown, the feature mining device 300 may also include:

[0177] The feedback unit 306 is used to record the feature name of the effective feature into the preset feature name list.

[0178] Correspondingly, the second acquisition unit 304 acquires the preset feature name list, and specifically, the second acquisition unit 304 reads the preset feature name list from the feedback unit 306 .

[0179] In a possible implementation, the feature analysis unit 303 may include:

[0180] A formal analysis unit 3031, used to perform formal validity analysis on the exploratory features;

[0181] The formal validity analysis includes at least one of the following: determining whether the format of the exploratory feature complies with a preset format, and determining whether the feature value of the exploratory feature is not empty.

[0182] In a possible implementation, the feature analysis unit 303 may include:

[0183] The substantive analysis unit 3032 is used to construct and execute an offline AB experiment and / or an online AB experiment based on the exploratory feature, obtain experimental results of the offline AB experiment and / or the online AB experiment, and perform substantive validity analysis on the exploratory feature according to the experimental results;

[0184] The substantive validity analysis includes determining the contribution rate of the exploratory features to the experimental results.

[0185] In a possible implementation, the substantial analysis unit 3032 may specifically include at least one of an offline analysis subunit and an online analysis subunit.

[0186] The offline analysis subunit is used to obtain historical operation data of the target service system, and to construct and execute the offline AB experiment based on the historical operation data and the exploratory features;

[0187] Wherein, the target service system includes a recommendation system for making project recommendations based on the preset large model.

[0188] The above-mentioned online analysis subunit is used to perform fusion training on the preset large model based on the exploratory features, and to construct and execute the online AB experiment by using the preset large model before the fusion training as a control group and the preset large model after the fusion training as an experimental group.

[0189] Based on the same inventive concept, one or more embodiments of this specification also provide another feature mining device, which is implemented based on an agent. In the field of artificial intelligence, an agent refers to a system or software entity that can perceive its environmental information and make decisions or perform actions autonomously based on the perceived information to achieve a specific goal.

[0190] like Figure 5 As shown, based on the Agent framework, the feature mining device 400 described in this embodiment may include the following modules: a planning module 401, an execution module 402, a profile module 403, a memory module 404, and an observation module 405.

[0191] Among them, the planning module 401 is used to obtain and store the execution plan corresponding to the feature mining task; the execution plan corresponds to the execution process of each step in the feature mining method described in the previous embodiment. The execution plan can be pre-written by relevant technical personnel in the form of a configuration file and input into the planning module 401.

[0192] The execution module 402 is used to execute the feature mining related operations according to the execution plan stored in the planning module 401, that is, to convert the above execution plan into actual operations to realize feature mining. The execution module 402 is pre-configured with a variety of application interfaces. In the process of executing the feature mining related operations by the above execution plan, external applications or services can be used by calling the corresponding application interfaces.

[0193] The profile module 403 is used to obtain and store the property configuration information of the current device to influence the behavior of the preset large model during the feature mining process performed by the execution module 402; the property configuration information includes information describing the role, goal, capability, etc. of the feature mining device.

[0194] The memory module 404 is the main storage module of the feature mining device 400 and is used to store information or data such as the original data to be mined and a list of preset feature names, so that the information or data can be read at any time when the execution module 402 performs related operations.

[0195] The observation module 405 is used to collect information about the environment in which the feature mining device 400 is located, including collecting experimental results obtained by the execution module 402 performing offline AB experiments and / or online AB experiments, and triggering the execution module to continue to perform substantive effectiveness analysis operations on exploratory features based on the experimental results.

[0196] Before actual application, each module of the feature mining device 400 may be configured through the following configuration process:

[0197] (1) Input the configuration prompt words into the configuration module 403 to form a configuration file;

[0198] The configuration prompt includes the attribute configuration information of the feature mining device 400, such as the role, goal, and capability; other related modules can determine the attribute configuration information by reading the above configuration file from the configuration module 403 to guide the actions and decisions of the related modules.

[0199] For example, the specific content of the configuration prompt word may be:

[0200] “You are a senior feature mining expert named “FeatureExpert”, focusing on mining valuable features for recommendation systems. Your mission is to identify and extract features from complex data sets that can significantly improve recommendation accuracy and user satisfaction. Using your advanced data analysis and machine learning skills, you can deeply understand the subtle relationships between data and transform these insights into practical features in recommendation algorithms. Your tasks include but are not limited to:

[0201] Data exploration: In-depth analysis of raw data sets to identify potentially informative features.

[0202] Feature Engineering: Use advanced algorithms for feature extraction, selection, and transformation to optimize the input of the recommendation system.

[0203] Model fusion: Assists in the development of recommendation models and ensures that the discovered features can be effectively used in the model, thereby improving the quality of recommendations.

[0204] Performance evaluation: Regularly evaluate the impact of features on the performance of the recommendation system through experiments and analysis to ensure that the feature set remains effective.

[0205] ...Ultimately, as a "FeatureExpert", your goal is to ensure that the recommendation algorithm is continuously improved through precise feature mining activities, giving users a personalized and profound recommendation experience. "

[0206] (2) storing the relevant data in the memory module 404;

[0207] Intrinsic data that needs to be read or called during the feature mining process, such as original data of the application scenario, a list of preset feature names, etc., can all be stored through the memory module 404.

[0208] (3) configuring each preset application interface in the execution module 402;

[0209] According to the interaction requirements between the execution module 402 and other modules or external applications during the feature mining process, the corresponding application programming interface (API) is configured in the execution module 402, including:

[0210] A large model interface (large model API), which is used to trigger a preset large model to perform exploratory feature mining and / or conventional feature mining to obtain exploratory features and / or conventional features when called by the execution module;

[0211] A memory read / write API, which is used to read the original data and the preset feature name list from the memory module when called by the execution module, or to record the feature names of the valid features in the exploratory features into the preset feature name list;

[0212] A training model interface (training model API), which is used to trigger the model training platform to perform fusion training on the preset large model based on the exploratory features when called by the execution module;

[0213] The AB experiment interface (AB experiment API) is used to trigger the AB experiment platform to construct and execute an offline AB experiment or an online AB experiment based on the exploratory features when called by the execution module.

[0214] A data analysis interface (data analysis API) is used to trigger a data analysis system to analyze the exploratory features and / or experimental results when called by the execution module.

[0215] In addition, the first mining prompt word and the second mining prompt word required in the feature mining method described in this embodiment can also be configured in the execution module 402 so that it can be read and used at any time during the execution process.

[0216] (4) configuring the observation target in the observation module 405;

[0217] In this embodiment, the observation targets (ie, observation objects) of the observation module 405 include AB experiment results, CTR indicators of the recommendation system, and the like.

[0218] (5) The execution plan corresponding to the feature mining process is configured in the planning module 401.

[0219] The execution plan corresponds to the execution process of each step in the feature mining method described in the previous embodiment, so as to guide the actions of the execution module 402 to implement the above feature mining method.

[0220] Optionally, in the above execution plan, the feature mining task can be decomposed into the following subtasks: original data acquisition, conventional feature mining (corresponding to the aforementioned steps 108 to 110), exploratory feature mining (corresponding to the aforementioned steps 102 to 104), formal validity analysis (corresponding to the aforementioned step 1062), substantive validity analysis (corresponding to the aforementioned step 1064), feedback learning (corresponding to the aforementioned step 112); wherein, the substantive validity analysis can specifically include: offline AB experiments (corresponding to the aforementioned steps 10642 to 10644), feature fusion training (corresponding to the aforementioned step 10646), online AB experiments (corresponding to the aforementioned step 10648), observation of experimental results (corresponding to the aforementioned step 1066), experimental decision-making, etc.

[0221] In the above-mentioned feature mining device 400 based on the Agent architecture, the configuration module 403 defines the role, task objectives, etc. of the entire feature mining device; in order to achieve the task objectives set in the configuration module 403, the planning module 401 decomposes complex tasks into a series of simple subtasks to form an execution plan; the execution module 402, the memory module 404 and the observation module 405 respectively provide relevant capabilities for the implementation of the above-mentioned execution plan.

[0222] In one possible implementation, Figure 6 As shown, the specific process of the feature mining device 400 implementing feature mining based on the execution plan is as follows:

[0223] Step 601 , the execution module 402 calls the “Memory read and write API” to obtain the original data stored in the memory module 404 .

[0224] Step 602 , the execution module 402 calls the “Memory read / write API” to obtain the preset feature name list stored in the memory module 404 , and integrates it with the second mining prompt word to form a conventional feature mining prompt word P2 .

[0225] Step 603, the execution module 402 calls the "big model API" to input the above-mentioned conventional feature mining prompt word P2 and the original data into the preset big model, so that the preset big model performs conventional feature mining on the original data according to the instructions of P2, and obtains the conventional features already listed in the preset feature name list.

[0226] Among them, the conventional features mined in the above step 603 can be directly applied to relevant service systems to implement services such as project recommendation and search.

[0227] Step 604, the execution module 402 calls the "big model API" to input the exploratory feature mining prompt word P1 and the original data into the preset big model, so that the preset big model performs exploratory feature mining on the original data according to the instruction of P1 to obtain one or more exploratory features.

[0228] Among them, the above-mentioned exploratory feature mining prompt word P1 is the first mining prompt word. The exploratory features mined in the above step 604 may contain invalid features, so they cannot be directly applied to the relevant service system to avoid affecting the user experience. In this regard, this embodiment first performs validity analysis on these exploratory features, including formal validity analysis and substantive validity analysis. Formal validity analysis is a simple and basic analysis, such as determining whether the feature format is correct, whether the feature value is a null value, etc.; this embodiment provides two specific implementation methods of formal validity analysis, namely the following steps 6051 and 6052. Substantive validity analysis is to analyze whether each exploratory feature can have a positive impact on the relevant service system, such as improving the CTR of the recommendation system; this embodiment provides two implementation methods of substantive validity analysis, namely the offline AB experiment in the following step 6061 and the online AB experiment in step 6062, and analyzes the experimental results through step 607 to obtain the substantive validity analysis results.

[0229] In step 6051, the execution module 402 calls the "big model API" and uses the reasoning and understanding capabilities of the preset big model to perform formal validity analysis on each exploratory feature.

[0230] When the amount of original data is small and the number of mined features is small, the exploratory features obtained in step 604 do not need to be stored in an additional storage structure and flow directly to step 6051 to perform formal validity analysis on them through a preset large model.

[0231] In scenarios with large amounts of data, a large number of exploratory features are mined. To facilitate subsequent use, they can be stored in a preset database. Accordingly, a formal validity analysis is performed on them in the manner described in step 6052 below.

[0232] Step 6052, the execution module 402 calls the "big model API" to generate database query prompt words through the preset big model, and the execution module 402 then calls the "data analysis API" to query and analyze the exploratory features stored in the preset database according to the database query prompt words through the data analysis platform.

[0233] The database query prompt word may be a prompt word based on Structured Query Language (SQL). The above data analysis platform may adopt a database-related data processing system such as an OLTP (Online Transaction Processing) system.

[0234] Step 6061, the execution module 402 calls the "AB experiment API", inputs the historical operation data of the relevant service system into the AB experiment platform, and creates and executes an offline AB experiment based on the historical operation data through the AB experiment platform.

[0235] The above historical operation data may be pre-stored in the memory module 404 , and the execution module 402 may read the historical operation data from the memory module 404 by calling the “Memory read and write API”.

[0236] Step 6062, the execution module 402 calls the "training model API" to perform fusion training on the preset large model through the model training platform, and integrates the exploratory features to be analyzed into the preset large model; after the training is completed, the execution module 402 calls the "AB experiment API" again to build and execute an online AB experiment based on the preset large model before and after the fusion training through the AB experiment platform.

[0237] Among them, the service system formed based on the preset large model before fusion training can be used as the control group of the online AB experiment; the service system formed based on the preset large model after fusion training can be used as the experimental group of the online AB experiment.

[0238] Step 607, the observation module 405 collects the AB experiment results (including offline AB experiment results and / or online AB experiment results), and triggers the execution module 402 to call the "data analysis API" to analyze the above AB experiment results through the data analysis platform.

[0239] The above analysis results of the AB experiment results, that is, the analysis results of the substantive effectiveness analysis, may include the effectiveness of each exploratory feature, the system traffic allocation plan based on the effective features (that is, how much traffic is allocated to the service system that applies the effective feature, and how much traffic is still allocated to the original service system), etc.

[0240] Step 608, executing the experiment decision, that is, the execution module 402, according to the analysis result of the above step 607, takes the AB experiment offline (stops) when all the system traffic is allocated to the service system applying the above effective features.

[0241] Step 609 , feedback and online learning, that is, the execution module 402 calls the “Memory read and write API” to write the above valid features into the preset feature name list stored in the memory module 404 .

[0242] After the above-mentioned valid features are stored in the preset feature name list, the next time the feature mining task is executed, the above-mentioned valid features will be used as the target features for conventional feature mining. There is no need to go through exploratory mining and complex effectiveness analysis processes, thus realizing automatic feedback, learning and accumulation of the feature mining device.

[0243] In summary, this embodiment provides a feature mining device based on an Agent framework, which can realize the automated feature mining requirements of various scenarios and fields by completing the configuration process of each model at one time, without the need for different manual design interventions for different scenarios or fields. The large model itself can use its understanding and reasoning ability of knowledge in various fields to perform exploratory mining on atomic information in the original data, and can realize feature mining of unstructured data such as long texts and pictures based on a multimodal large model, and realize feedback and online learning by recording the effective features obtained by exploratory mining and analysis into a preset feature name list. Therefore, the feature mining device provided in this embodiment, based on the large model and Agent technology, can realize the automated advancement of the entire feature mining process, and has universality in all fields.

[0244] Based on the same inventive concept, one or more embodiments of this specification also provide a recommendation system. Figure 7 As shown, the recommendation system may include:

[0245] A feature mining unit 701 is used to perform feature mining on the original data according to the feature mining method described in any of the above embodiments;

[0246] The item recommendation unit 702 is used to execute a preset recommendation algorithm according to the features mined by the feature mining unit to recommend items to the user.

[0247] In one possible implementation, the recommendation system 700 may further include a data collection unit for collecting original data of the application environment of the recommendation system. For example, when the recommendation system is applied to an e-commerce platform, product data, user data, transaction data, etc. may be collected and stored for use at any time by the feature mining unit 701.

[0248] Based on the same inventive concept, one or more embodiments of this specification further provide a recommendation system, including:

[0249] A feature mining device, used for performing feature mining on raw data;

[0250] An item recommendation unit, used to recommend items to users based on the features mined by the feature mining device;

[0251] The feature mining device may be the feature mining device 300 or the feature mining device 400 described in the previous embodiment.

[0252] The feature mining device and recommendation system provided in this embodiment complement each other, that is, feature mining based on the feature mining device provided in this embodiment can improve the calculation accuracy of the correlation between users and items in the recommendation system, and the recommendation results obtained by the recommendation system and the user's operation records on the recommendation results can be used as feedback information of the feature mining device, which is used to analyze and determine the effective features among the exploratory features mined by it.

[0253] See also Figure 8 , Figure 8 This is a structural block diagram of an electronic device provided in one or more embodiments of this specification. Figure 8 As shown, the electronic device 500 may include a processor 501 and a memory 502; the memory 502 may be coupled to the processor 501. It is worth noting that the Figure 8 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0254] In a possible implementation, the functions of the feature mining device 300 or the feature mining device 400 may be integrated into the processor 501. The processor 501 may be configured to perform the following operations:

[0255] Obtaining a first mining prompt word;

[0256] Calling a preset large model, performing exploratory feature mining on the original data based on the first mining prompt word, and obtaining at least one exploratory feature;

[0257] The validity analysis is performed on the exploratory features to determine the effective features.

[0258] In another possible implementation, the feature mining device 300 or the feature mining device 400 can be configured separately from the processor 501. For example, the feature mining device 300 or the feature mining device 400 can be configured as a chip connected to the processor 501, and feature mining is implemented through the control of the processor 501.

[0259] In a possible implementation, the functions of the recommendation system 700 may be integrated into the processor 501. The processor 501 may be configured to perform the following operations:

[0260] Perform feature mining on original data such as user data and / or project data according to the aforementioned feature mining method to obtain user features and / or project features;

[0261] Recommend items to target users based on the user characteristics and / or item characteristics.

[0262] In another possible implementation, the recommendation system 700 may be configured separately from the processor 501 . For example, the recommendation system 700 may be configured as a chip connected to the processor 501 , and item recommendations may be implemented under the control of the processor 501 .

[0263] In addition, in some optional implementations, the electronic device 500 may also include: a communication module, an input unit, an audio processor, a display, a power supply, etc. It is worth noting that the electronic device 500 does not necessarily have to include Figure 8 In addition, the electronic device 500 may also include Figure 8 For components not shown, reference may be made to the prior art.

[0264] In some optional implementations, the processor 501 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of the electronic device 500.

[0265] The memory 502 may be, for example, a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The memory 502 may store the information related to the feature mining device 300 or the feature mining device 400 or the recommendation system 700, and may also store programs for executing the related information. The processor 501 may execute the program stored in the memory 502 to implement information storage or processing.

[0266] The input unit can provide input to the processor 501. The input unit is, for example, a key or a touch input device. The power supply can be used to provide power to the electronic device 500. The display can be used to display display objects such as images and text. The display can be, for example, an LCD display, but is not limited thereto.

[0267] The memory 502 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 502 may also be some other type of device. The memory 502 includes a buffer memory (sometimes referred to as a buffer). The memory 502 may include an application / function storage unit for storing application programs and function programs or processes for executing the operation of the electronic device 500 through the processor 501.

[0268] The memory 502 may also include a data storage unit for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit of the memory 502 may include various drivers for the communication function of the computer device and / or for executing other functions of the computer device (such as a messaging application, a contact book application, etc.).

[0269] The communication module is a transmitter / receiver that sends and receives signals via an antenna. The communication module (transmitter / receiver) is coupled to the processor 501 to provide input signals and receive output signals, which may be the same as in a conventional mobile communication terminal.

[0270] Based on different communication technologies, multiple communication modules may be provided in the same computer device, such as a cellular network module, a Bluetooth module and / or a wireless local area network module, etc. The communication module (transmitter / receiver) is also coupled to a speaker and a microphone via an audio processor to provide an audio output via the speaker and receive an audio input from the microphone, thereby realizing a common telecommunication function. The audio processor may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor is also coupled to the processor 501, so that recording can be performed on the machine through the microphone, and the sound stored on the machine can be played through the speaker.

[0271] One or more embodiments of this specification also provide a computer-readable storage medium capable of implementing all the steps in the feature mining method or recommendation method in the above embodiments, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all the steps in the feature mining method or recommendation method in the above embodiments are implemented. The specific steps can be found in the above embodiments, and will not be repeated here.

[0272] Although one or more embodiments of this specification provide method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0273] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, devices (systems) or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0274] One or more embodiments of the present specification are described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to one or more embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0275] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0276] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0277] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0278] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of one or more embodiments of this specification, rather than to limit them. Although one or more embodiments of this specification have been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of one or more embodiments of this specification, and they should all be included in the scope of the claims and the specification of one or more embodiments of this specification.

[0279] One or more embodiments of this specification are described above in conjunction with optional implementation methods, but these implementation methods are only exemplary and serve only as an illustration. On this basis, multiple replacements and improvements can be made to one or more embodiments of this specification, all of which fall within the scope of protection of one or more embodiments of this specification.

Claims

1. A feature mining method, characterized in that: include: Obtaining a first mining prompt word; Calling a preset large model, performing exploratory feature mining on the original data based on the first mining prompt word, and obtaining at least one exploratory feature; The validity analysis is performed on the exploratory features to determine the effective features.

2. The method according to claim 1, characterized in that Also includes: Obtaining a second mining prompt word and a preset feature name list; The preset large model is called, and conventional feature mining is performed on the original data based on the second mining prompt word and the preset feature name list to obtain at least one conventional feature.

3. The method according to claim 2, characterized in that Also includes: The feature name of the valid feature is recorded in the preset feature name list.

4. The method according to claim 1, characterized in that: The validity analysis of the exploratory features includes: Conduct formal validity analysis on the exploratory features described; The formal validity analysis includes at least one of the following: determining whether the format of the exploratory feature complies with a preset format, and determining whether the feature value of the exploratory feature is not empty.

5. The method according to claim 1, characterized in that: The validity analysis of the exploratory features includes: Based on the exploratory features, construct and execute offline AB experiments and / or online AB experiments; Obtaining experimental results of the offline AB experiment and / or the online AB experiment, and performing substantial effectiveness analysis on the exploratory features according to the experimental results; The substantive validity analysis includes determining the contribution of the exploratory features to the experimental results.

6. The method according to claim 5, characterized in that The construction and execution of offline AB experiments include: Obtain historical operation data of the target service system; Constructing and executing the offline AB experiment according to the historical operation data and the exploratory features; Wherein, the target service system includes a recommendation system for making project recommendations based on the preset large model.

7. The method according to claim 5, characterized in that The construction and execution of online AB experiments include: Based on the exploratory features, the preset large model is subjected to fusion training; The preset large model before the fusion training is used as a control group, and the preset large model after the fusion training is used as an experimental group, to construct and execute the online AB experiment.

8. A recommendation method, characterized in that: include: According to any one of claims 1 to 7, feature mining is performed on user data and / or project data to obtain user features and / or project features; Recommend items to target users based on the user characteristics and / or item characteristics.

9. A feature mining device, characterized in that: include: A first acquisition unit is used to acquire a first mining prompt word A first mining unit is used to call a preset large model to perform exploratory feature mining on the original data based on the first mining prompt word to obtain at least one exploratory feature; The feature analysis unit is used to perform validity analysis on the exploratory features to determine valid features.

10. The device according to claim 9, characterized in that Also includes: A second acquisition unit, used to acquire a second mining prompt word and a preset feature name list; The second mining unit is used to call the preset large model, perform conventional feature mining on the original data based on the second mining prompt word and the preset feature name list, and obtain at least one conventional feature.

11. The device according to claim 10, characterized in that Also includes: The feedback unit is used to record the feature name of the effective feature into the preset feature name list.

12. The device according to claim 9, characterized in that The feature analysis unit comprises: A formal analysis unit, used for performing formal validity analysis on the exploratory features; The formal validity analysis includes at least one of the following: determining whether the format of the exploratory feature complies with a preset format, and determining whether the feature value of the exploratory feature is not empty.

13. The device according to claim 9, characterized in that The feature analysis unit comprises: A substantial analysis unit, configured to construct and execute an offline AB experiment and / or an online AB experiment based on the exploratory feature, obtain experimental results of the offline AB experiment and / or the online AB experiment, and perform substantial effectiveness analysis on the exploratory feature according to the experimental results; The substantive validity analysis includes determining the contribution of the exploratory features to the experimental results.

14. The device according to claim 13, characterized in that The substantive analysis unit comprises: An offline analysis subunit, configured to obtain historical operation data of a target service system, and to construct and execute the offline AB experiment according to the historical operation data and the exploratory features; Wherein, the target service system includes a recommendation system for making project recommendations based on the preset large model.

15. The device according to claim 13, characterized in that The substantive analysis unit comprises: The online analysis subunit is used to perform fusion training on the preset large model based on the exploratory features, and to construct and execute the online AB experiment by taking the preset large model before the fusion training as a control group and the preset large model after the fusion training as an experimental group.

16. A feature mining device, characterized in that: include: A planning module, used to obtain and store an execution plan formed based on the feature mining method according to any one of claims 1 to 7; The execution module is used to execute relevant operations according to the execution plan to realize feature mining of the original data.

17. The device according to claim 16, characterized in that Also includes at least one of the following: A memory module, used to store the raw data to be mined and / or a list of preset feature names; A configuration module, used to obtain and store attribute configuration information to influence the behavior of the preset large model during the feature mining process performed by the execution module; the attribute configuration information includes information describing the role, goal, and capability of the feature mining device; The observation module is used to collect the experimental results of the offline AB experiment and / or the online AB experiment, and trigger the execution module to perform the following operations: perform substantive effectiveness analysis on the exploratory features according to the experimental results.

18. The device according to claim 17, characterized in that The execution module is configured with at least one of the following application interfaces: A large model interface, which is used to trigger a preset large model to perform exploratory feature mining and / or conventional feature mining to obtain exploratory features and / or conventional features when called by the execution module; A memory read-write interface, used for reading the original data and the preset feature name list from the memory module when called by the execution module, or recording the feature names of the valid features in the exploratory features into the preset feature name list; A training model interface, used to trigger the model training platform to perform fusion training on the preset large model based on the exploratory features when called by the execution module; An AB experiment interface, used for triggering the AB experiment platform to construct and execute an offline AB experiment or an online AB experiment based on the exploratory features when called by the execution module; The data analysis interface is used to trigger the data analysis system to analyze the exploratory features and / or experimental results when called by the execution module.

19. A recommendation system, characterized in that: include: A feature mining unit, configured to perform feature mining on raw data according to the feature mining method according to any one of claims 1 to 7; The item recommendation unit is used to recommend items to users based on the features mined by the feature mining unit.

20. A recommendation system, characterized in that: include: A feature mining device, used for performing feature mining on raw data; An item recommendation unit, used to recommend items to users based on the features mined by the feature mining device; Wherein, the feature mining device includes the feature mining device described in any one of claims 9 to 15, or the feature mining device includes the feature mining device described in any one of claims 16 to 18.

21. An electronic device, characterized in that: The electronic device comprises: a memory for storing a computer program product; A processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, the method described in any one of claims 1 to 8 is implemented.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed, implement the method described in any one of claims 1 to 8.

Images