Feature processing method, system and device

Abstract

The embodiment of the invention provides a feature processing method, system and device, and the method comprises the steps: responding to target primary key information inputted in a feature registration page, constructing a target statement corresponding to the target primary key information, executing the target statement, obtaining an initial feature, and loading the initial feature to a feature processing page for display; in response to a processing instruction submitted for the initial feature loaded in the feature processing page, determining an associated feature, and constructing a target feature according to the associated feature and the initial feature; configuring an offline investigation task for the target feature, and constructing a model training sample corresponding to the target feature by executing the offline investigation task; the model training samples are stored in a sample table, and the model training samples stored in the sample table are used for training a business model.

Description

Technical Field

[0001] The embodiments of this specification relate to the technical field of feature processing, and particularly to a feature processing method, system and device. Background Art

[0002] With the development of computer and Internet technologies, machine learning models are applied in more and more scenarios. In order to enable machine learning models to have strong performance in business scenarios, they usually need to be fully trained before being deployed to business scenarios for use. Before model training, it is necessary to obtain and integrate features to provide TB-level training samples for machine learning models. In this process, it is necessary to first extract raw data from different HIVE tables or other storage systems, and then preprocess this part of the data and splice it to generate feature vectors that meet model training, so as to import the feature vectors into the machine learning system to complete the training of the machine learning model. In the prior art, in the sample construction stage before model training, developers need to manually write SQL statements to obtain the required data from different HIVE tables. After obtaining the data, it is also necessary to process the data in combination with predefined logic. This process not only consumes a large amount of human resources, but also has the risk of processing errors due to different data sources. Therefore, an effective solution is urgently needed to solve the above problems. Summary of the Invention

[0003] In view of this, the embodiments of this specification provide a feature processing method. One or more embodiments of this specification also relate to a feature processing system, a feature processing device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0004] According to the first aspect of the embodiments of this specification, a feature processing method is provided, including:

[0005] In response to the target primary key information entered on the feature registration page, construct a target statement corresponding to the target primary key information, execute the target statement to obtain an initial feature, and load it to the feature processing page for display;

[0006] In response to the processing instruction submitted for the initial feature loaded on the feature processing page, determine the associated features, and construct a target feature based on the associated features and the initial feature;

[0007] Configure an offline research task for the target feature, and construct a model training sample corresponding to the target feature by executing the offline research task;

[0008] Store the model training sample in a sample table, where the model training samples stored in the sample table are used to train a business model.

[0009] According to the second aspect of the embodiments of the present specification, a feature processing system is provided, including a client and a server, including:

[0010] The client is configured to receive the target primary key information entered on the feature registration page and send the target primary key information to the server;

[0011] The server is configured to construct a target statement corresponding to the target primary key information, execute the target statement to obtain an initial feature, load the initial feature onto the feature processing page and send it to the client for display;

[0012] The client is configured to receive a processing instruction submitted for the initial feature loaded on the feature processing page and send the processing instruction to the server;

[0013] The server is configured to determine an associated feature in response to the processing instruction, and construct a target feature based on the associated feature and the initial feature; configure an offline research task for the target feature, construct a model training sample corresponding to the target feature by executing the offline research task; store the model training sample in a sample table, where the model training samples stored in the sample table are used to train a business model.

[0014] According to the third aspect of the embodiments of the present specification, a feature processing device is provided, including:

[0015] A construction module, configured to construct a target statement corresponding to the target primary key information in response to the target primary key information entered on the feature registration page, execute the target statement to obtain an initial feature and load it onto the feature processing page for display;

[0016] A determination module, configured to determine an associated feature in response to a processing instruction submitted for the initial feature loaded on the feature processing page, and construct a target feature based on the associated feature and the initial feature;

[0017] A configuration module, configured to configure an offline research task for the target feature, construct a model training sample corresponding to the target feature by executing the offline research task;

[0018] A storage module, configured to store the model training sample in a sample table, where the model training samples stored in the sample table are used to train a business model.

[0019] According to the fourth aspect of the embodiments of the present specification, a computing device is provided, including:

[0020] A memory and a processor;

[0021] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned feature processing method are implemented.

[0022] According to the fifth aspect of the embodiments of the present specification, a computer-readable storage medium is provided, which stores computer-executable instructions. When the instructions are executed by a processor, the steps of the above-mentioned feature processing method are implemented.

[0023] According to the sixth aspect of the embodiments of the present specification, a computer program product is provided, including a computer program or instructions. When the computer program or instructions are executed by a processor, the steps of the above-mentioned feature processing method are implemented.

[0024] For the feature processing method provided in this embodiment, in order to save human resources and ensure that data processing is error-free, a feature processing platform can be provided to developers, so as to realize that in the sample construction stage before model training, in response to the target primary key information entered on the feature registration page, construct the target statement corresponding to the target primary key information, which does not need to be written separately by developers, effectively improving the data processing efficiency. After obtaining the target statement, the target statement can be executed to obtain the initial feature and load it to the feature processing page for display; in response to the processing instruction submitted for the initial feature loaded on the feature processing page, the associated feature can be determined, and the target feature can be constructed according to the associated feature and the initial feature; realizing feature splicing can be completed in combination with the processing instruction of the developer, making the feature processing more flexible and stable. On this basis, an offline research task is configured for the target feature, and the model training sample corresponding to the target feature is constructed by executing the offline research task; then the model training sample can be stored in the sample table, where the model training sample stored in the sample table is used to train the business model. It is realized that the construction of the sample can be completed without a large amount of manual processing, effectively improving the sample construction efficiency and reducing the cost at the same time, so that the downstream service can read the sample from the sample table at any time to complete model training and improve the model training efficiency. Description of the Drawings

[0025] Figure 1 is a flowchart of a feature processing method provided by an embodiment of the present specification;

[0026] Figure 2a is a schematic diagram of the first page of feature registration in a feature processing method provided by an embodiment of the present specification;

[0027] Figure 2b is a schematic diagram of a feature registration page in a feature processing method provided by an embodiment of the present specification;

[0028] Figure 2cIt is a schematic diagram of a feature processing page in a feature processing method provided by an embodiment of this specification;

[0029] Figure 2d It is a schematic diagram of a task configuration page in a feature processing method provided by an embodiment of this specification;

[0030] Figure 2e It is a schematic diagram of a sample viewing page in a feature processing method provided by an embodiment of this specification;

[0031] Figure 3 It is a schematic structural diagram of a feature processing system provided by an embodiment of this specification;

[0032] Figure 4 It is a process timing diagram of a feature processing method provided by an embodiment of this specification;

[0033] Figure 5 It is a schematic structural diagram of a feature processing device provided by an embodiment of this specification;

[0034] Figure 6 It is a structural block diagram of a computing device provided by an embodiment of this specification. Detailed implementation manners

[0035] In the following description, many specific details are set forth in order to provide a thorough understanding of this specification. However, this specification can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of this specification. Therefore, this specification is not limited by the specific implementations disclosed below.

[0036] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a", "the", and "said" used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more of the associated listed items.

[0037] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0038] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0039] First, the noun terms involved in one or more embodiments of this specification are explained.

[0040] SQL: SQL (Structured Query Language) is the standard language for managing and operating relational databases. Through it, operations such as data query, insertion, update, deletion, and definition and control of data structures can be performed, thereby helping users efficiently store, retrieve, and manage large amounts of information.

[0041] Feature: A feature refers to the attributes, characteristics, or qualities of an object or a system, which can be used to identify and describe a certain specific property of the object. In data analysis and machine learning, features are usually individual variables used to represent samples, helping the model understand the key factors of the data; they play a crucial role in classifying, predicting, or pattern recognizing data.

[0042] Feature concatenation: It is the process of combining features from multiple feature sets or data sources to form a larger feature vector. In machine learning and data analysis, this technique aims to integrate multi-dimensional information, enabling the model to understand the data from a broader perspective, improving prediction accuracy and generalization ability. This is usually achieved through a simple horizontal connection operation, that is, element-wise combination of multiple feature vectors.

[0043] Sample: In machine learning, a sample is an independent data point composed of features and labels (optional), used to train and evaluate models in order to make accurate predictions or classifications on unseen new data.

[0044] Codegen: Codegen (code generation) refers to the technology of automatically generating source code. By generating executable code for a specific programming environment or architecture, it simplifies the development process and reduces human errors, thereby improving software development efficiency and consistency.

[0045] Sample lake table: A sample lake table is a data management structure used to store a large amount of raw and processed sample data in a data lake. These data can be flexibly accessed and used by machine learning models and analysis tools to support large-scale data analysis and modeling tasks.

[0046] Offline Feature Generation: Offline feature generation refers to the process of extracting useful information from historical data through pre-computation and storage of features during the data processing and preparation stage, for use in subsequent model training and prediction processes. This process typically runs in batch mode and does not require real-time calculation of feature values, which can effectively improve the efficiency of complex feature engineering, ensure the use of high-quality and relatively complex feature sets to enhance model performance, and at the same time reduce the computational load on the online system.

[0047] Big Data Platform: A big data platform is an integrated technology framework designed specifically for processing and analyzing massive amounts of data. It integrates functions such as data storage, processing, management, analysis, and visualization, and supports the entire process from data acquisition to insight extraction. By using distributed storage and computing technologies such as Hadoop and Spark, these platforms can efficiently process batch and real-time data, achieve complex data integration and analysis, and at the same time provide security and permission management to help enterprises and organizations gain profound business insights from data to optimize decision-making and strategies.

[0048] Machine Learning Platform: A machine learning platform is an ecosystem of integrated tools and services designed to support the development, training, deployment, and management of machine learning projects. It provides functions such as automated data preparation, feature engineering, model training, evaluation, and management to simplify the model development process and accelerate the implementation of machine learning solutions. With the support of visual interfaces, pre-built models, and libraries (such as TensorFlow, PyTorch), these platforms enable data scientists and engineers to collaborate more efficiently, build, test, and deploy scalable machine learning applications, helping enterprises quickly obtain data-driven insights and business value.

[0049] In this specification, a feature processing method is provided. One or more embodiments of this specification simultaneously relate to a feature processing system, a feature processing device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail one by one in the following embodiments.

[0050] In practical applications, feature concatenation and feature processing are important prerequisites for model training. Usually before model training, algorithm engineers need to manually write SQL statements themselves to obtain the required feature fields from different HIVE tables, and then combine them with feature processing and transformation logic to obtain usable features, and finally write them into the sample table for the machine learning platform to read and use. Feature concatenation generally involves a large amount of feature reading and concatenation processing. Manually constructing SQL statements not only consumes a large amount of human resources but also has the risk of concatenation errors. Therefore, an effective solution is urgently needed to solve the above problems.

[0051] The feature processing method provided in this embodiment can provide a feature processing platform for developers in order to save human resources while ensuring that data processing is error-free. During the sample construction stage before model training, in response to the target primary key information entered on the feature registration page, a target statement corresponding to the target primary key information is constructed, which does not need to be written separately by developers, effectively improving the data processing efficiency. After obtaining the target statement, the target statement can be executed to obtain the initial features and loaded onto the feature processing page for display; in response to the processing instructions submitted for the initial features loaded on the feature processing page, the associated features can be determined, and the target features can be constructed based on the associated features and the initial features; the feature splicing can be completed in combination with the developers' processing instructions, making the feature processing more flexible and stable. On this basis, an offline research task is configured for the target features, and the model training samples corresponding to the target features are constructed by executing the offline research task; then the model training samples can be stored in the sample table, where the model training samples stored in the sample table are used to train the business model. It is realized that the construction of samples can be completed without a large amount of manual processing, effectively improving the sample construction efficiency and reducing costs at the same time, so that downstream services can read samples from the sample table at any time to complete model training and improve the model training efficiency.

[0052] See Figure 1 , Figure 1 shows a flowchart of a feature processing method provided according to an embodiment of this specification, which specifically includes the following steps.

[0053] Step S102, in response to the target primary key information entered on the feature registration page, construct a target statement corresponding to the target primary key information, execute the target statement to obtain the initial features and load them onto the feature processing page for display.

[0054] The feature processing method provided in this embodiment can be applied to the sample construction stage before the training of any business model. The business model can be a business recommendation model (recommending articles, videos, products, live streamers, etc. to users based on user attributes), an overdue prediction model (predicting the overdue risk of users based on user attributes, such as resource lease overdue risk, payment service overdue risk, etc.), a multi-modal information processing model (modifying multi-modal information (such as images, texts, videos) based on user attributes, such as P-picture, text polishing, video editing, etc.).

[0055] Specifically, for the sample construction method before the training of any business model, reference can be made to the feature processing method provided in this embodiment. This embodiment takes the business model as a business recommendation model as an example to illustrate the feature processing method for constructing samples for the business recommendation model. Descriptions of other scenarios are not elaborated in this embodiment.

[0056] Specifically, the feature registration page specifically refers to the page where users register initial features during sample construction. Through this page, users can be supported to specify primary key information to select specific initial features for subsequent feature processing, effectively improving the flexibility of sample construction for users. Correspondingly, the target primary key information specifically refers to the primary key information used to index the original features, and this target primary key information is specified by users according to their needs to read the required initial features from different source tables for subsequent processing. Correspondingly, the target statement specifically refers to the SQL statement constructed by combining the Codegen technology with the target primary key information, which is used to automatically construct code and execute it, enabling users to only configure registration information on the front end and automatically complete the reading of initial features. Correspondingly, the initial features specifically refer to the original features read from the source table according to the target statement, that is, the original state attributes of the data, such as features like height, weight, age, gender, or occupation of users within the reading platform, which are used for subsequent processing to construct samples. Specifically, the feature processing page refers to the page that supports users to process the initial features. Through the feature processing page, users can be supported to name the initial features, select the feature type, fill in the feature dimension, fill in the feature field type, fill in the business line information suitable for the features, etc., so that users can process the initial features as needed to convert them into a structure suitable for model training, thereby avoiding problems with incorrect feature splicing.

[0057] Based on this, in order to save human resources while ensuring that data processing is error-free, a feature processing platform can be provided to developers. During the sample construction stage before model training, in response to the target primary key information entered on the feature registration page, a target statement corresponding to the target primary key information is constructed, which does not need to be written separately by developers, effectively improving data processing efficiency. After obtaining the target statement, the target statement can be executed to obtain the initial features and load them onto the feature processing page for display; in response to the processing instructions submitted for the initial features loaded on the feature processing page, associated features can be determined, and target features can be constructed based on the associated features and the initial features; the realization of feature splicing can be completed in combination with the processing instructions of developers, making feature processing more flexible and stable. On this basis, an offline research task is configured for the target features, and the model training samples corresponding to the target features are constructed by executing the offline research task; then the model training samples can be stored in the sample table, where the model training samples stored in the sample table are used to train the business model. It is realized that the construction of samples can be completed without a large amount of manual processing, effectively improving the sample construction efficiency and reducing costs at the same time, so that downstream services can read samples from the sample table at any time to complete model training and improve model training efficiency.

[0058] Further, when constructing the target statement, in order to be able to automatically construct the target statement for reading the initial features, a statement generation algorithm can be used to automatically complete the construction of the statement. In this embodiment, the specific implementation method is as follows:

[0059] Receive the target primary key information entered through the feature registration page, and determine the statement template corresponding to the target primary key information; use the statement generation algorithm to insert the target primary key information into the statement template to obtain the target statement corresponding to the target primary key information.

[0060] Specifically, the statement template specifically refers to an SQL template string containing placeholders, which is used to insert user-defined target primary key information to complete the automatic construction of the target statement; correspondingly, the statement generation algorithm can be implemented using the MessageFormat.format method.

[0061] Based on this, when constructing the target statement corresponding to the target primary key information, in order to improve the feature processing efficiency, after receiving the target primary key information entered through the feature registration page, the statement template corresponding to the target primary key information can be determined; at this time, the statement generation algorithm can be used to insert the target primary key information into the statement template to obtain the target statement corresponding to the target primary key information according to the insertion result for subsequent use.

[0062] In practical applications, after determining the statement template, since the target primary key information is user-defined, the target primary key information to be replaced can be inserted into an array, and subsequently, the MessageFormat.format method can be used to call the target primary key information in the array to replace the placeholders in the statement template, so that the target statement can be automatically generated for subsequent initial feature loading and processing operations.

[0063] Specifically, the MessageFormat.format method generates an SQL statement by combining the template with the parameters. The process first defines a template string containing placeholders. Placeholders such as {0}, {1} represent the parameter positions, and can be appended with format types (such as number, date) and styles (such as #.## or date format). Subsequently, the parameters (target primary key information) are stored in an array or variable argument list in order. When calling the method, the template is parsed, the parameter values are replaced according to the index, and the data is processed according to the specified format rules. At the same time, special characters in the template need to be escaped (for example, a single quote needs to be represented by a double single quote), and finally, the target statement that meets the requirements is generated.

[0064] Furthermore, when executing the target statement, in order to be able to execute the target statement to read the initial features from the source table for use, it is necessary to determine the source table and the target table according to the target statement, and complete the loading of the initial features through the method of table merging. In this embodiment, the specific implementation method is as follows:

[0065] Execute the target statement to determine the source table and the target table, and determine the target column and the primary key column corresponding to the target primary key information in the source table; construct a temporary table according to the row data in the target column and the row data in the primary key column; merge the temporary table and the target table, construct an initial feature according to the merge result, and load the initial feature onto the feature processing page for display.

[0066] Specifically, the source table specifically refers to the table that needs to query data according to the target primary key information, and the target table specifically refers to the table that supplements key information for the data stored in the temporary table. It can be understood that the target table is the table that stores supplementary information; correspondingly, the temporary table specifically refers to the data table temporarily constructed after extracting the row data in the source table; the target column specifically refers to the specified column in the source table, which is determined by the target statement, and the primary key column specifically refers to the column corresponding to the target primary key information.

[0067] Based on this, after automatically generating the target statement corresponding to the target primary key information, in order to obtain the initial feature by executing the target statement for subsequent sample construction, during the execution of the statement, the source table and the target table can be determined first, and the target column and the primary key column corresponding to the target primary key information can be determined in the source table; based on this, a temporary table can be constructed according to the row data in the target column and the row data in the primary key column; since the data contained in the temporary table is the original data required for the execution of the current target statement, but there may be problems with its incompleteness. For example, when reading the user's height data, the user's name is missing, which may not be convenient for subsequent sample generation. Therefore, the temporary table and the target table that records supplementary data can be merged to construct the initial feature according to the merge result, so as to ensure the integrity of the initial feature during the initial feature construction stage, and then the initial feature can be loaded onto the feature processing page for display, so as to further process the initial feature subsequently.

[0068] During this process, in order to ensure the construction of the temporary table is more accurate, the target row data can be read and the temporary table can be constructed by means of data partitioning and sorting. In this embodiment, the specific implementation method is as follows:

[0069] Process the source table according to the target column and the primary key column to obtain a partial source table; divide the partial source table into multiple data partitions according to the preset partition information, and sort the row data in each data partition; read the target row data in each data partition according to the sorting result, and construct a temporary table based on the target row data.

[0070] Specifically, the local source table specifically refers to the data table obtained by retaining the row data corresponding to the target column and the primary key column in the source table; the preset partition information specifically refers to the partition information obtained by partitioning the source table according to the primary key using a window function (such as ROW_NUMBER()), and correspondingly, the target row data specifically refers to a certain row of data selected according to the set rules after sorting the row data in each data partition. For example, after sorting the row data in each data partition in ascending order according to the name identifier, the first row of data can be selected as the target row data. By integrating the target row data of each data partition, a temporary table can be obtained for subsequent construction of the initial features.

[0071] Based on this, after determining the target column and the primary key column associated with the target statement from the source table, considering that the source table contains a large amount of data, in order to accurately construct the initial features, the source table can be processed according to the target column and the primary key column first to obtain a local source table, so as to reduce the computational complexity caused by excessive data volume. Then, the local source table can be divided into multiple data partitions according to the preset partition information, and then the row data in each data partition can be sorted; according to the sorting result, the distribution of the row data in each data partition can be determined. Therefore, the target row data can be read from each data partition according to the sorting result, and a temporary table can be constructed based on the target row data for subsequent use in constructing the initial features after merging with the target table.

[0072] In practical applications, after generating the target statement SQL using the MessageFormat.format method, specific column data can be extracted from the specified source table by executing this statement, and then merged with another target table through a left outer join, thereby achieving the construction and display of the initial features on this basis. Specifically, the specific column (sourceColumnName) of the source table and the primary key column (primarykey) corresponding to the primary key information can be obtained through an inner subquery first, and the row number can be calculated for each partition (partitioned by primarykey). After sorting by sourceColumnName, only the first row of each partition is retained. Thereafter, this result can be used as a temporary table, with an alias, and joined with the target table containing note_id, so as to combine the processed data information with the target table to generate the initial features for use.

[0073] In this process, the purpose of the inner subquery is to filter out the first row of data for each primary key partition from the source table and sort it based on specific columns. When selecting columns, you can select the columns to be retained (sourceColumnName) and the primary key column (primarykey) from the source table. The primary key is used to define the basis for partitioning. When partitioning and sorting, window functions (such as ROW_NUMBER()) can be used to partition the source table by primarykey and then sort it by sourceColumnName within each partition. After sorting, a row number can be assigned to each sorted row within each partition, and the first record after sorting for each partition can be filtered out by WHERE row_num = 1. At this time, this part of the filtered row data can be stored in a temporary table, and then the associated fields between the temporary table and the target table can be determined, and the two data tables can be merged according to the associated fields to obtain the global table that matches the target statement. Based on this, the initial features used for model training can be constructed, and after subsequent processing, they can be used for model training. Among them, the process of constructing the initial features based on the global table can be understood as constructing vectors for the data in the global table to obtain the corresponding feature vectors as the initial features.

[0074] For example, when building samples for a business recommendation model, a feature registration home page as shown in Figure 2a can be displayed to the user first, supporting the user to click the "Original Feature Registration" control to perform the registration process for the specified features. After the user clicks the "Original Feature Registration" control, a feature registration page as shown in Figure 2b can be displayed, supporting the user to fill in the basic information of the features to be entered on this page, such as selecting the data source (including full amount (only HIVE), full amount + real-time (HIVE + Kafka), real-time (Kafka)), selecting the table name of the HIVE table, and selecting the registration fields, so as to enable the user to read the data corresponding to the specified primary key information from the specified data source table according to the requirements, and generate the initial features that meet the sample construction requirements for subsequent use. For example, the names, heights, and ages corresponding to n users can be read from the source table for subsequent sample construction and training a model that can recommend clothing products to users based on user attributes.

[0075] In summary, by automatically constructing the target statement corresponding to the target primary key information using the statement generation algorithm and executing the target statement to obtain the initial features, it is possible to quickly complete feature construction without manual intervention in the data preprocessing stage, thus effectively improving the feature processing efficiency.

[0076] Step S104, in response to a processing instruction submitted for the initial features loaded in the feature processing page, determine the associated features and construct target features based on the associated features and the initial features.

[0077] Specifically, after obtaining the initial features and loading them onto the feature processing page as described above, it is possible to support users in processing the initial features loaded onto the feature processing page as needed to achieve the processing of the initial features. On this basis, considering that the initial features may only contain some of the features for sample construction, it is also necessary to use associated features in order to construct complete features for sample construction. Therefore, users can be supported in splicing associated features when processing the initial features to construct complete target features for subsequent sample construction processing.

[0078] Among them, the processing instruction specifically refers to the feature processing instruction submitted by the user for the initial features. This instruction can clarify the associated features that the initial features need to be spliced with, and at the same time can clarify the information that needs to be added to the initial features, such as feature name, feature type, entity dimension, feature field type, applicable business line, etc., for subsequent splicing of the processed features and the associated features to obtain target features that can be used for sample construction. Correspondingly, the associated feature specifically refers to features of other dimensions that are spliced with the initial features. It can be understood that the associated feature is a feature of other dimensions that can still be used in the sample construction stage. For example, if the initial feature is the user's height and the model to be trained needs to have the ability to recommend clothing, then the user's weight can be selected as the associated feature to be spliced with the initial feature to make the constructed sample more in line with the model training requirements, so that the sample construction can be more accurate. Correspondingly, the target feature specifically refers to the vector expression obtained after splicing the associated feature and the initial feature.

[0079] Furthermore, when reading the associated features and splicing them with the initial features to construct the target features, feature processing can be combined with the processing instruction, and subsequent feature processing operations can be completed based on the results of the processing. In this embodiment, the specific implementation method is as follows:

[0080] Receive the processing instruction submitted for the initial features loaded in the feature processing page; screen the features to be processed from the initial features according to the processing instruction, and determine the feature configuration information to be added to the features to be processed; update the features to be processed based on the feature configuration information to obtain intermediate features; read the associated features from the association table corresponding to the processing instruction according to the attribute information of the intermediate features, and splice the associated features and the intermediate features to obtain the target features.

[0081] Specifically, the feature to be processed specifically refers to the features selected from the initial features that need to be processed. For example, if the initial feature is the height of n users, the feature to be processed can be the height of users over 18 years old, which is convenient for subsequent use of high-standard and more accurate features for sample construction. Correspondingly, the feature configuration information specifically refers to the information configured by the user for the feature to be processed, such as information related to the feature name, feature type, entity dimension, feature field type, applicable business line, etc. Correspondingly, the intermediate feature specifically refers to the vector representation obtained after processing the feature to be processed. The attribute information specifically refers to the attribute description corresponding to the intermediate feature. The association table specifically refers to the data table storing the features associated with the intermediate feature, and the associated feature is the vector representation that needs to be used read from the association table. For example, if the association table stores the weight and gender of n users, the weight of users over 18 years old can be read from this table according to the intermediate feature, so as to construct the target feature in combination with the intermediate feature for subsequent use.

[0082] Based on this, after presenting the feature processing page containing the initial features to the user, the user can process the loaded initial features according to the requirements. When receiving the processing instruction submitted for the initial features loaded in the feature processing page, at this time, the feature to be processed can be screened from the initial features according to the processing instruction to clarify the part that needs to be processed, and then the feature configuration information added for the feature to be processed can be determined according to the user's configuration operation; on this basis, the feature to be processed can be updated based on the feature configuration information to obtain the intermediate feature; on this basis, the associated feature can also be read from the association table corresponding to the processing instruction according to the attribute information of the intermediate feature, and the associated feature and the intermediate feature can be concatenated to obtain the target feature, so as to use the complete and multi-dimensional target feature for sample construction for subsequent use.

[0083] In practical applications, after obtaining the initial features, feature aggregation and feature processing need to be performed on the initial features and the associated features. In this process, in order to be able to complete the feature processing operation based on the user's processing instruction, a temporary view named src can be created first, and then the feature to be processed can be extracted from the initial features to construct the sss table; on this basis, the features in the sss table can be processed according to the user's processing instruction, and at the same time, another data table storing the specific attributes of the user can be associated, and then the user-defined function can be used to generate and process the features to obtain the complete target feature for subsequent sample construction processing.

[0084] In specific implementation, when processing the initial features, in addition to performing feature configuration processing, feature sorting can also be carried out. For example, users aged 1 - 12 can be marked as children, those aged 12 - 18 as youths, those aged 18 - 64 as adults, and those over 64 as the elderly according to age groups. Through this feature processing, the influence brought by age can be blurred. After constructing samples and training the model subsequently, the model can recommend different types of clothing for users in different stages, avoiding the problem of inaccurate prediction accuracy caused by recommending according to the exact age.

[0085] Continuing with the above example, after obtaining the initial features of the names, heights, and ages corresponding to n users respectively, it is possible to support users in processing the initial features. At this time, a feature processing page as shown in Figure 2c can be presented to the user, supporting the user in naming the feature name, selecting the feature type, setting the entity dimension, setting the field type, selecting the applicable business line, selecting the business scenario, setting the operating user information, etc. for any one of the initial features. For example, if the user filters out the heights of users over 18 years old from the initial features according to requirements, the to - be - processed features of the names, heights, and ages corresponding to m users can be obtained. At this time, the user can perform the above - mentioned processing on this part of the features to obtain intermediate features. Further, considering that the training of the business recommendation model may not meet the actual business requirements if only the height features of users are used, the user can also choose to associate and merge the features with the intermediate features. At this time, according to the user's operation, the data table recording the user weights is selected, and then the weight features corresponding to m users are read from this data table. By splicing the weight features and height features of m users, the body shape features corresponding to m users can be obtained for subsequent sample construction.

[0086] In summary, by processing the initial features, the expression of the initial features can be made more accurate. After merging them with the associated features, the integrity of the target features can be ensured, and more dimensions can be covered, thus making the subsequent constructed samples more accurate.

[0087] Step S106: Configure an offline research task for the target feature, and construct a model training sample corresponding to the target feature by executing the offline research task.

[0088] Specifically, after the above - mentioned processing of the initial features and construction of the target features, further, in order to be able to construct a model training sample based on the target features, an offline research task needs to be configured for the target feature to realize constructing the label corresponding to the target feature by executing the offline research task. By constructing the association relationship between the target feature and the label, the model training sample can be obtained, so that the downstream can complete the model training operation by combining the target feature and the label.

[0089] Among them, the offline research task specifically refers to a task that processes and analyzes the target feature in an offline state according to the configured target feature to generate labels. By pre-computing and storing the features, useful information is extracted from the target feature to construct the model training samples. And the constructed offline research task can run in batch mode without real-time calculation of feature values, thus effectively improving the feature processing efficiency and saving the manual labeling process. Correspondingly, the model training sample specifically refers to a sample pair constructed by combining the sample and the label, where the sample is the target feature and the label is the execution result of the offline research task. For example, if the offline research task is to analyze the dressing interests of users of different age groups, after obtaining the body characteristics of different users upstream, the offline research task can be configured and executed to count the corresponding dressing interests of this part of users, and use this as the label to construct the model training sample with the body characteristics of the users. After training the model, the model can output clothing products recommended to the user according to the height and weight input by the user.

[0090] Furthermore, when configuring the offline research task, in order to make the construction of the model training sample more automated, the user can complete the task configuration operation through the task creation page. In this embodiment, the specific implementation method is as follows:

[0091] Display the task creation page including the target feature, and receive the task creation request submitted for the target feature through the task creation page; display the task configuration page according to the task creation request, and receive the task parameters configured for the target feature through the task configuration page; construct the offline research task based on the task parameters, and configure the offline research task for the target feature.

[0092] Specifically, the task creation page specifically refers to a page that shows the user the offline research task that needs to be executed for the target feature. Correspondingly, the task configuration page specifically refers to a page that configures the specific content of the task for the target feature. Since different features need to configure different tasks to complete the construction of the samples, providing the task configuration page to the user can support the user to customize the task content. Correspondingly, the task parameters specifically refer to the parameters entered by the user when creating the offline research task.

[0093] Based on this, when performing task configuration processing operations, in order to improve the flexibility of task construction, a task creation page containing target features can be displayed. In the case of receiving a task creation request submitted for the target features through the task creation page, a task configuration page can be displayed according to the task creation request, and the user is supported to complete the task creation operation through the task configuration page. At this time, the task parameters configured for the target features through the task configuration page can be received; to implement the construction of an offline research task according to the task parameters and configure the offline research task for the target features. On this basis, by executing the offline research task, the label construction of the target features can be completed, so that model training samples can be constructed by combining the target features and the labels for subsequent storage and model training use.

[0094] In practical applications, the task parameters entered through the task configuration page can include set business line information, database name information, sample table name information, operator information, target feature information, task name information, description information, etc. This embodiment does not make any limitations here. Among them, the business line information is used to indicate the business line to which the business belongs, and multiple business lines are supported to share; the database name is used to clarify the business line sample lake table information; the feature is used to select the target feature to be processed this time; the task name is used to be bound to the sample name to represent the uniqueness of the sample and the feature; the description can record the relevant information of the sample; in addition, time information can also be set to control the task execution time.

[0095] Continuing with the above example, after obtaining the body features corresponding to m users respectively, the user can be supported to configure an offline research task for the body features of the m users for sample label construction; during this process, a task configuration page as shown in Figure 2d can be displayed according to the user's operation, and the user is supported to set task parameters for the body features through this page, such as setting business line information, database name information, sample table name information, operator information, target feature information, task name information, description information, etc. After setting the task parameters for the body features, an offline research task can be constructed according to the set task parameters; further, by executing the offline research task, the characteristics of the clothes worn by users with different body postures can be clarified, for example, it can be determined that a user with a height of 1.8 meters and a weight of 75 kilograms likes to wear jeans, a user with a height of 1.7 meters and a weight of 75 kilograms likes to wear sweatpants, and a user with a height of 1.6 meters and a weight of 55 kilograms likes to wear sports shoes. On this basis, model training samples can be constructed by combining the above body features and the task execution results for the training use of the business recommendation model.

[0096] In summary, by supporting users to customize the configuration of offline research tasks for target features, different tasks can be configured for different features, thereby ensuring that the task execution results can construct high-quality and highly accurate samples, making the model training effect better.

[0097] Step S108, storing the model training samples into a sample table, where the model training samples stored in the sample table are used to train a business model.

[0098] Specifically, after constructing the model training samples corresponding to the target features by performing the offline research task, considering that model training may not be directly connected to the feature processing platform, the model training samples can be stored in the sample table, so that when model training is required, the model training samples can be read from the sample table to train the business model.

[0099] Among them, the sample table specifically refers to a data table that stores model training samples separately. In practical applications, different sample tables can be set for business models with different prediction capabilities, so that the model training samples can be managed in partitions; at the same time, due to the different input formats of business models, when the sample table stores model training samples, they can also be stored according to the input format set by the model, thus saving the format conversion time and improving the model training efficiency.

[0100] In addition, after the sample table is constructed, it can be connected to the machine learning platform to complete the training of the business recommendation model. In this embodiment, the specific implementation method is as follows:

[0101] In response to a training instruction submitted for the business recommendation model, reading a first model training sample from the sample table; training the business recommendation model using the first model training sample, and detecting whether the trained business recommendation model meets the training stop condition; if not, reading a second model training sample from the sample table, using the second model training sample as the first model training sample, and performing the step of training the business recommendation model using the first model training sample; if so, using the trained business recommendation model as the target business recommendation model.

[0102] Specifically, the training instruction specifically refers to a training instruction submitted for the business recommendation model. Based on this training instruction, any first model training sample can be read from the sample table for model training; correspondingly, the business recommendation model specifically refers to a model with recommendation capabilities, such as being able to recommend products, clothing, articles, videos, audio, etc. according to user attribute features. Or recommend products, clothing, articles, videos, audio, etc. according to user behavior features. Correspondingly, the training stop condition specifically refers to the condition for stopping the training of the business recommendation model, including but not limited to the loss value comparison condition, the validation set verification condition, or the iteration number condition.

[0103] Among them, the loss value comparison condition is used to determine whether the loss value after each training of the model is less than the set loss value threshold; the validation set verification condition is used to determine whether the prediction accuracy after each training of the model is greater than the preset accuracy threshold; the number of iterations condition specifically refers to determining whether the training cycle of the model reaches the set number threshold. In specific implementation, the training stop condition can be selected according to actual needs, and this embodiment does not make any limitation here.

[0104] Based on this, after storing the model training samples in the sample table, the user can read the samples from the sample table at any time for model training. When it is necessary to train the business recommendation model, in response to the training instruction submitted for the business recommendation model, the first model training samples can be read from the sample table; at this time, the features in the first model training samples can be input into the business recommendation model for processing to obtain a prediction result, and then the loss value can be calculated based on the prediction result and the labels in the first model training samples, and the business recommendation model can be optimized according to the loss value. After the optimization is completed, it can be detected whether the business recommendation model meets the training stop condition; if not, the second model training samples can be read from the sample table, the second model training samples can be used as the first model training samples, and the step of training the business recommendation model using the first model training samples can be executed; until it meets the condition, the business recommendation model that meets the condition can be used as the target business recommendation model.

[0105] Following the above example, after constructing the model training samples based on the body posture features and labels, they can be stored in the sample table; at any time when it is necessary to use the samples stored in the sample table to train the business recommendation model (this model can recommend clothing to users according to the input body posture information), the model training samples can be read from the sample table to train the business recommendation model until the trained model meets the training stop condition, and then the trained model can be deployed to the clothing recommendation scenario. It can be realized that after receiving the user's input height and weight, corresponding clothing can be recommended to the user. For example, if the user's height is 1.8 meters and weight is 75 kilograms, jeans can be recommended to the user.

[0106] In addition, the samples recorded in the sample table can also be used by the user to verify the model, for verifying the prediction accuracy of the model, so as to decide whether to optimize the model.

[0107] In summary, by using the samples stored in the sample table for model training, the model training can be made more efficient, thus facilitating the use of downstream services.

[0108] In addition, after storing the model training samples in the sample table, it also supports the user to perform query processing on the samples. In this embodiment, the specific implementation method is as follows:

[0109] Receive a viewing instruction submitted for the sample table, and determine a target model training sample in the sample table according to the viewing instruction; determine the sample information of the target model training sample, construct a sample information list according to the sample information and display it; wherein, the sample information list includes the sample source information and sample configuration information of the target model training sample, the sample source information is used to record the data source of the target model training sample, and the sample configuration information is used to record the configuration details of the target model training sample.

[0110] Specifically, the viewing instruction specifically refers to an instruction submitted by the user when they need to view the sample table, which is used to select any target model training sample for viewing and supports the user in understanding the sample details information. Correspondingly, the sample information specifically refers to the sample details information corresponding to the target model training sample; including but not limited to the library name, table name, added feature information, backtracking period information, task instance information, training information, etc. corresponding to the sample; correspondingly, the sample source information is the source information representing the corresponding features of the sample; correspondingly, the sample configuration information specifically refers to the detailed information configured when processing the features.

[0111] Based on this, in order to facilitate the user to view the status of any sample, after receiving a viewing instruction submitted for the sample table, the target model training sample can be determined in the sample table according to the viewing instruction; on this basis, the sample information of the target model training sample can be determined, and a sample information list can be constructed and displayed according to the sample information; wherein, the sample information list includes the sample source information and sample configuration information of the target model training sample, the sample source information is used to record the data source of the target model training sample, and the sample configuration information is used to record the configuration details of the target model training sample.

[0112] For example, after the sample construction is completed through the above processing method, the user can also view the status of any sample at any time. According to the target model training sample selected by the user, a page as shown in Figure 2e can be displayed. Through this page, the library name, table name, added feature information, backtracking period information, task instance information, training information, etc. corresponding to the sample can be displayed.

[0113] The feature processing method provided in this embodiment can provide a feature processing platform for developers in order to save human resources while ensuring that data processing is error-free. During the sample construction stage before model training, in response to the target primary key information entered on the feature registration page, a target statement corresponding to the target primary key information is constructed, eliminating the need for developers to write it separately, effectively improving data processing efficiency. After obtaining the target statement, the target statement can be executed to obtain initial features and loaded onto the feature processing page for display; in response to a processing instruction submitted for the initial features loaded on the feature processing page, associated features can be determined, and target features can be constructed based on the associated features and the initial features; feature splicing can be completed in combination with the developer's processing instruction, making feature processing more flexible and stable. On this basis, an offline research task is configured for the target feature, and a model training sample corresponding to the target feature is constructed by executing the offline research task; then the model training sample can be stored in a sample table, where the model training samples stored in the sample table are used to train a business model. This enables the construction of samples to be completed without a large amount of manual processing, effectively improving the sample construction efficiency and reducing costs, so that downstream services can read samples from the sample table at any time to complete model training and improve model training efficiency.

[0114] Corresponding to the above method embodiment, this specification also provides an embodiment of a feature processing system. Figure 3 FIG. shows a schematic structural diagram of a feature processing system provided in an embodiment of this specification. As Figure 3 shown, the feature processing system 300 includes a client 310 and a server 320, including:

[0115] The client 310 is configured to receive the target primary key information entered on the feature registration page and send the target primary key information to the server;

[0116] The server 320 is configured to construct a target statement corresponding to the target primary key information, execute the target statement to obtain initial features, load the initial features onto the feature processing page and send them to the client for display;

[0117] The client 310 is configured to receive a processing instruction submitted for the initial features loaded on the feature processing page and send the processing instruction to the server;

[0118] The server 320 is configured to determine associated features in response to the processing instruction, and construct target features based on the associated features and the initial features; configure an offline research task for the target feature, construct a model training sample corresponding to the target feature by executing the offline research task; store the model training sample in a sample table, where the model training samples stored in the sample table are used to train a business model.

[0119] Further, when constructing the target statement, in order to automatically construct the target statement for reading the initial features, a statement generation algorithm can be used to automatically complete the construction of the statement. In this embodiment, the specific implementation method is as follows:

[0120] The client receives the target primary key information entered through the feature registration page and submits it to the server. The server determines the statement template corresponding to the target primary key information; uses the statement generation algorithm to insert the target primary key information into the statement template to obtain the target statement corresponding to the target primary key information.

[0121] Furthermore, when executing the target statement, in order to be able to execute the target statement to read the initial features from the source table for use, it is necessary to determine the source table and the target table according to the target statement, and complete the loading of the initial features through table merging. In this embodiment, the specific implementation method is as follows:

[0122] The server executes the target statement to determine the source table and the target table, and determines the target column and the primary key column corresponding to the target primary key information in the source table; constructs a temporary table according to the row data in the target column and the row data in the primary key column; merges the temporary table and the target table, constructs the initial features according to the merge result, and loads the initial features to the feature processing page and sends them to the client for display.

[0123] In this process, in order to ensure that the construction of the temporary table is more accurate, the target row data can be read and the temporary table can be constructed by means of data partitioning and sorting. In this embodiment, the specific implementation method is as follows:

[0124] The server processes the source table according to the target column and the primary key column to obtain a partial source table; divides the partial source table into multiple data partitions according to the preset partition information, and sorts the row data in each data partition; reads the target row data in each data partition according to the sorting result, and constructs a temporary table based on the target row data.

[0125] Further, when performing associated feature reading and splicing the initial features to construct the target features, feature processing can be combined with the processing instructions, and subsequent feature processing operations can be completed based on the results of the processing. In this embodiment, the specific implementation method is as follows:

[0126] The client receives a processing instruction submitted for the initial feature loaded in the feature processing page; the server filters the features to be processed from the initial features according to the processing instruction, and determines the feature configuration information to be added for the features to be processed; updates the features to be processed based on the feature configuration information to obtain intermediate features; reads associated features in the association table corresponding to the processing instruction according to the attribute information of the intermediate features, and splices the associated features and the intermediate features to obtain target features.

[0127] Further, when configuring the offline research task, in order to make the construction of the model training samples more automated, the user can complete the task configuration operation through the task creation page. In this embodiment, the specific implementation method is as follows:

[0128] The client displays a task creation page containing the target features, and receives a task creation request submitted for the target features through the task creation page; displays a task configuration page according to the task creation request, and receives task parameters configured for the target features through the task configuration page; constructs an offline research task based on the task parameters, and configures the offline research task for the target features.

[0129] In addition, after storing the model training samples in the sample table, it is also possible to support the user to perform query processing on the samples. In this embodiment, the specific implementation method is as follows:

[0130] The client receives a viewing instruction submitted for the sample table, and the server determines the target model training samples in the sample table according to the viewing instruction; determines the sample information of the target model training samples, constructs a sample information list according to the sample information and displays it; wherein, the sample information list includes the sample source information and sample configuration information of the target model training samples, the sample source information is used to record the data source of the target model training samples, and the sample configuration information is used to record the configuration details of the target model training samples.

[0131] In addition, after the sample table is constructed, it is possible to connect to the machine learning platform to complete the training of the business recommendation model. In this embodiment, the specific implementation method is as follows:

[0132] The server responds to a training instruction submitted for a business recommendation model, reads first model training samples from the sample table, trains the business recommendation model using the first model training samples, and detects whether the trained business recommendation model meets the training stop condition. If not, it reads second model training samples from the sample table, uses the second model training samples as the first model training samples, and executes the step of training the business recommendation model using the first model training samples. If so, it uses the trained business recommendation model as the target business recommendation model.

[0133] The feature processing system provided in this embodiment, in order to save human resources while ensuring that data processing is error-free, can provide a feature processing platform to developers, enabling the construction of target statements corresponding to target primary key information in response to the input of target primary key information on the feature registration page during the sample construction stage before model training, without the need for developers to write them separately, effectively improving data processing efficiency. After obtaining the target statements, the target statements can be executed to obtain initial features and loaded onto the feature processing page for display. In response to a processing instruction submitted for the initial features loaded on the feature processing page, associated features can be determined, and target features can be constructed based on the associated features and the initial features. The implementation of feature splicing can be completed in combination with the processing instructions of developers, making feature processing more flexible and stable. On this basis, an offline research task is configured for the target features, and model training samples corresponding to the target features are constructed by executing the offline research task. Then, the model training samples can be stored in the sample table, where the model training samples stored in the sample table are used to train the business model. It is realized that the construction of samples can be completed without a large amount of manual processing, effectively improving the sample construction efficiency and reducing costs at the same time, so that downstream services can read samples from the sample table at any time to complete model training and improve model training efficiency.

[0134] The above is a schematic solution of a feature processing system in this embodiment. It should be noted that the technical solution of this feature processing system and the technical solution of the above feature processing method belong to the same concept. For the details not described in detail in the technical solution of the feature processing system, reference can be made to the description of the technical solution of the above feature processing method.

[0135] The following combines the attached Figure 4 Take the application of the feature processing method provided in this specification in a business recommendation model as an example to further illustrate the feature processing method. Among them, Figure 4 shows a process timing diagram of a feature processing method provided in an embodiment of this specification.

[0136] The feature processing method is applied to a content application platform, which includes a client and a server. Among them, the client is the operating end that supports developers to select original features for sample construction according to requirements, and the server specifically refers to the provider of code generation service, feature processing service and server. As Figure 4 shown below:

[0137] The client receives the target primary key information entered through the feature registration page and submits it to the server.

[0138] The server determines the statement template corresponding to the target primary key information; uses the statement generation algorithm to insert the target primary key information into the statement template to obtain the target statement corresponding to the target primary key information.

[0139] The server executes the target statement to determine the source table and the target table, and determines the target column and the primary key column corresponding to the target primary key information in the source table.

[0140] The server processes the source table according to the target column and the primary key column to obtain a partial source table; divides the partial source table into multiple data partitions according to the preset partition information.

[0141] The server sorts the row data in each data partition; reads the target row data in each data partition according to the sorting result, and constructs a temporary table based on the target row data.

[0142] The server merges the temporary table and the target table, constructs an initial feature according to the merge result, loads the initial feature to the feature processing page, and sends it to the client for display.

[0143] The client receives the processing instruction submitted for the initial feature loaded in the feature processing page.

[0144] The server filters the features to be processed in the initial feature according to the processing instruction, and determines the feature configuration information added for the features to be processed.

[0145] The server updates the features to be processed based on the feature configuration information to obtain intermediate features; reads the associated features in the associated table corresponding to the processing instruction according to the attribute information of the intermediate features, and splices the associated features and the intermediate features to obtain the target features.

[0146] The client displays a task creation page containing the target features and receives a task creation request submitted for the target features through the task creation page.

[0147] The client displays a task configuration page according to the task creation request and receives the task parameters configured for the target features through the task configuration page.

[0148] The client constructs an offline research task based on task parameters and configures the offline research task for target features.

[0149] The server constructs model training samples corresponding to target features by executing the offline research task; stores the model training samples in a sample table.

[0150] The client submits a training instruction for a business recommendation model.

[0151] The server, in response to the training instruction, reads the model training samples from the sample table; uses the model training samples to train the business recommendation model until a target business recommendation model that meets the training conditions is obtained and deployed to the business recommendation scenario.

[0152] In summary, in order to save human resources and ensure that data processing is error-free, a feature processing platform can be provided to developers. During the sample construction stage before model training, in response to the target primary key information entered on the feature registration page, a target statement corresponding to the target primary key information is constructed, which does not need to be written separately by developers, effectively improving data processing efficiency. After obtaining the target statement, the target statement can be executed to obtain the initial features and loaded onto the feature processing page for display; in response to the processing instruction submitted for the initial features loaded on the feature processing page, the associated features can be determined, and the target features can be constructed based on the associated features and the initial features; the feature splicing can be completed in combination with the developer's processing instruction, making the feature processing more flexible and stable. On this basis, an offline research task is configured for the target features, and the model training samples corresponding to the target features are constructed by executing the offline research task; then the model training samples can be stored in the sample table, where the model training samples stored in the sample table are used to train the business model. It is realized that the construction of samples can be completed without a large amount of manual processing, effectively improving the sample construction efficiency and reducing costs, so that downstream services can read samples from the sample table at any time to complete model training and improve model training efficiency.

[0153] Corresponding to the above method embodiments, this specification also provides embodiments of a feature processing apparatus. Figure 5 The structural schematic diagram of a feature processing apparatus provided by an embodiment of this specification is shown. As Figure 5 shown, the apparatus includes:

[0154] A construction module 502, configured to construct a target statement corresponding to the target primary key information in response to the target primary key information entered on the feature registration page, execute the target statement to obtain initial features and load them onto the feature processing page for display;

[0155] A determination module 504, configured to determine associated features in response to a processing instruction submitted for the initial feature loaded in the feature processing page, and construct a target feature according to the associated features and the initial feature;

[0156] A configuration module 506, configured to configure an offline research task for the target feature, and construct a model training sample corresponding to the target feature by executing the offline research task;

[0157] A storage module 508, configured to store the model training sample in a sample table, wherein the model training samples stored in the sample table are used to train a business model.

[0158] In an optional embodiment, the constructing a target statement corresponding to the target primary key information in response to the target primary key information entered on the feature registration page includes:

[0159] Receiving the target primary key information entered through the feature registration page, and determining a statement template corresponding to the target primary key information; using a statement generation algorithm to insert the target primary key information into the statement template to obtain a target statement corresponding to the target primary key information.

[0160] In an optional embodiment, the executing the target statement to obtain an initial feature and loading it onto the feature processing page for display includes:

[0161] Executing the target statement to determine a source table and a target table, and determining a target column and a primary key column corresponding to the target primary key information in the source table; constructing a temporary table according to the row data in the target column and the row data in the primary key column; merging the temporary table and the target table, constructing an initial feature according to the merging result, and loading the initial feature onto the feature processing page for display.

[0162] In an optional embodiment, the determining associated features in response to a processing instruction submitted for the initial feature loaded in the feature processing page, and constructing a target feature according to the associated features and the initial feature includes:

[0163] Receiving a processing instruction submitted for the initial feature loaded in the feature processing page; screening features to be processed in the initial feature according to the processing instruction, and determining feature configuration information added for the features to be processed; updating the features to be processed based on the feature configuration information to obtain intermediate features; reading associated features in an association table corresponding to the processing instruction according to the attribute information of the intermediate features, and splicing the associated features and the intermediate features to obtain a target feature.

[0164] In an optional embodiment, the configuring an offline research task for the target feature includes:

[0165] Display a task creation page including the target feature, and receive a task creation request submitted for the target feature through the task creation page; display a task configuration page according to the task creation request, and receive task parameters configured for the target feature through the task configuration page; construct an offline research task based on the task parameters, and configure the offline research task for the target feature.

[0166] In an optional embodiment, the constructing a temporary table according to the row data in the target column and the row data in the primary key column includes:

[0167] Process the source table according to the target column and the primary key column to obtain a partial source table; divide the partial source table into multiple data partitions according to preset partition information, and sort the row data in each data partition; read target row data in each data partition according to the sorting result, and construct a temporary table based on the target row data.

[0168] In an optional embodiment, after the step of storing the model training samples into the sample table, the method further includes:

[0169] Receive a viewing instruction submitted for the sample table, determine target model training samples in the sample table according to the viewing instruction; determine the sample information of the target model training samples, construct a sample information list according to the sample information and display it; wherein, the sample information list includes the sample source information and sample configuration information of the target model training samples, the sample source information is used to record the data source of the target model training samples, and the sample configuration information is used to record the configuration details of the target model training samples.

[0170] In an optional embodiment, after the step of storing the model training samples into the sample table, the method further includes:

[0171] In response to a training instruction submitted for the business recommendation model, read first model training samples in the sample table; train the business recommendation model using the first model training samples, and detect whether the trained business recommendation model meets the training stop condition; if not, read second model training samples in the sample table, use the second model training samples as the first model training samples, and execute the step of training the business recommendation model using the first model training samples; if so, use the trained business recommendation model as the target business recommendation model.

[0172] The feature processing device provided in this embodiment can provide a feature processing platform for developers in order to save human resources and ensure error-free data processing and processing. During the sample construction stage before model training, in response to the target primary key information entered on the feature registration page, a target statement corresponding to the target primary key information is constructed, which does not need to be separately written by developers, effectively improving the data processing efficiency. After obtaining the target statement, the target statement can be executed to obtain the initial features and loaded onto the feature processing page for display; in response to the processing instructions submitted for the initial features loaded on the feature processing page, the associated features can be determined, and the target features can be constructed based on the associated features and the initial features; the feature splicing can be completed in combination with the developers' processing instructions, making the feature processing more flexible and stable. On this basis, an offline research task is configured for the target features, and the model training samples corresponding to the target features are constructed by executing the offline research task; then the model training samples can be stored in the sample table, where the model training samples stored in the sample table are used to train the business model. It is realized that the construction of samples can be completed without a large amount of manual processing, effectively improving the sample construction efficiency and reducing costs at the same time, so that the downstream services can read the samples from the sample table at any time to complete model training and improve the model training efficiency.

[0173] The above is a schematic solution of a feature processing device in this embodiment. It should be noted that the technical solution of this feature processing device and the technical solution of the above feature processing method belong to the same concept. For the details not described in detail in the technical solution of the feature processing device, reference can be made to the description of the technical solution of the above feature processing method.

[0174] Figure 6 FIG. shows a structural block diagram of a computing device 600 provided according to an embodiment of the present specification. The components of the computing device 600 include, but are not limited to, a memory 610 and a processor 620. The processor 620 is connected to the memory 610 through a bus 630, and the database 650 is used to store data.

[0175] The computing device 600 also includes an access device 640, which enables the computing device 600 to communicate via one or more networks 660. Examples of such networks include the Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0176] In one embodiment of the present specification, the above components of the computing device 600, as well as Figure 6 other components not shown, may also be connected to each other, for example, via a bus. It should be understood that Figure 6 the block diagram of the computing device shown is for illustrative purposes only and is not a limitation on the scope of the present specification. Those skilled in the art may add or replace other components as needed.

[0177] The computing device 600 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 600 can also be a mobile or stationary server.

[0178] Among them, the processor 620 is used to execute the following computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the above feature processing method are implemented.

[0179] The above is a schematic solution of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above feature processing method belong to the same concept. For the details not described in detail in the technical solution of the computing device, reference can be made to the description of the technical solution of the above feature processing method.

[0180] An embodiment of this specification also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the steps of the above feature processing method are implemented.

[0181] The above is a schematic solution of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above feature processing method belong to the same concept. For the details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solution of the above feature processing method.

[0182] An embodiment of this specification also provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed by a processor, the steps of the above feature processing method are implemented.

[0183] The above is a schematic solution of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above feature processing method belong to the same concept. For the details not described in detail in the technical solution of the computer program product, reference can be made to the description of the technical solution of the above feature processing method.

[0184] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain implementations, multitasking and parallel processing are also possible or may be advantageous.

[0185] The computer instructions include computer program code, which may be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, mobile hard disks, magnetic disks, optical disks, computer memories, read-only memories (ROMs), random access memories (RAMs), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0186] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of this specification are not limited by the described action sequence, because according to the embodiments of this specification, some steps may be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0187] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0188] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The alternative embodiments do not elaborate on all the details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can well understand and utilize this specification.

Claims

1. A feature processing method, characterized in that, Including: In response to the target primary key information entered on the feature registration page, construct a target statement corresponding to the target primary key information, execute the target statement to obtain an initial feature, and load it onto the feature processing page for display; In response to a processing instruction submitted for the initial feature loaded on the feature processing page, determine associated features, and construct a target feature based on the associated features and the initial feature; Configure an offline research task for the target feature, and construct a model training sample corresponding to the target feature by executing the offline research task; Store the model training sample in a sample table, where the model training samples stored in the sample table are used to train a business model.

2. The feature processing method according to claim 1, wherein The step of, in response to the target primary key information entered on the feature registration page, constructing a target statement corresponding to the target primary key information includes: Receive the target primary key information entered through the feature registration page, and determine a statement template corresponding to the target primary key information; Use a statement generation algorithm to insert the target primary key information into the statement template to obtain a target statement corresponding to the target primary key information.

3. The characteristic processing method according to claim 1, wherein The step of executing the target statement to obtain an initial feature and loading it onto the feature processing page for display includes: Execute the target statement to determine a source table and a target table, and determine a target column and a primary key column corresponding to the target primary key information in the source table; Construct a temporary table based on the row data in the target column and the row data in the primary key column; Merge the temporary table and the target table, construct an initial feature based on the merge result, and load the initial feature onto the feature processing page for display.

4. The feature processing method according to claim 1, wherein The step of, in response to a processing instruction submitted for the initial feature loaded on the feature processing page, determining associated features and constructing a target feature based on the associated features and the initial feature includes: Receive a processing instruction submitted for the initial feature loaded on the feature processing page; Filter the features to be processed in the initial feature according to the processing instruction, and determine the feature configuration information added for the features to be processed; Update the features to be processed based on the feature configuration information to obtain intermediate features; Read associated features from the associated table corresponding to the processing instruction according to the attribute information of the intermediate features, and splice the associated features and the intermediate features to obtain a target feature.

5. The feature processing method according to claim 1, wherein The step of configuring an offline research task for the target feature includes: Display a task creation page containing the target feature, and receive a task creation request submitted for the target feature through the task creation page; Display a task configuration page according to the task creation request, and receive task parameters configured for the target feature through the task configuration page; Construct an offline research task based on the task parameters, and configure the offline research task for the target feature.

6. The feature processing method according to claim 3, wherein The step of constructing a temporary table based on the row data in the target column and the row data in the primary key column includes: Process the source table according to the target column and the primary key column to obtain a partial source table; Divide the local source table into multiple data partitions according to the preset partition information, and sort the row data in each data partition; Read the target row data in each data partition according to the sorting result, and construct a temporary table based on the target row data.

7. The feature processing method according to any one of claims 1 to 6, characterized in that, After the step of storing the model training sample into the sample table, it further includes: Receive a viewing instruction submitted for the sample table, and determine the target model training sample in the sample table according to the viewing instruction; Determine the sample information of the target model training sample, and construct and display a sample information list according to the sample information; Wherein, the sample information list includes the sample source information and sample configuration information of the target model training sample, the sample source information is used to record the data source of the target model training sample, and the sample configuration information is used to record the configuration details of the target model training sample.

8. The feature processing method according to any one of claims 1 to 6, characterized in that After the step of storing the model training sample into the sample table, it further includes: In response to a training instruction submitted for the business recommendation model, read the first model training sample in the sample table; Use the first model training sample to train the business recommendation model, and detect whether the trained business recommendation model meets the training stop condition; If not, read the second model training sample in the sample table, use the second model training sample as the first model training sample, and execute the step of using the first model training sample to train the business recommendation model; If so, use the trained business recommendation model as the target business recommendation model.

9. A feature processing system, characterized in that It includes a client and a server, and includes: The client is used to receive the target primary key information entered on the feature registration page and send the target primary key information to the server; The server is used to construct a target statement corresponding to the target primary key information, execute the target statement to obtain the initial feature, load the initial feature onto the feature processing page and send it to the client for display; The client is used to receive a processing instruction submitted for the initial feature loaded on the feature processing page and send the processing instruction to the server; The server is used to determine the associated feature in response to the processing instruction, and construct the target feature according to the associated feature and the initial feature; configure an offline research task for the target feature, construct a model training sample corresponding to the target feature by executing the offline research task; store the model training sample into the sample table, wherein the model training sample stored in the sample table is used to train the business model.

10. A feature processing device, characterized in that, It includes: A construction module, configured to construct a target statement corresponding to the target primary key information in response to the target primary key information entered on the feature registration page, execute the target statement to obtain the initial feature and load it onto the feature processing page for display; A determination module, configured to determine the associated feature in response to a processing instruction submitted for the initial feature loaded on the feature processing page, and construct the target feature according to the associated feature and the initial feature; A configuration module, configured to configure an offline research task for the target feature and construct a model training sample corresponding to the target feature by executing the offline research task; A storage module, configured to store the model training sample in a sample table, wherein the model training samples stored in the sample table are used to train a business model.

11. A computing device, characterized in that, Comprising: A memory and a processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer-readable storage medium, characterized in that, It stores computer-executable instructions, and when the computer-executable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 8 are implemented.

13. A computer program product, characterized in that, Comprising a computer program or instructions, and when the computer program or instructions are executed by the processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images