Sample generation method, device, server and storage medium

Abstract

The embodiment of the present application provides a sample generation method, device, server and storage medium, wherein the method includes: obtaining business data based on the business log of a historical time period, wherein the business data includes multiple sample skeletons and a feature table, wherein the feature table records the sample features associated with the objects and businesses in the historical time period; querying the sample features corresponding to the sample skeleton from the feature table according to the feature identifier included in the sample skeleton; obtaining the original sample according to the sample skeleton and the sample features corresponding to the sample skeleton; obtaining sample data for training based on the original sample and the business scenario identifier in the original sample. The embodiment of the present application can improve the reusability of the sample generation scheme between different businesses.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically to a sample generation method, device, server, and storage medium. Background Art

[0002] With the development of machine learning, especially deep learning, machine learning models have been widely applied in numerous businesses. Using samples as input data for training and updating machine learning models, and optimizing sample generation schemes, has been a hot topic of research for those skilled in the art.

[0003] The current sample generation scheme is mainly designed based on the characteristics of the business, which makes it difficult to reuse the sample generation scheme between different businesses. It is often necessary to redesign the sample generation scheme for new businesses. Therefore, how to provide a new sample generation scheme to improve the reusability of the sample generation scheme between different businesses has become a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a sample generation method, apparatus, server, and storage medium to improve the reusability of sample generation solutions among different services.

[0005] To achieve the above objectives, the present invention provides the following technical solutions:

[0006] A sample generation method, comprising:

[0007] Based on the business logs of the historical time period, business data is obtained, wherein the business data includes multiple sample skeletons and a feature table, wherein the feature table records sample features associated with objects and businesses in the historical time period;

[0008] According to the feature identifier included in the sample skeleton, query the sample feature corresponding to the sample skeleton from the feature table;

[0009] According to the sample skeleton and the sample features corresponding to the sample skeleton, the original sample is obtained;

[0010] Based on the original samples and the business scenario identifiers in the original samples, sample data for training is obtained.

[0011] The present application also provides a sample generation device, including:

[0012] A business data acquisition module is used to acquire business data based on business logs of a historical time period. The business data includes multiple sample skeletons and a feature table. The feature table records sample features associated with objects and businesses in the historical time period.

[0013] A sample feature query module, configured to query the sample features corresponding to the sample skeleton from the feature table according to the feature identifiers included in the sample skeleton;

[0014] The original sample acquisition module is used to obtain the original sample based on the sample skeleton and the sample features corresponding to the sample skeleton;

[0015] The sample data acquisition module is used to obtain sample data for training based on the original samples and the business scenario identifiers in the original samples.

[0016] The present application also provides a sample generation device, including:

[0017] A feature generator is configured to obtain business data based on business logs for a historical time period, the business data including multiple sample skeletons and a feature table, the feature table recording sample features associated with objects and businesses within the historical time period; query the feature table for sample features corresponding to the sample skeleton based on feature identifiers included in the sample skeleton; and obtain the original sample based on the sample skeleton and the sample features corresponding to the sample skeleton;

[0018] The sample generator is used to obtain sample data for training based on the original samples and the business scenario identifiers in the original samples.

[0019] An embodiment of the present application also provides a server, comprising: at least one memory and at least one processor; the memory stores one or more computer-executable instructions, and the processor calls the one or more computer-executable instructions to execute the sample generation method as described above.

[0020] An embodiment of the present application further provides a storage medium, wherein the storage medium stores one or more computer-executable instructions, and the one or more computer-executable instructions are used to execute the sample generation method as described above.

[0021] In an embodiment of the present application, a sample is composed of a sample skeleton and sample features. Since the business data includes multiple sample skeletons and a feature table that records sample features, and the sample skeleton includes at least a feature identifier for querying sample features, after obtaining the business data, the embodiment of the present application can query the sample features corresponding to the sample skeleton from the feature table based on the feature identifier included in the sample skeleton, thereby associating the sample features corresponding to the sample skeleton on the basis of the sample skeleton to obtain the original sample, and then based on the original sample and the business scenario identifier in the original sample, obtain the sample data used for training in the corresponding business scenario. In this process, since the sample features are stored in the feature table, the sample features can be decoupled from the business logic and matched and associated through the feature identifier included in the sample skeleton. This allows the sample features of different businesses to be recorded in the feature table and matched and associated with the feature identifier of the sample skeleton to achieve sample generation. Different businesses can reuse the sample generation scheme of the embodiment of the present application to generate samples, which improves the reusability of the sample generation scheme provided by the embodiment of the present application between different businesses. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0023] Figure 1 A flow chart of a sample generation method provided in an embodiment of the present application;

[0024] Figure 2 Another flow chart of the sample generation method provided in an embodiment of the present application;

[0025] Figure 3 A block diagram of a sample generation component provided in an embodiment of the present application;

[0026] Figure 4 This is an example diagram of processing tiled samples and compressed samples provided in an embodiment of the present application;

[0027] Figure 5 A flowchart of compressing tiled original samples provided in an embodiment of the present application;

[0028] Figure 6 An example diagram of a tiled original sample provided in an embodiment of the present application;

[0029] Figure 7 An example diagram of a structured original sample provided in an embodiment of the present application;

[0030] Figure 8 This is an example diagram of the sample generation stage provided in an embodiment of the present application;

[0031] Figure 9 A block diagram of a sample generation device provided in an embodiment of the present application;

[0032] Figure 10 Another block diagram of the sample generation device provided in an embodiment of the present application;

[0033] Figure 11 Another block diagram of the sample generation device provided in an embodiment of the present application;

[0034] Figure 12 Another block diagram of a sample generation device provided in an embodiment of the present application;

[0035] Figure 13 Yet another block diagram of a sample generation device provided in an embodiment of the present application;

[0036] Figure 14 A block diagram of a server provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0037] As the technology of machine learning develops into a deeper stage, technical personnel in this field increasingly need a flexible and convenient sample generation solution to solve the many problems existing in the existing machine learning mechanism. Although the current sample generation solution based on streaming data can realize the timely training and updating of machine learning models, the sample generation solution is mainly designed according to the characteristics of the business and provided to the machine learning model in the business production link for training and updating. The data link in the business production link generally does not change frequently, which makes the sample design based on the premise that the data form of the sample no longer changes frequently, resulting in the various processing links of the sample data being more coupled with the specific business. When there is a need to train a machine learning model for a new business, it is necessary to create a new sample data stream for the new business and redesign the sample generation solution, which makes it difficult to reuse a set of sample generation solutions between different businesses.

[0038] The further problems that need to be solved at present are: the sample generation scheme is difficult to support the iteration of sample-rich features and iteration between samples, resulting in low iterativeness; at the same time, the sample data generated by the current sample generation scheme is highly coupled with the business, resulting in low versatility of the sample data.

[0039] In order to solve the problems existing in the prior art, the embodiments of the present application provide a new sample generation scheme to improve the reusability of the sample generation scheme between different businesses, and further improve the iterativeness of the sample generation scheme and the versatility of the sample data. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0040] In an optional implementation, Figure 1 The flow chart of the sample generation method provided by the embodiment of the present application is shown. The method can be applied to a server, which can be a server on the business side, for training and updating machine learning models for business, such as Figure 1 As shown, the method flow may include:

[0041] Step S100: Based on the business logs of a historical time period, business data is obtained, where the business data includes a plurality of sample skeletons and a feature table, where the feature table records sample features associated with objects and businesses within the historical time period.

[0042] Optionally, the business referred to in the embodiments of the present application may be any business to which a machine learning model is applied, such as a business for predicting the click-through rate of a creative script, a business for predicting the interests of an object, an information recommendation business, etc., and the embodiments of the present application are not limited thereto. It should be noted that the creative script is mainly disseminated in the form of text, sound, image, video, etc., and is the script content used for publicity and promotion of the object.

[0043] The business log of the historical time period can record: the behavior information of the objects associated with the business during the historical time period; taking the business involving showing creative scripts to objects as an example, the business log of the historical time period may include: the log of the objects watching the creative scripts during the historical time period, the log of the objects clicking on the creative scripts during the historical time period, etc.; that is, the business log of the historical time period comes from the various behavior information associated with the objects and the business during the historical time period recorded by the business system, and the business log of the historical time period can be stored in the database. It should be noted that the historical time period can be a historical time period measured in hours from the current time, and accordingly, the business data obtained based on the business log of the historical time period can provide real-time sample features for real-time training and updating of machine learning models; of course, the historical time period can also be a historical time period measured in days from the current time, etc., which is not limited by the embodiments of the present invention.

[0044] Optionally, an embodiment of the present application may extract information from business logs of a historical time period to obtain the business data, and the business data may be streaming data, such as a business data stream; in an embodiment of the present application, the business data may be composed of basic data constituting a sample. In one example, a sample generally includes three types of information: sample description information, sample features, and sample labels, and each type of information contains more fine-grained information; for example, sample features can be subdivided into more fine-grained features, such as object portrait features, daily features of the object and business association, real-time features of the object and business association, etc., and user portrait features include basic data such as user age, user city, and user gender; in an embodiment of the present application, business data may be composed of these basic data.

[0045] Furthermore, in an embodiment of the present application, the basic data in the business data can form multiple sample skeletons and feature tables corresponding to the historical time period. Taking the sample including three types of information: sample description information, sample features, and sample labels as an example, the sample skeleton can be considered as information in the sample other than the sample features, mainly including some basic sample description information (such as sample identification, sample label, etc.), and feature identification (identification such as ID) required for querying sample features; based on the business log of the historical time period, the embodiment of the present application can construct multiple samples, so the business data obtained based on the business log can include multiple sample skeletons;

[0046] In one example, taking a business involving displaying a creative script to an object as an example, an embodiment of the present application can extract basic data such as the creative script identifier, object identifier, whether the object clicked the creative script, the creative script display timestamp, and the object's device type from the business log of the historical time period, and organize the sample labels and sample identifiers, thereby merging these data to form a sample skeleton;

[0047] In an embodiment of the present application, the business data may further include a feature table for recording sample features of objects associated with businesses within a historical time period; optionally, one feature table may uniformly record multiple types of sample features associated with objects and businesses within a historical time period; in another optional implementation, an embodiment of the present application may also set up multiple types of feature tables, one type of feature table is used to record a type of sample features associated with objects and businesses within a historical time period, and different types of sample features are recorded in different types of feature tables. Specifically, an embodiment of the present application may extract various types of sample features associated with objects and businesses within a historical time period from a business log of a historical time period, and record different types of sample features in different types of feature tables. Taking the example of a business involving displaying a creative script to an object, the feature table may be, for example: an object feature table for recording object identifiers and object features corresponding to objects that viewed and / or clicked on the creative script within a historical time period, a creative script feature table for recording creative script identifiers and creative script features corresponding to creative scripts displayed and / or clicked within a historical time period;

[0048] Furthermore, the feature table can also be divided into a real-time feature table that records sample features in real time, and a daily feature table that records sample features on a daily basis, etc. Of course, the embodiment of the present invention does not exclude feature tables that record sample features on other time periods; accordingly, when a real-time feature table is used, the historical time period can be a historical time period from the present time, measured in hours; when a daily feature table is used, the historical time period can be a historical time period from the present time, measured in days; further, when multiple feature tables are set, the real-time feature table can have multiple types, and one type of real-time feature table can record a type of real-time sample features associated with an object and a business; the daily feature table can have multiple types, and one type of daily feature table can record a daily sample feature associated with an object and a business;

[0049] It should be noted that real-time sample features are relative to daily sample features. Real-time sample features are used to calculate sample features with a shorter time granularity. For example, real-time sample features generally represent sample features within the last few hours of object association with the business. For example, the features of the creative scripts clicked by the object in the last 3 hours belong to real-time sample features, while daily sample features are used to calculate sample features with a longer time granularity. For example, the features of the creative scripts clicked by the object in the last 3 days belong to daily sample features.

[0050] Step S110: According to the feature identifier included in the sample skeleton, query the sample features corresponding to the sample skeleton from the feature table.

[0051] In an embodiment of the present application, each sample skeleton may include at least a feature identifier for querying sample features. For example, each sample skeleton may include at least an object identifier for querying object features, a creative script identifier for querying creative script features, etc. Of course, each sample skeleton may also include other content, such as sample labels, sample identifiers, and other sample description information. Based on the feature identifiers included in each sample skeleton, the embodiment of the present application can query the sample features from the feature table to obtain the sample features corresponding to each sample skeleton.

[0052] As an example, taking the business involving displaying a creative script to an object as an example, a sample skeleton may include static feature identifiers such as a creative script identifier and an object identifier. Since the feature identifiers and sample features have a unique correspondence, such as each creative script corresponds to a unique creative script identifier, and each object corresponds to a unique object identifier, based on the feature identifiers contained in each sample skeleton, the embodiment of the present application can query the feature table to obtain the sample features corresponding to each sample skeleton;

[0053] In a more specific optional implementation, if a type of feature table records a type of sample features, such as an object feature table records object features associated with objects and businesses within a historical time period, and a creative script feature table records creative script features of creative scripts displayed to objects within a historical time period, and / or creative script features of creative scripts clicked by objects, then the embodiment of the present application can query and obtain sample features from the feature table of the corresponding type based on the feature type of the feature identifier contained in each sample skeleton. For example, the embodiment of the present application can query and obtain object features from the object feature table based on the object identifier contained in the sample skeleton, and query and obtain creative script features from the creative script feature table based on the creative script identifier contained in the sample skeleton.

[0054] Furthermore, to improve the query efficiency of sample features, the feature table can be in the form of key-value, where the key in the feature table is the feature identifier and the value is the content of the specific sample feature.

[0055] It can be understood that the sample mainly includes three types of information: sample description information, sample features and sample labels. On the basis that the sample skeleton includes other information in addition to the sample features, the embodiment of the present application further obtains the sample features corresponding to each sample skeleton, so as to prepare all the information that constitutes the sample.

[0056] Step S120: Obtain the original sample based on the sample skeleton and the sample features corresponding to the sample skeleton.

[0057] In an optional implementation, the embodiment of the present application may associate the sample features corresponding to the sample skeleton on the basis of the sample skeleton to form an original sample, wherein a sample skeleton and the sample features corresponding to the sample skeleton may form an original sample, so that the embodiment of the present application may form multiple original samples; optionally, the multiple original samples may be in the form of streaming data, for example, the multiple original samples may form an original sample data stream.

[0058] Step S130: Acquire sample data for training based on the original sample and the business scenario identifier in the original sample.

[0059] In order to provide sample data suitable for business scenario training, after obtaining the original samples, the embodiments of the present application can obtain sample data for training based on the original samples and the business scenario identifiers in the original samples; for example, when multiple original samples are obtained, the embodiments of the present application can aggregate the original samples with the same business scenario identifier according to the business scenario identifier of the original samples, so that the original samples of the same business scenario can be output to the training engine through one channel to obtain sample data suitable for use by the training engine of the corresponding business scenario; for example, to train machine learning models for multiple business scenarios, the embodiments of the present application can combine original samples with different business scenario identifiers based on the original samples and the business scenario identifiers in the original samples, so as to obtain sample data suitable for training in multiple business scenarios; the manner in which the embodiments of the present application obtain sample data for training based on the original samples and the business scenario identifiers in the original samples can be determined according to the training requirements of the actual business scenario, and the embodiments of the present application are not limited to this.

[0060] Furthermore, in an optional implementation, the data structure of the original sample obtained in step S120 may be a flat structure, that is, the data in the original sample is recorded in a manner similar to a two-dimensional table. For example, the sample features in the original sample are arranged in columns, and each column represents a type of feature. Based on the original sample with a flat structure, the embodiment of the present application may output the sample data for training in a flat structure in step S130.

[0061] In another optional implementation, the embodiment of the present application may further process the original sample of the tiled structure obtained in step S120 in terms of the dimension of the sample features and / or the dimension of the sample before executing step S130, and then execute step S130 on the processed original sample to obtain sample data for training.

[0062] Optionally, the sample data for training obtained in the embodiment of the present application may be in the form of streaming data. For example, step S130 may obtain a stream of sample data for training.

[0063] In an embodiment of the present application, a sample is composed of a sample skeleton and sample features. Since the business data includes multiple sample skeletons and a feature table that records sample features, and the sample skeleton includes at least a feature identifier for querying sample features, after obtaining the business data, the embodiment of the present application can query the sample features corresponding to the sample skeleton from the feature table based on the feature identifier included in the sample skeleton, thereby associating the sample features corresponding to the sample skeleton on the basis of the sample skeleton to obtain the original sample, and then based on the original sample and the business scenario identifier in the original sample, obtain the sample data used for training in the corresponding business scenario. In this process, since the sample features are stored in the feature table, the sample features can be decoupled from the business logic and matched and associated through the feature identifier included in the sample skeleton. This allows the sample features of different businesses to be recorded in the feature table and matched and associated with the feature identifier of the sample skeleton to achieve sample generation. Different businesses can reuse the sample generation scheme of the embodiment of the present application to generate samples, which improves the reusability of the sample generation scheme provided by the embodiment of the present application between different businesses.

[0064] In an optional implementation, Figure 2 Another process of the sample generation method provided by the embodiment of the present application is shown. Figure 2 , the process may include:

[0065] Step S200: Acquire business data based on the business logs of the historical time period. The business data includes multiple sample skeletons and a feature table. The feature table records sample features associated with objects and businesses in the historical time period.

[0066] Step S210: According to the feature identifier included in the sample skeleton, query the sample features corresponding to the sample skeleton from the feature table.

[0067] Step S220: Obtain the original sample based on the sample skeleton and the sample features corresponding to the sample skeleton.

[0068] Optionally, the introduction to step S200 to step S220 may refer to the description of the corresponding part above.

[0069] Step S230: Process the sample features in the multiple original samples, and / or process the multiple original samples.

[0070] Based on the obtained multiple original samples (the multiple original samples can form an original sample data stream), the embodiment of the present application can process the multiple original samples from the latitude of sample characteristics and / or from the latitude of samples.

[0071] From the perspective of sample features, processing the original samples can be considered as: taking the sample features in multiple original samples as the processing objects, and processing the sample features in multiple original samples to achieve the processing of the original samples;

[0072] From the perspective of samples, processing the original samples can be considered as: taking the original samples among multiple original samples as processing objects, and processing the original samples among multiple original samples to realize the processing of the original samples.

[0073] Step S240: Obtain sample data for training based on the original sample and the business scenario identifier in the original sample.

[0074] After the original sample is processed based on step S230, the embodiment of the present application can obtain sample data for training based on the processed original sample and the business scenario identifier in the original sample.

[0075] It should be noted that, through the processing of step S230, the embodiment of the present application can provide sample data for use in the training and updating of the machine learning model in the business production link in the stage of step S240, and can also provide sample data for use in the training and updating of the machine learning model in the business experiment link, which will be explained separately below.

[0076] It needs to be explained that in the actual operation of the business, for the security and stability of the business, each business has a master data stored in the database. Based on this master data, multiple real-time sample data stream outputs can be created to serve the machine learning model of the business production link, or some changes can be made to the master data to form a machine learning model that serves the business experimental link; generally speaking, the data link of the business production link will not change in the short term, so generally in the experimental link, after verifying that the changes in the data link are effective, the changes in the experimental link will be applied to the business production link.

[0077] On the one hand, the embodiments of the present application can provide sample data for training and updating for the machine learning model in the business production link. For the sake of convenience, the embodiments of the present application can refer to this sample data as production sample data. The generated sample data can be in the form of a data stream, such as multiple production sample data forming a production sample data stream.

[0078] The embodiment of the present application can process the sample features in the original sample from the dimension of sample features, and then obtain sample data for training based on the business scenario identifier in the original sample for the processed original sample to obtain production sample data; optionally, the embodiment of the present application can perform cross-combination and other processing on different sample features in multiple original samples to realize the processing of the original sample from the dimension of sample features;

[0079] It should be noted that cross-combining different sample features of multiple original samples is an operation in sample feature engineering. In theory, any different sample features can be cross-combined, but it is also necessary to analyze whether the cross-combination of sample features is beneficial to the training and updating of the machine learning model based on actual conditions; in one example, an embodiment of the present application can cross-combine creative script features with business features to characterize creative scripts in different business scenarios.

[0080] The embodiment of the present application may also process the original sample from the sample dimension, and then obtain sample data for training based on the business scenario identifier in the original sample for the processed original sample to obtain production sample data; Optionally, if the original sample obtained in step S120 is a flat data structure, the embodiment of the present application may compress the flat original sample to form a structured original sample, so as to realize the processing of the original sample from the sample dimension;

[0081] In an optional implementation of compressing original samples of a tiled structure, an embodiment of the present application may retain one copy of multiple identical sample features from multiple original samples and store it in a feature storage space, where a summary value of the sample feature is used to index the sample feature; at the same time, at least a sample identifier and a sample label in the original sample are associated with the summary value of the sample feature of the original sample, where the summary value indexes the sample feature of the original sample stored in the feature storage space;

[0082] After the original sample of the flat structure is compressed in the above manner, the original sample may not directly carry the sample features. Instead, the sample features stored in the feature storage space are indexed by the summary value of the sample features in the original sample, thereby realizing the association between the sample identifier and sample label in the original sample and the sample features stored in the feature storage space, thereby reducing the amount of data in the original sample; and, multiple copies of the same sample features in multiple original samples are retained in the feature storage space, which enables multiple original samples with the same sample features to share a copy of the sample features retained in the feature storage space through the summary value of the sample features, thereby greatly reducing the amount of data in the compressed original sample and realizing data reuse of the sample features;

[0083] Compared with the original samples with a flat structure, the throughput efficiency of the structured original samples is higher. For example, for N original samples with the same sample features, after structured compression in the embodiment of the present application, the N structured original samples do not carry specific sample features, but instead share a copy of the same sample features stored in the feature storage space through the summary value. As a result, the data volume of the N structured original samples can be greatly reduced, and the same sample features can be shared. It can be seen that the structured compression of original samples with the same sample features can enable multiple structured original samples to reuse the same sample features, thereby improving the data versatility of the sample features and greatly saving sample storage and computing resources.

[0084] In other possible implementations, the embodiments of the present application may also process the original samples from the sample dimension by filtering, deduplicating, and other processing.

[0085] It should be noted that the production sample data can be provided to the training engine in real time, so that the training engine can train the machine learning model in real time based on the production sample data; in order to achieve real-time training effects, the sample features in the embodiment of the present application may at least include real-time sample features; of course, the embodiment of the present application can also support training and updating the machine learning model with daily sample features, and the embodiment of the present application is not limited to this.

[0086] On the other hand, the embodiments of the present application can provide sample data for training and updating for the machine learning model in the business experiment phase, so as to be used for experimental verification of the machine learning model. For the convenience of explanation, the embodiments of the present application refer to this sample data as experimental sample data. The experimental sample data can be in the form of a data stream, such as multiple experimental sample data forming an experimental sample data stream; the experimental sample data is relative to the production sample data. Generally, the production sample data serves most of the online traffic, while the experimental sample data serves a small part of the traffic and is used to verify the experimental machine learning model; in the experimental phase, there is a need for frequent changes in sample data, for example, it is necessary to add new sample features to conduct experiments on the machine learning model, or to combine multiple samples to train the machine learning model, etc. Based on this, the embodiments of the present application can use sample features and samples as the dimensions of data change to process the original samples;

[0087] In an optional implementation, the embodiment of the present application may perform feature iteration on the sample features in the original sample to process the sample features in the original sample from the dimension of the sample features, and then obtain sample data for training based on the original sample and the business scenario identifier in the original sample for the processed original sample to obtain experimental sample data; in a more specific optional implementation description, feature iteration, such as adding new sample features, combining sample features, etc., takes adding new sample features to the original sample as an example, on the basis that the original sample has been associated with most of the sample features contained in the feature table, the embodiment of the present application may perform feature iteration processing on the sample features in the original sample to add new sample features.

[0088] In other optional implementations, the embodiments of the present application may perform sample iteration on multiple original samples to process the original samples from the sample dimension. The specific implementation can refer to the description of the relevant parts above. For example, the embodiments of the present application may perform structured compression on the flattened original samples to form structured original samples to process the original samples from the sample dimension. For example, the embodiments of the present application may also filter, deduplicate, and perform other processing on the original samples to process the original samples from the sample dimension.

[0089] In an optional implementation, taking filtering of original samples as an example, if the original sample is sampled as a negative sample, the embodiment of the present application can filter the original sample; in a more specific optional implementation, the embodiment of the present application can sample whether the original sample is a positive sample or a negative sample based on the sample label in the original sample; if the original sample is a negative sample, the embodiment of the present application can randomly discard the original sample according to a set probability to achieve filtering of the original sample; it should be noted that the sample label can identify whether the original sample is a positive sample or a negative sample. For example, after the creative script is displayed to the object, the sample label of the original sample of the object clicking on the creative script can be set to a first value (such as 1) and regarded as a positive sample. The sample label of the original sample of the object only browsing but not clicking on the creative script can be set to a second value (such as 0) and regarded as a negative sample; of course, the sample label can depend on specific business needs. For example, for short video businesses, the original sample of the creative script that the object browsed for a certain period of time can be regarded as a positive sample.

[0090] Whether generating production sample data or experimental sample data, the embodiment of the present application can obtain sample data for training based on the original sample and the business scenario identifier in the original sample after processing the original sample; in an optional implementation, the embodiment of the present application can aggregate original samples with the same business scenario identifier based on the business scenario identifier in the original sample to obtain sample data for the training engine suitable for the corresponding business scenario;

[0091] In another optional implementation, the embodiment of the present application may split the original sample based on the business scenario identifier in the original sample; it should be noted that different business scenario identifiers correspond to different business scenarios, for example, predicting object interests, live broadcast shopping, shopping lists, etc. belong to different business scenarios and may have different business scenario identifiers. When the business scenario involves presenting a creative script to an object, the user watches the creative script in which business scenario, and the original sample will carry the business scenario identifier of the business scenario in which the object watches the creative script; the embodiment of the present application may, based on the business scenario identifier of the original sample, split the original sample into multiple samples under different business scenarios when there are multiple business scenario identifiers in the original sample to output multiple channels of sample data; in a specific implementation, if an original sample has different business scenario identifiers, the embodiment of the present application may split the original sample into sample data of different business scenarios based on different business scenario identifiers;

[0092] In other optional implementations, the embodiments of the present application may merge multiple original samples identified by different business scenarios into one piece of sample data, so that the merged sample data can be applicable to the combination of multiple business scenarios. For example, for the original samples of predicted object interests and the original samples of live broadcast room shopping, in order to experiment the effect of merging the original samples of these two business scenarios together to train the machine learning model, the embodiments of the present application may merge the original samples of predicted object interests and the original samples of live broadcast room shopping into one piece of sample data.

[0093] It can be seen that in the embodiment of the present application, both production sample data and experimental sample data can be obtained based on the original samples. This approach not only ensures the independence of production sample data and experimental sample data, but also saves the resource consumption and data redundancy brought about by creating a new data link for business production or business experiments from beginning to end; and, by performing sample feature iteration or sample iterative processing on the original samples, the sample generation scheme provided in the embodiment of the present application can support sample-rich feature iteration and iteration between samples, thereby improving the iterativeness of the sample generation scheme provided in the embodiment of the present application.

[0094] It should also be noted that the sample generation scheme provided in the embodiment of the present application can have strong scalability, support the simulation of real-time sample features, and be used for real-time training and updating of machine learning models; for example, when the data of the object is static and cannot form a real-time data stream, the embodiment of the present application can store the original samples in chronological order in the database (the original samples stored in the database can be a flat data structure), and then obtain the original samples stored in chronological order from the database to obtain streaming original sample data; after processing the original samples and obtaining sample data for training based on the original samples and the business scenario identifiers in the original samples, the sample data can be streamed output to the training engine in chronological order, so that the machine learning model can be trained and updated based on the streaming sample data.

[0095] It should be noted that with the development of business, the behavior of objects under the business changes more frequently. The traditional way of offline training and updating of machine learning models on a daily basis can no longer meet the real-time requirements of machine learning model training and updating. Therefore, online learning mechanisms based on real-time streaming samples have emerged. There are many types of online learning mechanisms, among which ODL (Online Deep Learning) as a typical online learning mechanism has been widely used in many business scenarios and has enabled businesses to achieve significant business growth. The embodiment of the present invention is based on an online learning mechanism and can provide sample data for training and updating for machine learning models in real time; in an optional implementation, the embodiment of the present application can provide a sample generation component to generate production sample data and experimental sample data, which not only ensures the independence of production sample data and experimental sample data, but also saves the need to build a new data link for business production or business experiments from beginning to end; such as Figure 3 As shown, the sample generation component provided in the embodiment of the present application may include:

[0096] Feature Builder 01, Sample Pool 02, Sample Builder 03; Optionally, the Feature Builder and Sample Builder can be in software form, hardware form, or a combination of software and hardware form. The Sample Pool can be considered as a storage area for storing data, such as a database.

[0097] Combine Figure 3As shown, the embodiment of the present application is based on the business log of the historical time period. After obtaining the business data, the feature generator can query the feature table based on the feature identifier contained in the sample skeleton to obtain the sample features corresponding to each sample skeleton, so that the feature generator generates multiple original samples based on each sample skeleton and the sample features corresponding to each sample skeleton; in one example, for a sample skeleton, the feature generator can query the creative script feature and object feature corresponding to the sample skeleton from the creative script feature table and the object feature table based on the creative script identifier and object identifier contained in the sample skeleton, so that the sample skeleton and the creative script feature and object feature corresponding to the sample skeleton form an original sample; it should be noted that the feature generator can form multiple original samples of a business for the business (the multiple original samples can form a streaming original sample data stream), so that for multiple businesses, the feature generator can form original samples corresponding to each business;

[0098] On the one hand, the feature generator sends the original samples to the sample pool for storage. For example, the feature generator can send multiple original samples corresponding to each business to the sample pool for storage so that they can be reused in subsequent experimental stages. On the other hand, the feature generator can send the original samples to the sample generator in real time, and the sample generator generates sample data in real time for real-time training and updating of machine learning models.

[0099] Specifically, the sample generator may provide sample feature dimensions and / or operators of sample dimensions, thereby processing sample features in the original sample, and / or processing multiple original samples from the sample dimension. For example, the sample generator may cross-combine different sample features in multiple original samples, compress the original samples of the flat structure into structured original samples, etc.; after processing the original samples, the sample generator may obtain sample data for training based on the original samples and the business scenario identifiers in the original samples; the sample data generated by the sample generator may be transmitted to the training engine, so that the training engine trains and updates the machine learning module based on the sample data;

[0100] Furthermore, for the original samples stored in the sample pool, on the one hand, the feature generator can read the original samples from the sample pool, and perform feature iteration processing such as adding new sample features and feature combination on the sample features in the original samples, so that the original sample data stream after feature iteration is sent to the sample generator, and the sample generator performs further processing (the processing process of the sample generator here can refer to the processing process described above). After processing the original samples, the sample generator can obtain sample data for training based on the original samples and the business scenario identifiers in the original samples.

[0101] The embodiments of the present application support the implementation of the sample generation method provided by the embodiments of the present application by providing a sample generation component, so that different businesses can use the sample generation component provided by the embodiments of the present application to achieve sample generation, thereby improving the reusability of the sample generation solution provided by the embodiments of the present application between different businesses; and can improve the iterative nature of the sample generation solution provided by the embodiments of the present application and the data universality of the samples generated by the embodiments of the present application.

[0102] The structure of the sample data provided in the embodiment of the present application can be flat or structured after compression. Accordingly, the embodiment of the present application can provide flat samples or compressed samples. Among them, the sample features in the flat samples are stored in columns, which has the characteristics of versatility and strong readability, and can facilitate the access of new services, but its storage and calculation efficiency is not high. The compressed sample is a structured sample generated after compressing the flat sample on the basis of the flat sample. It has the advantages of saving storage space and high calculation efficiency.

[0103] In one example, Figure 4 As shown, in Figure 3 Based on the sample generation component shown, the original samples generated by the feature generator can be tiled samples (i.e., original samples with tiled structure), and accordingly, the tiled samples can be stored in the sample pool; after the tiled samples generated by the feature generator or the tiled samples stored in the sample pool are provided to the sample generator, the sample generator can perform structured compression on the tiled samples to generate compressed samples (i.e., the original samples structured after compression), and then obtain sample data for training based on the compressed samples and the business scenario identifiers in the compressed samples, thereby providing the sample data to the machine learning model for training and updating of the machine learning model; tiled samples and compressed samples can be applicable in different machine learning model training and update scenarios of the business.

[0104] In an optional implementation, Figure 5 An optional process of compressing the flattened original samples to form structured original samples is shown. This process can be performed before obtaining sample data for training based on the original samples and business scenario identifiers in the original samples; Figure 5 As shown, the process may include:

[0105] Step S300: At least the sample identifier and the sample label in the original sample are associated with the summary value of the sample feature of the original sample, and the summary value indexes the sample feature of the original sample stored in the feature storage space.

[0106] Step S310: retain one copy of the same sample features from the multiple original samples and store it in a feature storage space, and use the summary value of the sample feature to index the sample feature in the feature storage space.

[0107] In an optional implementation, the sample generator may obtain the original sample transmitted by the feature generator, or read the original sample from the sample pool, and the original sample may be a tiled structure; for multiple original samples, the sample generator may retain one copy of multiple identical sample features in the multiple original samples, and store them in a feature storage space, and use the summary value of the sample feature to index the sample feature in the feature storage space; at the same time, the sample generator may associate the sample identifier and sample label in the original sample with the summary value of the sample feature of the original sample, and use the summary value to index the sample feature of the original sample stored in the feature storage space.

[0108] It should be noted that the embodiments of the present application can support the compression of original samples with the same sample features in different dimensions. Taking the business of predicting object interests as an example, if the business involves displaying a creative script to the object, each time the object refreshes the business page, it will request the data of the creative script from the server. In this process, many sample features are generally updated once a day. For example, when the creative script data is requested multiple times for the same object, the object-side features are the same. Based on this, the embodiments of the present application can compress original samples with the same object features and the same creative script features from the dimensions of object features and creative script features.

[0109] In a specific implementation, the feature storage space for storing sample features may include: an object feature storage space for storing object features, and a creative script feature storage space for storing creative script features; thus, the embodiment of the present application may retain one copy of multiple identical object features in multiple original samples and store it in the object feature storage space, and use the summary value of the object feature in the object feature storage space to index the stored object features; retain one copy of multiple identical creative script features in multiple original samples and store them in the creative script feature storage space, and use the summary value of the creative script feature in the creative script feature storage space to index the stored creative script features; further, the embodiment of the present application may at least associate the sample identifier and sample label in the original sample with the summary value of the object feature of the original sample and the summary value of the creative script feature, the summary value of the object feature indexes the object feature of the original sample stored in the object feature storage space, and the summary value of the creative script feature indexes the creative script feature of the original sample stored in the creative script feature storage space;

[0110] After processing in the above manner, the original sample does not directly carry object features and creative script features. Instead, the object features are indexed in the object feature storage space and the creative script features are indexed in the creative script feature storage space through the summary value of the object features and the summary value of the creative script features, respectively, which can greatly reduce the data volume of the original sample; at the same time, only one of the multiple identical object features in the object feature storage space is retained, and only one of the multiple identical creative script features in the creative script feature storage space is retained, which can enable efficient data reuse of object features and creative script features.

[0111] In one example, taking a business involving presenting a creative script to an object, Figure 6 It shows multiple original samples of the flat structure obtained based on the business log of the historical time period, which can be used for reference. Figure 6 In the example, the sample identifier is used to uniquely identify the original sample; the sample label can be used to indicate whether the creative script corresponding to the sample is clicked by the object, wherein the sample label is 1, indicating that the creative script is clicked, and the sample label is 0, indicating that the creative script is not clicked; the object feature summary value is the summary value of the object feature; the object identifier and the object other features are two object features on the object side, wherein the object identifier is used to uniquely identify the object, and the object other features are the object features on the object side other than the object identifier, such as the sequence of categories to which the creative scripts clicked by the object in the historical time period belong; the creative script feature summary value is the summary value of the creative script feature; the creative script identifier and the creative script other features are two creative script features on the creative script side, wherein the creative script identifier is used to uniquely identify the creative script, and the creative script other features are the creative script features on the creative script side other than the creative script identifier, such as the subject words of the creative script, etc.

[0112] from Figure 6 It can be seen that the original samples 1001 and 1002 have the same object features u1, c1, c2; the original samples 1003, 1004 and 1005 have the same object features u2, c2, c3; similarly, the original samples 1001 and 1003 have the same creative script features a1, w1, w2; the original samples 1005 and 1006 have the same creative script features a4, w6, w7; further combined Figure 6 and Figure 7 As shown, the same object features u1, c1, c2 of the original samples 1001 and 1002 can be found in Figure 7 Only one copy of the object feature storage space is retained and indexed by the object feature summary value U1; the same object features u2, c2, c3 of the original samples 1003, 1004 and 1005 can be found in Figure 7Only one copy of the object feature storage space is retained and indexed by the object feature summary value U2; at the same time, the same creative script features a1, w1, w2 of the original samples 1001 and 1003 can be found in Figure 7 The creative script feature storage space shown in the figure only retains one copy and is indexed by the creative script feature summary value A1; the creative script features a4, w6, and w7 that are the same in the original samples 1005 and 1006 can be found in Figure 7 The creative script feature storage space shown is only retained in one copy and is indexed by the creative script feature summary value A4;

[0113] Furthermore, the different object features in the original samples can be stored independently in the object feature storage space and indexed by the summary value of the object feature, such as Figure 7 The object features u3 and c4 of the original sample 1006 are stored independently in the object feature storage space and indexed by the object feature summary value U3; the creative script features that are different from each other in the original sample can be stored independently in the creative script feature storage space and indexed by the summary value of the creative script feature, such as Figure 7 As shown, the creative script features a2, w3, and w4 of the original sample 1002 are independently stored in the creative script feature storage space and indexed by the creative script feature summary value A2. The creative script features a3 and w5 of the original sample 1004 are independently stored in the creative script feature storage space and indexed by the creative script feature summary value A3.

[0114] After storing the object features of the original sample in the object feature storage space and indexing them by the summary value of the object features, and storing the creative script features in the creative script feature storage space and indexing them by the summary value of the creative script features, the embodiment of the present application can index the sample identifier and sample label of the original sample by the summary value of the object features in the original sample, and index the object features stored in the object feature storage space by the summary value of the creative script features in the original sample, thereby achieving structured compression of the original sample; combined with Figure 7 As shown, the sample identifier 1003 and the sample label 1.0 of the original sample 1003 can be associated with the summary value U2 of the object feature and the summary value A1 of the creative script feature, so that the sample identifier 1003 and the sample label 1.0 of the original sample 1003 can index a1, w1, w2 in the creative script feature storage space through the summary value A1, and index u2, c2, c3 in the object feature storage space through the summary value U2.

[0115] The embodiment of the present application performs structured compression on the original samples, so that the data volume of the original samples is greatly reduced, and the same sample features in the sample feature storage space can be shared, thereby saving sample storage resources and improving sample computing efficiency.

[0116] In an optional implementation of the sample generation method provided in the embodiment of the present application, the sample generation stage can be as follows: Figure 8 Examples include the map phase, the subscribe phase, and the reduce phase;

[0117] Among them, the mapping stage is mainly used to form business data based on business logs of historical time periods, including obtaining sample skeletons and feature tables; the subscription stage is mainly reflected in two aspects, one is the subscription to different features, and the other is the subscription to different sample streams; the compression stage is mainly to dynamically compress the flattened original samples to provide structured samples.

[0118] The embodiment of the present application adopts the mapping-subscription-compression computing paradigm to provide a sample data stream for training a machine learning model, which can reduce the redundancy of sample data and meet iteration requirements. For example, when a new business needs to access the sample generation solution provided by the embodiment of the present application, the embodiment of the present application only needs to obtain the business data stream related to the business, so that the sample generator can reuse the existing sample feature system and quickly generate a sample data stream for machine learning model training. If the experiment has new sample features or samples from other business scenarios that affect the machine learning model, the embodiment of the present application only needs to merge the new sample features or new samples into the original sample data stream of the existing business (the original sample data stream of the existing business can be stored in a storage area such as a sample pool) to form a new sample data stream, without the need to create a new sample data stream from beginning to end.

[0119] Furthermore, the embodiments of the present application provide structured samples, which can reduce the cost of businesses accessing the sample generation solution provided by the embodiments of the present application and improve the training efficiency of machine learning models. For example, the sample features in the flat samples are stored by column, which is clear and easy to read, and has strong versatility. Each business can easily construct the sample data into a flat format. When it is necessary to improve the training efficiency of the machine learning model, the embodiments of the present application can compress the flat samples into a structured format, provide structured samples, and achieve efficient sample throughput.

[0120] The solution provided by the embodiment of the present application can achieve efficient reuse of sample data, save storage resources, and improve sample generation efficiency; at the same time, the concise structured samples also facilitate the access of various businesses and reduce the cost of business cold start; and the embodiment of the present application can support rich features and sample iteration requirements, and improve the experimental efficiency of machine learning models.

[0121] The above describes multiple embodiment schemes provided by the embodiments of the present application. The various optional methods introduced in each embodiment scheme can be combined and cross-referenced with each other without conflict, thereby extending a variety of possible embodiment schemes, which can all be considered as embodiment schemes disclosed and open in the embodiments of the present application.

[0122] The following describes the sample generation device provided in an embodiment of the present application. The sample generation device described below can be considered as the functional module required by the server to implement the sample generation method provided in an embodiment of the present application. The content of the sample generation device described below can be referenced in correspondence with the content of the sample generation method described above. In an optional implementation, the functional modules of the sample generation device described below can be respectively provided in the feature generator and sample generator of the sample generation component according to their functions.

[0123] Optional, Figure 9 An optional block diagram of a sample generation device provided in an embodiment of the present application is shown. Figure 9 As shown, the device may include:

[0124] The business data acquisition module 100 is used to acquire business data based on business logs of a historical time period. The business data includes multiple sample skeletons and a feature table. The feature table records sample features associated with objects and businesses in the historical time period.

[0125] A sample feature query module 110 is configured to query the sample feature corresponding to the sample skeleton from the feature table according to the feature identifier included in the sample skeleton;

[0126] The original sample obtaining module 120 is used to obtain the original sample according to the sample skeleton and the sample features corresponding to the sample skeleton;

[0127] The sample data acquisition module 130 is used to acquire sample data for training based on the original samples and the business scenario identifiers in the original samples.

[0128] Optional, Figure 10 Another optional block diagram of the sample generation device provided in the embodiment of the present application is shown, combined with Figure 9 and Figure 10 As shown, the sample generating device may further include:

[0129] The compression module 140 is used to associate at least the sample identification and sample label in the original sample with the summary value of the sample feature of the original sample before the sample data acquisition module 130 acquires the sample data for training based on the original sample and the scene identification in the original sample, and the summary value indexes the sample feature of the original sample stored in the feature storage space.

[0130] Optionally, the compression module 140 may also be configured to retain one copy of the same sample feature from multiple original samples and store the copy in a feature storage space, and use the summary value of the sample feature to index the sample feature in the feature storage space.

[0131] Optionally, the compression module 140 is configured to associate at least a sample identifier and a sample label in the original sample with a summary value of a sample feature of the original sample, where the summary value indexes the sample feature of the original sample stored in the feature storage space, including:

[0132] At least the sample identifier and sample label in the original sample are associated with the summary value of the object feature and the summary value of the creative script feature of the original sample, the summary value of the object feature indexes the object feature of the original sample stored in the object feature storage space, and the summary value of the creative script feature indexes the creative script feature of the original sample stored in the creative script feature storage space.

[0133] Optionally, the compression module 140 is configured to retain one copy of the same sample features from the multiple original samples and store the same features in the feature storage space, including:

[0134] Retain one copy of the same object features in multiple original samples and store it in the object feature storage space;

[0135] A copy of the same creative script features in multiple original samples is retained and stored in the creative script feature storage space.

[0136] Optionally, the compression module 140 is configured to index the sample feature in the feature storage space using the summary value of the sample feature, including:

[0137] Using the summary value of the object feature in the object feature storage space, indexing the stored object feature;

[0138] The creative script feature storage space uses the summary value of the creative script feature to index the stored creative script feature.

[0139] Optional, Figure 11 Another optional block diagram of the sample generation device provided in the embodiment of the present application is shown, combined with Figure 9 and Figure 11 As shown, the sample generating device may further include:

[0140] The sampling module 150 is used to sample whether the original sample is a positive sample or a negative sample based on the sample label in the original sample before the sample data acquisition module 130 acquires the sample data for training based on the original sample and the scene identifier in the original sample. If the original sample is a negative sample, the original sample is randomly discarded with a set probability.

[0141] Optional, Figure 12 Another optional block diagram of the sample generation device provided in the embodiment of the present application is shown, combined with Figure 9 and Figure 12 As shown, the sample generating device may further include:

[0142] Cross-combination module 160, used to cross-combine different sample features in multiple original samples;

[0143] The feature iteration module 170 is used to perform feature iteration processing on the sample features in the original sample to add new sample features.

[0144] Optionally, the cross-combination module 160 and the feature iteration module 170 may be used either alone or in combination.

[0145] Optionally, the sample data acquisition module 130 is configured to acquire sample data for training based on the original sample and the business scenario identifier in the original sample, including:

[0146] According to the business scenario identifier of the original sample, the original samples with the same business scenario identifier are aggregated to obtain sample data for the training engine suitable for the corresponding business scenario.

[0147] Optional, Figure 13 Another optional block diagram of the sample generation device provided in the embodiment of the present application is shown, combined with Figure 9 and Figure 13 As shown, the sample generating device may further include:

[0148] The original sample storage module 180 is used to store the original sample in the database;

[0149] The acquisition module 190 is configured to acquire the original sample from the database before the sample data acquisition module 130 acquires sample data for training based on the original sample and the business scenario identifier in the original sample.

[0150] Optionally, the original sample storage module 180 is configured to store the original sample in the database and includes:

[0151] Store the original samples in the database in chronological order;

[0152] The acquisition module 190 is configured to acquire the original sample from the database, including:

[0153] Read the raw samples stored in time order from the database to obtain the streaming raw samples.

[0154] Optionally, the embodiments of the present application may implement real-time training and updating of the machine learning model, and accordingly, the sample features may include real-time sample features.

[0155] The sample generation device provided in the embodiment of the present application can be set in a server. Optionally, the optional block diagram of the server can be as follows: Figure 14 As shown, it includes: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0156] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0157] Optionally, the communication interface 2 may be an interface of a communication module for performing network communication;

[0158] Optionally, processor 1 may be a CPU (central processing unit), a GPU (Graphics Processing Unit), an NPU (embedded neural network processor), an FPGA (Field Programmable Gate Array), a TPU (tensor processing unit), an AI chip, an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement an embodiment of the present application.

[0159] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0160] The memory 3 stores one or more computer-executable instructions, and the processor 1 calls the one or more computer-executable instructions to execute the sample generation method provided in the embodiment of the present application.

[0161] An embodiment of the present application further provides a storage medium, which can store one or more computer-executable instructions, and the one or more computer-executable instructions are used to execute the sample generation method provided in the embodiment of the present application.

[0162] Although the embodiments of the present application are disclosed above, the present application is not limited thereto. Any person skilled in the art may make various changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims.

Claims

1. A sample generation method, wherein: include: Based on the business logs of a historical time period, business data is obtained, the business data including multiple sample skeletons and a feature table, the feature table recording sample features associated with objects and businesses in the historical time period, the feature table being a single feature table that uniformly records multiple types of sample features associated with objects and businesses in the historical time period; or the feature table being a multi-category feature table, where one type of feature table is used to record one type of sample features associated with objects and businesses in the historical time period, and different types of sample features are recorded in different types of feature tables; According to the feature identifier included in the sample skeleton, query the sample feature corresponding to the sample skeleton from the feature table; According to the sample skeleton and the sample features corresponding to the sample skeleton, the original sample is obtained; Based on the original samples and the business scenario identifiers in the original samples, sample data for training is obtained.

2. The sample generation method according to claim 1, wherein: Before obtaining sample data for training based on the original samples and the scene identifiers in the original samples, the following steps are also included: At least the sample identifier and the sample label in the original sample are associated with the summary value of the sample feature of the original sample, and the summary value indexes the sample feature of the original sample stored in the feature storage space.

3. The sample generation method according to claim 2, wherein: Also includes: A copy of the same sample features in the plurality of original samples is retained and stored in a feature storage space, and the sample features are indexed in the feature storage space using the summary value of the sample features.

4. The sample generation method according to claim 3, wherein: The step of associating at least the sample identifier and the sample label in the original sample with the summary value of the sample feature of the original sample, wherein the summary value indexes the sample feature of the original sample stored in the feature storage space, includes: At least the sample identifier and sample label in the original sample are associated with the summary value of the object feature and the summary value of the creative script feature of the original sample, the summary value of the object feature indexes the object feature of the original sample stored in the object feature storage space, and the summary value of the creative script feature indexes the creative script feature of the original sample stored in the creative script feature storage space.

5. The sample generation method according to claim 4, wherein: The step of retaining one copy of the same sample features from the plurality of original samples and storing the same features in the feature storage space includes: Retain one copy of the same object features in multiple original samples and store it in the object feature storage space; A copy of the same creative script features in multiple original samples is retained and stored in the creative script feature storage space. The sample generation method according to claim 5 , wherein: Indexing the sample feature using the summary value of the sample feature in the feature storage space includes: Using the summary value of the object feature in the object feature storage space, indexing the stored object feature; The creative script feature storage space uses the summary value of the creative script feature to index the stored creative script feature.

7. The sample generation method according to claim 1, wherein: Before obtaining sample data for training based on the original samples and the business scenario identifiers in the original samples, the following steps are also included: Based on the sample label in the original sample, the original sample is sampled to determine whether it is a positive sample or a negative sample. If the original sample is a negative sample, the original sample is randomly discarded with a set probability.

8. The sample generation method according to claim 1, wherein: Before obtaining sample data for training based on the original samples and the scene identifiers in the original samples, the following steps are also included: Cross-combining different sample features in multiple original samples; And / or, performing feature iteration processing on sample features in the original sample to add new sample features.

9. The sample generation method according to any one of claims 1 to 8, wherein: The acquiring of sample data for training based on the original sample and the business scenario identifier in the original sample includes: According to the business scenario identifier of the original sample, the original samples with the same business scenario identifier are aggregated to obtain sample data for the training engine suitable for the corresponding business scenario.

10. The sample generation method according to claim 1, wherein: Also includes: Storing the original samples in a database; Before obtaining sample data for training based on the original samples and the business scenario identifiers in the original samples, the original samples are obtained from a database. The sample generation method according to claim 10 , wherein: Storing the original sample in the database includes: Store the original samples in the database in chronological order; The obtaining of the original sample from the database comprises: Read the raw samples stored in time order from the database to obtain the streaming raw samples.

12. The sample generation method according to any one of claims 1 to 8, or any one of claims 10 to 11, wherein: The sample characteristics include real-time sample characteristics.

13. A sample generating device, wherein: include: A business data acquisition module is configured to acquire business data based on business logs for a historical time period. The business data includes multiple sample skeletons and a feature table. The feature table records sample features of objects associated with businesses during the historical time period. The feature table is a single feature table that uniformly records multiple types of sample features associated with objects and businesses during the historical time period. Alternatively, the feature table is a multi-category feature table, where one type of feature table is used to record one type of sample features associated with objects and businesses during the historical time period, and different types of sample features are recorded in different types of feature tables. A sample feature query module, configured to query the sample features corresponding to the sample skeleton from the feature table according to the feature identifiers included in the sample skeleton; The original sample acquisition module is used to obtain the original sample based on the sample skeleton and the sample features corresponding to the sample skeleton; The sample data acquisition module is used to obtain sample data for training based on the original samples and the business scenario identifiers in the original samples.

14. A sample generating device, wherein: include: A feature generator is configured to obtain business data based on business logs for a historical time period, wherein the business data includes multiple sample skeletons and a feature table, wherein the feature table records sample features of objects associated with businesses within the historical time period, and wherein the feature table is a single feature table that uniformly records multiple types of sample features associated with objects and businesses within the historical time period; alternatively, the feature table is a multi-class feature table, wherein one class of feature table is used to record one class of sample features associated with objects and businesses within the historical time period, and different types of sample features are recorded in different types of feature tables; based on feature identifiers included in the sample skeleton, query the feature table for sample features corresponding to the sample skeleton; and obtain the original sample based on the sample skeleton and the sample features corresponding to the sample skeleton; The sample generator is used to obtain sample data for training based on the original samples and the business scenario identifiers in the original samples.

15. The sample generating device according to claim 14, wherein: The sample generator is also used to: before obtaining sample data for training based on the original sample and the business scenario identifier in the original sample, at least associate the sample identifier and sample label in the original sample with the summary value of the sample feature of the original sample, and the summary value indexes the sample feature of the original sample stored in the feature storage space.

16. The sample generating device according to claim 14 or 15, wherein: Also includes: The sample pool is used to store the original samples obtained by the feature generator.

17. A server, wherein: include: at least one memory and at least one processor; The memory stores one or more computer-executable instructions, and the processor calls the one or more computer-executable instructions to execute the sample generation method according to any one of claims 1 to 12.

18. A storage medium, wherein: The storage medium stores one or more computer-executable instructions, and the one or more computer-executable instructions are used to execute the sample generation method according to any one of claims 1 to 12.

Images