Sample data acquisition method and apparatus, device, and storage medium

Abstract

The application discloses a sample data acquisition method and device, equipment and a storage medium, and belongs to the field of machine learning. In the embodiment of the application, in response to a sample acquisition request, an object identifier of a sample object and a content identifier of sample content recommended for the sample object are acquired, the sample content being a label of the sample object when a training content recommendation model is trained. Based on the object identifier, an object feature of the sample object is acquired, the object feature including an object behavior and an object label. In the case that the sample content is recommended by a target recommendation engine for the sample object, based on the content identifier, a content feature of the sample content is acquired from the target recommendation engine, the content feature being a content feature used by the target recommendation engine when the sample content is recommended for the sample object, thereby improving the consistency of the object feature and the content feature, and avoiding the problem that the content recommendation effect of the trained content recommendation model is poor due to feature crossing.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning, and in particular to a sample data acquisition method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of computer technology, more and more manufacturers will use content recommendation models to recommend content. Training a content recommendation model requires a large amount of sample data, which includes object features and content features. The object features are the features of users who are recommended content.

[0003] In related technologies, in order to reduce the impact on online recommendation services, object features are often acquired online, and content features are acquired offline.

[0004] However, since the content features offline are not updated in real time, the consistency of the acquired object features and content features may be poor. When these object features and content features are used to train a content recommendation model, the content recommendation effect of the content recommendation model is poor. SUMMARY

[0005] Embodiments of the present application provide a sample data acquisition method, device, equipment and storage medium, which can acquire user features and content features with good consistency, thereby improving the content recommendation effect of the content recommendation model. The technical solutions are as follows:

[0006] On the one hand, a sample data acquisition method is provided, which includes:

[0007] In response to a sample acquisition request, an object identifier of a sample object and a content identifier of sample content recommended for the sample object are acquired;

[0008] Based on the object identifier, object features of the sample object are acquired, the object features including object behavior and object labels;

[0009] In the case where the sample content is recommended for the sample object by a target recommendation engine, content features of the sample content are acquired from the target recommendation engine based on the content identifier, the content features being the content features used by the target recommendation engine when recommending the sample content for the sample object;

[0010] The object features and the content features are used to train a content recommendation model.

[0011] On the one hand, a sample data acquisition device is provided, which includes:

[0012] an identifier obtaining module, configured to obtain, in response to a sample obtaining request, an object identifier of a sample object and a content identifier of sample content recommended for the sample object;

[0013] an object feature obtaining module, configured to obtain, based on the object identifier, an object feature of the sample object, the object feature comprising an object behavior and an object label;

[0014] a content feature obtaining module, configured to, in a case where the sample content is recommended for the sample object by a target recommendation engine, obtain, based on the content identifier, a content feature of the sample content from the target recommendation engine, the content feature being a content feature used by the target recommendation engine when recommending the sample content for the sample object;

[0015] wherein the object feature and the content feature are used to train a content recommendation model.

[0016] In a possible implementation, the identifier obtaining module is configured to obtain, in response to a sample obtaining request, the object identifier of the sample object and the content identifier of the sample content from a recommendation result cache, the recommendation result cache being configured to store object identifiers of objects for which content is recommended by different recommendation engines and content identifiers of content recommended by the recommendation engines.

[0017] In a possible implementation, the object feature obtaining module is configured to obtain, based on the object identifier, a real-time object behavior of the sample object from a real-time behavior database; and obtain, based on the object identifier, a historical object behavior and an object label of the sample object from an offline object database.

[0018] In a possible implementation, the content feature obtaining module is configured to, in a case where the sample content is recommended for the sample object by a target recommendation engine, send a content feature obtaining request to the target recommendation engine, the content feature obtaining request carrying the content identifier of the sample content; and obtain, from a content feature cache of the target recommendation engine, the content feature of the sample content, the content feature cache storing the content feature used by the target recommendation engine when recommending.

[0019] In a possible implementation, the content feature obtaining module is further configured to perform any one of the following:

[0020] in a case where the content feature cache stores at least two content features of the sample content, obtain, from the at least two content features, a content feature matching a target recommendation task, the target recommendation task being a recommendation task of recommending content to the sample object;

[0021] In a case where the content feature cache does not store the content feature of the sample content, the content feature of the sample content is obtained from a content feature dictionary based on the content identifier.

[0022] In a possible implementation, the apparatus further includes a storage module configured to store the object feature of the sample object and the content feature of the sample content in a bound manner, and the content feature of the sample content is used as a label corresponding to the object feature of the sample object when training the content recommendation model.

[0023] In a possible implementation, the content feature obtaining module is further configured to, in a case where the sample content is not recommended for the sample object by the target recommendation engine, obtain the content feature of the sample content from a content feature dictionary based on the content identifier.

[0024] In an aspect, a computer device is provided, which includes one or more processors and one or more memories, and the one or more memories store at least one computer program, which is loaded and executed by the one or more processors to implement the sample data obtaining method.

[0025] In an aspect, a computer readable storage medium is provided, which stores at least one computer program, which is loaded and executed by a processor to implement the sample data obtaining method.

[0026] In an aspect, a computer program product or computer program is provided, which includes program code stored in a computer readable storage medium, and a processor of a computer device reads the program code from the computer readable storage medium, and the processor executes the program code to enable the computer device to perform the sample data obtaining method.

[0027] Through the technical solutions provided by the embodiments of the present application, in response to a sample obtaining request, an object identifier of a sample object and a content identifier of sample content recommended for the sample object are obtained, and the sample content is a label of the sample object when training a content recommendation model. Based on the object identifier, an object feature of the sample object is obtained, and the object feature includes an object behavior and an object label. In a case where the sample content is recommended for the sample object by a target recommendation engine, based on the content identifier, a content feature of the sample content is obtained from the target recommendation engine, and the content feature is a content feature used by the target recommendation engine when recommending the sample content for the sample object, thereby realizing the unification of the object feature and the content feature in time sequence, improving the consistency of the object feature and the content feature, and avoiding the problem of feature crossing leading to a poor content recommendation effect of the trained content recommendation model. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0029] Figure 1 is a schematic diagram of an implementation environment of a sample data acquisition method provided by an embodiment of the present application;

[0030] Figure 2 is a flowchart of a sample data acquisition method provided by an embodiment of the present application;

[0031] Figure 3 is a flowchart of another sample data acquisition method provided by an embodiment of the present application;

[0032] Figure 4 is a flowchart of still another sample data acquisition method provided by an embodiment of the present application;

[0033] Figure 5 is a structural schematic diagram of a sample data acquisition device provided by an embodiment of the present application;

[0034] Figure 6 is a structural schematic diagram of a terminal provided by an embodiment of the present application;

[0035] Figure 7 is a structural schematic diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail with reference to the drawings.

[0037] In the present application, the terms "first", "second", etc. are used to distinguish the same or similar items with basically the same function, and it should be understood that there is no logical or time sequence dependency between "first", "second", "nth", and the number and execution order are not limited.

[0038] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge sub-models to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0039] Recommendation engines are information networks that proactively discover users' current or potential needs and actively push information to them. They mine user preferences and needs, and proactively recommend objects that users are interested in or need.

[0040] Feature crossing: For models that use past and present information to predict the future, feature crossing essentially means that features contain information about the future. In online inference, the information used to construct features can only come from the present or the past, so there is no feature crossing problem. However, in offline training, information from after the time of the sample may be mistakenly introduced during feature construction, leading to feature crossing.

[0041] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the object characteristics involved in this application were obtained with full authorization.

[0042] In related technologies, a callback service is built to obtain object features online, while content features are obtained from an offline content feature dictionary. This approach can lead to feature crossover issues due to the latency between online and offline methods. For example, when a recommendation engine is making content recommendations, it relies on a dictionary for date A from 0:00 to 7:00 and on a dictionary for date B from 7:00 to 24:00. The offline content feature dictionary is updated daily at 8:00, meaning it retrieves the dictionary for date B at 8:00, using the dictionary for date A before that. Therefore, when retrieving content features between 7:00 and 8:00, the recommendation engine might be using content features from date B for content recommendations, but the retrieved content features are from date A, resulting in feature crossover.

[0043] Figure 1 This is a schematic diagram illustrating the implementation environment of a sample data acquisition method provided in this application embodiment. See also...Figure 1 The implementation environment may include terminal 110 and server 140.

[0044] Terminal 110 is connected to server 140 via a wireless or wired network. Optionally, terminal 110 may be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. Terminal 110 has an application installed and running that supports sample data acquisition and content recommendation model training.

[0045] Server 140 is a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. A recommendation engine is deployed on Server 140, enabling online content recommendations.

[0046] Those skilled in the art will understand that the number of terminals and servers described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more, in which case other terminals may also be included in the above implementation environment. This application does not limit the number of terminals or the type of devices.

[0047] After introducing the implementation environment of the embodiments of this application, the application scenarios of the embodiments of this application will be described below.

[0048] The sample data acquisition method provided in this application embodiment can be applied to training any type of content recommendation model. For example, it can be applied to training an advertising recommendation model, a product recommendation model, or a news recommendation model.

[0049] Taking the technical solution provided in this application as an example in the scenario of training an advertising recommendation model, in response to a sample acquisition request, the object identifier of the sample object and the advertising identifier of the sample advertisement recommended for the sample object are obtained. The sample advertisement is recommended to the sample object by an advertising recommendation engine. Based on the object identifier, the object features of the sample object are obtained. These object features include object behavior and object tags, where object behavior refers to the interaction behavior between the object and the advertisement. When the sample advertisement is recommended to the sample object by a target advertising recommendation engine, the advertising features of the sample advertisement are obtained from the target advertising recommendation engine based on the advertising identifier. These advertising features are the advertising features used by the target advertising recommendation engine when recommending the sample advertisement to the sample object. The consistency between the advertising features and the object features is good, avoiding feature crossover. Subsequently, an advertising recommendation model can be trained based on the object features and the advertising features.

[0050] After introducing the implementation environment and application scenarios of the embodiments of this application, the method for obtaining sample data provided in the embodiments of this application will be described below. See [link to documentation]. Figure 2 Taking the terminal as the executing entity as an example, the method includes the following steps.

[0051] 201. In response to a sample acquisition request, the terminal acquires the object identifier of the sample object and the content identifier of the sample content recommended for that sample object.

[0052] The sample acquisition request is used to request the acquisition of object features and content features. The sample acquisition request can be triggered manually or automatically; this embodiment does not limit this. The sample object is an object that has undergone content recommendation, and the object identifier is used to uniquely identify the sample object. The sample content is the content recommended for the sample object, and the content identifier of the sample content is used to uniquely identify the sample content.

[0053] 202. Based on the object identifier, the terminal obtains the object characteristics of the sample object, which include object behavior and object tags.

[0054] Among them, the object features of the sample object are used to reflect the object characteristics of the sample object, the object behavior of the sample object refers to the interaction behavior between the sample object and multiple contents, and the object label of the sample object is used to classify the sample object.

[0055] 203. When the sample content is recommended by the target recommendation engine for the sample object, the terminal obtains the content features of the sample content from the target recommendation engine based on the content identifier. These content features are the content features used by the target recommendation engine when recommending the sample content for the sample object.

[0056] The object features and content features are used to train the content recommendation model. The content recommendation model then recommends content based on these features. During model training, the content features of the sample content can be used as annotations for the object features of the sample objects. A target recommendation engine, also known as a proprietary recommendation engine, refers to a recommendation engine built by the vendor itself. The process data of a target recommendation engine during recommendation can be obtained by the terminal. In contrast, a non-target recommendation engine, also known as a third-party recommendation engine, cannot have its process data obtained by the terminal.

[0057] The technical solution provided in this application, in response to a sample acquisition request, obtains the object identifier of the sample object and the content identifier of the sample content recommended for the sample object. The sample content is the annotation of the sample object during the training of the content recommendation model. Based on the object identifier, the object features of the sample object are obtained, including object behavior and object tags. When the sample content is recommended by a target recommendation engine for the sample object, based on the content identifier, the content features of the sample content are obtained from the target recommendation engine. These content features are the content features used by the target recommendation engine when recommending the sample content to the sample object, thereby achieving temporal unification of object features and content features, improving the consistency between object features and content features, and avoiding the problem of feature crossing leading to a deterioration in the content recommendation performance of the trained content recommendation model.

[0058] Steps 201-203 above are a brief introduction to the method for obtaining sample data provided in the embodiments of this application. The technical solutions provided in the embodiments of this application will be explained more clearly below with some examples. See [link to relevant documentation]. Figure 3 Taking the terminal as the executing entity as an example, the method includes the following steps.

[0059] 301. In response to a sample acquisition request, the terminal acquires the object identifier of the sample object and the content identifier of the sample content recommended for that sample object.

[0060] The sample acquisition request is used to request the acquisition of object features and content features. The sample acquisition request can be triggered manually or automatically, and this embodiment does not limit this. A sample object is an object that has undergone content recommendation, and an object identifier is used to uniquely identify the sample object. Sample content is the content recommended for the sample object, and a content identifier is used to uniquely identify the sample content. The number of sample objects and sample content can be one or more, and one sample object can correspond to one or more sample contents. A sample object refers to a sample user, that is, a user who has undergone content recommendation. Sample content refers to the content recommended to the sample object. The type of sample content is related to the content recommendation scenario. For example, in the scenario of recommending advertisements, the sample content is also a sample advertisement; in the scenario of recommending products, the sample content is also a sample product; in the scenario of recommending videos, the sample content is also a sample video. This embodiment does not limit the type of sample content. In some embodiments, the sample content is recommended to the sample object through a recommendation engine, which is an online recommendation engine. In some embodiments, the object is also referred to as a user, and the content is also referred to as an item.

[0061] In one possible implementation, in response to a sample acquisition request, the terminal obtains the object identifier of the sample object and the content identifier of the sample content from the recommendation result cache. The recommendation result cache is used to store the object identifiers of objects that use different recommendation engines for content recommendation and the content identifiers of the content recommended by the recommendation engines.

[0062] In this process, different recommendation engines store the content identifier of the recommended content and the object identifier of the object to which the content is pushed in the recommendation result cache after each recommendation. The object identifier and content identifier can then be retrieved from this cache. In this embodiment, the object corresponding to the object identifier stored in the recommendation result cache is called a sample object, and the content corresponding to the content identifier is called sample content. In some embodiments, the recommendation result cache stores streaming data; for example, it is a message queue, meaning it stores data in a first-in, first-out (FIFO) manner, such as a Kafka message queue.

[0063] In this implementation, the object identifier of the sample object and the content identifier of the sample content recommended for the sample object can be directly obtained from the recommendation result cache, and the acquisition efficiency of the object identifier and content identifier is relatively high.

[0064] For example, in response to a sample retrieval request, the terminal retrieves the object identifier of the sample object and the content identifier of the sample content from the recommendation result cache via a first service. This recommendation result cache is a local cache of the terminal. In some embodiments, a result retrieval service also runs on the terminal, which can be used to retrieve recommendation results (object identifier and content identifier) ​​from different recommendation engines back to the recommendation result cache.

[0065] The first service is a local service of the terminal, that is, an offline service. This first service is used to obtain object identifiers and content identifiers, and can also be used to obtain object characteristics. In this embodiment, the first service is named the ranker service.

[0066] In one possible implementation, in response to a sample acquisition request, the terminal acquires recommendation results from different recommendation engines. These results include the object identifier of the object for which content is recommended and the content identifier of the content recommended by the recommendation engine. The terminal then extracts the object identifier of the sample object and the content identifier of the sample content from these recommendation results.

[0067] The recommendation result can be either a real-time recommendation or a historical recommendation, and this application embodiment does not limit it.

[0068] In this implementation, in response to a sample acquisition request, recommendation results from different recommendation engines are obtained, and the object identifier of the sample object and the content identifier of the sample content are obtained from the recommendation results. The acquisition method is relatively simple.

[0069] For example, in response to a sample retrieval request, the terminal sends recommendation result retrieval requests to different recommendation engines. The terminal retrieves the recommendation results returned by the different recommendation engines and extracts the object identifier of the sample object and the content identifier of the sample content from the recommendation results.

[0070] 302. Based on the object identifier, the terminal obtains the object characteristics of the sample object, which include object behavior and object tags.

[0071] Among them, the object features of the sample object are used to reflect the object characteristics of the sample object, and the object behavior of the sample object refers to the interaction behavior between the sample object and multiple content, such as clicking, liking, disliking, and sharing. The object behavior can reflect the object's preference for the content to a certain extent. The object tags of the sample object are used to classify the sample object. The number of object tags of a sample object can be one or more, which is not limited in this embodiment. The object features of the sample object are the object features used when recommending the sample content to the sample object. It should be noted that the object features obtained in this embodiment are feature plaintext (Instance) rather than feature signature (Feature). Feature plaintext refers to the information that directly records the feature, while feature signature refers to the form obtained after abstracting the information that forms the feature. Compared with feature signature, feature plaintext has the advantages of better intuitiveness, strong scalability, good traceability, and cross-scenario use. The difference between feature plaintext and feature signature is shown in Table 1 below.

[0072] Table 1

[0073]

[0074] In one possible implementation, the terminal retrieves the real-time object behavior of the sample object from the real-time behavior database based on the object identifier. The terminal also retrieves the historical object behavior and object tags of the sample object from an offline object database based on the object identifier.

[0075] The real-time behavior database stores real-time object behavior, while the offline object database stores historical object behavior and object tags. The recommendation engine retrieves object features from these two databases when recommending content. Real-time object behavior refers to the object's behavior when content is recommended, while historical behavior refers to the object's behavior before content recommendation. Object tags are updated periodically. The real-time object behavior, historical object behavior, and object tags are the same as those used when recommending sample content to the sample object. In some embodiments, the real-time behavior database and the offline object database are Redis databases; however, other types of databases may be used in other possible implementations, and this application does not limit this.

[0076] In this implementation, object features of sample objects can be obtained from the real-time behavior database and the offline object database. Since the recommendation engine also obtains object features from the real-time behavior database and the offline object database when making content recommendations, this ensures that the method of obtaining object features and the data remain synchronized, thereby improving the accuracy of object features.

[0077] For example, the terminal uses the object identifier to query the real-time behavior database and obtains the real-time object behavior corresponding to that object identifier. The terminal also uses the object identifier to query the offline object database and obtains the historical object behavior and object tag corresponding to that object identifier.

[0078] For example, the terminal uses the object identifier to query the real-time behavior database through the first service, and obtains the real-time object behavior corresponding to the object identifier. The terminal also uses the object identifier to query the offline object database through the first service, and obtains the historical object behavior and object tag corresponding to the object identifier.

[0079] In one possible implementation, the terminal queries an object feature database based on the object identifier to obtain the object features of the sample object. The object feature database is used to store object features used when making content recommendations.

[0080] In this implementation, object features can be directly obtained from the object feature database, resulting in high efficiency in obtaining object features.

[0081] For example, the terminal uses the object identifier to query the object feature database and obtains the object features corresponding to that object identifier. When the recommendation engine performs content recommendation, it stores each object feature in the object feature database for subsequent queries.

[0082] Optionally, after step 302, the terminal may execute steps 303 or 304 as appropriate. This application embodiment does not limit the execution order of the steps.

[0083] 303. If the sample content is not recommended by the target recommendation engine for the sample object, the terminal obtains the content features of the sample content from the content feature dictionary based on the content identifier.

[0084] The object features and content features are used to train a content recommendation model. This model recommends content based on these features. During training, the content features of the sample content serve as annotations for the object features of the sample object. The target recommendation engine, also known as a proprietary recommendation engine, refers to a recommendation engine built by the vendor itself. The process data of the target recommendation engine during recommendation can be obtained by the terminal. In contrast, non-target recommendation engines, also known as third-party recommendation engines, cannot have their process data obtained by the terminal. If the sample content is not recommended by the target recommendation engine, the terminal cannot directly obtain the content features used to recommend the sample content from the recommendation engine that recommended it. The content feature dictionary stores the content features of multiple contents and is updated periodically. In some embodiments, content features include click-through rate, like rate, dislike rate, and share rate. It should be noted that, similar to the object features mentioned above, the content features obtained in this embodiment are plaintext features rather than feature signatures.

[0085] In one possible implementation, if the traffic identifier of the sample content indicates that the sample content was not recommended by the target recommendation engine for the sample object, the terminal uses the content identifier to query the content feature dictionary to obtain the content feature corresponding to the content identifier.

[0086] The traffic identifier is used to indicate the source of the sample content, that is, to identify the recommendation engine that recommended the sample content.

[0087] In this implementation, when the sample content is not recommended by the target recommendation engine for the sample object, the terminal can obtain content features from the content feature dictionary, ensuring the success rate of content feature acquisition.

[0088] For example, if the traffic identifier of the sample content indicates that the sample content was not recommended by the target recommendation engine, the terminal uses a second service to query the content feature dictionary using the content identifier to obtain the content feature corresponding to the content identifier. This second service is a local service of the terminal, i.e., an offline service, used to obtain the content feature. In this embodiment, the second service is named the kiddy service.

[0089] 304. When the sample content is recommended by the target recommendation engine for the sample object, the terminal obtains the content features of the sample content from the target recommendation engine based on the content identifier. These content features are the content features used by the target recommendation engine when recommending the sample content for the sample object.

[0090] In cases where the sample content is recommended by the target recommendation engine for the sample object, the terminal can directly obtain the content features used when recommending the sample content from the recommendation engine that recommended the sample content.

[0091] In one possible implementation, when the sample content is recommended by the target recommendation engine, the terminal sends a content feature retrieval request to the target recommendation engine, the request carrying the content identifier of the sample content. The terminal retrieves the content features of the sample content from the target recommendation engine's content feature cache, which stores the content features used by the target recommendation engine when making recommendations.

[0092] In this target recommendation engine, during the content recommendation process, the content features used for recommendation are stored in a content feature cache. In some embodiments, this content feature cache is maintained by a third service, which is an online service through which content features can be added or deleted from the content feature cache. In this embodiment, this third service is named the kiddy feature service.

[0093] In this implementation, when the sample content is recommended by the target recommendation engine for the sample object, the content features can be directly obtained from the content feature cache of the target recommendation engine, ensuring that the obtained content features are the same as the content features used by the target recommendation engine when making content recommendations, thus maintaining the consistency of content features.

[0094] For example, if the traffic identifier of the sample content indicates that the sample content was recommended by the target recommendation engine, the terminal sends a content feature retrieval request to the target recommendation engine through a first service. This content feature retrieval request carries the content identifier of the sample content. The target recommendation engine obtains the content feature retrieval request through a third service, retrieves the content identifier from the request, and uses the content identifier to query the content feature cache to obtain the content feature corresponding to the content identifier. The target recommendation engine then sends the content feature to the terminal through the third service. The terminal retrieves the content feature through the first service.

[0095] In addition to the above-described implementation methods, content features can also be obtained through the following methods.

[0096] In one possible implementation, if the sample content is recommended by a target recommendation engine for the sample object, the terminal sends a content feature retrieval request to the target recommendation engine, the content feature retrieval request carrying the content identifier of the sample content. If at least two content features of the sample content are stored in the content feature cache, the terminal retrieves the content features matching the target recommendation task from the at least two content features, the target recommendation task being a recommendation task to recommend content to the sample object.

[0097] The content feature cache stores at least two content features of the sample content, which means that the target recommendation engine has recommended the sample content at least twice in a short period of time. Since the time of recommendation of the sample content is different, the at least two content features of the sample content may also be different. For example, the click-through rate of the at least two content features may be different.

[0098] In this implementation, when at least two content features of the sample content are stored in the content feature cache, content features that match the target recommendation task can be selected from the at least two content features. These content features are the same content features used by the target recommendation engine when recommending the sample content to the sample object, ensuring the consistency between the content features used during model training and the content features used by the recommendation engine during recommendation.

[0099] For example, if the traffic identifier of the sample content indicates that the sample content was recommended by the target recommendation engine, the terminal sends a content feature retrieval request to the target recommendation engine through a first service. This request carries the content identifier of the sample content and the identifier of the target recommendation task. The target recommendation engine obtains the content feature retrieval request through a third service, retrieves the content identifier and the identifier of the target recommendation task from the request, and queries the content feature cache using the content identifier to obtain at least two content features corresponding to that identifier. The target recommendation engine then filters these at least two content features based on the identifier of the target recommendation task through the third service to obtain content features that match the identifier of the target recommendation task. The target recommendation engine then sends these content features to the terminal through the third service. The terminal retrieves these content features through the first service.

[0100] In one possible implementation, if the sample content is recommended by a target recommendation engine, the terminal sends a content feature retrieval request to the target recommendation engine, the request carrying the content identifier of the sample content. If the content feature cache does not store the content feature of the sample content, the terminal retrieves the content feature of the sample content from the content feature dictionary based on the content identifier.

[0101] In this case, the content feature cache does not store the content features of the sample content, meaning that the interval between the sample acquisition request and the target recommendation engine recommending the sample content to the sample object is relatively long, and the content features of the sample content stored in the content feature cache have been covered by other content features.

[0102] In this implementation, even if the content features of the sample content are not stored in the content feature cache, the content features of the sample content can be obtained from the content feature dictionary based on the content identifier, ensuring that the content features of the sample content can be obtained.

[0103] For example, if the traffic identifier of the sample content indicates that the sample content was recommended by the target recommendation engine, the terminal sends a content feature retrieval request to the target recommendation engine through a first service. This content feature retrieval request carries the content identifier of the sample content. The target recommendation engine obtains the content feature retrieval request through a third service, retrieves the content identifier from the request, and uses the content identifier to query the content feature cache. If the content feature corresponding to the content identifier is not found, the target recommendation engine sends a content feature loss message to the terminal through the third service. This message indicates that the content feature of the sample content was not found. Upon receiving the content feature loss message, the terminal uses a second service to query the content feature dictionary using the content identifier to obtain the content feature corresponding to the content identifier.

[0104] 305. The terminal binds and stores the object features of the sample object with the content features of the sample content. The content features of the sample content are used as the annotations corresponding to the object features of the sample object when training the content recommendation model.

[0105] Among them, binding storage includes feature splicing and disk dumping. Feature splicing refers to splicing content features and object features, while disk dumping refers to storing the spliced ​​features.

[0106] The following will combine Figure 4 The above steps 301-305 will be explained.

[0107] See Figure 4In response to a sample acquisition request, the terminal obtains the object identifier of the sample object and the content identifier of the sample content from the recommendation result cache (kafka1) through the first service (ranker service). The terminal then uses the object identifier to query the real-time behavior database to obtain the real-time object behavior corresponding to that object identifier. Next, the terminal uses the object identifier to query the offline object database to obtain the historical object behavior and object tags corresponding to that object identifier. If the traffic identifier of the sample content indicates that the sample content was not recommended by the target recommendation engine, the terminal uses the second service (kiddy service) to query the content feature dictionary (item dictionary) using the content identifier to obtain the content features corresponding to that content identifier. Finally, the terminal uses the second service (kiddy service) to bind and store the object feature and content feature in the training data cache (kafka2). When the traffic identifier of the sample content indicates that the sample content was recommended by the target recommendation engine for the sample object, the terminal sends a content feature retrieval request to the target recommendation engine through the first service (ranker service). This content feature retrieval request carries the content identifier of the sample content. The target recommendation engine obtains the content feature retrieval request through the third service (kiddy feature service), retrieves the content identifier from the request, and uses the content identifier to query the content feature cache to obtain the content feature corresponding to the content identifier. The target recommendation engine also provides recall, ranking, and content recommendation services. The target recommendation engine sends the content feature to the terminal through the third service (kiddy feature service). The terminal obtains the content feature through the first service (ranker service). The terminal binds the object feature and content feature through the first service (ranker service) and stores them in the training data cache (kafka2).

[0108] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0109] The technical solution provided in this application, in response to a sample acquisition request, obtains the object identifier of the sample object and the content identifier of the sample content recommended for the sample object. The sample content is the annotation of the sample object during the training of the content recommendation model. Based on the object identifier, the object features of the sample object are obtained, including object behavior and object tags. When the sample content is recommended by a target recommendation engine for the sample object, based on the content identifier, the content features of the sample content are obtained from the target recommendation engine. These content features are the content features used by the target recommendation engine when recommending the sample content to the sample object, thereby achieving temporal unification of object features and content features, improving the consistency between object features and content features, and avoiding the problem of feature crossing leading to a deterioration in the content recommendation performance of the trained content recommendation model.

[0110] Figure 5 This is a schematic diagram of the structure of a sample data acquisition device provided in an embodiment of this application. See also... Figure 5 The device includes: an identifier acquisition module 501, an object feature acquisition module 502, and a content feature acquisition module 503.

[0111] The identifier acquisition module 501 is used to acquire the object identifier of the sample object and the content identifier of the sample content recommended for the sample object in response to the sample acquisition request.

[0112] The object feature acquisition module 502 is used to acquire the object features of the sample object based on the object identifier. The object features include object behavior and object label.

[0113] The content feature acquisition module 503 is used to acquire the content features of the sample content from the target recommendation engine based on the content identifier when the sample content is recommended by the target recommendation engine for the sample object. The content features are the content features used by the target recommendation engine when recommending the sample content for the sample object.

[0114] The object feature and the content feature are used to train the content recommendation model.

[0115] In one possible implementation, the identifier acquisition module 501 is used to retrieve the object identifier of the sample object and the content identifier of the sample content from the recommendation result cache in response to the sample acquisition request. The recommendation result cache is used to store the object identifiers of objects that use different recommendation engines for content recommendation and the content identifiers of the content recommended by the recommendation engines.

[0116] In one possible implementation, the object feature acquisition module 502 is used to obtain the real-time object behavior of the sample object from the real-time behavior database based on the object identifier. Based on the object identifier, it also obtains the historical object behavior and object tags of the sample object from an offline object database.

[0117] In one possible implementation, the content feature acquisition module 503 is configured to send a content feature acquisition request to the target recommendation engine when the sample content is recommended by the target recommendation engine for the sample object. The content feature acquisition request carries the content identifier of the sample content. The module then retrieves the content features of the sample content from the content feature cache of the target recommendation engine, which stores the content features used by the target recommendation engine when making recommendations.

[0118] In one possible implementation, the content feature acquisition module 503 is further configured to perform any of the following:

[0119] If the content feature cache stores at least two content features of the sample content, the content feature that matches the target recommendation task is obtained from the at least two content features. The target recommendation task is a recommendation task that recommends content to the sample object.

[0120] If the content features of the sample content are not stored in the content feature cache, the content features of the sample content are retrieved from the content feature dictionary based on the content identifier.

[0121] In one possible implementation, the device further includes a storage module for binding and storing the object features of the sample object with the content features of the sample content, wherein the content features of the sample content serve as annotations corresponding to the object features of the sample object when training the content recommendation model.

[0122] In one possible implementation, the content feature acquisition module 503 is further configured to, when the sample content is not recommended for the sample object by the target recommendation engine, acquire the content features of the sample content from the content feature dictionary based on the content identifier.

[0123] It should be noted that the sample data acquisition device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the sample data acquisition device and the sample data acquisition method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0124] The technical solution provided in this application, in response to a sample acquisition request, obtains the object identifier of the sample object and the content identifier of the sample content recommended for the sample object. The sample content is the annotation of the sample object during the training of the content recommendation model. Based on the object identifier, the object features of the sample object are obtained, including object behavior and object tags. When the sample content is recommended by a target recommendation engine for the sample object, based on the content identifier, the content features of the sample content are obtained from the target recommendation engine. These content features are the content features used by the target recommendation engine when recommending the sample content to the sample object, thereby achieving temporal unification of object features and content features, improving the consistency between object features and content features, and avoiding the problem of feature crossing leading to a deterioration in the content recommendation performance of the trained content recommendation model.

[0125] This application provides a computer device for performing the above-described method. This computer device can be implemented as a terminal or a server. The structure of the terminal will be described below:

[0126] Figure 6 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. The terminal 600 can be a smartphone, tablet computer, laptop computer, or desktop computer. The terminal 600 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0127] Typically, terminal 600 includes one or more processors 601 and one or more memories 602.

[0128] Processor 601 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 601 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0129] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 are used to store at least one computer program, which is executed by the processor 601 to implement the sample data acquisition method provided in the method embodiments of this application.

[0130] In some embodiments, the terminal 600 may optionally include a peripheral device interface 603 and at least one peripheral device. The processor 601, memory 602, and peripheral device interface 603 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 603 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607, and a power supply 608.

[0131] Peripheral interface 603 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 601 and memory 602. In some embodiments, processor 601, memory 602 and peripheral interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 601, memory 602 and peripheral interface 603 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0132] The radio frequency (RF) circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 604 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 604 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc.

[0133] Display screen 605 is used to display a user interface (UI). This UI may include graphics, text, icons, video, and any combination thereof. When display screen 605 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 601 for processing. In this case, display screen 605 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard.

[0134] The camera assembly 606 is used to capture images or videos. Optionally, the camera assembly 606 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal.

[0135] The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 601 for processing, or input to the radio frequency circuit 604 to realize voice communication.

[0136] Power supply 608 is used to supply power to the various components in terminal 600. Power supply 608 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery.

[0137] In some embodiments, the terminal 600 further includes one or more sensors 609. The one or more sensors 609 include, but are not limited to, an accelerometer 610, a gyroscope 611, a pressure sensor 612, an optical sensor 613, and a proximity sensor 614.

[0138] Accelerometer 610 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established with terminal 600.

[0139] The gyroscope sensor 611 can detect the orientation and rotation angle of the terminal 600. The gyroscope sensor 611 can work in conjunction with the accelerometer sensor 610 to collect the user's 3D movements on the terminal 600.

[0140] The pressure sensor 612 can be installed on the side bezel of the terminal 600 and / or on the lower layer of the display screen 605. When the pressure sensor 612 is installed on the side bezel of the terminal 600, it can detect the user's grip signal on the terminal 600, and the processor 601 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 612. When the pressure sensor 612 is installed on the lower layer of the display screen 605, the processor 601 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 605.

[0141] An optical sensor 613 is used to collect ambient light intensity. In one embodiment, a processor 601 can control the display brightness of a display screen 605 based on the ambient light intensity collected by the optical sensor 613.

[0142] The proximity sensor 614 is used to detect the distance between the user and the front of the terminal 600.

[0143] Those skilled in the art will understand that Figure 6 The structure shown does not constitute a limitation on terminal 600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0144] The aforementioned computer equipment can also be implemented as a server. The structure of a server is described below:

[0145] Figure 7This is a schematic diagram of a server structure provided in an embodiment of this application. The server 700 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 701 and one or more memories 702. The one or more memories 702 store at least one computer program, which is loaded and executed by the one or more processors 701 to implement the methods provided in the various method embodiments described above. Of course, the server 700 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 700 may also include other components for implementing device functions, which will not be elaborated upon here.

[0146] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a computer program that can be executed by a processor to complete the sample data acquisition method in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0147] In an exemplary embodiment, a computer program product or computer program is also provided, which includes program code stored in a computer-readable storage medium. The processor of a computer device reads the program code from the computer-readable storage medium and executes the program code, causing the computer device to perform the above-described method for acquiring sample data.

[0148] In some embodiments, the computer program involved in the present application embodiments may be deployed and executed on a computer device, or executed on multiple computer devices located in one location, or executed on multiple computer devices distributed in multiple locations and interconnected through a communication network. Multiple computer devices distributed in multiple locations and interconnected through a communication network may constitute a blockchain system.

[0149] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0150] The above are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method of acquiring sample data, characterized by, The method comprises: in response to a sample acquisition request, acquiring an object identifier of a sample object and a content identifier of sample content recommended for the sample object; based on the object identifier, acquiring real-time object behavior of the sample object from a real-time behavior database, the real-time object behavior being object behavior of the sample object when content is recommended for the sample object; based on the object identifier, acquiring historical object behavior of the sample object and an object label from an offline object database, the historical behavior being object behavior of the sample object before content is recommended for the sample object, the real-time object behavior, the historical object behavior and the object label forming object features; in a case where the sample content is recommended for the sample object by a target recommendation engine, based on the content identifier, acquiring content features of the sample content from the target recommendation engine, the content features being content features used by the target recommendation engine when recommending the sample content for the sample object, the target recommendation engine acquiring object features from the real-time behavior database and the offline object database when recommending content; wherein the object features and the content features are used to train a content recommendation model.

2. The method of claim 1, wherein, The response to the sample acquisition request, acquiring the object identifier of the sample object and the content identifier of the sample content recommended for the sample object comprises: in response to a sample acquisition request, acquiring an object identifier of a sample object and a content identifier of sample content recommended for the sample object from a recommendation result cache, the recommendation result cache being used to store object identifiers of objects for which content is recommended using different recommendation engines and content identifiers of content recommended by the recommendation engines.

3. The method of claim 1, wherein, The in a case where the sample content is recommended for the sample object by a target recommendation engine, based on the content identifier, acquiring content features of the sample content from the target recommendation engine comprises: in a case where the sample content is recommended for the sample object by a target recommendation engine, sending a content feature acquisition request to the target recommendation engine, the content feature acquisition request carrying a content identifier of the sample content; acquiring content features of the sample content from a content feature cache of the target recommendation engine, the content feature cache storing content features used by the target recommendation when recommending content.

4. The method of claim 3, wherein, The in a case where the sample content is recommended for the sample object by a target recommendation engine, sending a content feature acquisition request to the target recommendation engine, the content feature acquisition request carrying a content identifier of the sample content; acquiring content features of the sample content from a content feature cache of the target recommendation engine, the content feature cache storing content features used by the target recommendation when recommending content. ​ ​ 5. The method of claim 1, wherein, In the case that the sample content is recommended by the target recommendation engine for the sample object, after the content feature of the sample content is obtained from the target recommendation engine based on the content identifier, the method further comprises: Binding and storing the object feature of the sample object and the content feature of the sample content, and the content feature of the sample content is used as a label corresponding to the object feature of the sample object when the content recommendation model is trained.

6. The method of claim 1, wherein, After the object feature of the sample object is obtained based on the object identifier, the method further comprises: In the case that the sample content is not recommended by the target recommendation engine for the sample object, the content feature of the sample content is obtained from the content feature dictionary based on the content identifier.

7. A sample data acquisition apparatus characterized by comprising: The device comprises: An identifier obtaining module configured to obtain an object identifier of a sample object and a content identifier of a sample content recommended for the sample object in response to a sample obtaining request; An object feature obtaining module configured to obtain real-time object behavior of the sample object from a real-time behavior database based on the object identifier, the real-time object behavior being object behavior of the sample object when the sample object is recommended content, and obtain historical object behavior and an object label of the sample object from an offline object database based on the object identifier, the historical behavior being object behavior of the sample object before the sample object is recommended content, the real-time object behavior, the historical object behavior and the object label forming an object feature; A content feature obtaining module configured to obtain a content feature of the sample content from a target recommendation engine based on the content identifier in the case that the sample content is recommended by the target recommendation engine for the sample object, the content feature being a content feature used by the target recommendation engine to recommend the sample content for the sample object, and the target recommendation engine obtaining the object feature from the real-time behavior database and the offline object database when recommending content; The object feature and the content feature are used to train a content recommendation model.

8. A computer device, comprising: The computer device comprises one or more processors and one or more memories, and at least one computer program is stored in the one or more memories, the computer program is loaded and executed by the one or more processors to implement the sample data obtaining method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer program, and the computer program is loaded and executed by the processor to implement the sample data obtaining method of any one of claims 1 to 6.

Images