Method and apparatus for constructing training sample, electronic device, and storage medium

Abstract

The application discloses a method and device for constructing training samples, electronic equipment and a storage medium. The method comprises the following steps: obtaining the information of clicks and payment commodities of a plurality of users in a predetermined time period, determining the information of clicks and payment commodities belonging to a first target query word as a first positive sample set generated in the predetermined time period, and the commodities from the same session in the first positive sample set belong to the same positive sample commodity list; respectively performing a deduplication processing on each positive sample commodity list, adding a first type identifier to each commodity in each deduplicated positive sample commodity list, selecting negative sample commodities for each positive sample commodity according to the attributes of each positive sample commodity in the deduplicated positive sample commodity list, constituting a corresponding negative sample commodity list, and constituting a first negative sample set based on the negative sample commodity list. The embodiment of the application can obtain high-quality training samples.

Description

Technical Field

[0001] This application relates to the fields of machine learning and e-commerce platform technology, and in particular to a method, apparatus, electronic device and computer-readable storage medium for constructing training samples. Background Technology

[0002] E-commerce platforms' search engines retrieve products based on search queries. These retrieved products undergo coarse and fine ranking processes before being displayed for users to choose from and purchase. For more information on coarse and fine ranking, please refer to [link / reference]. Figure 1 The overall strategy in the coarse ranking stage is to filter and sort the recall results based on their relevance to the query, with highly relevant products moving to the fine ranking stage. The fine ranking stage can comprehensively consider the impact of various factors on the ranking, such as historical sales volume or current popularity of products, which can give exposure opportunities to products more likely to be selected and purchased by users; or considering seller promotional factors, prioritizing the exposure of promotional products, and so on. The recall and exposure strategies described above are applicable to search engines or recommendation systems on various online platforms.

[0003] One of the core issues in coarse-ranking is the selection of negative samples for the coarse-ranking model. Due to the limited log data that the system can record, existing approaches mostly construct positive and negative samples based on user behavior (such as searching keywords, browsing exposed products, clicking products, saving products, purchasing products, etc.). For example, for a series of actions after a user enters a search term, the product ultimately purchased by the user is used as the positive sample for that search, while products exposed but not purchased and / or not clicked are used as negative samples. These positive and negative samples reflect the user's selection tendency to some extent and can be used as training samples for the coarse-ranking model. However, the diversity of negative samples determined in the above way is poor, and there is a significant difference between this and the actual data distribution processed by the coarse-ranking model, making the training effect unsatisfactory. Currently, there is considerable room for improvement in the composition of training samples for coarse-ranking models in relevant scenarios, which is one of the hot topics and difficulties for researchers, and the training effect of coarse-ranking models needs to be improved. Summary of the Invention

[0004] In view of the above, embodiments of this application provide a method, apparatus, electronic device, and computer-readable storage medium for constructing training samples to solve at least one of the above technical problems.

[0005] In a first aspect, embodiments of this application provide a method for constructing training samples, including:

[0006] Get information on clicked and paid products from multiple users within a predetermined time period, and determine the clicked and paid products that belong to the first target query term as the first positive sample set generated within the predetermined time period. Products from the same session in the first positive sample set belong to the same positive sample product list.

[0007] Each positive sample product list is deduplicated to obtain a deduplicated positive sample product list.

[0008] Add a first-class identifier to each product in each deduplicated positive sample product list, and determine that each product in the preset negative sample candidate pool has a second-class identifier.

[0009] Based on the attributes of each positive sample product in the deduplicated positive sample product list, R negative sample products are selected from the negative sample candidate pool for each positive sample product to form a corresponding negative sample product list. The first negative sample set corresponding to the first positive sample set is formed based on one or more negative sample product lists. When selecting R negative sample products for the current positive sample product, the selection is based on the first type identifier of the current positive sample product and the second type identifier of the products in the negative sample candidate pool, so that there is no duplicate product information in the first negative sample set, where R is a preset step size.

[0010] Secondly, embodiments of this application provide an apparatus for constructing training samples, comprising:

[0011] The positive sample processing module is used to obtain information on clicked and paid products of multiple users within a predetermined time period, and to determine the information on clicked and paid products belonging to the first target query term as the first positive sample set generated within the predetermined time period. Products from the same session in the first positive sample set belong to the same positive sample product list.

[0012] The deduplication module is used to deduplicatize each positive sample product list separately to obtain a deduplicated positive sample product list.

[0013] The identification processing module is used to add a first-class identifier to each product in the deduplicated positive sample product list and determine that each product in the preset negative sample candidate pool has a second-class identifier.

[0014] The negative sample processing module is used to select R negative sample products from the negative sample candidate pool for each positive sample product based on the attributes of each positive sample product in the deduplicated positive sample product list, forming a corresponding negative sample product list. The first negative sample set corresponding to the first positive sample set is formed based on one or more negative sample product lists. When selecting information of R products for the current positive sample product, the selection is based on the first type identifier of the current positive sample product and the second type identifier of the products in the negative sample candidate pool, so that there is no duplicate product information in the first negative sample set, where R is a preset step size.

[0015] Thirdly, embodiments of this application provide an electronic device, the electronic device including: a processor and a memory storing computer program instructions; the electronic device executes the computer program instructions to implement the steps of the method described above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the steps of the method described above.

[0017] Fifthly, embodiments of this application provide a computer program product including computer program instructions that, when executed by a processor, implement the steps of the method described above.

[0018] This application embodiment designs a dynamic step size mechanism for constructing model training samples. It removes duplicates of the same type of goods in the positive sample set, which can reduce the number of negative samples required for the same type of goods, thereby reducing or eliminating free positive samples. It adds appropriate labels to positive sample goods and candidate negative sample goods, which can make the selection of negative sample goods random and non-repeating, thereby fundamentally optimizing the sample quality and realizing a high-quality model training process. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings of the embodiments of this application will be briefly described below.

[0020] Figure 1 This is a schematic diagram of a recommender system model.

[0021] Figure 2 This is a diagram illustrating the situation where free positive samples occur when selecting negative sample products for positive sample products.

[0022] Figure 3 This is a flowchart illustrating a method for constructing training samples according to an embodiment of this application.

[0023] Figure 4 This is a schematic diagram illustrating the principle of constructing training samples using embodiments of this application.

[0024] Figure 5 This is a structural block diagram of an apparatus for constructing training samples according to an embodiment of this application.

[0025] Figure 6 This is a schematic diagram of an electronic device used to implement the method for constructing training samples in the embodiments of this application. Detailed Implementation

[0026] The principles and spirit of this application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement this application, and are not intended to limit the scope of this application in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0027] Those skilled in the art will recognize that embodiments of this application can be implemented as a system, apparatus, device, method, computer-readable storage medium, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0028] According to embodiments of this application, a method, apparatus, electronic device, and storage medium for constructing training samples are proposed.

[0029] In this document, any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any names in the figures are for distinction only and have no limiting meaning.

[0030] The principles and spirit of this application are explained in detail below with reference to several representative embodiments.

[0031] In this field, after a user enters a search term and makes a request, the search engine inputs a set of products (items) related to the search term into a coarse ranking model for further filtering, and then sends them to a fine ranking model. After fine ranking, the products are exposed and presented to the user. The system can record the user's behavior log (such as clicks, favorites, adding to cart, payment, etc.). Generally speaking, the user behavior log may include one or more of the following product information: products that were exposed and clicked after recall, products that were exposed but not clicked after recall, and products that were not exposed and not clicked after recall.

[0032] On the one hand, user behavior logs are relatively easy to obtain log data. Therefore, when building model training samples, most common systems will select positive and negative samples from user behavior logs. For example, they will use products that users have clicked in the exposure table as positive samples and recall products that have not been exposed and / or recall products that have been exposed but not clicked as negative samples to train coarse-ranking models.

[0033] While the above sample composition may represent some of the user’s preferences when choosing products, it is not actually consistent with the facts. This is because the “negative samples” selected by this standard are actually high-quality products (essentially positive samples, not negative samples) that were filtered out by the coarse-ranking model during the recall phase. Therefore, the composition of the negative samples deviates greatly from the actual situation.

[0034] On the other hand, the inventors of this application have discovered that when constructing training samples for the coarse-ranking model, a list of products clicked or purchased by the user can be used as a positive sample list, and several products can be selected from the corresponding product categories in the system's product library to form a negative sample list for training the coarse-ranking model.

[0035] While this construction method can more realistically and accurately reflect the composition of positive and negative samples and is convenient for training using the ListWise Approach in Learning to Rank (LTR) techniques, it has a drawback. Because a certain number of negative sample products need to be selected from the product database for each positive sample product, and these negative sample products must belong to the same category as the positive sample products, a sufficient number of products in the product database are required as a candidate pool for negative samples. If a user clicks on or purchases a large number of products, but the number of products in the negative sample candidate pool is small, some positive sample products may not be able to find corresponding negative sample products, causing these positive sample products to become detached positive samples. Figure 2 As shown, in the column of positive sample items on the left, the last positive sample item lacks a corresponding negative sample item, thus becoming a detached positive sample. The overall quality of such training samples is poor, and when the number of detached positive samples is large, it will significantly affect the training effect of the coarse-ranking model. This situation is prone to occur in second-hand goods trading systems because, unlike the large-scale inventory of new goods trading platforms, the inventory of various second-hand goods in second-hand trading platforms is limited. For example, the inventory of popular second-hand mobile phone models may be large, making it less likely for detached positive samples to appear, while the inventory of less popular second-hand mobile phone models may be very small, making it easier for detached positive samples to appear.

[0036] Figure 3 This illustration shows a flowchart of a method for constructing training samples according to an embodiment of this application. The method includes the following steps:

[0037] S101, obtain information on clicked and paid products of multiple users within a predetermined time period, and determine the clicked and paid products that belong to the first target query term as the first positive sample set generated within the predetermined time period. Products from the same user session in the first positive sample set belong to the same positive sample product list.

[0038] S102, Perform deduplication on each positive sample product list to obtain a deduplicated positive sample product list;

[0039] S103, Add a first-class identifier to each product in each deduplicated positive sample product list.

[0040] Each product in the pre-defined negative sample candidate pool is identified as having a second-class identifier;

[0041] S104, based on the attributes of each positive sample product in the deduplicated positive sample product list, select R negative sample products for each positive sample product in the negative sample candidate pool to form a corresponding negative sample product list. Based on one or more negative sample product lists, construct the first negative sample set corresponding to the first positive sample set. When selecting R negative sample products for the current positive sample product, the selection is based on the first type identifier of the current positive sample product and the second type identifier of the products in the negative sample candidate pool, so that there is no duplicate product information in the first negative sample set, where R is a preset step size.

[0042] According to embodiments of this application, data on user clicks and payment behaviors occurring on a given day can first be obtained from system logs. This typically includes a large amount of data on clicks and payment behaviors by a large number of users under different queries, involving a large number of products of different types and attributes. Following the processing method of embodiments of this application, products corresponding to click behaviors and payment behaviors belonging to the same target query term (i.e., the same query) are identified as a positive sample set. Here, "click behaviors and payment behaviors belonging to the same query" may originate from a single user's single search session, multiple different search sessions by the same user, or multiple search sessions by multiple different users, and so on. In embodiments of this application, products belonging to the same session within the positive sample set are compiled into a positive sample product list. That is, under the same query, each session can obtain a positive sample product list, and these positive sample product lists constitute the positive sample set for that query.

[0043] Regarding sessions (sometimes also called search sessions or user sessions), a session is generally defined as the interaction process between a user and a network system over a period of time. A session can record a series of behaviors that occur during the time period from the start of a search to the end of the search, such as searching for products, browsing products, adding products to favorites, placing an order, and exiting.

[0044] Next, according to the embodiments of this application, deduplication is performed on each positive sample product list. For example, three products with the same attribute in the first positive sample product list corresponding to the first session are deduplicated, resulting in the first positive sample product list containing only one product with that attribute. The reason for deduplication is that within a single session, clicks on similar types of products generally reflect similar user intentions, while in different sessions, the same product clicked by different users may indicate completely different user intentions. Therefore, deduplication of the positive sample product list according to the embodiments of this application can reduce the number of positive sample products and retain and reflect the actual user intentions to the greatest extent possible, thereby improving sample quality.

[0045] Furthermore, in order to obtain suitable negative sample products, embodiments of this application propose a "dynamic step size mechanism", that is, according to a pre-configured step size (denoted as R, where R can be any positive integer), R qualified negative sample products are selected from the negative sample candidate pool for a single positive sample product in the positive sample product list, thereby obtaining the corresponding negative sample product list, and then obtaining the corresponding negative sample product set. The above processing can reduce the occurrence of free positive samples.

[0046] Specifically, the "dynamic step size mechanism" requires adding an identity identifier (or tag, ID, number) to each product in the positive sample product set, and also to products in the negative sample candidate pool. The former and the latter can be the same type of identifier or different types of identifiers. For example, assuming the step size is R = 5 (it can also be 7, 12, 23, etc.), for products in the deduplicated positive sample product list, such as the first positive sample product and the second positive sample product, 5 negative sample products need to be selected from the negative sample candidate pool for the first positive sample product, and 5 negative sample products also need to be selected for the second positive sample product. When selecting 5 negative sample products for the first positive sample product (or the second positive sample product), one approach is to select based on the attributes of the first positive sample product. For example, the attributes of the selected 5 negative sample products can be the same as or similar to those of the first positive sample product. Alternatively, selection can be based on the product category to which the first positive sample product belongs; that is, selecting 5 negative sample products from the same product category in the negative sample candidate pool. As an example, negative sample products from the negative sample candidate pool that share the same third-level category as the first positive sample product can be selected to form a high-quality negative sample product list. Furthermore, when selecting 5 negative sample products for the first positive sample product, the identifiers carried by the first positive sample product and the identifiers carried by the products in the negative sample candidate pool are also used to distinguish different products based on these identifiers. The goal is to select 5 different negative sample products for the first positive sample product and 5 different negative sample products for the second positive sample product, resulting in 10 distinct negative sample products.

[0047] In this way, five negative sample items can be selected for each item in the positive sample item list to form a negative sample item list. Finally, the negative sample item list forms a negative sample set, which does not contain duplicate items, thus forming a high-quality training sample.

[0048] As can be seen, this application embodiment designs a dynamic step size mechanism for constructing model training samples. According to this mechanism, the positive sample set can be determined based on click and purchase information in user behavior logs, and the entire product library of the system can be used as a negative sample candidate pool to determine the negative sample set. First, the positive sample product list is deduplicated, which reduces the number of required negative samples, thereby reducing or eliminating detached positive samples. Second, identifiers are added to the positive sample products and candidate negative sample products to ensure that the negative sample products selected for each positive sample product are different, and that no duplicate products appear in the negative sample set. This effectively makes the selection of negative sample products random and non-repeating, providing more learning information for the model training process and improving the quality of the model.

[0049] For example, even if a user clicks on or purchases multiple products with the same attributes under the same query, the negative sample set found for the search request of this application embodiment will not contain the same products, thus providing as much valuable learning information as possible for the model training stage. In addition, the embodiments of this application determine the number of negative samples according to a preset step size R. On the basis of avoiding free positive samples, it can also compensate for the reduction in the number of negative samples after deduplication, thereby minimizing the impact of related processing on the sample composition and improving the overall quality of the samples in terms of structure or composition.

[0050] Regarding the preset step size, a larger step size results in better sample quality and better model training performance. In some embodiments of this application, the step size can optionally be dynamically adjusted based on at least one of the following factors: the size of the negative sample candidate pool, the size of the system's product library, whether the products involved are popular, the required accuracy of model training, etc. For example, to improve the training effect of the coarse-ranking model, the higher the required training accuracy of the coarse-ranking model, and / or, the larger the size of the negative sample candidate pool, the larger the step size can be set.

[0051] Optionally, deduplication can be performed as follows: if there are two or more products with the same attributes in a positive sample product list, retain the information of one of the two or more products with the same attributes, and remove the information of the remaining products.

[0052] Optionally, the attribute includes at least one of the following: product type, product category, product model, and product device type. Therefore, products with the same attributes can be at least one of the following: products of the same type, products of the same category (such as the same first-level, second-level, or third-level category), products of the same model, or products of the same device type.

[0053] For example, in deduplication, if a positive sample product list has 3 products belonging to the same third-level category, then after deduplication, there will only be 1 product under that third-level category.

[0054] In some embodiments of this application, optionally, a first-class identifier can be added to each product in each deduplicated positive sample product list in the following manner:

[0055] S202, randomly sort multiple lists of deduplicated positive sample products;

[0056] S204, sequentially process multiple deduplicated positive sample product lists containing products with the same attributes.

[0057] The items are numbered sequentially.

[0058] For example, suppose the first set of positive samples for the first query contains 3 positive sample product lists (already deduplicated). These 3 lists are then randomly sorted as follows:

[0059] ·List 2: cate1-item4, cate2-item5, cate3-item6;

[0060] ·List 3: cate1-item7, cate2-item8;

[0061] ·List 1: cate1-item1, cate2-item2, cate3-item3;

[0062] Here, "cate1-item4" represents a product item 4, whose category is cate1.

[0063] This example illustrates sorting products of the same category. When executing step S204, it is necessary to number the products of the same category across lists. In this example, Arabic numerals are used as an example. Specifically, according to the random sorting of the three lists (List 2 → List 3 → List 1), the sequential numbering of products belonging to category 1 is as follows: item4(1), item7(2), item1(3), where the numbers in parentheses are the numbers. Similarly, the product numbers of category 2 are as follows: item5(1), item8(2), item2(3), and the product numbers of category 3 are as follows: item6(1), item3(2).

[0064] For ease of understanding, the following is a summary, with the first type of identifier in parentheses:

[0065] ·List 2: cate1-item4(1), cate2-item5(1), cate3-item6(1);

[0066] ·List 3: cate1-item7(2), cate2-item8(2);

[0067] ·List 1: cate1-item1(3), cate2-item2(3), cate3-item3(2).

[0068] Optionally, the second type of identifier can be set as: a number for products with the same attributes and randomly sorted in the negative sample candidate pool.

[0069] For example, suppose there are 5 products in the negative sample candidate pool that belong to category 1, and they are randomly sorted as follows:

[0070] cate1-item12, cate1-item11, cate1-item15, cate1-item13, cate1-item14; After adding the second type of identifier, it looks like this, with the second type of identifier in parentheses, taking the number as an example:

[0071] cate1-item12(1), cate1-item11(2), cate1-item15(3), cate1-item13(4), cate1-item14(5).

[0072] In an embodiment of this application, optionally, when both the first type of identifier and the second type of identifier use numerical codes, negative sample products can be selected through the following processing:

[0073] S302, Determine that the current positive sample product has a first-class attribute;

[0074] S304, using the following formula, determine the number of the negative sample product corresponding to the current positive sample product:

[0075] M = ceil(N / R) (1)

[0076] In equation (1), ceil() is the rounding up operation, M is the number of the current positive sample product in the deduplicated positive sample product list, N is the number of the product with the first type of attribute in the negative sample candidate pool, and the number of values ​​of N is R.

[0077] S306, from the products with the first type of attribute in the negative sample candidate pool, obtain the information of R products with number N, and use them as the R negative sample products corresponding to the current positive sample product.

[0078] For example, suppose the current positive sample product is item4 in the positive sample product list "List 2" mentioned in the previous example, and we can determine that its category is cate1. Then, we also need to look for negative sample products under cate1 in the negative sample candidate pool.

[0079] It is known that item4 is numbered 1. Assuming the pre-configured step size R = 20, according to equation (1):

[0080] 1 = ceil(N / 20) (2)

[0081] Where R = 20 means that 20 negative sample items need to be selected for item4 in this example. According to formula (2), the numbers N of the 20 negative sample items should be 1, 2, 3...20.

[0082] Then, from the negative sample candidate pool, select products numbered 1 to 20 from products in category cate1 and add them to the negative sample product list.

[0083] As can be seen, by using equation (1), the corresponding number of the positive sample product in the negative sample candidate pool can be determined, which also determines the corresponding negative sample product. In application, multiple positive sample product lists can be obtained based on user behavior logs. By performing the above operation on each product in each positive sample product list, the corresponding negative sample product lists can be obtained, which are sample pairs for model training. This can be used for high-quality training of coarse-ranking models in product recommendation systems and product search engines.

[0084] Optionally, the positive and negative sample sets obtained by using at least one of the above embodiments of this application can be used to train a coarse-ranking model according to the ListWise method in the LTR ranking technology, which can reduce the occurrence of free positive samples, improve the quality of training samples, and improve the overall training effect.

[0085] The above describes various implementations of the method for constructing training samples according to embodiments of this application. The following provides specific examples to describe optional processing procedures of embodiments of this application.

[0086] As an example, see reference Figure 4 The query is "AirPods", where each square represents a product. First, the operator can obtain the set of positive sample products for this query on that day based on user behavior log data. Then, the set is divided into multiple positive sample product lists according to the session, and duplicates are removed based on product attributes.

[0087] For ease of description, Figure 4 The first column on the left illustrates nine positive sample products from five sessions, all belonging to the same third-level category, "AirPods1". Figure 4 The nine positive sample items in the first column are as follows:

[0088] a) Items 1, 2, and 3 come from session1 and belong to list1;

[0089] b) Items 4 and 5 come from session2 and belong to list2;

[0090] c) Item 6, from session3, belongs to list3;

[0091] d) Items 7 and 8 come from session4 and belong to list4;

[0092] e) Item 9, from session 5, belongs to list 5;

[0093] Performing deduplication on each of the five lists will result in each list containing only one product under that third-level category, represented as follows: Figure 4 The second column contains 5 positive sample items. It should be noted that these 5 lists are randomly sorted; after deduplication, the 5 positive sample items are numbered to obtain... Figure 4 Items 1 through 5 in the second column.

[0094] Figure 4 The example sets the step size to 2, which means that two negative sample products need to be selected from the negative sample candidate pool for each of the five positive sample products.

[0095] Here, we use equation (1) described above to calculate the item number of the negative sample. For example, for Figure 4 According to formula (1), the positive sample product numbered 1 in the second column is 1 = ceil(N / 2), so the value of N is 1 and 2. Therefore, in the negative sample candidate pool, the products numbered 1 and 2 should be selected under the third-level category "AirPods1" as the two negative sample products corresponding to the positive sample product numbered 1.

[0096] For example, regarding Figure 4 The positive sample product numbered 5 in the second column, according to formula (1): 5 = ceil(N / 2), gives N values ​​of 9 and 10. Therefore, in the negative sample candidate pool, products numbered 9 and 10 under the third-level category "AirPods1" should be selected as the two negative sample products corresponding to the positive sample product numbered 5. Following the above method, it can be... Figure 4 The 5 positive sample items in the second column were selected to form a total of 10 negative sample items, such as... Figure 4 The 10 items in the third column.

[0097] Similarly, for other third-level categories involved in the positive sample product set of this query (i.e., "AirPods"), such as "AirPodspro", a similar approach can be executed. Figure 4 The example processing yields positive and negative sample products under the "AirPodspro" category.

[0098] After all processing is completed, the set of positive sample products and the set of negative sample products corresponding to the query can be obtained. It should be noted that, since the embodiments of this application have designed a complete processing mechanism to determine negative sample products based on product attributes, two types of product identifiers, and a preset step size, the determined negative sample products can be random and distinct from each other. Therefore, there are no duplicate products in either the set of positive sample products or the set of negative sample products, which can fundamentally improve the diversity and quality of the samples.

[0099] In addition, to facilitate the training of the coarse-ranking model using the ListWise method, positive sample items belonging to the same session in the set of positive sample items can be made into a positive sample item list, and the corresponding negative sample items can be made into a negative sample item list, which can be used as positive and negative sample list pairs in the training samples.

[0100] Corresponding to the method embodiments provided in this application, this application also provides an apparatus for constructing training samples. Figure 5 A structural block diagram of an apparatus 100 for constructing training samples according to an embodiment of this application is shown, which includes:

[0101] Positive sample processing module 110 is used to obtain information on clicked and paid products of multiple users within a predetermined time period, and to determine the information on clicked and paid products belonging to the first target query term as the first positive sample set generated within the predetermined time period. Products from the same session in the first positive sample set belong to the same positive sample product list.

[0102] The deduplication module 120 is used to perform deduplication on each positive sample product list to obtain a deduplicated positive sample product list.

[0103] The identification processing module 130 is used to add a first-class identifier to each product in each deduplicated positive sample product list and to determine that each product in the preset negative sample candidate pool has a second-class identifier.

[0104] The negative sample processing module 140 is used to select R negative sample products for each positive sample product in the negative sample candidate pool according to the attributes of each positive sample product in the deduplicated positive sample product list, to form a corresponding negative sample product list, and to form a first negative sample set corresponding to the first positive sample set based on one or more negative sample product lists. When selecting information of R products for the current positive sample product, the selection is based on the first type identifier of the current positive sample product and the second type identifier of the products in the negative sample candidate pool, so that there is no duplicate product information in the first negative sample set, where R is a preset step size.

[0105] Optionally, the deduplication module 120 is used to retain the information of one of the two or more products with the same attributes when there are two or more products with the same attributes in a single positive sample product list, and remove the information of the remaining products.

[0106] Optionally, the identification processing module 130 includes:

[0107] The sorting submodule is used to randomly sort multiple lists of deduplicated positive sample products.

[0108] The numbering processing submodule is used to sequentially number products with the same attributes contained in multiple deduplicated positive sample product lists.

[0109] Optionally, the second type of identifier is a number for products with the same attributes and randomly sorted in a preset negative sample candidate pool.

[0110] Optionally, the negative sample processing module 140 includes:

[0111] The attribute determination submodule is used to determine whether the current positive sample product has the first type of attribute;

[0112] The number calculation submodule is used to determine the number of the negative sample product corresponding to the current positive sample product using the following formula:

[0113] M = ceil(N / R),

[0114] Where ceil() is the rounding up operation, M is the number of the current positive sample product in the deduplicated positive sample product list, N is the number of the product with the first type of attribute in the negative sample candidate pool, and the number of values ​​of N is R.

[0115] The product information acquisition submodule is used to acquire information of R products with the first type of attribute from the negative sample candidate pool, and use them as the R negative sample products corresponding to the current positive sample product.

[0116] Optionally, the attribute includes at least one of the following: product type, product category, product model, and product device type.

[0117] Optionally, the attribute includes the product's third-level category.

[0118] Optionally, the first positive sample set and the corresponding first negative sample set are used to train a coarse-ranking model in the search system according to the document list method in the ranking learning technique.

[0119] This application also provides an electronic device, including: a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the method for constructing training samples according to any of the above embodiments.

[0120] Figure 6 A schematic diagram of the hardware structure of an embodiment of the electronic device provided in this application is shown.

[0121] like Figure 6 As shown, the electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0122] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0123] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is non-volatile solid-state memory.

[0124] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to this disclosure.

[0125] The processor 601 implements any of the methods for constructing training samples in the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0126] In one example, the electronic device may also include a communication interface 603 and a bus 610. For example, Figure 6As shown, the processor 601, memory 602, and communication interface 603 are connected via bus 610 and communicate with each other. The electronic device in this embodiment can be a server or other computing device, or it can be a cloud server.

[0127] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0128] Bus 610 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0129] Furthermore, in conjunction with the methods for constructing training samples in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the methods for constructing training samples in the above embodiments.

[0130] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0131] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Machine-readable media can include non-transitory computer-readable storage media, such as electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, and can also include radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0132] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0133] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0134] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for constructing a training sample, characterized by, include: Get information on clicked and paid products from multiple users within a predetermined time period, and determine the clicked and paid products that belong to the first target query term as the first positive sample set generated within the predetermined time period. Products from the same session in the first positive sample set belong to the same positive sample product list. Each positive sample product list is deduplicated. If there are two or more products with the same attributes in a single positive sample product list, the information of one of the two or more products with the same attributes is retained, and the information of the remaining products is removed to obtain a deduplicated positive sample product list. Multiple deduplicated positive sample product lists are randomly sorted. By sequentially numbering the products with the same attributes contained in the multiple deduplicated positive sample product lists, a first-class identifier is added to each product in each deduplicated positive sample product list. It is determined that each product in the preset negative sample candidate pool has a second-class identifier. The second-class identifier is the number of the products with the same attributes and random sorting in the preset negative sample candidate pool. Based on the attributes of each positive sample product in the deduplicated positive sample product list, R negative sample products are selected for each positive sample product in the negative sample candidate pool to form a corresponding negative sample product list. The first negative sample set corresponding to the first positive sample set is formed based on one or more negative sample product lists; wherein, the negative sample candidate pool is the full product library; Based on the attributes of each positive sample product in the deduplicated positive sample product list, R negative sample products are selected from the negative sample candidate pool for each positive sample product, including: It has been determined that the current positive sample product has a first-class attribute; The negative sample item number corresponding to the current positive sample item is determined using M=ceil(N / R): Where ceil() is the round-up operation, M is the number of the current positive sample product in the deduplicated positive sample product list, N is the number of the product with the first type of attribute in the negative sample candidate pool, and the number of values ​​of N is R. Among the products with the first type of attribute in the negative sample candidate pool, information of R products with number N is obtained and used as the R negative sample products corresponding to the current positive sample product.

2. The method of any one of claim 1, characterized in that, The attributes include at least one of the following: product type, product category, product model, and product device model.

3. The method of any one of claim 1, wherein, The attributes include the product's third-level category.

4. The method of any one of claim 1, wherein, The first positive sample set and the corresponding first negative sample set are used to train a coarse-ranking model in the search system based on the document list method in the ranking learning technique.

5. An apparatus for constructing a training sample, the apparatus comprising: include: The positive sample processing module is used to obtain information on clicked and paid products of multiple users within a predetermined time period, and to determine the information on clicked and paid products belonging to the first target query term as the first positive sample set generated within the predetermined time period. Products from the same session in the first positive sample set belong to the same positive sample product list. The deduplication module is used to deduplicatize each positive sample product list. If there are two or more products with the same attributes in a single positive sample product list, the information of one of the two or more products with the same attributes is retained, and the information of the remaining products is removed to obtain a deduplicated positive sample product list. The sorting submodule is used to randomly sort multiple lists of deduplicated positive sample products. The numbering processing submodule is used to sequentially number the products with the same attributes contained in multiple deduplicated positive sample product lists, add a first type of identifier to each product in each deduplicated positive sample product list, and determine that each product in the preset negative sample candidate pool has a second type of identifier. The second type of identifier is the number of products with the same attributes and randomly sorted in the preset negative sample candidate pool. The negative sample processing module is used to select R negative sample products for each positive sample product in the negative sample candidate pool according to the attributes of each positive sample product in the deduplicated positive sample product list, thereby forming a corresponding negative sample product list. The first negative sample set corresponding to the first positive sample set is formed based on one or more negative sample product lists; wherein, the negative sample candidate pool is the full product library. The negative sample processing module includes: The attribute determination submodule is used to determine whether the current positive sample product has the first type of attribute; The number calculation submodule is used to determine the number of the negative sample product corresponding to the current positive sample product using M = ceil(N / R): Where ceil() is the round-up operation, M is the number of the current positive sample product in the deduplicated positive sample product list, N is the number of the product with the first type of attribute in the negative sample candidate pool, and the number of values ​​of N is R. The product information acquisition submodule is used to acquire information of R products with the first type of attribute from the negative sample candidate pool, and use them as the R negative sample products corresponding to the current positive sample product.

6. An electronic device, comprising: The electronic device includes a processor and a memory storing computer program instructions; when the electronic device executes the computer program instructions, it implements the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, perform the method as described in any one of claims 1-4.

8. A computer program product, characterised in that, It includes computer program instructions that, when executed by a processor, implement the method as described in any one of claims 1-4.

Images