Training sample generation method and device, electronic equipment and storage medium

Abstract

The application discloses a kind of training sample generation method, device, electronic equipment and storage medium.The method comprises: based on the user behavior data of system in predetermined time period, according to the multiple search sessions under at least first query word, generate multiple positive sample commodity list;Determine the one or more attributes of multiple commodities under the first query word;In the case where the multiple attributes are included under the first query word, determine the proportion of the number of commodities under each attribute in the multiple attributes in the total number of commodities under the first query word;According to the multiple attributes and the proportion corresponding to the multiple attributes, select multiple commodities in negative sample candidate pool to construct multiple negative sample commodity list.Using the embodiment of the application, high-quality training samples can be obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning and e-commerce platforms, and particularly relates to a training sample generation method and device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] The search engine of an e-commerce platform recalls commodities according to a search request (query), and the recalled commodities are exposed after rough sorting, fine sorting and other processing, for users to select and purchase. Regarding rough sorting and fine sorting, refer to Figure 1 The overall strategy in the rough sorting stage is to filter and sort the recall results according to the relevance of the recall results to the query, and the commodities with high relevance enter the fine sorting stage. Various factors can be considered to affect the sorting in the fine sorting stage, for example, the historical sales or current heat of the commodities can be considered to make the commodities that are more likely to be selected and purchased by users have exposure opportunities, and for example, the seller promotion factor is considered to give priority to exposure of promotion commodities, and the like. The recall exposure strategy described above is applicable to the search engine or recommendation system of various network platforms.

[0003] Among them, one of the core problems in rough sorting processing is the negative sample selection problem of the rough sorting model. Due to the limited log data that the system can record, most of the existing methods are based on user behavior (such as search keywords, browsing and exposing commodities, clicking commodities, collecting commodities, and paying for purchasing commodities) to construct positive and negative samples, for example, a series of behaviors after a user inputs a search word, and the commodity that the user finally pays for purchasing is the positive sample for this search, and the exposed but not paid and / or not clicked commodity is the negative sample. Such positive and negative samples reflect the selection tendency of the user to a certain extent, and can be used as training samples of the rough sorting model. However, the negative samples determined in the above manner have poor diversity, and have large differences from the data distribution actually processed by the rough sorting model, and the training effect is difficult to satisfy people. At present, there is a large optimization space for the training sample composition problem of the rough sorting model in related scenarios, and it is one of the hotspots and difficulties for researchers to study, and the training effect of the rough sorting model needs to be improved. SUMMARY

[0004] Therefore, the embodiments of the present application provide a method and device for constructing training samples, an electronic device and a computer readable storage medium, to solve at least one of the above technical problems.

[0005] In a first aspect, the embodiments of the present application provide a training sample generation method, comprising:

[0006] generating a plurality of positive sample commodity lists according to a plurality of search sessions under at least a first query word based on user behavior data of the system in a predetermined time period;

[0007] determining one or more attributes of a plurality of commodities under the first query word;

[0008] In a case where the first query word includes multiple attributes, determine a proportion of a quantity of goods under each attribute in the multiple attributes in a total quantity of goods under the first query word;

[0009] According to the multiple attributes and the proportions corresponding to the multiple attributes, select multiple goods in a negative sample candidate pool to construct multiple negative sample good lists.

[0010] In a second aspect, an embodiment of the present application provides a training sample generation apparatus, comprising:

[0011] A positive sample processing module is configured to generate multiple positive sample good lists according to multiple search sessions under at least a first query word based on user behavior data of a system in a predetermined time period;

[0012] An attribute processing module is configured to determine one or more attributes of multiple goods under the first query word;

[0013] A proportion processing module is configured to, in a case where the first query word includes multiple attributes, determine a proportion of a quantity of goods under each attribute in the multiple attributes in a total quantity of goods under the first query word;

[0014] A negative sample processing module is configured to select multiple goods in a negative sample candidate pool according to the multiple attributes and the proportions corresponding to the multiple attributes to construct multiple negative sample good lists.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory storing computer program instructions; the electronic device implements steps of the method as described above when executing the computer program instructions.

[0016] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer storage medium storing computer program instructions, the computer program instructions being executed by a processor to implement steps of the method as described above.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer program instructions, the computer program instructions being executed by a processor to implement steps of the method as described above.

[0018] Based on the embodiments of the present application, the negative sample goods can be determined according to the proportion of the quantity of different attribute positive sample goods in a long time period, which can be more consistent with the distribution of the goods actually processed by the system model. The positive and negative sample good lists obtained by using the embodiments of the present application can essentially and stably improve the quality of model training. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings in the embodiments of the present application will be briefly introduced.

[0020] Figure 1 is a model schematic diagram of a recommendation system.

[0021] Figure 2 is a flow schematic diagram of a training sample generation method of the embodiments of the present application.

[0022] Figure 3 is a principle schematic diagram of constructing a training sample by using the embodiments of the present application.

[0023] Figure 4 is a structural block diagram of an apparatus for constructing a training sample of the embodiments of the present application.

[0024] Figure 5 is a schematic diagram of an electronic device for implementing the training sample generation method of the embodiments of the present application. DETAILED DESCRIPTION

[0025] The principles and spirits of the present application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present application, and do not limit the scope of the present application in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0026] Those skilled in the art know that the embodiments of the present application can be implemented as a system, an apparatus, a device, a method, a computer readable storage medium or a computer program product. Therefore, the present disclosure can be embodied as a complete hardware, a complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0027] According to the embodiments of the present application, a training sample generation method, apparatus, storage medium and computing device are provided.

[0028] In this document, the number of any element in the drawings is used for example and not limitation, and any name in the drawings is only used for distinction and does not have any limiting meaning.

[0029] The principles and spirits of the present application will be described below with reference to several representative embodiments of the present application.

[0030] The inventors of the present application have found that in the training phase of the coarse ranking model, the listwise approach in the learning to rank (LTR) technology is overall superior to other methods. In constructing the training sample of the coarse ranking model, the list of user clicks and paid goods can be used as the positive sample list, and a plurality of goods of the same category can be selected from the system goods library to form the negative sample list, and a sample pair of the positive and negative sample lists is generated for training the coarse ranking model. Specifically, for the same query, if the user clicks and pays for 10 goods in one search session, the number of negative sample goods obtained under the corresponding category of the goods library can be a multiple of 10, for example, 50 or 100. In this way, if the number of goods clicked and paid by the user in one search behavior is large, the number of negative samples is correspondingly large, and vice versa. If the number of goods clicked and paid by the user is small, the number of negative samples is correspondingly small. Similarly, if the goods clicked by the user cover many categories, the negative samples are rich in categories, the diversity of the samples is good, and vice versa. If the user repeatedly clicks the same (or similar) goods, a large number of similar negative sample goods will appear, the diversity of the samples is limited, and the quality of the samples is unstable.

[0031] Figure 2 A flowchart of a training sample generation method of an embodiment of the present application is shown, which includes the following steps:

[0032] S101, based on the user behavior data of the system in a predetermined time period, generating a plurality of positive sample goods lists according to a plurality of search sessions under at least a first query word;

[0033] S102, determining one or more attributes of a plurality of goods under the first query word;

[0034] S103, in the case where the first query word includes a plurality of attributes, determining the proportion of the number of goods under each attribute in the total number of goods under the first query word in the plurality of attributes;

[0035] S104, selecting a plurality of goods from the negative sample candidate pool according to the plurality of attributes and the proportions corresponding to the plurality of attributes to construct a plurality of negative sample goods lists.

[0036] According to the embodiments of the present application, firstly, the data of user click and payment behaviors occurring in a period of time can be obtained from the system log, which usually contains a large amount of click and payment behaviors of a large number of users under different queries, involving a large number of different kinds or different attributes of goods. According to the embodiments of the present application, a plurality of positive sample goods lists can be generated according to the query words, i.e. queries, contained in the log data. For example, assuming that the query1 and query2 are contained in the user behavior log in the period of time, and assuming that p1 sessions are contained under the query1, the goods clicked and paid by the user in each session can constitute a positive sample goods list, so that p1 positive sample goods lists can be obtained. Under the query2, assuming that p2 sessions are contained, and for the same reason, p2 positive sample goods lists can be obtained.

[0037] On the other hand, taking the query1 as an example, for all or part of the goods under the query1, the attribute information of the goods (such as the type of the goods, the category to which the goods belong, the model, the model number or the brand, etc.) can be obtained, and the number of goods under each attribute can be obtained. For example, assuming that the number of goods with the first attribute is q1, and the number of goods with the second attribute is q2, the proportions of the number of goods under the first attribute and the second attribute in the total number (denoted as Q) of all or part of the goods under the query1 can be obtained, which are q1 / Q and q2 / Q respectively. Then, according to the values of q1 / Q and q2 / Q, the goods with the corresponding attributes and the corresponding numbers can be obtained from the preset negative sample candidate pool as negative sample goods, which can constitute a plurality of negative sample goods lists under the query1.

[0038] It can be seen that the embodiments of the present application select negative sample goods according to the proportion of the number of positive sample goods of different attributes under the same query in the total number of goods. The reason for this processing is that if the log data of a long time period such as 2 days, 5 days, 7 days or more (not only the current day) is taken as the processing object, the number of goods of various attributes clicked and paid by different users within this time period can reflect the characteristics of the attribute goods as sample data to some extent. For example, the user behavior log under a query within a week can be analyzed. It is assumed that it is found that the number of times a certain model of intelligent device is clicked by different users is significantly more than the number of times other models of intelligent device are clicked. This indicates that the certain model of intelligent device accounts for a large proportion (or weight) in the positive sample as a whole. Therefore, it can be inferred that the certain model of intelligent device accounts for a large proportion in the negative sample under the query. Therefore, the negative sample goods are determined according to the proportion of the number of positive sample goods of different attributes within a long time period, which can be more consistent with the distribution of goods actually processed by the system model. The positive and negative sample goods list obtained by the embodiments of the present application is used to train the coarse ranking model of the system, which can stably improve the quality of model training.

[0039] In relation to a session (also sometimes referred to as a search session or a user session), the interactive process between a user and a network system within a period of time is generally referred to as a "session". A session can record a series of behaviors of a user within a time period from starting a search to leaving the search, such as querying goods, browsing goods, collecting goods, paying for purchase, and exiting.

[0040] According to the embodiments of the present application, the selection of the plurality of goods in the negative sample candidate pool according to the plurality of attributes and the plurality of proportions corresponding to the plurality of attributes in step S104 can be implemented by one of the following three methods.

[0041] Method one: selecting negative sample goods according to system setting

[0042] S202, after determining the proportion of the number of goods of each attribute in the plurality of attributes of the goods under the first query term in the total number of goods under the first query term, obtaining a plurality of negative sample goods in a negative sample candidate pool, wherein the plurality of negative sample goods contain the same attributes as the plurality of attributes under the first query term, and the proportion of the number of goods of different attributes in the plurality of negative sample goods is the same as the proportion corresponding to the plurality of attributes under the first query term;

[0043] S204, constructing at least one negative sample goods list based on the plurality of negative sample goods.

[0044] For example, assuming that the full inventory of the system is the negative sample candidate pool, and the user behavior log within 10 days is examined, which contains several queries. First, for the query A of interest, on the one hand, several positive sample product lists under the query A can be generated, and each positive sample product list includes the information of the products clicked and paid by the user in a single session under the query A.

[0045] On the other hand, for the query A, the attribute information of the clicked and paid products under the query A from all users can be obtained. After classifying and counting the attribute information, the quantity ratio of the first model mobile phone and the second model mobile phone under the query A can be obtained as A1:A2.

[0046] In the negative sample candidate pool, according to the ratio A1:A2, two types of mobile phones with the quantity ratio A1:A2 are obtained from the product set of the first model and the second model, respectively, to form the negative sample product list. The obtaining method can be selected according to the system settings, for example, a plurality of first model mobile phones can be selected after excluding certain products in the negative sample candidate pool according to the predetermined exclusion conditions, and / or a plurality of second model mobile phones can be selected in the negative sample candidate pool according to the predetermined priority rules. Then, if necessary, random selection can be continued until the quantity of the two types of mobile phones meets the requirement of the ratio A1:A2.

[0047] In this way, a plurality of negative sample product lists can be obtained, which can be assigned to the obtained positive sample product lists one by one (or more than one to one, one to more than one), to form positive and negative sample pairs, which can be used for high-quality training of the coarse ranking model in the product recommendation system (or product search engine).

[0048] • Method two: randomly selecting negative sample products

[0049] S302, after determining the proportion of the quantity of products under each attribute in the plurality of attributes of the product under the first query term in the total quantity of products under the first query term, determining that the quantity of products under the first to Nth attributes under the first query term corresponds to the proportion K1:K2:…K N , wherein K i represents the percentage of the quantity of products of the ith attribute, i=1~N, N is a positive integer;

[0050] S304, in the product set of the first to Nth attributes in the negative sample candidate pool, respectively randomly obtaining information of at least K i products;

[0051] S306, constructing at least one negative sample product list based on the information of the at least K i products.

[0052] For example, assuming that the second-hand commodity library in the system is the negative sample candidate pool, and the user behavior log in the past two weeks (14 days) is examined, first, for the query B of interest, on the one hand, a number of positive sample commodity lists under the query B can be generated, and each positive sample commodity list includes the information of the commodities clicked and paid by the user in a single session under the query B.

[0053] On the other hand, the attribute information of the clicked and paid commodities under the query B from all users can be obtained, and the attribute information is classified and counted to obtain, for example, 300 first-brand tablet computers, 80 second-brand tablet computers, 20 third-brand tablet computers, and 10 fourth-brand tablet computers under the query B, and then the quantity ratio of the three brands of tablet computers under the query B can be obtained as 30:8:2:1.

[0054] In the negative sample candidate pool, according to the ratio of 30:8:2:1, four brands of tablet computers with the quantity ratio of 30:8:2:1 are randomly obtained from the tablet computer product sets of the first brand, the second brand, the third brand, and the fourth brand, respectively, to form a negative sample commodity list, and are matched to the positive sample commodity list under the query B randomly or according to a preconfigured mapping mode (one-to-one, one-to-many, or many-to-one) to form a positive-negative sample pair, which can be used for high-quality training of the coarse ranking model in the commodity recommendation system.

[0055] • Method three: determining negative sample commodities according to the identification carried by the commodities

[0056] S402, after determining the proportion of the number of commodities under each attribute in the plurality of attributes of the commodities under the first query term in the total number of commodities under the first query term, determining that the first query term includes an i-th attribute and an i+1-th attribute, and the proportion corresponding to the number of commodities under the i-th attribute and the i+1-th attribute is K i :K i+1 ;

[0057] S404, constructing a plurality of temporary lists, each temporary list including at least one commodity of the i-th attribute and at least one commodity of the i+1-th attribute;

[0058] S406, adding a first type of identification to the plurality of temporary lists, adding a second type of identification to a plurality of commodities in the negative sample candidate pool having the i-th attribute, and adding a third type of identification to a plurality of commodities in the negative sample candidate pool having the i+1-th attribute;

[0059] S408, for the jth temporary list in the plurality of temporary lists, based on the first type identifier corresponding to the jth temporary list and the second type identifier carried by the goods in the negative sample candidate pool, K i negative sample goods are determined; and based on the first type identifier corresponding to the jth temporary list and the third type identifier carried by the goods in the negative sample candidate pool, K i+1 negative sample goods are determined.

[0060] S410, based on the K i negative sample goods and the K i+1 negative sample goods, a first negative sample goods list is constructed so that there is no repeated goods information in the first negative sample goods list.

[0061] In the embodiments of the present application, the role of constructing a temporary list is mainly to determine a suitable negative sample goods list for each positive sample goods list. The following will be described in detail.

[0062] As an example, assuming that the second-hand goods library in the system is the negative sample candidate pool, and the user behavior log in, for example, 7 days is investigated, first, for the query C of interest, on the one hand, a plurality of positive sample goods lists under the query C can be generated, wherein a single positive sample goods list contains the information of the goods clicked and paid by the user in a single session under the query C.

[0063] On the other hand, the attribute information of the clicked and paid goods from all users under the query C is obtained, and the attribute information is classified and counted. In this example, the product attributes are divided by "model of electronic products", and three types of attributes of goods under the query C are taken as an example for illustration. The statistical information of the three types of attributes is as follows:

[0064] • The first type of attribute: the first type of wireless earphone, the number of which is K1;

[0065] • The second type of attribute: the second type of wireless earphone, the number of which is K2;

[0066] • The third type of attribute: the third type of wireless earphone, the number of which is K3;

[0067] Therefore, the quantity ratio of the three types of wireless earphones under the query C is K1:K2:K3.

[0068] On the other hand, in this example, assuming that the number of positive sample goods lists under the query C in the 7-day user behavior log is 1000, at least 1000 temporary lists can be constructed, wherein each temporary list includes at least three goods belonging to the three types of attributes described above.

[0069] Then, four IDs are added, for example, respectively:

[0070] • Adding IDs to the temporary list, different IDs corresponding to different temporary lists;

[0071] • Adding IDs to all or part of the first type of wireless earphones in the second-hand goods library, different IDs corresponding to different second-hand products;

[0072] • Adding IDs to all or part of the second type of wireless earphones in the second-hand goods library, different IDs corresponding to different second-hand products;

[0073] • Adding IDs to all or part of the third type of wireless earphones in the second-hand goods library, different IDs corresponding to different second-hand products.

[0074] Among them, the above "different second-hand products" can be understood as second-hand products with the same attributes (such as the same model) but different device serial numbers, such as second-hand wireless earphones of the same model sold by different users on the second-hand trading platform.

[0075] Then, for the first temporary list, according to the ID of the first temporary list and the ID of the first type of wireless earphones in the second-hand goods library, K1 first type of wireless earphones can be determined, wherein there is no repeated product information, which can be used as high-quality negative samples;

[0076] Similarly, according to the ID of the first temporary list and the ID of the second type of wireless earphones in the second-hand goods library, K2 second type of wireless earphones can be determined, wherein there is no repeated product information, which can be used as high-quality negative samples;

[0077] Similarly, according to the ID of the first temporary list and the ID of the third type of wireless earphones in the second-hand goods library, K3 third type of wireless earphones can be determined, wherein there is no repeated product information, which can be used as high-quality negative samples;

[0078] At this point, based on the K1 first type of wireless earphones, K2 second type of wireless earphones and K3 third type of wireless earphones obtained above, a negative sample product list can be constructed. The negative sample product list can be matched to the positive sample product list under the query C randomly or according to a preconfigured mapping mode (one-to-one, one-to-many or many-to-one) to form a positive and negative sample pair, which can be used for high-quality training of the coarse ranking model in the product recommendation system.

[0079] It can be seen that according to the above at least one embodiment of the present application, a certain number of temporary lists are constructed according to certain rules, and by means of the temporary list ID, the ID of the different attribute goods in the negative sample candidate pool and the statistical data of the positive sample goods, a certain number of negative sample goods of a certain attribute are selected, which not only realizes the non-repetition of each negative sample good, but also realizes the controllability of the attributes and the number of negative sample goods to a certain extent, and the mutual balance between each negative sample good list obtained, which actually optimizes the composition of negative samples in essence, greatly and stably improves the overall quality of training samples, and can significantly improve the overall performance of the goods recommendation system.

[0080] The list of positive and negative samples obtained by the above at least one embodiment of the present application can train a rough ranking model according to the ListWise method in the LTR technology, stably obtain high-quality positive and negative sample lists, and improve the overall model training effect.

[0081] As an example, the following describes the operation process of obtaining negative samples when the first type of identification, the second type of identification and the third type of identification are all Arabic numerals.

[0082] Specifically, based on the first type of identification corresponding to the jth temporary list and the second type of identification carried by the goods in the negative sample candidate pool, K i negative sample goods are determined, which can be realized according to the following steps:

[0083] The number of negative sample goods corresponding to each ith attribute goods in the jth temporary list is determined by the following formula:

[0084] M = ceil(N' / K i )

[0085] Where ceil() is the rounding up operation, M is the number corresponding to the jth temporary list, N' is the number of the ith attribute goods in the negative sample candidate pool, and the number of values of N' is K i .

[0086] Similarly, based on the first type of identification corresponding to the jth temporary list and the third type of identification carried by the goods in the negative sample candidate pool, K i+1 negative sample goods are determined, which can be realized according to the following steps:

[0087] The number of negative sample goods corresponding to each ith attribute goods in the jth temporary list is determined by the following formula:

[0088] M = ceil(N" / K i )

[0089] Wherein, N" is the digital number of the (i+1)th attribute commodity in the negative sample candidate pool, and the number of values of N" is K i+1 .

[0090] For the convenience of understanding, with reference to Figure 3 , an illustrative example is provided below to describe the specific implementation process of an embodiment of the present application.

[0091] The illustrative example is based on a user behavior log of 7 days, which contains a large number of queries. Taking the query "AirPods" as an example, the commodity attributes are divided according to the three-level category to which the commodity belongs. The statistical result is that there are three three-level categories under the query "AirPods", which are AirPods1, AirPods2 and AirPodspro, and the number ratio of commodities belonging to the above three three-level categories in the 7-day user behavior log data is 2:5:64.

[0092] When constructing the positive sample commodity list, in the 7-day user behavior log, the commodities clicked and paid by the user in each session under the query "AirPods" are used to construct a positive sample commodity list. It is assumed that 200 positive sample commodity lists are obtained.

[0093] Before constructing the negative sample commodity list, 200 temporary lists are constructed and numbered 1-200. For example, Figure 3 , each temporary list contains the labels of three three-level categories, and can also contain three commodity information (belonging to three three-level categories respectively). The following is an example:

[0094] For example, temporary list 1 includes 1 AirPods1 (corresponding to number 1), 1 AirPods2 (corresponding to number 1) and 1 AirPodspro (corresponding to number 1);

[0095] For example, temporary list 2 includes 1 AirPods1 (corresponding to number 2), 1 AirPods2 (corresponding to number 2) and 1 AirPodspro (corresponding to number 2);

[0096] For example, temporary list 100 Figure 3 includes 1 AirPods1 (corresponding to number 100), 1 AirPods2 (corresponding to number 100) and 1 AirPodspro (corresponding to number 100).

[0097] According to the number of the temporary list and the quantity ratio 2:5:64, the number N' of negative sample goods under each three-level category is calculated. Among them, the quantity ratio of the negative sample goods obtained under the above three three-level categories (AirPods1, AirPods2, AirPodspro) should comply with 2:5:64.

[0098] Specifically, for AirPods1 numbered 1, the formula M = ceil(N' / K i ) is calculated, M = 1, K i = 2, and the following is obtained:

[0099] 1 = ceil(N' / 2)

[0100] The value of N' is 1 and 2, that is, under the three-level category AirPods1 in the negative sample candidate pool, the goods numbered 1 and 2 should be selected as two negative sample goods.

[0101] Similarly, for AirPods2 numbered 1, the formula M = ceil(N' / K i ) is calculated, M = 1, K i = 5, and the following is obtained:

[0102] 1 = ceil(N' / 5)

[0103] The value of N' is 1-5, that is, under the three-level category AirPods2 in the negative sample candidate pool, the goods numbered 1-5 should be selected as five negative sample goods.

[0104] Similarly, for AirPodspro numbered 1, the formula M = ceil(N' / K i ) is calculated, M = 1, K i = 64, and the following is obtained:

[0105] 1 = ceil(N' / 64)

[0106] The value of N' is 1-64, that is, under the three-level category AirPodspro in the negative sample candidate pool, the goods numbered 1-64 should be selected as 64 negative sample goods.

[0107] So far, a negative sample goods list can be constructed, which includes a total of 71 goods, namely: the goods numbered 1 and 2 under the three-level category AirPods1, the goods numbered 1-5 under the three-level category AirPods2, and the goods numbered 1-64 under the three-level category AirPodspro. The negative sample goods list can be combined with any positive sample goods list to form a sample pair for training the coarse ranking model.

[0108] It should be noted that in the embodiments of the present application, the goods under various attributes in the negative sample candidate pool are numbered after random sorting, so the selected negative sample goods according to the number are random and non-repeating. This processing can make the distribution of negative samples as close to the actual situation as possible, so that the model can learn more sample attributes in the training process, and essentially improve the overall training effect of the system.

[0109] Similarly, for AirPods1 numbered 2, in the calculation formula M = ceil(N' / K i ), M = 2, K i = 2, we get:

[0110] 2 = ceil(N' / 2)

[0111] The value of N' is 3 and 4, that is, under the third-level category AirPods1 in the negative sample candidate pool, the goods numbered 3 and 4 should be selected as the two negative sample goods.

[0112] Similarly, for AirPods2 numbered 2, in the calculation formula M = ceil(N' / K i ), M = 2, K i = 5, we get:

[0113] 2 = ceil(N' / 5)

[0114] The value of N' is 5-10, that is, under the third-level category AirPods2 in the negative sample candidate pool, the goods numbered 5-10 should be selected as the five negative sample goods.

[0115] It can be understood that the subsequent algorithm process is similar to the foregoing, which will not be described here. After processing, a new negative sample goods list can be constructed, and the number of goods in the list is also 71.

[0116] As can be seen, in the exemplary example, 200 negative sample goods lists can be constructed, each of which contains 71 goods of three attributes. Since the numbers of the goods in the newly constructed negative sample goods list and the already constructed negative sample goods list are different, there will be no repeated goods in different negative sample goods lists, ensuring that high-quality sample pairs can always be generated.

[0117] In addition, corresponding to the method embodiments provided by the present application, the embodiments of the present application also provide a training sample generation device. Figure 4 The structural block diagram of the training sample generation device 100 of the embodiments of the present application is shown, which includes:

[0118] The positive sample processing module 110 is configured to generate a plurality of positive sample product lists based on user behavior data of the system within a predetermined time period according to a plurality of search sessions under at least a first query word.

[0119] The attribute processing module 120 is configured to determine one or more attributes of a plurality of products under the first query word.

[0120] The proportion processing module 130 is configured to determine, in the case that the first query word includes a plurality of attributes, a proportion of a number of products under each attribute in the plurality of attributes in a total number of products under the first query word.

[0121] The negative sample processing module 140 is configured to select a plurality of products from a negative sample candidate pool according to the plurality of attributes and the proportions corresponding to the plurality of attributes to construct a plurality of negative sample product lists.

[0122] Optionally, the plurality of positive sample product lists and the plurality of negative sample product lists are used to train a coarse ranking model in a product search system by a Listwise method in a ranking learning LTR technique.

[0123] Optionally, the predetermined time period involves at least two natural days.

[0124] Optionally, the user behavior data includes attribute information of products clicked by a user and attribute information of products paid by the user.

[0125] Optionally, the attribute information includes at least one of the following: a type of a product, a category of a product, a model of a product, and a machine type of a product.

[0126] Optionally, the positive sample product list includes information of products clicked and paid by a user in a single search session.

[0127] Optionally, the selecting a plurality of products from a negative sample candidate pool according to the plurality of attributes and the proportions corresponding to the plurality of attributes includes:

[0128] Obtaining a plurality of negative sample products in a negative sample candidate pool, wherein the plurality of negative sample products contain the same attributes under the first query word, and a proportion of a number of products under different attributes in the plurality of negative sample products is the same as the proportions corresponding to the plurality of attributes under the first query word; and constructing at least one negative sample product list based on the plurality of negative sample products.

[0129] Optionally, the negative sample processing module 140 includes:

[0130] The proportion determining sub-module is configured to determine the proportion of the number of the commodities of the first to Nth types of attributes under the first query word as K1:K2:…K N , wherein K i represents the percentage of the number of the commodities of the ith type of attribute, i=1~N, N is a positive integer; in the set of commodities of the first to Nth types of attributes in the negative sample candidate pool, information of at least K i commodities is randomly obtained respectively;

[0131] The negative sample commodity list constructing sub-module is configured to construct at least one negative sample commodity list based on the information of the at least K i commodities.

[0132] Optionally, the negative sample processing module 140 comprises:

[0133] The attribute determining sub-module is configured to determine that the first query word comprises an ith type of attribute and an i+1th type of attribute, and the proportion of the number of the commodities of the ith type of attribute and the i+1th type of attribute is K i :K i+1 .

[0134] The temporary list constructing sub-module is configured to construct a plurality of temporary lists, and each temporary list comprises at least one commodity of the ith type of attribute and at least one commodity of the i+1th type of attribute.

[0135] The identification processing sub-module is configured to add a first type of identification to the plurality of temporary lists, add a second type of identification to a plurality of commodities of the ith type of attribute in the negative sample candidate pool, and add a third type of identification to a plurality of commodities of the i+1th type of attribute in the negative sample candidate pool.

[0136] The negative sample commodity determining sub-module is configured to, for a jth temporary list in the plurality of temporary lists, determine K i negative sample commodities based on the first type of identification corresponding to the jth temporary list and the second type of identification carried by the commodities in the negative sample candidate pool, and determine K i+1 negative sample commodities based on the first type of identification corresponding to the jth temporary list and the third type of identification carried by the commodities in the negative sample candidate pool.

[0137] The negative sample commodity list generating sub-module is configured to generate a first negative sample commodity list based on the K i negative sample commodities and the K i+1 negative sample commodities, so that there is no repeated commodity information in the first negative sample commodity list.

[0138] Optionally, the number of the plurality of temporary lists is greater than or equal to the number of the plurality of positive sample commodity lists.

[0139] Optionally, the first type of identifier, the second type of identifier and the third type of identifier are all digital numbers, wherein the first type of identifier, the second type of identifier and the third type of identifier are independently numbered.

[0140] Optionally, the negative sample commodity determination submodule comprises:

[0141] A first calculation unit is configured to determine the negative sample commodity number corresponding to each i-th type of attribute commodity in the j-th temporary list by using the following formula:

[0142] M = ceil(N' / K i ),

[0143] wherein ceil() is a rounding up operation, M is a digital number corresponding to the j-th temporary list, N' is a digital number of the i-th type of attribute commodity in the negative sample candidate pool, and the number of values of N' is K i

[0144] A second calculation unit is configured to determine the negative sample commodity number corresponding to each i+1-th type of attribute commodity in the j-th temporary list by using the following formula:

[0145] M = ceil(N" / K i ),

[0146] wherein N" is a digital number of the i+1-th type of attribute commodity in the negative sample candidate pool, and the number of values of N" is K i+1

[0147] Optionally, the attribute of the commodity comprises a three-level category of the commodity.

[0148] Correspondingly, the present application also provides an electronic device comprising a processor and a memory storing computer program instructions; the processor implements the steps of the training sample generation method of any one of the above embodiments when executing the computer program instructions.

[0149] Figure 5 A hardware structure schematic diagram of one embodiment of the electronic device provided by the present application is shown, which can comprise a processor 601 and a memory 602 storing computer program instructions.

[0150] Specifically, the processor 601 can comprise a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0151] ​​The memory 602 can include mass storage for data or instructions. As an example and not by way of limitation, the memory 602 can include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc (e.g., a compact disc (CD) or a digital versatile disc (DVD)), a solid-state drive (SSD), a USB drive, or a combination of two or more of these. Where appropriate, the memory 602 can include removable or non-removable (or fixed) media. Where appropriate, the memory 602 can be internal or external to the integrated gateway disaster recovery appliance. In particular embodiments, the memory 602 is non-volatile, solid-state memory.

[0152] The memory can include read-only memory (ROM), random-access memory (RAM), magnetic disk storage mediums, optical storage mediums, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software that, when executed (by one or more processors), is operable to access the data and / or execute the instructions as described with respect to the methods according to the present disclosure.

[0153] The processor 601 implements any one of the training sample generation methods in the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0154] In one example, the electronic device can further include a communication interface 603 and a bus 610. As shown, the processor 601, the memory 602, and the communication interface 603 are connected through the bus 610 and complete communication among each other. The electronic device in the embodiments of the present application can be a server or other computing device, or a cloud server. Figure 5

[0155] The communication interface 603 is mainly used to realize the communication among the modules, devices, units and / or equipment in the embodiments of the present application.

[0156] ​Bus 610 includes hardware, software, or both, to couple components of the online data traffic metering device to each other and to couple components to other parallel components for communication and transferring of data. While bus 610 is shown for the sake of clarity as a single bus, it can include one or more buses operating together. Bus 610 can be implemented using any suitable type of bus structure including, for example, a data bus with multiple bit lines, an address bus, a control bus, a memory bus, a peripheral bus, a local bus, a global bus, a Fiber Channel bus, and so on, using any suitable bus architecture. Although a specific bus is described and shown, this disclosure can apply to any bus or interconnect, as appropriate.

[0157] In addition, in combination with the training sample generation method in the above-mentioned embodiments, the embodiments of the present application can provide a computer storage medium to implement. The computer storage medium has computer program instructions stored thereon; the computer program instructions are executed by a processor to implement any one of the training sample generation methods in the above-mentioned embodiments.

[0158] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the above-mentioned embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0159] The functional blocks shown in the structural block diagrams described above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are program or code segments used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave in a transmission medium or communication link. The "machine-readable medium" can include any medium capable of storing or transmitting information. The machine-readable medium can include a non-transitory computer-readable storage medium, such as an electronic circuit, a semiconductor memory device, a ROM, a flash memory, an erasable ROM (EROM), a floppy disk, a CD-ROM, an optical disk, a hard disk, an optical fiber medium, and a radio frequency (RF) link, and so on. The code segments can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0160] It is also important to note that the examples mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the examples, or in an order different from the examples, or several steps can be performed simultaneously.

[0161] The above generally describes the aspects of the present application with reference to a flowchart and / or a block diagram of the method, device (system) and computer program product according to embodiments of the present application. It should be understood that each block of the flowchart and / or block diagram and a combination of blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus enable the implementation of the functions / acts specified in one or more blocks of the flowchart and / or block diagram. The processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each block of the block diagram and / or flowchart and a combination of blocks in the block diagram and / or flowchart can also be implemented by special hardware to perform the specified functions or acts, or can be implemented by a combination of special hardware and computer instructions.

[0162] The above is only a specific implementation of the present application, and those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, module and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here. It should be understood that the protection scope of the present application is not limited to this, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A method for training sample generation, characterized in that, The method comprises the following steps: generating a plurality of positive sample product lists based on user behavior data of the system within a predetermined time period according to a plurality of search sessions under at least a first query word; determining one or more attributes of a plurality of products under the first query word; determining the proportion of the number of products under each attribute in the plurality of attributes in the total number of products under the first query word in the case that the plurality of attributes are included under the first query word; selecting a plurality of products in the negative sample candidate pool according to the plurality of attributes and the proportion corresponding to the plurality of attributes to construct a plurality of negative sample product lists; selecting a plurality of products in the negative sample candidate pool according to the plurality of attributes and the proportion corresponding to the plurality of attributes, comprising: determining that the first query word includes an i-th category attribute and an i+1-th category attribute, wherein a proportion of a number of commodities under the i+1-th category attribute corresponding to the i-th category attribute is K i :K i+1 ; constructing a plurality of temporary lists, each temporary list including at least one i-th attribute product and at least one i+1-th attribute product; adding a first type of identification to the plurality of temporary lists, adding a second type of identification to the plurality of products in the negative sample candidate pool having the i-th attribute, and adding a third type of identification to the plurality of products in the negative sample candidate pool having the i+1-th attribute; For a jth temporary list in the plurality of temporary lists, K i negative sample commodities are determined based on the first type identifier corresponding to the jth temporary list and the second type identifier carried by the commodities in the negative sample candidate pool; and K i+1 negative sample commodities are determined based on the first type identifier corresponding to the jth temporary list and the third type identifier carried by the commodities in the negative sample candidate pool. Based on the K i negative sample commodities and the K i+1 negative sample commodities, a first negative sample commodity list is generated so that there is no repeated commodity information in the first negative sample commodity list.

2. The method of claim 1, wherein: the plurality of positive sample product lists and the plurality of negative sample product lists are used to train a coarse ranking model in a product search system using a document list Listwise method in a ranking learning LTR technology.

3. The method of claim 1, wherein, The predetermined time period involves at least two natural days.

4. The method of claim 1, wherein: the user behavior data includes attribute information of products clicked by the user and attribute information of products paid by the user.

5. The method of claim 4, wherein: the attribute information includes at least one of the following: type of product, category of product, model of product, and model of product.

6. The method of claim 1, wherein: the positive sample product list includes information of products clicked and paid by the user in a single search session.

7. The method of claim 1, wherein, The number of the plurality of temporary lists is greater than or equal to the number of the plurality of positive sample product lists.

8. The method of claim 1, wherein, The first type of identification, the second type of identification, and the third type of identification are all digital numbers, wherein the first type of identification, the second type of identification, and the third type of identification are independently numbered.

9. The method of claim 8, wherein: Based on the first type of identifier corresponding to the j-th temporary list and the second type of identifier carried by the goods in the negative sample candidate pool, K is determined. i One negative sample item, including: the negative sample product number corresponding to each i-th attribute product in the j-th temporary list is determined using the following formula: M = ceil(N' / K i ), Wherein ceil() is a rounding-up operation, M is the number corresponding to the jth temporary list, N' is the number of the i-th attribute commodity of the negative sample candidate pool, and the number of values of N' is K i ; and Based on the first type of identifier corresponding to the j-th temporary list and the third type of identifier carried by the goods in the negative sample candidate pool, K is determined. i+1 One negative sample item, including: the negative sample product number corresponding to each i+1-th attribute product in the j-th temporary list is determined using the following formula: M = ceil(N" / K i ), Wherein, N" is the digital number of the (i+1)th attribute commodity in the negative sample candidate pool, the number of values of N" is K i+1 .

10. The method according to any one of claims 1-9, characterized in that, The attributes of the product include the three-level category of the product.

11. A training sample generation apparatus, characterized by comprising: The method comprises the following steps: a positive sample processing module for generating a plurality of positive sample product lists based on user behavior data of the system within a predetermined time period according to a plurality of search sessions under at least a first query word; an attribute processing module for determining one or more attributes of a plurality of products under the first query word; a negative sample processing module for selecting a plurality of products in the negative sample candidate pool according to the plurality of attributes and the proportion corresponding to the plurality of attributes to construct a plurality of negative sample product lists. The proportional processing module is configured to determine, in a case where the first query word comprises a plurality of attributes, a proportion of a quantity of commodities in each attribute in the plurality of attributes in a total quantity of commodities in the first query word. The negative sample processing module is configured to select a plurality of commodities from a negative sample candidate pool according to the plurality of attributes and the proportions corresponding to the plurality of attributes, to construct a plurality of negative sample commodity lists. The negative sample processing module comprises: The attribute determining sub-module is configured to determine that the first query word comprises an i-th attribute and an i+1-th attribute, and the proportion of the number of commodities corresponding to the i-th attribute and the i+1-th attribute is K i :K i+1 ; A temporary list construction submodule is configured to construct a plurality of temporary lists, each temporary list comprising at least one commodity of an ith attribute and at least one commodity of an (i+1)th attribute. An identification processing submodule is configured to add a first type of identification to the plurality of temporary lists, a second type of identification to a plurality of commodities in the negative sample candidate pool that have the ith attribute, and a third type of identification to a plurality of commodities in the negative sample candidate pool that have the (i+1)th attribute. The negative sample commodity determination sub-module is configured to, for the jth temporary list in the plurality of temporary lists, determine K i negative sample commodities based on the first type identifier corresponding to the jth temporary list and the second type identifier carried by the commodities in the negative sample candidate pool; and determine K i+1 negative sample commodities based on the first type identifier corresponding to the jth temporary list and the third type identifier carried by the commodities in the negative sample candidate pool. The negative sample commodity list generation submodule is configured to generate a first negative sample commodity list based on the K i negative sample commodities and the K i+1 negative sample commodities, so that there is no repeated commodity information in the first negative sample commodity list.

12. An electronic device, comprising: The electronic device comprises a processor and a memory storing computer program instructions; and the electronic device implements the method of any one of claims 1-10 when executing the computer program instructions.

13. A computer-readable storage medium, characterized in that, The computer storage medium stores computer program instructions, and the computer program instructions are executed by a processor to perform the method of any one of claims 1-10.

14. A computer program product, characterised in that, The computer program instructions are executed by a processor to perform the method of any one of claims 1-10.

Images