Method and device for adjusting article display position

Abstract

The invention relates to a method and device for adjusting article display positions and a computer readable storage medium, and relates to the technical field of computers. The method for adjusting the article display positions comprises the steps of obtaining an initial article sequence generated in response to a search operation of a user at the current moment, wherein the initial article sequence comprises a plurality of articles arranged at a plurality of display positions; determining target feature information used for adjusting display positions of the plurality of articles in the initial article sequence, wherein the target feature information comprises user feature information of the user at the current moment and attribute feature information of a search text used by the search operation at the current moment; and according to the target feature information, adjusting the display positions of the plurality of articles in the initial article sequence. According to the method and the device, accurate flow regulation and control can be realized, and the article display accuracy aiming at different users and / or different search operations is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method and device for adjusting a display position of an item, and a computer storable medium. Background Art

[0002] In the e-commerce search scenario, in the relevant traffic distribution technology, in response to the user's search operation or search request, multiple items arranged in sequence in multiple display positions are displayed to the user. In the process of traffic distribution, traffic control is performed by adjusting the arrangement order of multiple items displayed to the user in multiple display positions. How to achieve accurate traffic control is urgently needed. Summary of the invention

[0003] The present disclosure proposes a solution that can achieve precise traffic control and improve the accuracy of item display for different users and / or different search operations.

[0004] According to a first aspect of the present disclosure, a method for adjusting a display position of an item is provided, comprising: obtaining an initial item sequence generated in response to a search operation of a user at a current moment, the initial item sequence comprising a plurality of items arranged in a plurality of display positions; determining target feature information for adjusting the display positions of the plurality of items in the initial item sequence, the target feature information comprising user feature information of the user at the current moment and attribute feature information of a search text used in the search operation at the current moment; and adjusting the display positions of the plurality of items in the initial item sequence according to the target feature information.

[0005] In some embodiments, adjusting the display positions of the multiple items in the initial item sequence according to the target feature information includes: determining the score of each item in each display position using a reinforcement learning model according to the target feature information; and adjusting the display positions of the multiple items in the initial item sequence by comparing the scores of different items in each display position.

[0006] In some embodiments, determining the score of each item in each display position based on the target feature information using a reinforcement learning model includes: determining the score of each item in each display position under each business goal using multiple reinforcement learning models that control different business goals based on the target feature information, where each reinforcement learning model controls one business goal; and fusing multiple scores of each item in each display position under multiple business goals to obtain a score for each item in each display position.

[0007] In some embodiments, fusing multiple scores of each item at each display location under multiple business objectives to obtain a score of each item at each display location includes: acquiring multiple weights corresponding to the multiple business objectives; and using the multiple weights corresponding to the multiple business objectives to fuse the multiple scores of each item at each display location under the multiple business objectives to obtain a score of each item at each display location.

[0008] In some embodiments, obtaining multiple weights corresponding to the multiple business objectives includes: using a cross entropy method CEM to maximize an indicator related to the search operation, and determining the multiple weights corresponding to the multiple business objectives.

[0009] In some embodiments, the target feature information also includes the degree of completion of the adjustment target for each item. Based on the target feature information, multiple reinforcement learning models that control different business targets are used to determine the score of each item in each display position under each business target, including: based on the user feature information of the user at the current moment, the attribute feature information of the search text used for the search operation at the current moment, and the degree of completion of the adjustment target for each item, the reinforcement learning model that controls each business target is used to determine the score of each item in each display position under each business target.

[0010] In some embodiments, when the user behavior corresponding to the business goal controlled by each reinforcement learning model occurs, the value of the reward function of each reinforcement learning is a first value, and when the user behavior corresponding to the business goal controlled by each reinforcement learning model does not occur, the value of the reward function of each reinforcement learning is a second value. Determining the score of each item in each display position under each business goal based on the target feature information and using multiple reinforcement learning models that control different business goals includes: determining the value of the reward function of each item in each display position under each business goal based on the target feature information and using the reinforcement learning model that controls each business goal; determining the score of each item in each display position under each business goal based on the value of the reward function of each item in each display position under each business goal.

[0011] In some embodiments, the target feature information also includes item feature information and context feature information of each item at each display location. Determining the score of each item at each display location using a reinforcement learning model based on the target feature information includes: for each item, determining an output value of a value function network of the reinforcement learning model as a first output value based on the user feature information, the attribute feature information, and the context feature information corresponding to each item; for each item, determining an output value of an advantage function network of the reinforcement learning model as a second output value based on the user feature information, the attribute feature information, the context feature information corresponding to each item, and the item feature information of each item at each display location; determining the score of each item at each display location based on the first output value and the second output value corresponding to each item.

[0012] In some embodiments, the target feature information also includes bilateral feature information between the user and the multiple items, and the bilateral feature information serves as input to the value function network and the advantage function network of the reinforcement learning model.

[0013] In some embodiments, the bilateral feature information includes historical behavior feature information of the user with respect to the plurality of items.

[0014] In some embodiments, the plurality of items include a plurality of first items and a plurality of second items, and adjusting the display positions of the plurality of items in the initial item sequence by comparing the scores of different items at each display position includes: for each first item, determining at least one second item arranged before each first item in the initial item sequence as at least one reference item; determining the first reference item in the initial item sequence having a score lower than the score corresponding to each first item by comparing the score of each first item at the display position of each reference item in the initial item sequence with the score of each reference item at the display position of the initial item sequence, as the target item corresponding to each first item; adjusting the display position of each first item in the initial item sequence so that the display position of each first item is located before the display position of the target item corresponding to each first item; and repeatedly performing the operations of determining at least one reference item, determining the target item, and adjusting the display position of each first item until the display position of any first item is located before the display position of the target item corresponding to the first item.

[0015] According to a second aspect of the present disclosure, there is provided a device for adjusting a display position of an item, comprising: an acquisition module, configured to acquire an initial item sequence generated in response to a search operation of a user at a current moment, the initial item sequence comprising a plurality of items arranged in a plurality of display positions; a determination module, configured to determine target feature information for adjusting the display positions of the plurality of items in the initial item sequence, the target feature information comprising user feature information of the user at the current moment and attribute feature information of a search text used in the search operation at the current moment; and an adjustment module, configured to adjust the display positions of the plurality of items in the initial item sequence according to the target feature information.

[0016] According to a third aspect of the present disclosure, there is provided a device for adjusting a display position of an item, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the method for adjusting a display position of an item as described in any of the above embodiments based on instructions stored in the memory.

[0017] According to a sixth aspect of the present disclosure, a computer storable medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, the method for adjusting the display position of an item as described in any of the above embodiments is implemented.

[0018] In the above embodiments, accurate traffic control can be achieved, thereby improving the accuracy of item display for different users and / or different search operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0020] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0021] Figure 1 is a flow chart showing a method for adjusting an item display position according to some embodiments of the present disclosure;

[0022] Figure 2 is a schematic diagram showing how to determine multiple weights corresponding to multiple business objectives using CEM according to some embodiments of the present disclosure;

[0023] Figure 3 is a schematic diagram showing a flow chart of determining a score of each item in each display position according to some embodiments of the present disclosure;

[0024] Figure 4 is a block diagram showing an apparatus for adjusting an item display position according to some embodiments of the present disclosure;

[0025] Figure 5is a block diagram showing an apparatus for adjusting an item display position according to some other embodiments of the present disclosure;

[0026] Figure 6 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0027] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0028] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0029] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0030] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0031] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0032] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0033] Figure 1 is a flowchart illustrating a method for adjusting an item display position according to some embodiments of the present disclosure.

[0034] like Figure 1 As shown, the method for adjusting the display position of an item includes: step S110: obtaining an initial item sequence generated in response to a user's search operation at the current moment, the initial item sequence including multiple items arranged in multiple display positions; step S120, determining target feature information for adjusting the display positions of multiple items in the initial item sequence, the target feature information including user feature information of the user at the current moment and attribute feature information of the search text used in the search operation at the current moment; and step S130, adjusting the display positions of multiple items in the initial item sequence according to the target feature information. In some embodiments, the method for adjusting the display position of an item is performed by a device for adjusting the display position of an item.

[0035] In the above embodiment, in the process of adjusting the display position of items to achieve traffic regulation, not only personalized user feature information for different users is introduced, but also personalized attribute feature information of search text for different search operations is introduced, so that personalized adjustments when adjusting the display positions of multiple items are made, and a finer request granularity is used for traffic regulation, thereby achieving personalized traffic regulation or traffic distribution, which can improve the accuracy of item display for different users and / or different search operations, and make traffic distribution more efficient.

[0036] In step S110, an initial item sequence generated in response to a user's search operation at the current moment is obtained, and the initial item sequence includes multiple items arranged in multiple display positions. For example, in the initial item sequence, multiple items are identified by SKUs of multiple items. In some embodiments, the multiple items include multiple first items and multiple second items. In some embodiments, the first item is a candidate item, and the second item is a non-candidate item. For example, the first item is a new item to be regulated, and the second item is an item other than the new item to be regulated.

[0037] In step S120, target feature information for adjusting display positions of multiple items in the initial item sequence is determined, where the target feature information includes user feature information of the user at the current moment and attribute feature information of the search text used in the search operation at the current moment.

[0038] For example, user characteristic information includes user portrait characteristic information such as gender, age group, etc. For another example, attribute characteristic information of search text includes at least one of search intent information (intent rating), semantic breadth and fuzziness of search text, whether the brand of search text matches the brand of item, whether search text contains brand, and frequency of search text being searched.

[0039] In step S130, the display positions of the multiple items in the initial item sequence are adjusted according to the target feature information. For example, the initial item sequence after the display positions of the multiple items are adjusted is called the target item sequence. In some embodiments, the multiple items can also be displayed to the user according to the target item sequence in response to the user's search operation at the current moment.

[0040] In some embodiments, adjusting the display positions of the plurality of items in the initial item sequence according to the target feature information may be achieved in the following manner.

[0041] First, based on the target feature information, a reinforcement learning model is used to determine the score of each item in each display position. The display positions of the item arrangement are the action space of the reinforcement learning model. For example, the reinforcement learning model is a DQN (Deep Q-Network) model.

[0042] In some embodiments, the state of the reinforcement learning model includes user feature information of the user at the current moment in the target feature information and attribute feature information of the search text used in the search operation at the current moment.

[0043] In some embodiments, the action space of the reinforcement learning model is 100 discrete values ​​from 0 to 99, corresponding to the first to the 100th display positions. In some embodiments, the reinforcement learning model can select the best display position for each item, that is, select the best action, represented by a * = argmax a g * (s t+1 ,a). a represents action, s t+1 represents the state of the reinforcement learning model at time t+1, g * () represents the output of the reinforcement learning model in state s t+1 The score of taking action a, a * To maximize the score.

[0044] Then, by comparing the scores of different items at each display position, the display positions of the multiple items in the initial item sequence are adjusted.

[0045] In some embodiments, after the display positions of multiple items in the initial item sequence are adjusted, some user operations may cause the target feature information to change, so that during the next search, the display positions are adjusted again using the changed target feature information.

[0046] In some embodiments, the plurality of items include a plurality of first items and a plurality of second items, and by comparing the scores of different items at each display position, adjusting the display positions of the plurality of items in the initial item sequence includes the following steps. In some embodiments, the first item is a candidate item, and the second item is a non-candidate item. For example, the first item is a new item to be regulated, and the second item is an item other than the new item to be regulated.

[0047] First, for each first object, at least one second object arranged before each first object in the initial object sequence is determined as at least one reference object.

[0048] Secondly, by comparing the score of each first object at the display position of each reference object in the initial object sequence with the score of each reference object at the display position in the initial object sequence, the first reference object in the initial object sequence whose score is lower than the score corresponding to each first object is determined as the target object corresponding to each first object.

[0049] Then, in the initial item sequence, the display position of each first item is adjusted so that the display position of each first item is located before the display position of the target item corresponding to each first item.

[0050] Finally, the operations of determining at least one reference object, determining the target object, and adjusting the display position of each first object are repeatedly performed until the display position of any first object is located before the display position of the target object corresponding to the first object. For example, each first object is moved forward in a decreasing direction of the display position until the display position of the first second object with a lower score than that of each first object is found, and each first object is inserted into the display position.

[0051] In the above embodiment, a reinforcement learning model is used to score each item in each display position, and then the display positions of multiple items in the initial item sequence are adjusted by comparing the scores of different items in each display position. In this way, the reinforcement learning model can be used to learn target feature information, so that the advantages and disadvantages of each item in each display position can be evaluated, and the display position can be accurately adjusted, and personalized traffic control can be further achieved, and the accuracy of item display for different users and / or different search operations can be further improved.

[0052] In some embodiments, the score of each item in each display position may be determined based on the target feature information using a reinforcement learning model in the following manner.

[0053] First, based on the target feature information, multiple reinforcement learning models that control different business objectives are used to determine the score of each item in each display position under each business objective, and each reinforcement learning model controls a business objective. For example, the multiple business objectives controlled by multiple reinforcement learning models include click objectives, add-to-cart objectives, and order objectives. For another example, the multiple business objectives may also include collection objectives and sharing objectives.

[0054] In some embodiments, the target feature information also includes the degree of completion of the adjustment target for each item. The score of each item in each display position under each business target can be determined by using the reinforcement learning model that controls each business target based on the user feature information of the user at the current moment, the attribute feature information of the search text used for the search operation at the current moment, and the degree of completion of the adjustment target for each item. For example, the degree of completion of the adjustment target for each item in the target feature information serves as the state of the reinforcement learning model that controls each business target. The degree of completion of the adjustment target for each item belongs to a type of total feedback feature of the regulation or adjustment target. For example, the adjustment target includes increasing the exposure of the item by a specified amount. For example, the adjustment target of item 1 is to increase the exposure of item 1 by 100. If 80 exposures have been increased, the degree of completion of the adjustment target of item 1 is 80%.

[0055] By using the user characteristic information of the user at the current moment, the attribute characteristic information of the search text used in the search operation at the current moment, and the completion degree of the adjustment target for each item as the state information of the reinforcement learning model, the accuracy of the reinforcement learning model in learning the score of each item in each display position under each business goal can be further improved, thereby further improving the accuracy of the display position adjustment.

[0056] In some embodiments, when the user behavior corresponding to the business goal controlled by each reinforcement learning model occurs, the value of the reward function of each reinforcement learning is a first value, and when the user behavior corresponding to the business goal controlled by each reinforcement learning model does not occur, the value of the reward function of each reinforcement learning is a second value. In this case, according to the target feature information, using multiple reinforcement learning models that control different business goals, determining the score of each item in each display position under each business goal includes: according to the target feature information, using the reinforcement learning model that controls each business goal, determining the value of the reward function of each item in each display position under each business goal; according to the value of the reward function of each item in each display position under each business goal, determining the score of each item in each display position under each business goal.

[0057] For example, the first value is 1 and the second value is -1. Taking multiple business goals including click goals, add-to-cart goals and order goals as an example, the reward function or reward and punishment function can be expressed as That is, when there is a click behavior, a purchase-add-to-cart behavior, or an order behavior, the value of the reward function is 1. When there is no such behavior (i.e., only exposure), the value of the reward function is -1.

[0058] According to the value of the reward function of each item in each display position under each business objective, the score of each item in each display position under each business objective can be expressed as g(s t ,a)=reward t+1 +λ×max a g(s t+1 ,a) or the update process can be expressed as g(s t ,a t )←g(s t ,a t )+α(reward t+1 +λ×max a g(s t+1 ,a)-g(s t ,a t )).s t is the state at time t, a represents the action, g(s t ,a) means in s t The long-term benefit that can be obtained by taking action a, reward t+1 Indicates in s t The immediate reward that can be obtained by taking action a, λ is the discount factor, max a g(s t+1 ,a) represents the maximum long-term benefit that can be obtained at time t+1, and α is the update parameter. The process can refer to the Qlearning network.

[0059] Then, multiple scores of each item in each display position under various business objectives are integrated to obtain a score of each item in each display position.

[0060] In the above embodiment, by controlling multiple reinforcement learning models of different business objectives, determining the score of each item in each display position under each business objective, and then integrating the scores of each item under multiple business objectives, multiple business objectives can be balanced. While achieving personalized traffic regulation, the adjusted display positions can adapt to different business objectives, thereby improving the versatility and universality of the adjustment, and further improving the accuracy of item display in different business objective scenarios for different users and / or different search operations.

[0061] In some embodiments, the multiple scores of each item at each display position under various business objectives may be integrated to obtain the score of each item at each display position in the following manner: For example, the multiple scores of each item at each display position under various business objectives may be integrated by a decision fusion module of the device for adjusting the display position of the item.

[0062] First, multiple weights corresponding to the multiple business objectives are obtained. In some embodiments, CEM (Cross-Entropy Method) can be used to maximize the index related to the search operation to determine the multiple weights corresponding to the multiple business objectives. In some embodiments, the index related to the search operation includes at least one of UCVR and UV (Unique Visitor) value. UCVR = total order lines / search UV, UV value = GMV / search UV.

[0063] Figure 2 is a schematic diagram showing how to determine multiple weights corresponding to multiple business objectives using CEM according to some embodiments of the present disclosure.

[0064] like Figure 2 As shown in the figure, in the offline state, with the goal of maximizing the indicators related to the search operation such as UCVR and UV value, CEM is used to perform multi-objective parameter search, and the weight vector w composed of multiple weights is continuously updated to obtain the updated weight vector w′, and this cycle is repeated until convergence to obtain multiple weights w1, w2 and w3 corresponding to multiple business objectives. These multiple weights can also be called weighting factors.

[0065] The cross entropy method uses a linear deterministic strategy to solve the problem of multi-objective parameter search. It believes that π M (s) = M φ (s), where M represents the Markov process, π represents the strategy, and S represents the state. represents the decision space. The goal of this process is to maximize Among them, S(M) represents the benefits obtained in this Markov decision process, represents the mean, r t Represents the immediate reward at time t. The problem to be solved is regarded as a black box, expressed as M→S(M), that is, the Markov process is regarded as a Markov reward process.

[0066] The core idea of ​​CEM is to maintain a distribution of possible optimal solutions, and then update the distribution based on sampling and querying the sampling values ​​of the black box. Consider w = vec (M) as a vector, and sample n samples in each round. The sampling process is expressed as i∈[n], which means that n samples follow a normal distribution with a mean of μ t , the variance is Then rollout is performed to obtain the corresponding values ​​S(w1), S(w2),…, S(w n ), the largest value , get the collection of its index Then the distribution is updated to find the optimal parameters. The updating process includes updating the mean at time t+1 and updating the variance, which are expressed as The process of updating the variance adds noise Z t+1 This is to prevent the variance from converging too quickly.

[0067] Then, using multiple weights corresponding to the multiple business goals, multiple scores of each item in each display position under the multiple business goals are merged to obtain the score of each item in each display position. In some embodiments, taking the multiple business goals including click goals, add-to-cart goals and order goals as an example, the multiple scores of an item in a certain display position under the click goal, add-to-cart goal and order goal are respectively represented as g click (s t ,a),g cart (s t ,a) and g order (s t ,a), can also be called target revenue. click means click target, cart means add-to-cart target, and order means order target. t represents the state of the reinforcement learning model at time t. a represents the action of an item in a certain display position.

[0068] In some embodiments, multiple weights corresponding to multiple business objectives are used to perform weighted operations on multiple scores of each item in each display position under the multiple business objectives to obtain the score of each item in each display position. For example, the score of an item in a certain display position can be expressed as gain(s t ,a)=w1×g click (s t ,a)+w2×g cart (s t ,a)+w3×g order (s t ,a).

[0069] In some embodiments, the target feature information also includes the item feature information and context feature information of each item at each display location, which can be achieved by using a reinforcement learning model based on the target feature information to determine the score of each item at each display location.

[0070] First, for each item, the output value of the value function network of the reinforcement learning model is determined as the first output value according to the user feature information, the attribute feature information and the context feature information corresponding to each item. In some embodiments, the context feature information corresponding to each item is the context information between other items in the initial item sequence except for each item and each item, including relevant features of other items. For example, the context feature information corresponding to each item is also used as the state of the reinforcement learning model. The value function network learns the state information.

[0071] Then, for each item, the output value of the advantage function network of the reinforcement learning model is determined as the second output value according to the user feature information, the attribute feature information, the context feature information corresponding to each item, and the item feature information of each item at each display position. For example, the item feature information of each item at each display position is used as an action, so that the advantage function network learns state and action information.

[0072] Finally, the score of each item at each display position is determined according to the first output value and the second output value corresponding to each item.

[0073] In some embodiments, the process of determining the score of each item in each display position using a reinforcement learning model including a value function network and an advantage function network may be a processing process of a reinforcement learning model that controls each business objective. In this case, based on the first output value and the second output value corresponding to each item, the score of each item in each display position under each business objective is first determined, and then the scores under multiple business objectives are fused to obtain the score of each item in each display position. The specific fusion process can be referred to the aforementioned embodiment and will not be repeated here.

[0074] In some embodiments, the target feature information also includes bilateral feature information between the user and the multiple items, and the bilateral feature information is used as input to the value function network and the advantage function network of the reinforcement learning model. The bilateral feature information includes the historical behavior feature information of the user for the multiple items. For example, the bilateral feature information is used as the state of the reinforcement learning model. The historical behavior feature information may include the behavior information of the items, item categories, stores, etc. that the user has clicked, added to a purchase, or ordered. The historical behavior feature information may characterize the user's sequence preference.

[0075] Figure 3 is a schematic diagram showing a flow chart of determining a score of each item in each display position according to some embodiments of the present disclosure.

[0076] like Figure 3As shown, taking the display positions of the candidate items for regulation including positions 0 to L as an example, combined with the reinforcement learning model including the value function network and the advantage function network, the process of determining the score of each display position of the candidate items for regulation in 0 to L is explained.

[0077] refer to Figure 3 , query represents the attribute feature information of the search text, user represents the user feature information, Represents the historical behavior characteristics of users on items other than the candidate items for regulation. Represents the historical behavior characteristics of users on the candidate items for regulation, search t Represents the contextual feature information of the regulation candidate items, which together serve as the state of the reinforcement learning model. It represents the item features of the candidate items at each display position in 0~L, and represents the actions of the reinforcement learning model. query and user are fixed attributes. and Together they constitute the user's historical behavior.

[0078] Figure 3 The left subnetwork is the value function network, input query, user, and search t , and get the output V(s) of the value function network t ). Figure 3 The right subnetwork is the advantage function network, input query, user, search t and Get the output of the advantage function network According to the output V(s t ) and the output of the advantage function network The scores of the candidate items at each display position from 0 to L can be determined and expressed as Q corresponds to the previous g.

[0079] After testing, the method of adjusting the display position of items disclosed in the present invention has different arrangement orders of the display positions of multiple items displayed to different users when different users use the same or different search texts to search, which reflects that the method of adjusting the display position of items disclosed in the present invention can achieve personalized traffic regulation. For example, the candidate items include red dress, black dress and white dress. When three different users search for "dress", the order displayed to user 1 is red dress, black dress and white dress, the order displayed to user 2 is white dress, black dress and red dress, and the order displayed to user 3 is red dress, white dress and black dress.

[0080] It can be seen that for different users searching for "dress" or the same user searching for "dress" at different times, the ranking results of candidate items displayed to the user are different. For example, users who like white dresses (learned based on the user's historical behavior) will put white dresses in the front, so as to improve the user's click rate. Compared with the related art that displays the same order of item sequences to different users, the present disclosure can achieve personalized traffic control and improve the accuracy of item display for different users and / or different search operations.

[0081] Figure 4 is a block diagram showing an apparatus for adjusting an item display position according to some embodiments of the present disclosure.

[0082] like Figure 4 As shown, the device 4 for adjusting the display position of an item includes an acquisition module 41 , a determination module 42 and an adjustment module 43 .

[0083] The acquisition module 41 is configured to acquire an initial item sequence generated in response to a user's search operation at the current moment, wherein the initial item sequence includes a plurality of items arranged in a plurality of display positions, for example, by performing the following steps: Figure 1 Step S110 is shown.

[0084] The determination module 42 is configured to determine target feature information for adjusting the display positions of the plurality of items in the initial item sequence, the target feature information including user feature information of the user at the current moment and attribute feature information of the search text used in the search operation at the current moment, for example, Figure 1 Step S120 is shown.

[0085] The adjustment module 43 is configured to adjust the display positions of the plurality of items in the initial item sequence according to the target feature information, for example, by performing the following steps: Figure 1 Step S130 is shown.

[0086] In some embodiments, the adjustment module 43 is further configured to determine the score of each item in each display position based on the target feature information using a reinforcement learning model; and adjust the display positions of the multiple items in the initial item sequence by comparing the scores of different items in each display position.

[0087] In some embodiments, the adjustment module 43 is further configured to determine the score of each item in each display position under each business objective based on the target feature information and using multiple reinforcement learning models that control different business objectives, where each reinforcement learning model controls one business objective; and to fuse the multiple scores of each item in each display position under multiple business objectives to obtain the score of each item in each display position.

[0088] In some embodiments, the adjustment module 43 is further configured to obtain multiple weights corresponding to the multiple business objectives; using the multiple weights corresponding to the multiple business objectives, the multiple scores of each item in each display position under the multiple business objectives are integrated to obtain the score of each item in each display position.

[0089] In some embodiments, the adjustment module 43 is further configured to determine a plurality of weights corresponding to the plurality of business objectives by using a cross entropy method CEM with the goal of maximizing an indicator related to the search operation.

[0090] In some embodiments, the target feature information also includes the completion degree of the adjustment target for each item, and the adjustment module 43 is further configured to determine the score of each item in each display position under each business goal based on the user feature information of the user at the current moment, the attribute feature information of the search text used for the search operation at the current moment, and the completion degree of the adjustment target for each item, using the reinforcement learning model that controls each business goal.

[0091] In some embodiments, when the user behavior corresponding to the business goal controlled by each reinforcement learning model occurs, the value of the reward function of each reinforcement learning is a first value, and when the user behavior corresponding to the business goal controlled by each reinforcement learning model does not occur, the value of the reward function of each reinforcement learning is a second value. The adjustment module 43 is also configured to determine the value of the reward function of each item in each display position under each business goal according to the target feature information and using the reinforcement learning model that controls each business goal; and determine the score of each item in each display position under each business goal according to the value of the reward function of each item in each display position under each business goal.

[0092] In some embodiments, the target feature information also includes item feature information and context feature information of each item at each display location, and the adjustment module 43 is also configured to determine, for each item, an output value of a value function network of a reinforcement learning model as a first output value based on the user feature information, the attribute feature information, and the context feature information corresponding to each item; for each item, determine an output value of an advantage function network of a reinforcement learning model as a second output value based on the user feature information, the attribute feature information, the context feature information corresponding to each item, and the item feature information of each item at each display location; and determine a score of each item at each display location based on the first output value and the second output value corresponding to each item.

[0093] In some embodiments, the target feature information further includes bilateral feature information between the user and the multiple items, and the bilateral feature information is used as input to the value function network and the advantage function network of the reinforcement learning model. For example, the bilateral feature information includes historical behavior feature information of the user with respect to the multiple items.

[0094] In some embodiments, the multiple items include multiple first items and multiple second items, and the adjustment module 43 is further configured to determine, for each first item, at least one second item that is arranged before each first item in the initial item sequence as at least one reference item; determine, by comparing the score of each first item at the display position of each reference item in the initial item sequence with the score of each reference item at the display position in the initial item sequence, the first reference item in the initial item sequence whose score is lower than the score corresponding to each first item as the target item corresponding to each first item; adjust the display position of each first item in the initial item sequence so that the display position of each first item is located before the display position of the target item corresponding to each first item; and repeat the operations of determining at least one reference item, determining the target item, and adjusting the display position of each first item until the display position of any first item is located before the display position of the target item corresponding to the any first item.

[0095] Figure 5 is a block diagram showing an apparatus for adjusting an item display position according to some other embodiments of the present disclosure.

[0096] like Figure 5As shown, the device 5 for adjusting the display position of an item includes a memory 51; and a processor 52 coupled to the memory 51. The memory 51 is used to store instructions for executing the method for adjusting the display position of an item corresponding to an embodiment. The processor 52 is configured to execute the method for adjusting the display position of an item in any of the embodiments of the present disclosure based on the instructions stored in the memory 51.

[0097] Figure 6 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0098] like Figure 6 As shown, the computer system 60 may be embodied in the form of a general purpose computing device. The computer system 60 includes a memory 610, a processor 620, and a bus 600 that connects the various system components.

[0099] The memory 610 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions for executing at least one of the corresponding embodiments of the method for adjusting the display position of an item. The non-volatile storage medium includes, but is not limited to, a disk memory, an optical memory, a flash memory, etc.

[0100] The processor 620 can be implemented by a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, etc. Discrete hardware components. Accordingly, each module such as the judgment module and the determination module can be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or can be implemented by a dedicated circuit that executes corresponding steps.

[0101] The bus 600 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0102] The computer system 60 may also include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650, the memory 610, and the processor 620 may be connected via a bus 600. The input / output interface 630 may provide a connection interface for input / output devices such as a display, a mouse, and a keyboard. The network interface 640 may provide a connection interface for various networked devices. The storage interface 650 may provide a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.

[0103] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.

[0104] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0105] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to work in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0106] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0107] Through the method and device for adjusting the display position of an item and the computer-storable medium in the above-mentioned embodiments, accurate flow control can be achieved, thereby improving the accuracy of item display for different users and / or different search operations.

[0108] So far, the method and device for adjusting the display position of an item according to the present disclosure, and the computer storable medium have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

Claims

1. A method for adjusting an item display position, comprising: Acquire an initial item sequence generated in response to a search operation of a user at a current moment, the initial item sequence comprising a plurality of items arranged in a plurality of display positions; Determine target feature information for adjusting display positions of the plurality of items in the initial item sequence, the target feature information including user feature information of the user at the current moment and attribute feature information of a search text used in a search operation at the current moment; According to the target feature information, display positions of the plurality of items in the initial item sequence are adjusted.

2. The method for adjusting the display position of an item according to claim 1, wherein: Adjusting the display positions of the plurality of items in the initial item sequence according to the target feature information includes: Determine the score of each item at each display position using a reinforcement learning model based on the target feature information; By comparing the scores of different items at each display position, the display positions of the multiple items in the initial item sequence are adjusted.

3. The method for adjusting the display position of an item according to claim 2, wherein: According to the target feature information, using the reinforcement learning model, determining the score of each item in each display position includes: Determine the score of each item in each display position under each business objective using multiple reinforcement learning models controlling different business objectives according to the target feature information, each reinforcement learning model controlling one business objective; The multiple scores of each item in each display position under various business objectives are integrated to obtain the score of each item in each display position.

4. The method for adjusting the display position of an item according to claim 3, wherein: The scores of each item in each placement are integrated under various business objectives to obtain the scores of each item in each placement, including: Acquire multiple weights corresponding to the multiple business objectives; By using multiple weights corresponding to the multiple business objectives, multiple scores of each item at each display position under the multiple business objectives are integrated to obtain the score of each item at each display position.

5. The method for adjusting the display position of an item according to claim 4, wherein: Acquiring a plurality of weights corresponding to the plurality of business objectives includes: A cross entropy method CEM is used to determine a plurality of weights corresponding to the plurality of business objectives with the goal of maximizing an indicator related to the search operation.

6. The method for adjusting the display position of an item according to claim 3, wherein: The target feature information also includes the completion degree of the adjustment target for each item. According to the target feature information, using multiple reinforcement learning models that control different business objectives, the score of each item in each display position under each business objective is determined, including: Based on the user characteristic information of the user at the current moment, the attribute characteristic information of the search text used in the search operation at the current moment, and the completion degree of the adjustment target for each item, the score of each item in each display position under each business goal is determined by using the reinforcement learning model that controls each business goal.

7. The method for adjusting the display position of an item according to claim 3, wherein: When the user behavior corresponding to the business goal controlled by each reinforcement learning model occurs, the value of the reward function of each reinforcement learning is a first value, and when the user behavior corresponding to the business goal controlled by each reinforcement learning model does not occur, the value of the reward function of each reinforcement learning is a second value. According to the target feature information, using multiple reinforcement learning models that control different business goals, determining the score of each item in each display position under each business goal includes: According to the target feature information, using a reinforcement learning model that controls each business objective, determining the value of a reward function for each item in each display position under each business objective; According to the value of the reward function of each item in each display position under each business objective, the score of each item in each display position under each business objective is determined.

8. The method for adjusting the display position of an item according to claim 2, wherein: The target feature information also includes the item feature information and context feature information of each item at each display position. According to the target feature information, using the reinforcement learning model, determining the score of each item at each display position includes: For each item, determining an output value of a value function network of a reinforcement learning model as a first output value according to the user feature information, the attribute feature information, and the context feature information corresponding to each item; For each of the items, determining an output value of an advantage function network of a reinforcement learning model as a second output value according to the user feature information, the attribute feature information, the context feature information corresponding to each of the items, and the item feature information of each of the items at each display position; The score of each item at each display position is determined according to the first output value and the second output value corresponding to each item.

9. The method for adjusting the display position of an item according to claim 8, wherein: The target feature information also includes bilateral feature information between the user and the multiple items, and the bilateral feature information serves as input to the value function network and the advantage function network of the reinforcement learning model.

10. The method for adjusting the display position of an item according to claim 9, wherein: The bilateral feature information includes historical behavior feature information of the user with respect to the multiple items.

11. The method for adjusting the display position of an item according to any one of claims 2 to 10, wherein: The plurality of items include a plurality of first items and a plurality of second items, and adjusting the display positions of the plurality of items in the initial item sequence by comparing the scores of different items at each display position includes: For each first object, determining at least one second object arranged before each first object in the initial object sequence as at least one reference object; By comparing the score of each first object at the display position of each reference object in the initial object sequence with the score of each reference object at the display position in the initial object sequence, determining the first reference object in the initial object sequence whose score is lower than the score corresponding to each first object as the target object corresponding to each first object; In the initial item sequence, adjusting the display position of each first item so that the display position of each first item is located before the display position of the target item corresponding to each first item; The operations of determining at least one reference item, determining a target item, and adjusting a display position of each first item are repeatedly performed until a display position of any first item is located before a display position of a target item corresponding to the first item.

12. A device for adjusting the display position of an item, comprising: An acquisition module is configured to acquire an initial item sequence generated in response to a search operation of a user at a current moment, wherein the initial item sequence includes a plurality of items arranged in a plurality of display positions; a determination module configured to determine target feature information for adjusting display positions of the plurality of items in the initial item sequence, the target feature information comprising user feature information of the user at the current moment and attribute feature information of a search text used in a search operation at the current moment; The adjustment module is configured to adjust the display positions of the multiple items in the initial item sequence according to the target feature information.

13. A device for adjusting the display position of an item, comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the method for adjusting the display position of an item according to any one of claims 1 to 11 based on instructions stored in the memory.

14. A computer storable medium having computer program instructions stored thereon, wherein when the instructions are executed by a processor, the method for adjusting the display position of an item according to any one of claims 1 to 11 is implemented.

Images