Method and device for recommending data

By introducing an absorption state in the Markov decision-making process, processing user browsing records, calculating push probability and strategies, the problem of low recommendation accuracy caused by user departure behavior is solved, and the effect of the recommendation system is improved.

CN113761325BActive Publication Date: 2024-11-01ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010497018.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-06-03
Publication Date
2024-11-01
Estimated Expiration
2040-06-03

AI Technical Summary

Technical Problem

In the prior art, the user's departure behavior leads to the cumulative income in the recommended product process being terminated, resulting in low accuracy of the recommended product.

Method used

By introducing an absorption state in the Markov decision-making process (MDP), the user's browsing records are sorted according to the preset time series, browsing records with a distance from the end point smaller than the preset value, and the push probability and strategy are calculated to model the user. browsing behavior and push the best content.

Benefits of technology

It improves the accuracy of recommended products, avoids the problem of cumulative income termination caused by user departure behavior, and improves the effectiveness of the recommendation system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113761325B_ABST
    Figure CN113761325B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and device for recommending data. The method comprises: sorting the browsing records of at least one user according to a preset time sequence to obtain at least one group of browsing records; obtaining browsing records whose distance from the end point of the preset time sequence is less than a preset value from the sorted at least one group of browsing records; obtaining corresponding push probabilities according to the browsing records; and obtaining corresponding push strategies according to the push probabilities. The present invention solves the technical problem that due to the user's leaving behavior in the prior art, the accumulated income obtained based on the browsing operation in the process of recommending products is terminated, resulting in a low accuracy rate of recommended products.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a method and device for recommending data. Background Art

[0002] The serialized recommendation scenario continuously recommends items to users so that they can continue to browse, with the goal of maximizing the cumulative benefits. However, in reality, users may leave at any time, resulting in the termination of the benefit accumulation process.

[0003] Currently, no effective solution has been proposed to the problem that due to the user's leaving behavior in the prior art, the accumulated revenue obtained based on the browsing operation in the process of recommending products is terminated, resulting in a low accuracy rate of recommended products. Summary of the invention

[0004] The embodiments of the present invention provide a method and device for recommending data, so as to at least solve the technical problem that due to the user's leaving behavior in the prior art, the accumulated revenue obtained based on the browsing operation in the process of recommending products is terminated, resulting in low accuracy of the recommended products.

[0005] According to one aspect of an embodiment of the present invention, a method for recommending data is provided, including: sorting browsing records of at least one user according to a preset time series to obtain at least one group of browsing records; obtaining browsing records whose distance from an end point of the preset time series is less than a preset value from the at least one group of sorted browsing records; obtaining corresponding push probabilities according to the browsing records; and obtaining corresponding push strategies according to the push probabilities.

[0006] Optionally, obtaining the corresponding push probability based on the browsing history includes: obtaining the browsing status of the browsing history; obtaining the probability of the user transferring from each browsing state to the absorbing state when performing a specific action based on the browsing status, wherein the absorbing state is the state when the user leaves while browsing the recommended object; and determining the probability as the push probability.

[0007] Further, optionally, the method also includes: obtaining a reward value for performing a specific action in the absorbing state, and a value of a first optimal value function in the absorbing state.

[0008] Optionally, obtaining a corresponding push strategy based on the push probability includes: calculating the push probability, reward value and value of the absorption state and each browsing state according to a preset time series to obtain the push strategy, wherein the push strategy includes: a push object and a way of displaying the push object through the client; wherein the push object includes: a first type of push object or a second type of push object; the first type of push object is an object with the same attributes as the push object in at least one group of browsing records; the second type of push object is an object with different attributes from the push object in at least one group of browsing records.

[0009] Optionally, the method also includes: when the push object is a first type of push object, displaying a push object with the same attributes as those in at least one group of browsing records through the client; when the push object is a second type of push object, displaying a push object with different attributes from those in at least one group of browsing records through the client.

[0010] Optionally, the push strategy also includes: associating corresponding push objects according to the user's historical consumption capacity.

[0011] Optionally, the first category push object or the second category push object includes: at least one of: a product recommended by at least one person, merchandise in an online shopping platform, a recommended route in a navigation platform, and a travel product in a travel product platform.

[0012] According to one aspect of an embodiment of the present invention, another method for recommending data is provided, including: obtaining the last browsing state in the browsing history of at least one user, and determining the browsing state as an absorbing state; respectively obtaining the probability of executing a specific action when each browsing state in the browsing history is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state; calculating according to the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state; and recommending corresponding content to the user according to the value of the optimal value function and the optimal strategy.

[0013] Optionally, the probability of performing a specific action when each browsing state in the browsing record is transferred to the absorbing state is obtained respectively, and the reward value of each browsing state and the value of the first optimal value function in the absorbing state include: obtaining the first reward value for performing a specific action in the first browsing state, the first state probability of transferring from performing the specific action in the first browsing state to the absorbing state, and the value of the first optimal value function in the absorbing state.

[0014] Further, optionally, before respectively obtaining a first reward value for performing a specific action in a first browsing state, a first state probability of transferring from the first browsing state to the absorbing state by performing the specific action, and a value of a first optimal value function in the absorbing state, the method further includes: obtaining a state space, wherein the state space includes a set of browsing card serial numbers and absorbing states; obtaining an action space, wherein the action space includes a set of recommended content; obtaining a state transition matrix, wherein the probability of transition between adjacent states in the state transition matrix when performing a specific action and the probability of transferring from the current state to the absorbing state to perform the specific action are greater than 0, and the remaining probabilities are all 0.

[0015] Optionally, calculations are performed based on the probability, reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy for each browsing state, including: calculations are performed based on the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state; and iterative calculations are performed forward in an order in a preset state space based on the first optimal strategy to obtain the value of the optimal value function and the optimal strategy for the user when browsing.

[0016] Further, optionally, calculations are performed based on the first reward value, the first state probability, and the values ​​of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state, including: calculations are performed based on the product of the first state probability and the value of the first optimal value function and the first reward value to obtain the first reward value at the maximum value; determining the first reward value at the maximum value as the value of the optimal value function in the first browsing state; and determining the variable of the first reward value at the maximum value as the first optimal strategy.

[0017] Optionally, iterative calculations are performed forward in an order in a preset state space according to the first optimal strategy to obtain the value of the optimal value function and the optimal strategy for the user when browsing, including: obtaining a second reward value for performing a specific action in the second browsing state, a second probability for performing the specific action when transferring from the second browsing state to the first browsing state, and a third probability for performing the specific action when transferring from the second browsing state to the absorbing state; wherein, in the preset state space, the second browsing state is before the first browsing state; and calculating according to the product of the second reward value, the second probability, and the third probability and the value of the first optimal value function to obtain the second optimal strategy in the second browsing state and the value of the optimal value function in the second browsing state.

[0018] According to one aspect of an embodiment of the present invention, a method for recommending data is also provided, the method comprising: displaying a plurality of products on a display page of an online shopping platform; collecting operation records for each product displayed on the display page, wherein the operation records comprise at least one of the following: sorting browsing records of at least one user browsing the product according to a preset time series, obtaining at least one group of browsing records, and sorting transaction records of each product according to a preset time series, obtaining at least one group of transaction records; filtering out browsing records whose distance from an end point of the preset time series is less than a preset value from the operation records; obtaining corresponding push probabilities based on the browsing records obtained after filtering; and determining push strategies based on the push probabilities.

[0019] According to one aspect of an embodiment of the present invention, a method for recommending data is also provided, the method comprising: displaying a set of driving routes from a departure point to a destination in a navigation page, wherein the driving route set comprises a plurality of driving routes; collecting operation records for each driving route displayed in the navigation page, wherein the operation records comprise: sorting the driving routes selected by at least one user according to a preset time series to obtain at least one group of driving records; filtering out from the operation records driving records whose distance from an end point of the preset time series is less than a preset value; obtaining corresponding push probabilities based on the driving records obtained after filtering; and determining push strategies based on the push probabilities.

[0020] According to one aspect of an embodiment of the present invention, a method for recommending data is also provided, the method comprising: displaying product information of a plurality of tourism products on a display page of a tourism platform, wherein the product information comprises: a name of the tourism product and a travel route; collecting operation records of each tourism product, wherein the operation records comprise at least one of the following: sorting browsing records of at least one user browsing the tourism product according to a preset time series, obtaining at least one group of browsing records, and sorting transaction records of each tourism product according to a preset time series, obtaining at least one group of transaction records; filtering browsing records whose distance from an end point of the preset time series is less than a preset value from the operation records; obtaining corresponding push probabilities based on the browsing records obtained after filtering; and determining push strategies based on the push probabilities.

[0021] According to one aspect of an embodiment of the present invention, there is also provided a device for recommending data, including: a sorting module, used to sort the browsing records of at least one user according to a preset time series to obtain at least one group of browsing records; a first acquisition module, used to obtain, from the at least one group of sorted browsing records, browsing records whose distance from the end point of the preset time series is less than a preset value; a second acquisition module, used to obtain a corresponding push probability based on the browsing records; and a third acquisition module, used to obtain a corresponding push strategy based on the push probability.

[0022] According to another aspect of an embodiment of the present invention, there is also provided a device for recommending data, comprising: a first acquisition module, used to obtain the last browsing state in the browsing history of at least one user, and determine the browsing state as an absorbing state; a second acquisition module, used to respectively obtain the probability of executing a specific action when each browsing state in the browsing history is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state; a calculation module, used to perform calculations based on the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state; and a recommendation module, used to recommend corresponding content to the user based on the value of the optimal value function and the optimal strategy.

[0023] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided. The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the method for recommending data.

[0024] According to yet another aspect of an embodiment of the present invention, a processor is provided, and the processor is used to run a program, wherein the method for recommending data is executed when the program is run.

[0025] In an embodiment of the present invention, an absorbing state is added to the MDP, and at least one group of browsing records is obtained by sorting the browsing records of at least one user according to a preset time series; browsing records whose distance from the end point of the preset time series is less than a preset value are obtained from the sorted at least one group of browsing records; corresponding push probabilities are obtained according to the browsing records; and corresponding push strategies are obtained according to the push probabilities, thereby achieving the purpose of modeling the user browsing behavior into a Markov decision process containing an absorbing state and converting it into an SSP problem to quickly solve the optimal strategy, thereby achieving the technical effect of improving the accuracy of recommended products, and further solving the technical problem that due to the user's leaving behavior in the prior art, the accumulated revenue obtained based on the browsing operation in the process of recommending products is terminated, thereby resulting in low accuracy of recommended products. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 It is a hardware structure block diagram of a computer terminal of a method for recommending data according to an embodiment of the present invention;

[0028] Figure 2 is a flow chart of a method for recommending data according to Embodiment 1 of the present invention;

[0029] Figure 3 is a flow chart of a method for recommending data according to Embodiment 2 of the present invention;

[0030] Figure 4 is a schematic diagram of a finite state machine in a method for recommending data according to Embodiment 2 of the present invention;

[0031] Figure 5 is a flow chart of a method for recommending data according to Embodiment 3 of the present invention;

[0032] Figure 6 is a flow chart of a method for recommending data according to a fourth embodiment of the present invention;

[0033] Figure 7 is a flow chart of a method for recommending data according to Embodiment 5 of the present invention;

[0034] Figure 8 is a schematic diagram of a device for recommending data according to a sixth embodiment of the present invention;

[0035] Fig. 9 FIG. 4 is a schematic diagram of a device for recommending data according to Embodiment 7 of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] Technical terms involved in this application:

[0039] Greedy: Greedy strategy is a strategy that takes the best option under the current state in each step.

[0040] SSP: Stochastic Shortest Path is a method for finding the shortest path given a Markov decision process.

[0041] Example 1

[0042] According to an embodiment of the present invention, a method embodiment for recommending data is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a computer terminal of a method for recommending data according to an embodiment of the present invention. Figure 1 As shown, the computer terminal 10 may include one or more (only one is shown in the figure) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0044] The memory 104 can be used to store software programs and modules of application software, such as program instructions / modules corresponding to the method for recommending data in the embodiment of the present invention. The processor 102 executes various functional applications and recommends data by running the software programs and modules stored in the memory 104, that is, implements the method for recommending data of the above-mentioned application program. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0045] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0046] Under the above operating environment, this application provides Figure 2 The method for the recommended data is shown. Figure 2 1 is a flow chart of a method for recommending data according to Embodiment 1 of the present invention. The method for recommending data provided in the embodiment of the present application is specifically as follows:

[0047] Step S202, sorting the browsing records of at least one user according to a preset time sequence to obtain at least one group of browsing records;

[0048] In the above step S202 of the present application, the preset time sequence may be a time sequence consisting of multiple time points from the moment when the user leaves the browsing to the preset moment; the browsing record of at least one user may include: at least one of the records of pages clicked by the user, the records of pages collected by the user, the records of users adding to the shopping cart, and the records of users checking out;

[0049] The browsing records of at least one user are sorted by a preset time sequence, that is, the browsing records of at least one user are sorted according to the order (reverse order) of the time points to obtain at least one group of browsing records, wherein each group of browsing records can be the browsing records of one user after being sorted by the preset time sequence.

[0050] Specifically, the preset time sequence may be as follows: the time when the user leaves the browsing is recorded as T, and the time forward to the preset time may be: T-1, ..., 3, 2, 1; in the embodiment of the present application, the position of the preset time in the preset time sequence may be 1; that is, the preset time sequence S = {1, 2, 3, ..., T-1, T} is obtained;

[0051] The browsing records at each time point are obtained according to the preset time sequence {1, 2, 3, ... T-1, T}, and the browsing records at each time point are sorted according to the preset time sequence.

[0052] Step S204, obtaining browsing records whose distance from the end point of the preset time sequence is less than a preset value from at least one group of sorted browsing records;

[0053] In the above step S204 of the present application, based on the preset time series in step S202, the end point of the preset time series can be obtained as T; the browsing records whose distance from the end point of the preset time series is less than the preset value are obtained, that is, the TopN browsing records before T.

[0054] Step S206, obtaining corresponding push probability according to the browsing history;

[0055] In the above step S206 of the present application, by obtaining the last browsing state in the historical browsing record, the browsing state when the user leaves is determined as the absorbing state. According to the absorbing state, the browsing length of the user is increased in a forward calculation manner, and the probability of performing a specific action (i.e., the action of the user leaving when browsing the recommended object in the embodiment of the present application) when each browsing state in the historical browsing record is transferred to the absorbing state is obtained respectively.

[0056] In one feasible method, obtaining the corresponding push probability based on the browsing history includes: obtaining the browsing status of the browsing history; obtaining the probability of the user transferring from each browsing state to the absorbing state when performing a specific action based on the browsing status, wherein the absorbing state is the state when the user leaves when browsing the recommended object; and determining the probability as the push probability.

[0057] Furthermore, the method for recommending data provided in the embodiment of the present application also includes: obtaining a reward value for performing a specific action in the absorbing state, and a value of a first optimal value function in the absorbing state.

[0058] Specifically, by obtaining the last browsing state in the historical browsing record, the browsing state when the user leaves is determined as the absorbing state. According to the absorbing state, the user's browsing length is increased in a forward calculation manner, and the probability of performing a specific action (that is, the action of the user leaving when browsing the recommended object in the embodiment of the present application) when each browsing state in the historical browsing record is transferred to the absorbing state is obtained respectively, the reward value of each browsing state and the value of the first optimal value function in the absorbing state.

[0059] Step S208: Obtain a corresponding push strategy according to the push probability.

[0060] In the above step S208 of the present application, based on the push probability obtained in step S206, the reward value of each browsing state and the value of the first optimal value function in the absorption state are also obtained; the push strategy is obtained by calculation based on the push probability, reward value and value.

[0061] In one achievable manner, obtaining a corresponding push strategy based on the push probability includes: calculating the push probability, reward value, and value of the absorption state and each browsing state based on a preset time series to obtain a push strategy, wherein the push strategy includes: a push object and a method of displaying the push object through a client; wherein the push object includes: a first-category push object or a second-category push object; the first-category push object is an object with the same attributes as those of a push object in at least one group of browsing records; the second-category push object is an object with different attributes from those of a push object in at least one group of browsing records.

[0062] Among them, according to the probability of executing a specific action when each browsing state is transferred to the absorbing state obtained in step S206, the reward value of each browsing state and the value of the first optimal value function in the absorbing state, the value of the optimal value function and the optimal strategy in each browsing state are calculated respectively in a forward calculation manner, so as to recommend the corresponding content to the user based on the obtained value of the optimal value function and the optimal strategy, so as to avoid the situation where the push accuracy is reduced when the user leaves midway.

[0063] In an embodiment of the present invention, at least one group of browsing records is obtained by sorting the browsing records of at least one user according to a preset time series; the browsing records whose distance from the end point of the preset time series is less than a preset value are obtained from the sorted at least one group of browsing records; the corresponding push probability is obtained according to the browsing record; the corresponding push strategy is obtained according to the push probability, thereby achieving the purpose of modeling the user browsing behavior into a Markov decision process containing an absorbing state and converting it into an SSP problem to quickly solve the optimal strategy, thereby achieving the technical effect of improving the accuracy of recommended products, and further solving the technical problem that due to the user's leaving behavior in the prior art, the accumulated income obtained based on the browsing operation in the process of recommending products is terminated, thereby resulting in low accuracy of recommended products.

[0064] Further, optionally, the method for recommending data provided in the embodiment of the present application also includes:

[0065] In the embodiments of the present application, the methods for displaying the push object through the client include the following:

[0066] Method 1:

[0067] In the case where the push object is a first-category push object, displaying, through the client, a push object having the same attributes as the push object in at least one set of browsing records;

[0068] wherein the first type of push object is obtained through at least one of the records of pages clicked by the user, the records of pages collected by the user, the records of the pages added to the shopping cart by the user, and the records of the pages checked out by the user, and the push objects with the same attributes are displayed to the user through the client according to the first type of push objects;

[0069] For example, assuming that the first type of push object is perfume A of brand A, push objects with the same attributes may include: perfumes B, C and D of brand A; perfume of brand B; and fragrances and solid perfumes related to perfume.

[0070] By associating push objects with the same attributes, users can be shown more types and qualities of the same type of products, thereby expanding the user's range of preferences.

[0071] Method 2:

[0072] In the case where the push object is a second type of push object, a push object having attributes different from those of the push objects in at least one set of browsing records is displayed through the client.

[0073] wherein the second type of push object is obtained through at least one of the records of pages clicked by the user, the records of pages collected by the user, the records of the user adding to the shopping cart, and the records of the user checking out, and the push objects with different attributes are displayed to the user through the client according to the second type of push objects;

[0074] For example, assuming that the second type of push object is perfume A of brand A, push objects with different attributes may include: skin care products, cosmetics, and pharmaceutical cosmetics of brand A.

[0075] By associating push objects with different attributes, users can be shown more types and qualities of the same type of products, thereby expanding the user's preference range and increasing the user's chance of transaction.

[0076] Optionally, the push strategy also includes: associating corresponding push objects according to the user's historical consumption capacity.

[0077] In the embodiment of the present application, the historical consumption capacity of the user can be obtained through the historical purchase record of the user; the push objects associated with the historical consumption capacity may include: commodities that meet the historical transaction prices in the historical purchase records, or commodities that have a price difference with the historical transaction prices in the historical purchase records;

[0078] Specifically, the push strategy in the embodiment of the present application can also be implemented in the following ways:

[0079] Method 1: Recommend products that match the historical transaction prices in historical purchase records;

[0080] By evaluating the historical purchase records, recommend products that match the historical transaction prices in the historical purchase records to the user; for example, if the user usually buys cheap jewelry with a price distribution between [50,150], the price of the recommended products can be between [50,150];

[0081] Method 2: Products with a price difference from the historical transaction price in the historical purchase records;

[0082] Recommend products with a price difference from the historical transaction price in the historical purchase record to the user. The products with a price difference can be products with a price greater than the historical transaction price. For example, the products that the user usually buys are affordable jewelry with a price distribution between [50,150]. At this time, the recommended products with a price difference can be products in brand counters, that is, products with a price between [500,3000]. The recommendation method with a price difference from the historical transaction price is called reverse recommendation, that is, recommending products that do not conform to the user's online shopping habits, so that more products that the user may be interested in and have a transaction possibility can be obtained through reverse recommendation.

[0083] Through the above-mentioned reverse recommendation, the user's browsing history and final transaction record are obtained in a random recommendation manner, and the user's possible shopping habits and product preferences recorded in the system are further improved, so as to adjust the user's push strategy so that the user's browsing history and transaction record have more intersections, improve the push accuracy, and provide users with more, more valuable, and higher-quality products.

[0084] Optionally, the first category push object or the second category push object includes: at least one of: a product recommended by at least one person, merchandise in an online shopping platform, a recommended route in a navigation platform, and a travel product in a travel product platform.

[0085] The product recommended by at least one person may be a product recommended by various experts on the online platform, that is, a currently popular product and / or a product with a high rate of favorable comments;

[0086] The products on the online shopping platform may be the products associated with the online live broadcast;

[0087] The recommended routes in the navigation platform can be routes recommended for autonomous driving or daily navigation. For example, when a user wants to go to location A, a route with less congestion but longer distance and shorter time (including expressway routes and non-expressway routes) is recommended to the user, or scenic spots, dining and entertainment locations, refueling / charging locations on the route, or preferred routes uploaded online by other users;

[0088] The travel products in the travel product platform may include at least one of the following products or a combination of products: hotels, transportation, catering, and attractions. For example, if a user wants to go to place A, the platform recommends the user special travel and travel products in place A, catering products in place A that meet the user's previous dining habits and preferences (e.g., shopping coupons, checkout coupons), and attractions products in place A (e.g., attractions peripheral products, tickets).

[0089] Among them, products surrounding scenic spots can include handicrafts, special snacks, etc.

[0090] The above-mentioned first type of push objects or second type of push objects can be used to enrich the shopping needs of users when browsing online.

[0091] Example 2

[0092] According to one aspect of an embodiment of the present invention, another method for recommending data is provided. Figure 3 FIG. 1 is a flow chart of a method for recommending data according to Embodiment 2 of the present invention. Figure 3 As shown, the method for recommending data provided in the embodiment of the present application is specifically as follows:

[0093] Step S302, obtaining the last browsing state in the browsing record of at least one user, and determining the browsing state as an absorbing state;

[0094] In the above step S302 of the present application, since the method for recommending data provided in the embodiment of the present application can be applied to the serialized recommendation scenario, in order to avoid the situation where the push accuracy is reduced when the recommendation is made again due to the user leaving halfway, the browsing status of the last time in the historical browsing record is obtained, and the browsing status when the user leaves is determined as the absorption status. According to the absorption status, the browsing length of the user is increased by forward calculation.

[0095] Step S304, respectively obtaining the probability of executing a specific action when each browsing state in the browsing record is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state;

[0096] In the above step S304 of the present application, based on the absorption state determined in step S302, the probability of executing a specific action when each browsing state in the historical browsing record is transferred to the absorption state, the reward value of each browsing state and the value of the first optimal value function in the absorption state are obtained respectively.

[0097] in, Figure 4 is a schematic diagram of a finite state machine in a method for recommending data according to Embodiment 1 of the present invention, such as Figure 4 As shown, S represents the set of browsing states and absorption states in the historical browsing records. S can be expressed as:

[0098] S = {1, 2, 3, ..., T, A}; A represents the absorbing state.

[0099] Step S306, performing calculations based on the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state;

[0100] In the above step S306 of the present application, based on the probability of performing a specific action when each browsing state is transferred to the absorbing state obtained in step S304, the reward value of each browsing state and the value of the first optimal value function in the absorbing state, the value of the optimal value function and the optimal strategy in each browsing state are calculated respectively in a forward calculation manner.

[0101] Step S308: recommend corresponding content to the user based on the value of the optimal value function and the optimal strategy.

[0102] In the above step S308 of the present application, the corresponding content is recommended to the user based on the value of the optimal value function and the optimal strategy obtained in step S306, so as to avoid the situation where the push accuracy is reduced when the recommendation is made again due to the user leaving midway.

[0103] The method for recommending data provided in the embodiment of the present application can be applied to online shopping platforms, wherein, by continuously recommending products to users in a serialized recommendation scenario, users can continue to browse, thereby maximizing the cumulative revenue, thereby improving users' acquisition of more and more accurate product information on the online shopping platform, and avoiding the situation where the push accuracy is reduced when the recommendation is made again due to the user leaving midway.

[0104] In an embodiment of the present invention, an absorbing state is added to the MDP, by obtaining the last browsing state in the browsing history of at least one user, and determining the browsing state as the absorbing state; respectively obtaining the probability of executing a specific action when each browsing state in the browsing history is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state; calculating according to the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state; recommending corresponding content to the user according to the value of the optimal value function and the optimal strategy, thereby achieving the purpose of modeling the user browsing behavior into a Markov decision process containing an absorbing state, converting it into an SSP problem to quickly solve the optimal strategy, thereby achieving the technical effect of improving the accuracy of recommended products, and further solving the technical problem that due to the user's leaving behavior in the prior art, the accumulated income obtained based on the browsing operation in the process of recommending products is terminated, thereby resulting in low accuracy of recommended products.

[0105] Optionally, the method for recommending data provided in an embodiment of the present application also includes: obtaining a reward value for performing a specific action in an absorbing state, wherein the reward value is 0; obtaining a value of a first optimal value function in an absorbing state, wherein the value of the first optimal value function is 0.

[0106] Among them, the reward value for performing a specific action in the absorbing state is expressed as: R A,a =0;

[0107] The value of the first optimal value function in the absorbing state is expressed as: V * (s=A)=0.

[0108] In the present embodiment, R A,a =0 and V * (s=A)=0 exists as a known condition, and the optimal strategy in each browsing state is calculated based on the known condition.

[0109] Optionally, in step S304, the probability of executing a specific action when each browsing state in the browsing record is transferred to the absorbing state is obtained respectively, and the reward value of each browsing state and the value of the first optimal value function in the absorbing state include:

[0110] Step S3041, respectively obtaining a first reward value for executing a specific action in a first browsing state, a first state probability of transferring from executing the specific action in the first browsing state to an absorbing state, and a value of a first optimal value function in the absorbing state.

[0111] Specifically, in the embodiment of the present application, since the optimal strategy and the value of the optimal value function in each browsing state are calculated forward in the absorption state, here S represents the set of each browsing state and absorption state in the historical browsing record, and S = {1, 2, 3, ..., T, A}, the first browsing state is T. Therefore, the first reward value for executing a specific action in the first browsing state is recorded as: R T,a , the probability of transferring from the first browsing state to the absorption state by executing a specific action is recorded as: P T,a,A .

[0112] Further, optionally, before respectively obtaining in step S3041 a first reward value for performing a specific action in the first browsing state, a first state probability of transferring from the first browsing state to the absorbing state by performing the specific action, and a value of a first optimal value function in the absorbing state, the method for recommending data provided in an embodiment of the present application further includes:

[0113] Step S3038, obtaining a state space, wherein the state space includes a set of browse card serial numbers and absorption states;

[0114] Specifically, by Figure 4 It can be seen that in the embodiment of the present application, S represents the set of various browsing states and absorption states in the historical browsing records, that is, the state space, which is expressed as: S = {1, 2, 3, ..., T, A}.

[0115] Step S3039, obtaining an action space, wherein the action space includes a recommended content set;

[0116] Specifically, the action space can be written as: the set of recommended content M = {a1, a2, a3, ..., a k}.

[0117] Step S3040, obtaining a state transfer matrix, wherein the probability of transition between adjacent states in the state transfer matrix when executing a specific action and the probability of transitioning from the current state to the absorbing state when executing a specific action are greater than 0, and the remaining probabilities are all 0.

[0118] Specifically, in the embodiment of the present application, the state transfer matrix is ​​similar to a diagonal matrix, which has the properties of an upper triangular matrix, that is, the change of the value function of a state will not affect the value function of the state behind it. Therefore, it is possible to iterate forward from the absorbing state, and each round of iteration can actually obtain the value of the optimal value function corresponding to the current state and the desired optimal strategy, where, due to the coherence of S, such as Figure 4 As shown, in the embodiment of the present application, the probability P of the transition of adjacent states when performing a specific action is set, where only {P i,a,i+1 |i∈S∧i<T,a∈M}∪{P i,a,A |i∈S, a∈M} is greater than 0, all other elements are 0, and for i<T, P i,a,i+1 +P i,a,A =1, and P T,a,A =1.

[0119] Optionally, in step S306, the calculation is performed based on the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state, including:

[0120] Step S3061, calculating according to the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state;

[0121] Further, optionally, in step S3061, the calculation is performed based on the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state, including:

[0122] S30611, calculating the first reward value based on the product of the first state probability and the value of the first optimal value function, to obtain the first reward value at the maximum value;

[0123] S30612, determining the first reward value at the maximum value as the value of the optimal value function in the first browsing state;

[0124] S30613, determining the variable of the first reward value at the maximum value as the first optimal strategy.

[0125] Specifically, from the record in step S3041, it can be seen that the first reward value is recorded as: R T,a , the probability of transferring from the first browsing state to the absorption state by executing a specific action is recorded as: P T,a,A , the value of the first optimal value function is expressed as: V * (s=A)=0, so the first optimal strategy is denoted as π * (s=T), the value of the first optimal value function is denoted as V * (s=T), the calculation process of the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state is specifically as follows:

[0126]

[0127]

[0128] Step S3062, iterative calculation is performed forward according to the first optimal strategy in the order of the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing.

[0129] Specifically, based on the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state calculated in step S3061, iterative calculation is performed forward according to the first optimal strategy in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing, as follows:

[0130] Further, optionally, in step S3062, iterative calculation is performed forward according to the first optimal strategy in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing, including:

[0131] Step S30621, respectively obtaining a second reward value for performing a specific action in the second browsing state, a second probability for performing the specific action when the second browsing state is transferred to the first browsing state, and a third probability for performing the specific action when the second browsing state is transferred to the absorbing state; wherein the second browsing state is before the first browsing state in the preset state space;

[0132] Step S30622, calculating based on the product of the second reward value, the second probability, the third probability and the value of the first optimal value function, to obtain the second optimal strategy in the second browsing state and the value of the optimal value function in the second browsing state.

[0133] Specifically, based on S={1, 2, 3, ..., T, A}, combined with step S30621 and step S30622, the second browsing state may be T-1. Therefore, the forward iterative calculation involved in the embodiment of the present application is specifically as follows:

[0134]

[0135]

[0136] Among them, π * (s=T-1) is the second optimal strategy in the second browsing state; V * (s=T-1) is the value of the optimal value function in the second browsing state.

[0137] Similarly, the optimal strategies and optimal value functions of each state obtained by other forward iterations are as follows:

[0138]

[0139]

[0140] Example 3

[0141] According to one aspect of an embodiment of the present invention, another method for recommending data is provided. Figure 5 FIG. 1 is a flow chart of a method for recommending data according to Embodiment 3 of the present invention. Figure 5 As shown, the method includes:

[0142] Step S502, displaying a plurality of products on a display page of an online shopping platform;

[0143] In the above step S502 of the present application, the data recommendation method provided in the embodiment of the present application can be applied to an online shopping platform, wherein the display page of the online shopping platform is used to display multiple products to each user, wherein a portion of each displayed product is associated with the historical browsing record of each user.

[0144] The online shopping platform in the embodiment of the present application provides various products for users to choose from. When users operate the products displayed on the display page of the online shopping platform, browsing information, transaction information, evaluation information, and forwarding information will be generated, among which the browsing information may include the length of stay when browsing the product, the number of views, etc. After the user clicks on the product, the operation of collection, comment and transaction will be entered. The above operations are all recorded as reference data for whether the product is recommended to other customers or targeted recommendation. It should be emphasized that it is not limited to this.

[0145] Step S504, collecting operation records of each product displayed on the display page, wherein the operation records include at least one of the following: at least one set of browsing records obtained by sorting browsing records of at least one user browsing the product according to a preset time sequence, and at least one set of transaction records obtained by sorting transaction records of each product according to a preset time sequence;

[0146] In the above step S504 of the present application, based on the multiple products displayed in the display page in step S502, the preset time sequence may be a time sequence consisting of multiple time points from the moment when the user leaves the browsing to the preset moment;

[0147] The browsing record of at least one user browsing a product may include: a record of pages clicked by at least one user on the browsed product; in addition, the browsing record of at least one user browsing a product may also include: a record of pages collected by at least one user on the browsed product, a record of pages recommended to user A by other users, and a record of at least one user adding the browsed product to a shopping cart;

[0148] For example, the browsing record of at least one user browsing a product can be expressed as: user A browsed product a at time A, browsed product b at time B, browsed product c at time C, ... through an online shopping platform; based on the products browsed at the above-mentioned moments, a record of the pages clicked by user A on the browsed products is formed; similarly, page records of products a, b, c... collected by user A can be obtained, and / or collection records of other users including user A who have collected products a, b, c... can be obtained; page collection records of other users recommending at least one of the products a, b, c... browsed by user A; and page records of user A adding at least one of products a, b, c... to the shopping cart, and / or records of other users including user A who have added at least one of products a, b, c... to the shopping cart.

[0149] The transaction record for each product may include at least one of: a record that at least one user has checked out for the browsed product;

[0150] For example, a record of user A checking out at least one of products a, b, c, ..., and / or a record of other users including user A checking out at least one of products a, b, c, ...

[0151] Among them, collecting the operation records of each product displayed in the display page includes: obtaining at least one group of browsing records generated by at least one user when browsing products using a client or a web page, wherein the at least one group of browsing records can be obtained after sorting through a preset time series; and obtaining the transaction records of at least one user for each product using a client or a web page segment, wherein the transaction records can be obtained after sorting through a preset time series.

[0152] Specifically, the preset time sequence may be as follows: the time when the user leaves the browsing is recorded as T, and the time forward to the preset time may be: T-1, ..., 3, 2, 1; in the embodiment of the present application, the position of the preset time in the preset time sequence may be 1; that is, the preset time sequence S = {1, 2, 3, ..., T-1, T} is obtained;

[0153] Obtain browsing records of at least one user browsing products at each time point according to a preset time sequence {1, 2, 3, ... T-1, T}, and sort the browsing records at each time point according to the preset time sequence. Also, obtain transaction records of at least one user for each product at each time point, and sort the transaction records of each product at each time point according to the preset time sequence.

[0154] Step S506, filtering out browsing records whose distance from the end point of the preset time sequence is less than a preset value from the operation records;

[0155] In the above step S506 of the present application, based on the preset time series in step S504, the end point of the preset time series can be obtained as T; the browsing records whose distance from the end point of the preset time series is less than the preset value are obtained, that is, the TopN browsing records before T.

[0156] Step S508, obtaining the corresponding push probability according to the browsing records obtained after screening;

[0157] In the above step S508 of the present application, by obtaining the last browsing status in the browsing history, the browsing status when the user leaves is determined as the absorbing status, and based on the absorbing status, the browsing length of the user is increased in a forward calculation manner, and the probability of performing a specific action (i.e., the action of the user leaving when browsing the recommended object in the embodiment of the present application) when each browsing status in the historical browsing record is transferred to the absorbing status is obtained respectively.

[0158] In one feasible method, obtaining the corresponding push probability based on the browsing history includes: obtaining the browsing status of the browsing history; obtaining the probability of the user transferring from each browsing state to the absorbing state when performing a specific action based on the browsing status, wherein the absorbing state is the state when the user leaves when browsing the recommended object; and determining the probability as the push probability.

[0159] Furthermore, the method for recommending data provided in the embodiment of the present application also includes: obtaining a reward value for performing a specific action in the absorbing state, and a value of a first optimal value function in the absorbing state.

[0160] Specifically, by obtaining the last browsing state in the historical browsing record, the browsing state when the user leaves is determined as the absorbing state. According to the absorbing state, the user's browsing length is increased in a forward calculation manner, and the probability of performing a specific action (that is, the action of the user leaving when browsing the recommended object in the embodiment of the present application) when each browsing state in the historical browsing record is transferred to the absorbing state is obtained respectively, the reward value of each browsing state and the value of the first optimal value function in the absorbing state.

[0161] Step S510: determining a push strategy according to the push probability.

[0162] In the above step S510 of the present application, based on the push probability obtained in step S508, the reward value of each browsing state and the value of the first optimal value function in the absorption state are also obtained; the push strategy is obtained by calculation based on the push probability, reward value and value.

[0163] In one achievable manner, obtaining a corresponding push strategy based on the push probability includes: calculating the push probability, reward value, and value of the absorption state and each browsing state based on a preset time series to obtain a push strategy, wherein the push strategy includes: a push object and a method of displaying the push object through a client; wherein the push object includes: a first-category push object or a second-category push object; the first-category push object is an object with the same attributes as those of a push object in at least one group of browsing records; the second-category push object is an object with different attributes from those of a push object in at least one group of browsing records.

[0164] Among them, according to the probability of executing a specific action when each browsing state is transferred to the absorbing state obtained in step S508, the reward value of each browsing state and the value of the first optimal value function in the absorbing state, the value of the optimal value function and the optimal strategy in each browsing state are calculated respectively in a forward calculation manner, so as to recommend the corresponding content to the user based on the obtained value of the optimal value function and the optimal strategy, so as to avoid the situation where the push accuracy is reduced when the user leaves midway.

[0165] Optionally, the method for recommending data provided in an embodiment of the present application also includes: obtaining a reward value for performing a specific action in an absorbing state, wherein the reward value is 0; obtaining a value of a first optimal value function in an absorbing state, wherein the value of the first optimal value function is 0.

[0166] Among them, the reward value for performing a specific action in the absorbing state is expressed as: R A,a =0;

[0167] The value of the first optimal value function in the absorbing state is expressed as: V * (s=A)=0.

[0168] In the present embodiment, R A,a =0 and V * (s=A)=0 exists as a known condition, and the optimal strategy in each browsing state is calculated based on the known condition.

[0169] Optionally, the probability of executing a specific action when each browsing state in the browsing record is transferred to the absorbing state is obtained respectively, and the reward value of each browsing state and the value of the first optimal value function in the absorbing state include:

[0170] A first reward value for executing a specific action in a first browsing state, a first state probability of transferring from executing the specific action in the first browsing state to an absorbing state, and a value of a first optimal value function in the absorbing state are obtained respectively.

[0171] Specifically, in the embodiment of the present application, since the optimal strategy and the value of the optimal value function in each browsing state are calculated forward in the absorption state, here S represents the set of each browsing state and absorption state in the historical browsing record, and S = {1, 2, 3, ..., T, A}, the first browsing state is T. Therefore, the first reward value for executing a specific action in the first browsing state is recorded as: R T,a , the probability of transferring from the first browsing state to the absorption state by executing a specific action is recorded as: P T,a,A .

[0172] Further, optionally, before respectively obtaining a first reward value for performing a specific action in the first browsing state, a first state probability of transferring to the absorbing state by performing the specific action in the first browsing state, and a value of a first optimal value function in the absorbing state, the method for recommending data provided in the embodiment of the present application further includes:

[0173] Step S1, obtaining a state space, wherein the state space includes a set of browse card serial numbers and absorption states;

[0174] Specifically, in the embodiment of the present application, S represents the set of browsing states and absorption states in the historical browsing record, that is, the state space, which is expressed as: S = {1, 2, 3, ..., T, A}.

[0175] Step S2, obtaining an action space, wherein the action space includes a recommended content set;

[0176] Specifically, the action space can be written as: the set of recommended content M = {a1, a2, a3, ..., a k}.

[0177] Step S3, obtaining a state transfer matrix, wherein the probability of transition between adjacent states in the state transfer matrix when executing a specific action and the probability of transitioning from the current state to the absorbing state when executing a specific action are greater than 0, and the remaining probabilities are all 0.

[0178] Specifically, in the embodiment of the present application, the state transfer matrix is ​​similar to a diagonal matrix, which has the properties of an upper triangular matrix, that is, the change of the value function of a state will not affect the value function of the state behind it. Therefore, it is possible to iterate forward from the absorbing state, and each round of iteration can actually obtain the value of the optimal value function corresponding to the current state and the desired optimal strategy. In this embodiment, due to the continuity of S, the probability P of the transition of adjacent states when performing a specific action is set, where only {P i,a,i+1 |i∈S∧i<T,a∈M}∪{P i,a,A |i∈S, a∈M} is greater than 0, all other elements are 0, and for i<T, P i,a,i+1 +P i,a,A =1, and P T,a,A =1.

[0179] Optionally, the calculation is performed based on the probability, the reward value, and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state, including:

[0180] Step S1, calculating according to the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state;

[0181] Further, optionally, in step S1, the calculation is performed based on the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state, including:

[0182] S11, calculating the first reward value based on the product of the first state probability and the value of the first optimal value function and obtaining the first reward value at the maximum value;

[0183] S12, determining the first reward value at the maximum value as the value of the optimal value function in the first browsing state;

[0184] S13, determining the variable of the first reward value at the maximum value as the first optimal strategy.

[0185] Specifically, the first reward value is recorded as: R T,a , the probability of transferring from the first browsing state to the absorption state by executing a specific action is recorded as: P T,a,A , the value of the first optimal value function is expressed as: V *(s=A)=0, so the first optimal strategy is denoted as π * (s=T), the value of the first optimal value function is denoted as V * (s=T), the calculation process of the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state is specifically as follows:

[0186]

[0187]

[0188] Optionally, iterative calculation is performed forward according to the first optimal strategy in the order of the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing.

[0189] Specifically, based on the obtained first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state, iterative calculation is performed forward according to the first optimal strategy in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing, as follows:

[0190] Further, optionally, iterative calculation is performed forward according to the first optimal strategy in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing, including:

[0191] respectively obtaining a second reward value for performing a specific action in a second browsing state, a second probability for performing the specific action when transferring from the second browsing state to the first browsing state, and a third probability for performing the specific action when transferring from the second browsing state to the absorbing state; wherein the second browsing state is before the first browsing state in the preset state space;

[0192] The second optimal strategy in the second browsing state and the value of the optimal value function in the second browsing state are obtained by calculating the product of the second reward value, the second probability, the third probability and the value of the first optimal value function.

[0193] Specifically, based on S={1, 2, 3, ..., T, A}, the second browsing state may be T-1. Therefore, the forward iterative calculation involved in the embodiment of the present application is specifically as follows:

[0194]

[0195]

[0196] Among them, π * (s=T-1) is the second optimal strategy in the second browsing state; V * (s=T-1) is the value of the optimal value function in the second browsing state.

[0197] Similarly, the optimal strategies and optimal value functions of each state obtained by other forward iterations are as follows:

[0198]

[0199]

[0200] In an embodiment of the present invention, a plurality of products are displayed on a display page of an online shopping platform; and an operation record of each product displayed on the display page is collected, wherein the operation record includes at least one of the following: at least one group of browsing records obtained by sorting browsing records of at least one user browsing the product according to a preset time series, and at least one group of transaction records obtained by sorting transaction records of each product according to a preset time series; and browsing records whose distance from an end point of the preset time series is less than a preset value are screened from the operation records; and corresponding push probabilities are obtained according to the browsing records obtained after screening; and a push strategy is determined according to the push probability, thereby achieving the purpose of modeling the user browsing behavior into a Markov decision process containing an absorbing state and converting it into an SSP problem to quickly solve the optimal strategy, thereby achieving the technical effect of improving the accuracy of recommended products, and further solving the technical problem that due to the user's leaving behavior in the prior art, the accumulated income obtained based on the browsing operation in the process of recommending products is terminated, thereby leading to a low accuracy of the recommended products.

[0201] Example 4

[0202] According to one aspect of an embodiment of the present invention, another method for recommending data is provided. Figure 6 FIG. 4 is a flow chart of a method for recommending data according to Embodiment 4 of the present invention. Figure 6 As shown, the method includes:

[0203] Step S602, displaying a set of driving routes from a departure point to a destination on a navigation page, wherein the driving route set includes a plurality of driving routes;

[0204] In the above step S602 of the present application, the navigation page in the embodiment of the present application provides a set of driving routes formed by multiple driving routes from the departure point to the destination (for example, from A to B) for the user to choose. During the user's operation of the driving route, browsing information, evaluation information, and forwarding information will be generated, wherein the browsing information may include the duration of driving the at least one driving route, the number of times the at least one driving route is recommended, etc. After the user clicks on at least one driving route, the operation of collection, commenting, and forwarding will be entered. The above operations are all recorded as reference data for whether the driving route is recommended to other customers or for targeted recommendation. It should be emphasized that it is not limited to this.

[0205] Step S604, collecting the operation record of each driving route displayed in the navigation page, wherein the operation record includes: sorting the driving routes selected by at least one user according to a preset time sequence to obtain at least one set of driving records;

[0206] In the above step S604 of the present application, based on the set of driving routes from the departure place to the destination displayed in the navigation page in step S602, the preset time sequence can be a time sequence consisting of multiple time points from the time when the user leaves during browsing to the preset time;

[0207] The at least one set of driving records may include: a driving route selected by at least one user; in addition, the browsing records of at least one user browsing products may also include: at least one of: a record of pages collected by at least one user for the browsed products, a record of pages collected by other users for the products browsed by user A, and a record of at least one user adding the browsed products to a shopping cart;

[0208] For example, at least one group of driving records may be represented as: a driving record formed by a historical driving route selected by user A after the user A selects a departure point A and a destination B through a navigation page; similarly, driving records of at least one driving route from the departure point A to the destination B that user A has collected may be obtained, and / or driving records collected by other users including user A who have collected at least one driving route from the departure point A to the destination B; and driving records collected by other users for at least one driving route from the departure point A to the destination B that user A has browsed.

[0209] Specifically, the preset time sequence may be as follows: the time when the user leaves the browsing is recorded as T, and the time forward to the preset time may be: T-1, ..., 3, 2, 1; in the embodiment of the present application, the position of the preset time in the preset time sequence may be 1; that is, the preset time sequence S = {1, 2, 3, ..., T-1, T} is obtained;

[0210] According to the preset time sequence {1, 2, 3, ... T-1, T}, the driving record of the travel route selected by at least one user for each trip is obtained, and the driving records of the travel route selected for each trip are sorted according to the preset time sequence.

[0211] Step S606, filtering out driving records whose distance from the end point of the preset time series is less than a preset value from the operation records;

[0212] In the above step S606 of the present application, based on the preset time series in step S604, the end point of the preset time series can be obtained as T; the driving records whose distance from the end point of the preset time series is less than the preset value are obtained, that is, the TopN driving records before T.

[0213] Step S608, obtaining a corresponding push probability according to the driving records obtained after screening;

[0214] In the above step S608 of the present application, by obtaining the last browsing status in the driving record, the browsing status when the user leaves is determined as the absorbing status, and based on the absorbing status, the browsing length of the user is increased by forward calculation, and the probability of performing a specific action (i.e., the action of the user leaving when browsing the recommended object in the embodiment of the present application) when each browsing status in the historical driving record is transferred to the absorbing status is obtained respectively.

[0215] In one feasible method, based on the driving records obtained after screening, obtaining the corresponding push probability includes: obtaining the browsing status of the driving records; obtaining the probability of the user transferring from each browsing state to the absorption state when performing a specific action based on the browsing status, wherein the absorption state is the state when the user leaves when browsing the recommended object; and determining the probability as the push probability.

[0216] Furthermore, the method for recommending data provided in the embodiment of the present application also includes: obtaining a reward value for performing a specific action in the absorbing state, and a value of a first optimal value function in the absorbing state.

[0217] Specifically, by obtaining the browsing status of the last driving record in the historical driving record, the browsing status when the user leaves is determined as the absorbing state. According to the absorbing state, the user's browsing length is increased in a forward calculation manner, and the probability of performing a specific action (that is, the action of the user leaving when browsing the recommended object in the embodiment of the present application) when each browsing state in the historical browsing record is transferred to the absorbing state is obtained respectively, the reward value of each browsing state and the value of the first optimal value function in the absorbing state.

[0218] Step S610: determining a push strategy according to the push probability.

[0219] In the above step S610 of the present application, based on the push probability obtained in step S608, the reward value of each browsing state and the value of the first optimal value function in the absorption state are also obtained; the push strategy is obtained by calculation based on the push probability, reward value and value.

[0220] In one achievable manner, obtaining a corresponding push strategy based on the push probability includes: calculating the push probability, reward value, and value of the absorption state and each browsing state based on a preset time series to obtain a push strategy, wherein the push strategy includes: a push object and a method of displaying the push object through a client; wherein the push object includes: a first-category push object or a second-category push object; the first-category push object is an object with the same attributes as those of a push object in at least one group of browsing records; the second-category push object is an object with different attributes from those of a push object in at least one group of browsing records.

[0221] Among them, according to the probability of executing a specific action when each browsing state is transferred to the absorbing state obtained in step S608, the reward value of each browsing state and the value of the first optimal value function in the absorbing state, the value of the optimal value function and the optimal strategy in each browsing state are calculated respectively in a forward calculation manner, so as to recommend the corresponding content to the user based on the obtained value of the optimal value function and the optimal strategy, so as to avoid the situation where the push accuracy is reduced when the user leaves midway.

[0222] Optionally, the method for recommending data provided in an embodiment of the present application also includes: obtaining a reward value for performing a specific action in an absorbing state, wherein the reward value is 0; obtaining a value of a first optimal value function in an absorbing state, wherein the value of the first optimal value function is 0.

[0223] Among them, the reward value for performing a specific action in the absorbing state is expressed as: R A,a =0;

[0224] The value of the first optimal value function in the absorbing state is expressed as: V * (s=A)=0.

[0225] In the present embodiment, R A,a =0 and V * (s=A)=0 exists as a known condition, and the optimal strategy in each browsing state is calculated based on the known condition.

[0226] Optionally, the probability of executing a specific action when each browsing state in the driving record is transferred to the absorbing state is obtained respectively, and the reward value of each browsing state and the value of the first optimal value function in the absorbing state include:

[0227] A first reward value for executing a specific action in a first browsing state, a first state probability of transferring from executing the specific action in the first browsing state to an absorbing state, and a value of a first optimal value function in the absorbing state are obtained respectively.

[0228] Specifically, in the embodiment of the present application, since the optimal strategy and the value of the optimal value function in each browsing state are calculated forward in the absorption state, here S represents the set of each browsing state and absorption state in the historical browsing record, and S = {1, 2, 3, ..., T, A}, the first browsing state is T. Therefore, the first reward value for executing a specific action in the first browsing state is recorded as: R T,a , the probability of transferring from the first browsing state to the absorption state by executing a specific action is recorded as: P T,a,A .

[0229] Further, optionally, before respectively obtaining a first reward value for performing a specific action in the first browsing state, a first state probability of transferring to the absorbing state by performing the specific action in the first browsing state, and a value of a first optimal value function in the absorbing state, the method for recommending data provided in the embodiment of the present application further includes:

[0230] Step S1, obtaining a state space, wherein the state space includes a set of browse card serial numbers and absorption states;

[0231] Specifically, in the embodiment of the present application, S represents the set of browsing states and absorption states in the historical browsing record, that is, the state space, which is expressed as: S = {1, 2, 3, ..., T, A}.

[0232] Step S2, obtaining an action space, wherein the action space includes a recommended content set;

[0233] Specifically, the action space can be written as: the set of recommended content M = {a1, a2, a3, ..., a k}.

[0234] Step S3, obtaining a state transfer matrix, wherein the probability of transition between adjacent states in the state transfer matrix when executing a specific action and the probability of transitioning from the current state to the absorbing state when executing a specific action are greater than 0, and the remaining probabilities are all 0.

[0235] Specifically, in the embodiment of the present application, the state transfer matrix is ​​similar to a diagonal matrix, which has the properties of an upper triangular matrix, that is, the change of the value function of a state will not affect the value function of the state behind it. Therefore, it is possible to iterate forward from the absorbing state, and each round of iteration can actually obtain the value of the optimal value function corresponding to the current state and the desired optimal strategy. In this embodiment, due to the continuity of S, the probability P of the transition of adjacent states when performing a specific action is set, where only {P i,a,i+1 |i∈S∧i<T,a∈M}∪{P i,a,A |i∈S, a∈M} is greater than 0, all other elements are 0, and for i<T, P i,a,i+1 +P i,a,A=1, and P T,a,A =1.

[0236] Optionally, the calculation is performed based on the probability, the reward value, and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state, including:

[0237] Step S1, calculating according to the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state;

[0238] Further, optionally, in step S1, the calculation is performed based on the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state, including:

[0239] S11, calculating the first reward value based on the product of the first state probability and the value of the first optimal value function and obtaining the first reward value at the maximum value;

[0240] S12, determining the first reward value at the maximum value as the value of the optimal value function in the first browsing state;

[0241] S13, determining the variable of the first reward value at the maximum value as the first optimal strategy.

[0242] Specifically, the first reward value is recorded as: R T,a , the probability of transferring from the first browsing state to the absorption state by executing a specific action is recorded as: P T,a,A , the value of the first optimal value function is expressed as: V * (s=A)=0, so the first optimal strategy is denoted as π * (s=T), the value of the first optimal value function is denoted as V * (s=T), the calculation process of the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state is specifically as follows:

[0243]

[0244]

[0245] Optionally, iterative calculation is performed forward according to the first optimal strategy in the order of the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing.

[0246] Specifically, based on the obtained first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state, iterative calculation is performed forward according to the first optimal strategy in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing, as follows:

[0247] Further, optionally, iterative calculation is performed forward according to the first optimal strategy in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing, including:

[0248] respectively obtaining a second reward value for performing a specific action in a second browsing state, a second probability for performing the specific action when transferring from the second browsing state to the first browsing state, and a third probability for performing the specific action when transferring from the second browsing state to the absorbing state; wherein the second browsing state is before the first browsing state in the preset state space;

[0249] The second optimal strategy in the second browsing state and the value of the optimal value function in the second browsing state are obtained by calculating the product of the second reward value, the second probability, the third probability and the value of the first optimal value function.

[0250] Specifically, based on S={1, 2, 3, ..., T, A}, the second browsing state may be T-1. Therefore, the forward iterative calculation involved in the embodiment of the present application is specifically as follows:

[0251]

[0252]

[0253] Among them, π * (s=T-1) is the second optimal strategy in the second browsing state; V * (s=T-1) is the value of the optimal value function in the second browsing state.

[0254] Similarly, the optimal strategies and optimal value functions of each state obtained by other forward iterations are as follows:

[0255]

[0256]

[0257] In an embodiment of the present invention, a set of driving routes from a departure point to a destination is displayed on a navigation page, wherein the driving route set includes a plurality of driving routes; operation records of each driving route displayed on the navigation page are collected, wherein the operation records include: sorting the driving routes selected by at least one user according to a preset time series to obtain at least one set of driving records; filtering out driving records whose distance from an end point of the preset time series is less than a preset value from the operation records; obtaining corresponding push probabilities based on the filtered driving records; and determining push strategies based on the push probabilities, thereby achieving the purpose of modeling user browsing behavior into a Markov decision process containing an absorbing state and converting it into an SSP problem to quickly solve the optimal strategy, thereby achieving the technical effect of improving the accuracy of recommended products, and further solving the technical problem that due to the user's leaving behavior in the prior art, the accumulated revenue obtained based on the browsing operation in the process of recommending products is terminated, thereby leading to low accuracy of recommended products.

[0258] Example 5

[0259] According to one aspect of an embodiment of the present invention, another method for recommending data is provided. Figure 7 FIG. 1 is a flow chart of a method for recommending data according to Embodiment 5 of the present invention. Figure 7 As shown, the method includes:

[0260] Step S702, displaying product information of multiple tourism products on a display page of the tourism platform, wherein the product information includes: tourism product name and tourism route;

[0261] In the above step S702 of the present application, the tourism platform in the embodiment of the present application provides various tourism products for users to choose from. During the user's operation of the tourism product, browsing information, transaction information, evaluation information, and forwarding information will be generated, wherein the browsing information may include the length of stay and the number of views when browsing the product. After the user clicks on the tourism product, the user will enter the collection, comment, and transaction operations. The above operations are all recorded as reference data for whether the tourism product is recommended to other customers or for targeted recommendations. It should be emphasized that it is not limited to this.

[0262] Step S704, collecting operation records of each tourism product, wherein the operation records include at least one of the following: at least one set of browsing records obtained by sorting browsing records of at least one user browsing the tourism product according to a preset time sequence, and at least one set of transaction records obtained by sorting transaction records of each tourism product according to a preset time sequence;

[0263] In the above step S704 of the present application, based on the product information of multiple travel products displayed on the display page of the travel platform in step S702, the preset time sequence can be a time sequence consisting of multiple time points from the moment when the user leaves during browsing to the preset moment;

[0264] The browsing record of at least one user browsing the travel product may include: a record of pages clicked by at least one user on the browsed travel product; in addition, the browsing record of at least one user browsing the travel product may also include: a record of pages collected by at least one user on the browsed travel product, a record of pages collected by other users for the travel product browsed by user A, and a record of at least one user adding the browsed travel product to a shopping cart;

[0265] For example, the browsing record of at least one user browsing tourism products can be expressed as: user A browsed tourism product a at time A, browsed tourism product b at time B, browsed tourism product c at time C, ... through the tourism platform; based on the tourism products browsed at the above-mentioned moments, a page record of the products clicked by user A is formed; similarly, page records of tourism products a, b, c... collected by user A can be obtained, and / or collection records of other users including user A who have collected tourism products a, b, c... can be obtained; page collection records of other users recommending at least one of the tourism products a, b, c... browsed by user A; and page records of user A adding at least one of tourism products a, b, c... to the shopping cart, and / or records of other users including user A who have added at least one of tourism products a, b, c... to the shopping cart.

[0266] The transaction record of each travel product may include: at least one of the records of at least one user having checked out for the browsed travel product;

[0267] For example, a record of user A checking out at least one of travel products a, b, c, ..., and / or a record of other users including user A checking out at least one of travel products a, b, c, ...

[0268] Among them, collecting the operation records of each tourism product displayed in the display page includes: obtaining at least one group of browsing records generated by at least one user when browsing tourism products using a client or a web page, wherein the at least one group of browsing records can be obtained after sorting through a preset time series; and obtaining the transaction records of at least one user for each tourism product when using a client or a web page segment, wherein the transaction records can be obtained after sorting through a preset time series.

[0269] Specifically, the preset time sequence may be as follows: the time when the user leaves the browsing is recorded as T, and the time forward to the preset time may be: T-1, ..., 3, 2, 1; in the embodiment of the present application, the position of the preset time in the preset time sequence may be 1; that is, the preset time sequence S = {1, 2, 3, ..., T-1, T} is obtained;

[0270] Obtain browsing records of at least one user browsing products at each time point according to a preset time sequence {1, 2, 3, ... T-1, T}, and sort the browsing records at each time point according to the preset time sequence. Also, obtain transaction records of at least one user for each product at each time point, and sort the transaction records of each product at each time point according to the preset time sequence.

[0271] Step S706, filtering out browsing records whose distance from the end point of the preset time sequence is less than a preset value from the operation records;

[0272] In the above step S706 of the present application, based on the preset time series in step S704, the end point of the preset time series can be obtained as T; the browsing records whose distance from the end point of the preset time series is less than the preset value are obtained, that is, the TopN browsing records before T.

[0273] Step S708, obtaining the corresponding push probability according to the browsing records obtained after screening;

[0274] In the above step S708 of the present application, by obtaining the last browsing status in the browsing history, the browsing status when the user leaves is determined as the absorbing status, and based on the absorbing status, the browsing length of the user is increased by forward calculation, and the probability of performing a specific action (i.e., the action of the user leaving when browsing the recommended object in the embodiment of the present application) when each browsing status in the historical browsing record is transferred to the absorbing status is obtained respectively.

[0275] In one feasible method, obtaining the corresponding push probability based on the browsing history includes: obtaining the browsing status of the browsing history; obtaining the probability of the user transferring from each browsing state to the absorbing state when performing a specific action based on the browsing status, wherein the absorbing state is the state when the user leaves when browsing the recommended object; and determining the probability as the push probability.

[0276] Furthermore, the method for recommending data provided in the embodiment of the present application also includes: obtaining a reward value for performing a specific action in the absorbing state, and a value of a first optimal value function in the absorbing state.

[0277] Specifically, by obtaining the last browsing state in the historical browsing record, the browsing state when the user leaves is determined as the absorbing state. According to the absorbing state, the user's browsing length is increased in a forward calculation manner, and the probability of performing a specific action (that is, the action of the user leaving when browsing the recommended object in the embodiment of the present application) when each browsing state in the historical browsing record is transferred to the absorbing state is obtained respectively, the reward value of each browsing state and the value of the first optimal value function in the absorbing state.

[0278] Step S710: determining a push strategy according to the push probability.

[0279] In the above step S710 of the present application, based on the push probability obtained in step S708, the reward value of each browsing state and the value of the first optimal value function in the absorption state are also obtained; the push strategy is obtained by calculation based on the push probability, reward value and value.

[0280] In one achievable manner, obtaining a corresponding push strategy based on the push probability includes: calculating the push probability, reward value, and value of the absorption state and each browsing state based on a preset time series to obtain a push strategy, wherein the push strategy includes: a push object and a method of displaying the push object through a client; wherein the push object includes: a first-category push object or a second-category push object; the first-category push object is an object with the same attributes as those of a push object in at least one group of browsing records; the second-category push object is an object with different attributes from those of a push object in at least one group of browsing records.

[0281] Among them, according to the probability of executing a specific action when each browsing state is transferred to the absorbing state obtained in step S708, the reward value of each browsing state and the value of the first optimal value function in the absorbing state, the value of the optimal value function and the optimal strategy in each browsing state are calculated respectively in a forward calculation manner, so as to recommend the corresponding content to the user based on the obtained value of the optimal value function and the optimal strategy, so as to avoid the situation where the push accuracy is reduced when the user leaves midway.

[0282] Optionally, the method for recommending data provided in an embodiment of the present application also includes: obtaining a reward value for performing a specific action in an absorbing state, wherein the reward value is 0; obtaining a value of a first optimal value function in an absorbing state, wherein the value of the first optimal value function is 0.

[0283] Among them, the reward value for performing a specific action in the absorbing state is expressed as: R A,a =0;

[0284] The value of the first optimal value function in the absorbing state is expressed as: V * (s=A)=0.

[0285] In the present embodiment, R A,a =0 and V * (s=A)=0 exists as a known condition, and the optimal strategy in each browsing state is calculated based on the known condition.

[0286] Optionally, the probability of executing a specific action when each browsing state in the browsing record is transferred to the absorbing state is obtained respectively, and the reward value of each browsing state and the value of the first optimal value function in the absorbing state include:

[0287] A first reward value for executing a specific action in a first browsing state, a first state probability of transferring from executing the specific action in the first browsing state to an absorbing state, and a value of a first optimal value function in the absorbing state are obtained respectively.

[0288] Specifically, in the embodiment of the present application, since the optimal strategy and the value of the optimal value function in each browsing state are calculated forward in the absorption state, here S represents the set of each browsing state and absorption state in the historical browsing record, and S = {1, 2, 3, ..., T, A}, the first browsing state is T. Therefore, the first reward value for executing a specific action in the first browsing state is recorded as: R T,a , the probability of transferring from the first browsing state to the absorption state by executing a specific action is recorded as: P T,a,A .

[0289] Further, optionally, before respectively obtaining a first reward value for performing a specific action in the first browsing state, a first state probability of transferring to the absorbing state by performing the specific action in the first browsing state, and a value of a first optimal value function in the absorbing state, the method for recommending data provided in the embodiment of the present application further includes:

[0290] Step S1, obtaining a state space, wherein the state space includes a set of browse card serial numbers and absorption states;

[0291] Specifically, in the embodiment of the present application, S represents the set of browsing states and absorption states in the historical browsing record, that is, the state space, which is expressed as: S = {1, 2, 3, ..., T, A}.

[0292] Step S2, obtaining an action space, wherein the action space includes a recommended content set;

[0293] Specifically, the action space can be written as: the set of recommended content M = {a1, a2, a3, ..., a k}.

[0294] Step S3, obtaining a state transfer matrix, wherein the probability of transition between adjacent states in the state transfer matrix when executing a specific action and the probability of transitioning from the current state to the absorbing state when executing a specific action are greater than 0, and the remaining probabilities are all 0.

[0295] Specifically, in the embodiment of the present application, the state transfer matrix is ​​similar to a diagonal matrix, which has the properties of an upper triangular matrix, that is, the change of the value function of a state will not affect the value function of the state behind it. Therefore, it is possible to iterate forward from the absorbing state, and each round of iteration can actually obtain the value of the optimal value function corresponding to the current state and the desired optimal strategy. In this embodiment, due to the continuity of S, the probability P of the transition of adjacent states when performing a specific action is set, where only {P i,a,i+1 |i∈S∧i<T,a∈M}∪{P i,a,A |i∈S, a∈M} is greater than 0, all other elements are 0, and for i<T, P i,a,i+1 +P i,a,A =1, and P T,a,A =1.

[0296] Optionally, the calculation is performed based on the probability, the reward value, and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state, including:

[0297] Step S1, calculating according to the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state;

[0298] Further, optionally, in step S1, the calculation is performed based on the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state, including:

[0299] S11, calculating the first reward value based on the product of the first state probability and the value of the first optimal value function and obtaining the first reward value at the maximum value;

[0300] S12, determining the first reward value at the maximum value as the value of the optimal value function in the first browsing state;

[0301] S13, determining the variable of the first reward value at the maximum value as the first optimal strategy.

[0302] Specifically, the first reward value is recorded as: R T,a , the probability of transferring from the first browsing state to the absorption state by executing a specific action is recorded as: P T,a,A , the value of the first optimal value function is expressed as: V *(s=A)=0, so the first optimal strategy is denoted as π * (s=T), the value of the first optimal value function is denoted as V * (s=T), the calculation process of the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state is specifically as follows:

[0303]

[0304]

[0305] Optionally, iterative calculation is performed forward according to the first optimal strategy in the order of the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing.

[0306] Specifically, based on the obtained first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state, iterative calculation is performed forward according to the first optimal strategy in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing, as follows:

[0307] Further, optionally, iterative calculation is performed forward according to the first optimal strategy in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing, including:

[0308] respectively obtaining a second reward value for performing a specific action in a second browsing state, a second probability for performing the specific action when transferring from the second browsing state to the first browsing state, and a third probability for performing the specific action when transferring from the second browsing state to the absorbing state; wherein the second browsing state is before the first browsing state in the preset state space;

[0309] The second optimal strategy in the second browsing state and the value of the optimal value function in the second browsing state are obtained by calculating the product of the second reward value, the second probability, the third probability and the value of the first optimal value function.

[0310] Specifically, based on S={1, 2, 3, ..., T, A}, the second browsing state may be T-1. Therefore, the forward iterative calculation involved in the embodiment of the present application is specifically as follows:

[0311]

[0312]

[0313] Among them, π * (s=T-1) is the second optimal strategy in the second browsing state; V * (s=T-1) is the value of the optimal value function in the second browsing state.

[0314] Similarly, the optimal strategies and optimal value functions of each state obtained by other forward iterations are as follows:

[0315]

[0316]

[0317] In an embodiment of the present invention, product information of a plurality of tourism products is displayed on a display page of a tourism platform, wherein the product information includes: a name of the tourism product and a travel route; an operation record of each tourism product is collected, wherein the operation record includes at least one of the following: at least one group of browsing records is obtained by sorting browsing records of at least one user browsing the tourism product according to a preset time series, and at least one group of transaction records is obtained by sorting transaction records of each tourism product according to a preset time series; browsing records whose distance from an end point of the preset time series is less than a preset value are screened from the operation records; corresponding push probabilities are obtained according to the browsing records obtained after screening; and a push strategy is determined according to the push probability, thereby achieving the purpose of modeling the user browsing behavior into a Markov decision process containing an absorbing state and converting it into an SSP problem to quickly solve the optimal strategy, thereby achieving the technical effect of improving the accuracy of the recommended products, and further solving the technical problem that the accumulated revenue obtained based on the browsing operation in the process of recommending products is terminated due to the user's leaving behavior in the prior art, thereby resulting in a low accuracy of the recommended products.

[0318] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0319] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method for recommending data according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a non-volatile storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0320] Example 6

[0321] According to one aspect of the embodiments of the present invention, a device for the method for recommending data in Embodiment 1 is also provided. Figure 8 is a schematic diagram of a device for recommending data according to Embodiment 6 of the present invention. Figure 8 As shown, it includes: a sorting module 82, which is used to sort the browsing records of at least one user according to a preset time sequence to obtain at least one group of browsing records; a first acquisition module 84, which is used to obtain browsing records whose distance from the end point of the preset time sequence is less than a preset value from the sorted at least one group of browsing records; a second acquisition module 86, which is used to obtain the corresponding push probability according to the browsing records; and a third acquisition module 88, which is used to obtain the corresponding push strategy according to the push probability.

[0322] Example 7

[0323] According to an embodiment of the present invention, a device for implementing the method for recommending data in the above embodiment 2 is also provided. Fig. 9 is a schematic diagram of a device for recommending data according to Embodiment 7 of the present invention. Fig. 9 As shown, the device for recommending data provided by the embodiment of the present application includes: a first acquisition module 92, which is used to obtain the last browsing state in the browsing history of at least one user, and determine the browsing state as an absorbing state; a second acquisition module 94, which is used to respectively obtain the probability of executing a specific action when each browsing state in the browsing history is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state; a calculation module 96, which is used to calculate according to the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state; a recommendation module 98, which is used to recommend corresponding content to the user according to the value of the optimal value function and the optimal strategy.

[0324] Example 8

[0325] According to one aspect of an embodiment of the present invention, a device corresponding to a method for recommending data in embodiment 3 is provided, the device comprising: a display module, used to display a plurality of products in a display page of an online shopping platform; a collection module, used to collect operation records of each product displayed in the display page, wherein the operation records comprise at least one of the following: at least one group of browsing records obtained by sorting browsing records of at least one user browsing the product according to a preset time sequence, and at least one group of transaction records obtained by sorting transaction records of each product according to a preset time sequence; a screening module, used to screen browsing records whose distance from an end point of the preset time sequence is less than a preset value from the operation records; an acquisition module, used to obtain corresponding push probabilities based on the browsing records obtained after screening; and a recommendation module, used to determine a push strategy based on the push probability.

[0326] Example 9

[0327] According to one aspect of an embodiment of the present invention, a device corresponding to a method for recommending data in Example 4 is provided, the device comprising: a display module, used to display a set of driving routes from a departure point to a destination in a navigation page, wherein the driving route set comprises a plurality of driving routes; a collection module, used to collect operation records of each driving route displayed in the navigation page, wherein the operation records comprise: sorting the driving routes selected by at least one user according to a preset time sequence to obtain at least one group of driving records; a screening module, used to screen the operation records to obtain driving records whose distance from an end point of the preset time sequence is less than a preset value; an acquisition module, used to obtain a corresponding push probability based on the driving records obtained after screening; and a recommendation module, used to determine a push strategy based on the push probability.

[0328] Example 10

[0329] According to one aspect of an embodiment of the present invention, there is provided an apparatus for a method for recommending data corresponding to Embodiment 5, the apparatus comprising: a display module for displaying product information of a plurality of tourism products in a display page of a tourism platform, wherein the product information comprises: a name of the tourism product and a tourism route; a collection module for collecting operation records of each tourism product, wherein the operation records comprise at least one of the following: at least one group of browsing records obtained by sorting browsing records of at least one user browsing the tourism product according to a preset time series, and at least one group of transaction records obtained by sorting transaction records of each tourism product according to a preset time series; a screening module for screening browsing records whose distance from an end point of the preset time series is less than a preset value from the operation records; an acquisition module for acquiring corresponding push probabilities according to the browsing records obtained after screening; and a recommendation module for determining a push strategy according to the push probability.

[0330] Embodiment 11

[0331] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided, the non-volatile storage medium including a stored program, wherein when the program is running, a method for controlling a device where the non-volatile storage medium is located to execute the recommended data in the above-mentioned embodiment 1 or embodiment 2 is provided.

[0332] Example 12

[0333] According to yet another aspect of an embodiment of the present invention, a processor is provided. The processor is used to run a program. When the program is run, the method for recommending data in the above-mentioned embodiment 1 or embodiment 2 is executed.

[0334] Embodiment 13

[0335] The embodiment of the present invention further provides a non-volatile storage medium. Optionally, in this embodiment, the non-volatile storage medium can be used to store the program code executed by the method for recommending data provided in the first embodiment.

[0336] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0337] Optionally, in this embodiment, the non-volatile storage medium is configured to store program codes for executing the following steps: obtaining the last browsing state in the historical browsing record, and determining the browsing state as an absorbing state; obtaining the probability of performing a specific action when each browsing state in the historical browsing record is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state; calculating based on the probability, reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy for each browsing state; recommending corresponding content to the user based on the value of the optimal value function and the optimal strategy.

[0338] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: obtaining a reward value for performing a specific action in an absorbing state, wherein the reward value is 0; obtaining a value of a first optimal value function in an absorbing state, wherein the value of the first optimal value function is 0.

[0339] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: respectively obtaining the probability of performing a specific action when each browsing state in the historical browsing record is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state include: respectively obtaining the first reward value for performing a specific action in the first browsing state, the first state probability of transferring from performing the specific action in the first browsing state to the absorbing state, and the value of the first optimal value function in the absorbing state.

[0340] Further, optionally, in this embodiment, the non-volatile storage medium is configured to store program codes for executing the following steps: obtaining a state space before respectively obtaining a first reward value for performing a specific action in the absorbing state, a first state probability of transferring from the absorbing state to the first state by performing the specific action, and a value of the first optimal value function in the first state, wherein the state space includes a set of browsed card numbers and absorbing states; obtaining an action space, wherein the action space includes a set of recommended content; obtaining a state transition matrix, wherein the probability of transition between adjacent states in the state transition matrix when performing a specific action and the probability of transferring from the current state to the absorbing state to perform the specific action are greater than 0, and the remaining probabilities are all 0.

[0341] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: calculating according to the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state, including: calculating according to the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state; iterating forward according to the order in the preset state space according to the first optimal strategy to obtain the value of the optimal value function and the optimal strategy when the user is browsing.

[0342] Further, optionally, in the present embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: calculating according to the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state, including: calculating according to the product of the first state probability and the value of the first optimal value function and the first reward value to obtain the first reward value at the maximum value; determining the first reward value at the maximum value as the value of the optimal value function in the first browsing state; determining the variable of the first reward value at the maximum value as the first optimal strategy.

[0343] Optionally, in this embodiment, the non-volatile storage medium is configured to store program code for executing the following steps: performing iterative calculations forward in an order in a preset state space according to a first optimal strategy to obtain the value of the optimal value function of the user when browsing and the optimal strategy including: obtaining a second reward value for performing a specific action in a second browsing state, a second probability for performing a specific action when transferring from the second browsing state to the first browsing state, and a third probability for performing a specific action when transferring from the second browsing state to the absorbing state; wherein, in the preset state space, the second browsing state is before the first browsing state; calculating according to the product of the second reward value, the second probability, and the third probability and the value of the first optimal value function to obtain a second optimal strategy in the second browsing state and the value of the optimal value function in the second browsing state.

[0344] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0345] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0346] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0347] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0348] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0349] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a non-volatile storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned non-volatile storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0350] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for recommending data, characterized in that: include: sorting the browsing records of at least one user according to a preset time sequence to obtain at least one set of browsing records; Obtaining browsing records whose distance from the end point of the preset time sequence is less than a preset value from the at least one group of sorted browsing records, wherein the preset time sequence is a time sequence consisting of multiple time points from the moment when the user leaves the browsing process to the preset moment; Obtaining a corresponding push probability according to the browsing history; Obtaining a corresponding push strategy according to the push probability; Among them, obtaining the corresponding push probability based on the browsing history includes: obtaining the browsing status of the browsing history; obtaining the probability of the user transferring from each browsing status to the absorption state when performing a specific action based on the browsing status, wherein the absorption state is the state when the user leaves when browsing the recommended object; and determining the probability as the push probability.

2. The method according to claim 1, characterized in that: The method further comprises: A reward value for executing a specific action in the absorbing state and a value of a first optimal value function in the absorbing state are obtained.

3. The method according to claim 2, characterized in that The acquiring of the corresponding push strategy according to the push probability includes: The push probability, the reward value and the value of the absorption state and each browsing state are calculated according to the preset time sequence to obtain the push strategy, wherein the push strategy includes: a push object and a method of displaying the push object through a client; Among them, the push object includes: a first type of push object or a second type of push object; the first type of push object is an object with the same attributes as the push object in the at least one group of browsing records; the second type of push object is an object with different attributes from the push object in the at least one group of browsing records.

4. The method according to claim 3, characterized in that The method further comprises: In the case where the push object is a first type of push object, displaying, through the client, a push object with the same attributes as the push object in the at least one group of browsing records; In the case that the push object is a push object of the second type, a push object having different attributes from those of the push objects in the at least one group of browsing records is displayed through the client.

5. The method according to claim 3, characterized in that: The push strategy also includes: associating corresponding push objects according to the user's historical consumption capacity.

6. A method for recommending data, characterized in that: include: Acquire the last browsing state in the browsing history of at least one user, and determine the browsing state as an absorbing state; Respectively obtaining the probability of executing a specific action when each browsing state in the browsing record is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state; Calculating according to the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state; Recommending corresponding content to the user according to the value of the optimal value function and the optimal strategy; The method comprises: obtaining browsing records whose distance from the end point of a preset time sequence is less than a preset value from the sorted browsing records, wherein the preset time sequence is a time sequence consisting of multiple time points from the moment when the user leaves the browsing process to a preset time.

7. The method according to claim 6, characterized in that The respectively obtaining the probability of executing a specific action when each browsing state in the browsing record is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state include: A first reward value for performing a specific action in a first browsing state, a first state probability of transferring to an absorbing state by performing the specific action in the first browsing state, and a value of a first optimal value function in the absorbing state are respectively obtained.

8. The method according to claim 7, characterized in that Before respectively acquiring a first reward value for performing a specific action in a first browsing state, a first state probability of transferring to an absorbing state by performing the specific action in the first browsing state, and a value of a first optimal value function in the absorbing state, the method further includes: Acquire a state space, wherein the state space includes a set of browse card serial numbers and the absorption state; Acquire an action space, wherein the action space includes a recommended content set; A state transfer matrix is ​​obtained, wherein the probability of transition between adjacent states in the state transfer matrix when performing the specific action and the probability of transition from the current state to the absorbing state when performing the specific action are greater than 0, and the remaining probabilities are all 0.

9. The method according to claim 7, characterized in that: The calculation based on the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state includes: Calculating according to the first reward value, the first state probability and the value of the first optimal value function to obtain a first optimal strategy in the first browsing state and a value of the optimal value function in the first browsing state; According to the first optimal strategy, iterative calculation is performed forward in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing.

10. The method according to claim 9, characterized in that The calculating according to the first reward value, the first state probability and the value of the first optimal value function to obtain the first optimal strategy in the first browsing state and the value of the optimal value function in the first browsing state includes: Calculating the first reward value according to the product of the first state probability and the value of the first optimal value function and obtaining the first reward value at the maximum value; Determining the first reward value at the maximum value as the value of the optimal value function in the first browsing state; The variable of the first reward value at the maximum value is determined as the first optimal strategy.

11. The method according to claim 9, characterized in that According to the first optimal strategy, iterative calculation is performed forward in the order in the preset state space to obtain the value of the optimal value function and the optimal strategy when the user is browsing, including: respectively obtaining a second reward value for executing the specific action in a second browsing state, a second probability for executing the specific action when transferring from the second browsing state to the first browsing state, and a third probability for executing the specific action when transferring from the second browsing state to the absorbing state; wherein the second browsing state is before the first browsing state in the preset state space; The second optimal strategy in the second browsing state and the value of the optimal value function in the second browsing state are obtained by calculating the product of the second reward value, the second probability, the third probability and the value of the first optimal value function.

12. A method for recommending data, characterized in that: include: Display multiple products on the display page of the online shopping platform; Collecting operation records of each product displayed in the display page, wherein the operation records include at least one of the following: at least one set of browsing records obtained by sorting browsing records of at least one user browsing products according to a preset time sequence, and at least one set of transaction records obtained by sorting transaction records of each product according to a preset time sequence, wherein the preset time sequence is a time sequence consisting of multiple time points from the moment when the user leaves the browsing process to the preset moment; Filtering the operation records to obtain operation records whose distance from the end point of the preset time series is less than a preset value; According to the operation records obtained after screening, the corresponding push probability is obtained; Determining a push strategy according to the push probability; Among them, according to the operation records obtained after screening, the corresponding push probability is obtained, including: obtaining the status of the operation record; according to the status, obtaining the probability of the user transferring from each state to the absorption state when performing a specific action, wherein the absorption state is the state when the user leaves when browsing the recommended object; and determining the probability as the push probability.

13. A method for recommending data, characterized in that: include: Displaying a set of driving routes from a departure point to a destination on a navigation page, wherein the set of driving routes includes a plurality of driving routes; Collecting an operation record of each driving route displayed in the navigation page, wherein the operation record includes: sorting the driving routes selected by at least one user according to a preset time sequence to obtain at least one set of driving records; Filtering the operation records to obtain operation records whose distance from the end point of the preset time sequence is less than a preset value from the operation records, wherein the preset time sequence is a time sequence consisting of multiple time points from the moment when the user leaves the browsing to the preset moment; According to the operation records obtained after screening, the corresponding push probability is obtained; Determining a push strategy according to the push probability; Among them, according to the operation records obtained after screening, the corresponding push probability is obtained, including: obtaining the status of the operation record; according to the status, obtaining the probability of the user transferring from each state to the absorption state when performing a specific action, wherein the absorption state is the state when the user leaves when browsing the recommended object; and determining the probability as the push probability.

14. A method for recommending data, characterized in that: include: Displaying product information of a plurality of travel products on a display page of a travel platform, wherein the product information includes: a name of the travel product and a travel route; Collecting operation records of each tourism product, wherein the operation records include at least one of the following: at least one set of browsing records obtained by sorting browsing records of at least one user browsing the tourism product according to a preset time sequence, and at least one set of transaction records obtained by sorting transaction records of each tourism product according to a preset time sequence; Filtering the operation records to obtain operation records whose distance from the end point of the preset time sequence is less than a preset value from the operation records, wherein the preset time sequence is a time sequence consisting of multiple time points from the moment when the user leaves the browsing to the preset moment; According to the operation records obtained after screening, the corresponding push probability is obtained; Determining a push strategy according to the push probability; Among them, according to the operation records obtained after screening, the corresponding push probability is obtained, including: obtaining the status of the operation record; according to the status, obtaining the probability of the user transferring from each state to the absorption state when performing a specific action, wherein the absorption state is the state when the user leaves when browsing the recommended object; and determining the probability as the push probability.

15. A device for recommending data, characterized in that: include: A sorting module, used to sort the browsing records of at least one user according to a preset time sequence to obtain at least one group of browsing records; A first acquisition module is used to acquire, from the sorted at least one group of browsing records, browsing records whose distance from the end point of the preset time sequence is less than a preset value, wherein the preset time sequence is a time sequence consisting of multiple time points from the moment when the user leaves the browsing process to the preset moment; A second acquisition module, used to acquire a corresponding push probability according to the browsing history; A third acquisition module is used to acquire a corresponding push strategy according to the push probability; Among them, the second acquisition module is used to obtain the corresponding push probability based on the browsing history through the following steps: obtaining the browsing status of the browsing history; obtaining the probability of the user transferring from each browsing state to the absorption state when performing a specific action based on the browsing status, wherein the absorption state is the state when the user leaves when browsing the recommended object; and determining the probability as the push probability.

16. A device for recommending data, characterized in that: include: A first acquisition module, used for acquiring the last browsing state in the browsing history of at least one user, and determining the browsing state as an absorbing state; A second acquisition module is used to respectively acquire the probability of executing a specific action when each browsing state in the browsing record is transferred to the absorbing state, the reward value of each browsing state and the value of the first optimal value function in the absorbing state; A calculation module, used to calculate according to the probability, the reward value and the value of the first optimal value function to obtain the value of the optimal value function and the optimal strategy in each browsing state; A recommendation module, used for recommending corresponding content to users according to the value of the optimal value function and the optimal strategy; The device is also used to perform the following steps: obtaining browsing records whose distance from the end point of a preset time sequence is less than a preset value from the sorted browsing records, wherein the preset time sequence is a time sequence consisting of multiple time points from the moment when the user leaves the browsing process to the preset time.

17. A non-volatile storage medium, the non-volatile storage medium comprising a stored program, wherein: A method for recommending data according to any one of claims 1 to 14, controlling the device where the non-volatile storage medium is located to execute when the program is running.

18. A processor for running a program, wherein: When the program is run, the method for recommending data described in any one of claims 1 to 14 is executed.