A recommendation method, computing device and storage medium

Abstract

The present application embodiment discloses a recommendation method, computing device and storage medium. A specific implementation of the method includes: obtaining user operation data from a first terminal; in response to the user operation data, calling at least one recommendation algorithm to calculate at least one first recommendation result; displaying user information and the algorithm identifier of the called recommendation algorithm on the display device of the computer device or the second terminal; in response to the operation of the algorithm identifier, displaying the first recommendation result information calculated by the corresponding recommendation algorithm on the display device of the computer device or the second terminal. The recommendation method of this implementation can realize the tracking of the intermediate process and intermediate results of the recommendation method, analyze the performance of the recommendation method, and facilitate personnel to analyze and correct in real time.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and more specifically, to a recommendation method, a computing device, and a storage medium. Background Art

[0002] With the development of the Internet, the number of Internet users has increased rapidly, and the number of Internet websites providing content services to users has also increased. In order to better operate Internet websites and serve Internet website users, personalized information (content) recommendation systems based on user preferences have emerged.

[0003] The recommended results of the recommendation systems in the prior art are often attached to a specific product, and most of the recommended results are only displayed on the user's interactive interface (i.e., the user's electronic device). This display method makes the recommendation system similar to a black box. Operators have no way of knowing what user characteristics and behavioral attributes the recommendation result is based on, and it is difficult to perceive the recommendation logic and strategy in the recommendation system, making it difficult to explain the reason for the recommendation result. Summary of the invention

[0004] The purpose of the present application is to provide a recommendation method, a computing device and a storage medium to solve at least one of the problems existing in the prior art.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] The first aspect of the present application provides a recommendation method, which is applied to a computer device, comprising:

[0007] Acquiring user operation data from the first terminal;

[0008] In response to the user operation data, calling at least one recommendation algorithm to calculate and obtain at least one first recommendation result;

[0009] Displaying user information and an algorithm identifier of the called recommendation algorithm on a display device of the computer device or a second terminal;

[0010] In response to the operation on the algorithm identifier, first recommendation result information calculated by the corresponding recommendation algorithm is displayed on the display device of the computer device or the second terminal.

[0011] In one embodiment, in response to the user operation data, calling at least one recommendation algorithm to calculate at least one first recommendation result includes:

[0012] In response to the user operation data, the user type, application scenario and recommendation strategy are determined in sequence, and at least one recommendation algorithm is called according to the recommendation strategy to calculate and obtain at least one first recommendation result.

[0013] In one embodiment, the method further comprises:

[0014] Displaying the determined type identifier of the user type, the determined scenario identifier of the application scenario, and the determined policy identifier of the recommended policy on the display device of the computer device or the second terminal;

[0015] In response to the operation of the type identifier, scenario identifier or policy identifier, the corresponding user type information, application scenario information or recommended policy information is displayed on the display device of the computer device or the second terminal.

[0016] In one embodiment, the method further comprises:

[0017] De-duplicate and sort the at least one first recommendation result to generate a second recommendation result.

[0018] In one embodiment, the method further comprises:

[0019] Displaying a processing identifier of the deduplication and sorting process on a display device of the computer device or a second terminal;

[0020] In response to the operation on the processing identifier, deduplication and sorting processing information is displayed on the display device of the computer device or the second terminal.

[0021] In one embodiment, the method further comprises:

[0022] Displaying a result identification of the second recommendation result on the display device of the computer device or the second terminal;

[0023] In response to the operation on the result identifier, second recommendation result information is displayed on the display device of the computer equipment or the second terminal.

[0024] In one embodiment, the method further comprises:

[0025] Sending a second recommendation result to the first terminal;

[0026] User feedback information from the first terminal is obtained, and a contribution value of the recommended strategy is calculated according to the user feedback information and displayed on a display device of the computer device or a second terminal.

[0027] A second aspect of the present application provides a computer device, comprising:

[0028] An acquisition module, used to acquire user operation data from the first terminal;

[0029] A recommendation module, configured to, in response to the user operation data, call at least one recommendation algorithm to calculate and obtain at least one first recommendation result;

[0030] An interactive module is used to display user information and the algorithm identifier of the called recommendation algorithm on the display device of the computer device or the second terminal, and in response to the operation of the algorithm identifier, display the first recommendation result information calculated by the corresponding recommendation algorithm on the display device of the computer device or the second terminal.

[0031] In one embodiment, the recommendation module is used to respond to the user operation data and call at least one recommendation algorithm to calculate at least one first recommendation result, including: responding to the user operation data, determining the user type, application scenario and recommendation strategy in turn, and calling at least one recommendation algorithm according to the recommendation strategy to calculate at least one first recommendation result.

[0032] In one embodiment, the interaction module is also used to display the type identification of the determined user type, the scenario identification of the determined application scenario, and the policy identification of the determined recommended strategy on the display device of the computer device or the second terminal; in response to the operation of the type identification, scenario identification or policy identification, the corresponding user type information, application scenario information or recommended strategy information is displayed on the display device of the computer device or the second terminal.

[0033] In one embodiment, the recommendation module is further used to perform deduplication and sorting processing on the at least one first recommendation result to generate a second recommendation result.

[0034] In one embodiment, the interactive module is also used to display the processing identifier of the deduplication and sorting processing on the display device of the computer device or the second terminal; in response to the operation of the processing identifier, the deduplication and sorting processing information is displayed on the display device of the computer device or the second terminal.

[0035] In one embodiment, the interaction module is further used to display a result identifier of the second recommendation result on the display device of the computer device or the second terminal; in response to an operation on the result identifier, the second recommendation result information is displayed on the display device of the computer device or the second terminal.

[0036] In one embodiment, the computer device further includes a sending module and a computing module;

[0037] The sending module is used to send the second recommendation result to the first terminal;

[0038] The acquisition module is further used to acquire user feedback information from the first terminal;

[0039] The calculation module is used to calculate the contribution value of the recommendation strategy according to the user feedback information;

[0040] The interaction module is further used to display the contribution value of the recommended strategy on the display device of the computer equipment or the second terminal.

[0041] The third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the present application.

[0042] The beneficial effects of this application are as follows:

[0043] In response to the existing problems, this application provides a recommendation method, computing device and storage medium. The recommendation method can clearly and intuitively display the different attributes and preferences of different users by achieving the systematization and visualization of the recommendation process of the recommendation method, completing the tracking of the intermediate process and intermediate results of the recommendation method, and tracking the execution process of the recommendation method and analyzing the performance of the recommendation method, so as to facilitate the operation personnel to analyze the method more macroscopically and make timely improvements and corrections. In addition, the recommendation method can also trace the recommendation results based on multiple intermediate results in the execution process, which helps to provide a recommendation explanation for the recommendation results, and at the same time measure the contribution of different recommendation strategies, so as to facilitate the ABTest evaluation of the performance of the recommendation method and the operation content decision. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings.

[0045] Figure 1 An exemplary system architecture diagram is shown in which one embodiment of the present application can be applied.

[0046] Figure 2 A flow chart showing a recommended method in one embodiment of the present application.

[0047] Figure 3 A business flow diagram showing a recommendation method in one embodiment of the present application.

[0048] Figure 4 A schematic diagram showing a directed acyclic graph in one embodiment of the present application.

[0049] Figure 5 A schematic diagram of the structure of a computer device according to an embodiment of the present application is shown.

[0050] Figure 6 A schematic diagram showing the structure of a computer system that implements a computer device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0051] In order to more clearly illustrate the present application, the present application is further described below in conjunction with the embodiments and drawings. Similar components in the drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the content specifically described below is illustrative rather than restrictive, and should not be used to limit the scope of protection of the present application.

[0052] It should be noted that, in the description of the present application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of further restrictions, the elements defined by the statement "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0053] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the proposed method of the present application may be applied.

[0054] like Figure 1 As shown, the system architecture 100 includes first terminal devices 101, 102, 103, a network 104, a server 105, a network 106 and second terminal devices 107, 108, 109, the network 104 is used to provide a medium for a communication link between the first terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, optical fiber cables, and 5G networks, etc. The network 106 is used to provide a medium for a communication link between the server 105 and the second terminal devices 107, 108, 109. The network 106 may include various connection types, such as wired, wireless communication links, optical fiber cables, and 5G networks, etc.

[0055] The user can use the first terminal device 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as image recognition applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0056] The first terminal devices 101, 102, 103 and the second terminal devices 107, 108, 109 can be hardware or software. When the first terminal devices 101, 102, 103 and the second terminal devices 107, 108, 109 are hardware, they can be various electronic devices with display screens and supporting image recognition, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the first terminal devices 101, 102, 103 and the second terminal devices 107, 108, 109 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0057] The server 105 may be a server providing various services. In a specific embodiment, the server 105 is a data server loaded with a recall result database. The recall result database calls a corresponding recall algorithm to calculate the user's historical behavior data, thereby obtaining different calculation results, and saves the calculation results in the recall result database.

[0058] In a specific embodiment, the server 105 may be a server that provides various services, such as a background server that provides support for e-commerce applications on the first terminal devices 101, 102, and 103. The background server may call at least one recommendation algorithm based on the operation data from the first terminal device, generate at least one first recommendation result, and display user information and the algorithm identifier of the called recommendation algorithm on the background server or the second terminal provided with a display device, and display the first recommendation result information.

[0059] It should be noted that the recommendation method provided in the embodiment of the present application is generally executed by the server 105 , and accordingly, the computer device used for the recommendation method is generally set in the server 105 .

[0060] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0061] It should be understood that Figure 1 The number of the first terminal device, network, server and second terminal device in the embodiment is only for illustration. According to the implementation requirements, there may be any suitable number of terminal devices, networks and servers.

[0062] Please continue to refer to Figure 2 , Figure 2 A flowchart showing an embodiment of a recommendation method applied to a computer device according to the present application may include the following steps:

[0063] S201, obtaining user operation data from a first terminal;

[0064] In a specific embodiment, the execution subject (for example, Figure 1 The server 105 shown in the figure can obtain the user operation data from the first terminal locally or remotely through the network 104. In an example where the object of a recommendation method is a commodity or service, the user operation data may include the user clicking, purchasing, browsing, forwarding, and sharing a certain commodity or service on an e-commerce application of the first terminal (e.g., the user's electronic device).

[0065] S202: In response to user operation data, call at least one recommendation algorithm to calculate and obtain at least one first recommendation result.

[0066] In a specific embodiment, the execution subject (for example, Figure 1 The server 105 shown in the figure is provided with a recall result database, wherein the recall result database preliminarily calls at least one corresponding recall algorithm offline for the user's historical behavior data to perform offline calculations, thereby obtaining different offline calculation results, and saves the different offline calculation results in the recall result database for the recommendation algorithm to calculate and obtain at least one first recommendation result. By setting up the recall result database and analyzing the user's historical behavior data, the accuracy of the recommendation method can be improved.

[0067] In a specific embodiment, the recall result database uses the memory computing framework Spark to perform offline recall operations, which can further improve computing efficiency. In another embodiment, the algorithms included in the recall result database include: user-based collaborative filtering algorithm (User-CF), item-based collaborative filtering algorithm (Item-CF), user-based collaborative filtering algorithm considering time context factors (time-User-CF), item-based collaborative filtering algorithm considering time context factors (time-Item-CF), ALS algorithm, Word2vec algorithm and matrix decomposition algorithm.

[0068] In a specific embodiment, the execution subject (for example, Figure 1The server 105 shown in the figure responds to the user operation data, combines the offline calculation results stored in the recall result database and the user operation data, and calls at least one recommendation algorithm to calculate at least one first recommendation result, wherein the recommendation algorithm may include LGBM+LR algorithm (LGBM, Light GBM; LR, Logistic Regression), DNN (Deep Neural Networks) model, DeepFM model (DeepFM model is generated by combining FM (Factorization Machine) model and DNN model), association rule algorithm and sorting algorithm. The above-mentioned various recommendation algorithms can use the offline calculation results in the recall result database to calculate the first recommendation result, thereby obtaining the corresponding first recommendation result.

[0069] S203: Display the user information and the algorithm identifier of the called recommendation algorithm on a display device of the computer device or a second terminal.

[0070] In a specific embodiment, the execution subject (for example, Figure 1 The server 105 shown in the figure) displays the user information and the algorithm identifier of the recommendation algorithm called on the display device of the computer device or sends it to the second terminal (such as the electronic device of the operator) through the network. It is understandable that it can also be directly used in the execution subject of the recommendation method of this embodiment (for example, Figure 1 The display device is set on the server 105 shown in the figure to directly display locally. The user information includes user attribute information (such as the user's name, gender, age, etc.), user behavior information (such as click, purchase, browse, forward, share, etc. operations and product information) and prediction information (such as purchase intention). In addition, the algorithm identifier can be, for example, a display device of a computer device or a virtual button on the display interface of the second terminal.

[0071] In a specific embodiment, the execution subject (for example, Figure 1 The server 105 shown in the figure can establish a multi-dimensional system of user portraits in advance. The user portraits mainly include explicit portraits and implicit portraits, where the explicit portraits mainly include information that is easy to obtain or can be obtained through simple statistical analysis, such as the user's gender, age, number of views, purchase frequency, etc. The implicit portrait of the user portrait mainly includes user information obtained through in-depth analysis and mining of the user's historical behavior, such as demand forecasting analysis results based on the user's timeline search data, label systems based on goods or services, and user preference analysis results based on user historical behavior.

[0072] In a specific embodiment, the user portrait of the multi-dimensional system can be visualized through a visual display interface (such as a data dashboard). By classifying the explicit portrait and implicit portrait labels in the above user portrait, the corresponding pluggable display modules are customized respectively, and the corresponding modules can be operated to perform visual presentation. This embodiment is provided with a user portrait main interface and an optional pane of a visualization function module, and its specific display logic is:

[0073] 1. Display the user's basic information in a list format on the user portrait main interface, such as user avatar, name, gender, age, location, etc.

[0074] 2. In the optional pane of the visualization function module, the information module (such as implicit portrait) obtained by the server through statistical analysis and mining prediction is displayed, including but not limited to the user operation behavior distribution description module, the user preference analysis module based on the item label system and the user demand prediction module based on the user timeline search data.

[0075] Among them, the user operation behavior distribution description module mainly conducts statistical analysis on user behavior data, and displays the user's login frequency, click frequency, purchase frequency, forwarding frequency, sharing frequency, function module usage frequency, average stay time in function modules, etc. through line graphs and bar graphs.

[0076] The user preference analysis module based on the item tag system mainly conducts in-depth mining and analysis of the user's historical behavior logs, and displays the user's preference rankings for preset tag categories through line charts and bar charts. The tags can be preset tags for goods or services, such as "skin care products", "face creams", "new models", "Internet celebrities", etc.

[0077] The user demand prediction module based on user timeline search data combines the user's historical search content and historical purchase of goods or services, uses natural language processing technology to analyze the user's historical search content and match it with the preset label content, and uses association rule mining technology to predict the goods or services that the user may purchase in the future, exploring the user's purchase intention.

[0078] This embodiment displays the user portrait from the perspective of the operator based on the user multi-dimensional portrait such as the user portrait main interface and the optional pane of the visualization function module as described above, such as by displaying it in the form of a radar chart. The dimensions of the radar chart include but are not limited to the user purchasing power based on the user's historical purchasing behavior, the user's purchasing potential based on the prediction of the user's future purchasing behavior, the user's integrity based on the user's historical transaction return record, etc.

[0079] S204: In response to the operation on the algorithm identifier, first recommendation result information calculated by the corresponding recommendation algorithm is displayed on the display device of the computer device or the second terminal.

[0080] In a specific embodiment, the first recommendation result information may include the ranking of commodities with different recommendation probabilities calculated according to the recommendation algorithm and their corresponding commodity information. In one example, the operator clicks a virtual button corresponding to the algorithm identifier on the display device of the computer device or the second terminal, and the execution subject (for example, Figure 1 The server 105 shown responds and displays the first recommendation result information calculated by the recommendation algorithm on the display device of the computer equipment or the second terminal.

[0081] By displaying user information and the algorithm identifier of the called recommendation algorithm on the display device of the computer equipment or the second terminal, the operating personnel can track the intermediate process and intermediate results of the recommendation method, that is, the adopted recommendation algorithm and the first recommendation result information. The recommendation results can also be traced based on multiple intermediate results in the execution process, which helps to provide a recommendation explanation for the recommendation result, so that the operating personnel can analyze the method more macroscopically and make timely improvements and corrections.

[0082] In a specific embodiment, step S202 may specifically include:

[0083] In response to user operation data, the user type, application scenario and recommendation strategy are determined in sequence, and at least one recommendation algorithm is called according to the recommendation strategy to calculate and obtain at least one first recommendation result.

[0084] In a specific embodiment, user types include active users, new users, and tourists; application scenarios include operational recommendation scenarios, personalized recommendation scenarios, and related product or service recommendation scenarios; recommendation strategies include product or service tag preference strategies, hot recommendation strategies, and novelty recommendation strategies. Figure 3-4 The example shown is an example of an execution subject (eg, Figure 1 The server 105 shown responds to a user clicking on a certain product and determines that the user type is an "active user" and the application scenario of the product clicked by the user is a "personalized recommendation scenario". The server 105 calls the recommendation algorithm corresponding to the recommendation strategy, such as the LGBM+LR algorithm, the DeepFM model and the association rule algorithm, according to the recommendation strategy of "preference strategy of the label corresponding to the product or service" and "novelty recommendation strategy" to calculate and obtain three first recommendation results.

[0085] In a specific embodiment, the recommendation method further includes:

[0086] Displaying the determined type identifier of the user type, the determined scenario identifier of the application scenario, and the determined policy identifier of the recommended policy on a display device of the computer device or a second terminal;

[0087] In response to the operation of the type identification, scenario identification or policy identification, the corresponding user type information, application scenario information or recommended policy information is displayed on the display device of the computer equipment or the second terminal.

[0088] The second terminal is, for example, an electronic device of an operator. The type identifier, scenario identifier, and strategy identifier may be, for example, virtual buttons on a display interface. In one example, an operator clicks a virtual button corresponding to a type identifier, scenario identifier, or strategy identifier on a display device of a computer device or a second terminal to execute the recommendation method of this embodiment (for example, Figure 1 The server 105 shown in the figure responds and displays the corresponding information. For example, the operator clicks the virtual button corresponding to the user type identification, and the user type corresponding to the user can be displayed as "active user".

[0089] By displaying the type identification of the determined user type, the scenario identification of the determined application scenario and the strategy identification of the determined recommendation strategy on the display device of the computer equipment or the second terminal, the operating personnel can clearly understand the intermediate result of the recommendation method, that is, the user's type information, the application scenario and the recommendation strategy provided by the recommendation method, etc., so that the operator can know for whom the recommendation is made, in what scenario, and how the recommendation is made. The different attributes and preferences of different users are clearly and intuitively displayed, and the recommendation reasons are provided for the first recommendation result, which is convenient for the operating personnel to conduct a comprehensive analysis.

[0090] In a specific embodiment, the recommendation method further includes:

[0091] S205: De-duplicate and sort at least one first recommendation result to generate a second recommendation result.

[0092] In this embodiment, the execution subject (for example, Figure 1 The server 105 shown in the figure removes duplicates and sorts the first recommendation results calculated by the recommendation algorithms. For example, the same product or service calculated by different recommendation algorithms is removed from duplicates, and then the multiple first recommendation results after removal of duplicates are sorted, so as to integrate and obtain the second recommendation result. In another specific embodiment, corresponding weights are set for all recommendation algorithms, and the first recommendation results calculated by recommendation algorithms with different weights are removed from duplicates and sorted, so as to obtain the second recommendation result. This embodiment can adjust the second recommendation result by adjusting the weights corresponding to different recommendation algorithms in real time according to the operation of the recommendation method, thereby improving the accuracy of the recommendation method.

[0093] In a specific embodiment, the recommendation method further includes:

[0094] S206. Displaying a processing mark of deduplication and sorting processing on the display device of the computer device or the second terminal; wherein the processing mark may be, for example, a virtual button on the display device of the computer device or the display interface of the second terminal.

[0095] S207: In response to the operation on the processing identifier, display the deduplication and sorting processing information on the display device of the computer device or the second terminal. The operation in response to the processing identifier may be, for example, clicking a virtual button corresponding to the processing identifier on the display device of the computer device or the second terminal; in addition, the deduplication and sorting processing information may include deduplication of product or service information and its corresponding recommendation probability, and the ranking of each product or service in the first recommendation results corresponding to different recommendation algorithms.

[0096] By displaying the processing identifier of deduplication and sorting processing on the display device of the computer equipment or the second terminal, the operating personnel can clearly understand the products deduplicated by the recommendation method and the detailed process of deduplication and sorting, that is, understand the deduplication and sorting process of the recommendation method, and further provide a recommendation explanation for the second recommendation result.

[0097] In a specific embodiment, the recommendation method further includes:

[0098] S208: Displaying a result identifier of the second recommendation result on the display device of the computer device or the second terminal; wherein the result identifier may be, for example, a virtual button on the display device of the computer device or the display interface of the second terminal.

[0099] S209: In response to the operation on the result identifier, display the second recommendation result information on the display device of the computer device or the second terminal, wherein the second recommendation result information may include commodities and related information arranged from high to low according to the recommendation probability, and the recommendation probability corresponding to the commodity.

[0100] By displaying the intermediate result, the first recommendation result and the second recommendation result on the display device of the computer equipment or the second terminal, the operating personnel can clearly and intuitively understand the execution process of the recommendation method from beginning to end, which is convenient for the operating personnel to analyze the method more macroscopically and improve and correct the recommendation method in a timely manner.

[0101] In a specific embodiment, the recommendation method further includes:

[0102] S210: Send a second recommendation result to the first terminal.

[0103] In this embodiment, the execution subject (for example, Figure 1The server 105 shown in the figure sends the second recommendation result to the first terminal (such as the user's electronic device) through the network, so that the user can clearly understand the second recommendation result actually recommended by the recommendation method.

[0104] S211: Obtain user feedback information from the first terminal, calculate a contribution value of the recommended strategy according to the user feedback information, and display it on a display device of a computer device or a second terminal.

[0105] In this embodiment, the user provides feedback according to the second recommendation result sent by the server, for example, the user clicks a virtual button of "interested" or "not interested" displayed on the first terminal, and the execution subject (for example, Figure 1 The server 105 shown in the figure records the feedback mark fed back by the user at the corresponding second recommendation result according to the positive feedback or negative feedback generated by the user on the second recommendation result, and synchronously displays it on the display device of the computer device or the display interface of the second terminal. In addition, the collected user feedback data is statistically analyzed, and the contribution of the recommendation strategy corresponding to the second recommendation result to the recommendation method is evaluated from multiple dimensions such as different time periods and different user groups. The calculation formula of the contribution of the recommendation strategy is as follows:

[0106]

[0107] Among them, C tb It is expressed as the contribution of the recommended strategy, T pi It indicates that the second recommendation result calculated corresponding to the recommendation strategy i is accepted by the user, that is, the number of results in which the user generates positive feedback, and N represents the total number of recommendation strategies.

[0108] This embodiment analyzes the contribution of different recommendation strategies to the recommendation method, so that operators can flexibly adjust different recommendation strategies. Developers can tune the recommendation algorithm in a timely manner, which is beneficial to the subsequent optimization and iteration of the recommendation algorithm.

[0109] by Figure 3-4 Taking the embodiment shown as an example, starting from the interaction between the user and the server 105, the execution subject (for example, Figure 1The server 105 shown in the figure first constructs an interactive user portrait for visual presentation, and calls at least one corresponding recall algorithm offline for offline calculation for the user's historical behavior data through the recall result database in advance, thereby obtaining different offline calculation results, and storing the different offline calculation results in the recall result database. Subsequently, in response to the user's operation (click, forward, share) of a certain product or service, the user type (such as active user, new user and tourist) and the application scenario of the product clicked by the user (such as operation recommendation, personalized recommendation and related item recommendation) are judged, thereby obtaining the recommendation strategy corresponding to the user type in the application scenario, and then calling the recommendation algorithm corresponding to the recommendation strategy to calculate and obtain multiple first recommendation results, deduplicate and sort the multiple first recommendation results to obtain the second recommendation result, and send the second recommendation result to the first terminal (such as the user's electronic device), and display the user information in this recommendation method, the algorithm identifier of the called recommendation algorithm, the type identifier, the scenario identifier, the strategy identifier, the processing identifier and the result identifier on the display device of the computer device or the second terminal.

[0110] The recommendation method of this embodiment is a recommendation method triggered by the user's operation. Specifically, it visualizes who is recommended, what method is used, how to recommend, what the recommendation result is, what the intermediate result is, etc., so as to realize the systematization and visualization of the recommendation process of the recommendation method, complete the tracking of the intermediate process and intermediate results of the recommendation method, and can clearly and intuitively display the different attributes and preferences of different users. At the same time, it tracks the execution process of the recommendation method and analyzes the performance of the recommendation method, which is convenient for operators to analyze the method more macroscopically and make timely improvements and corrections. In addition, the recommendation method can also trace the recommendation results based on multiple intermediate results in the execution process. Based on the scene trigger, algorithm trigger and user feedback information obtained by the result traceability, it is helpful to discover user preference scenes and recommendation algorithm modules with high value efficiency, which is helpful to optimize and improve the diversity and accuracy of the recommendation system, and help provide the recommendation explanation of the recommendation result. At the same time, it measures the contribution of different recommendation strategies, which is conducive to the ABTest evaluation of the performance of the recommendation method and the operation content decision. Furthermore, by tracing the second recommendation result and providing the relevant recommendation reasons for the second recommendation result, the interpretability of the recommendation method is enhanced. This recommendation method not only intuitively reflects the execution process of the recommendation method, but also provides effective data support, which helps to discover application scenarios and recommendation algorithms with high value efficiency and optimize the diversity and accuracy of the recommendation method.

[0111] In a specific embodiment, Figure 4As shown, the display device of the computer device or the second terminal displays the above identifications, namely, user information, algorithm identification, user type identification, scenario identification, strategy identification, processing identification and result identification, in the form of a directed acyclic graph. In graph theory, if a directed graph cannot start from a vertex and return to the vertex through several edges, it is called a directed acyclic graph.

[0112] According to the execution process of the above-mentioned actual recommendation method, the recommendation method is abstracted into modules to generate a directed acyclic graph, such as Figure 4 This is a directed acyclic graph diagram of the recommendation method for a certain user, which is mainly divided into two stages, as shown below:

[0113] 1) Module abstraction and encapsulation. For example, user types and application scenarios are abstracted and encapsulated into user type modules and application scenario modules respectively, and different recommendation strategies and different recommendation algorithms are encapsulated into different recommendation strategy modules and different recommendation algorithm modules respectively.

[0114] Before abstracting and encapsulating different modules, maintain the following information for different modules:

[0115] a) Assign a unique event node ID to each abstract encapsulation module, for example, configure the identification code 01 for the user type module, and maintain a set of event node IDs of the previous level and a set of event node IDs of the next level. That is to say, a single module can perceive the set of event node IDs of the previous level module and the set of event node IDs of the next level module.

[0116] b) For each abstract module, the key information output of the module is retained, that is, in response to the operation of the module, the key information output of the module can be displayed in real time, such as retaining the user type result in the user type module, retaining the application scenario set of the user in the scenario identification module, retaining the recommendation strategy set of the recommendation method in each recommendation strategy module, and retaining the sorting result in each recommendation algorithm module, that is, retaining the intermediate result. The above-mentioned intermediate results are uniformly packaged and put into the second recommendation result set as the birth source and recommendation explanation of the second recommendation result for reference by operators.

[0117] 2) Dynamically construct a directed acyclic graph. According to the execution order of the recommended method, a directed acyclic graph is constructed through the chain relationship of the unique event node ID corresponding to each abstract encapsulation module, thereby recording the actual execution status of the recommended method.

[0118] In a specific embodiment, the recommendation method generates a second recommendation result in response to a single operation of a user, and displays it visually on a display device of a computer device or a second terminal, and displays the user information in this recommendation method, the algorithm identifier, type identifier, scenario identifier, strategy identifier, processing identifier and result identifier of the called recommendation algorithm through a directed acyclic graph, and displays the final recommendation result list, which specifically includes:

[0119] 1) Display content of directed acyclic graph

[0120] a) A directed acyclic graph legend, which displays the directed acyclic graph legend of the execution process of the recommendation method generated by the user's operation on the display device of the computer equipment or the display interface of the second terminal, so as to present the name information of the unique event node ID of each abstract module in the recommendation process, as well as the execution order.

[0121] b) Event node output data, design buttons for nodes of directed acyclic graphs of different abstract modules, so as to control the display of key information output by the node, which is in a non-display state by default. For example, corresponding virtual buttons are set in each recommendation algorithm module, and operators can click different virtual buttons to trigger the operation of connecting to the database to find the key information of the corresponding module, so that the page pops up the key information of the module corresponding to the virtual button. It is understandable that when the key information of each module is empty, a short data alarm prompts.

[0122] 2) The second recommendation result list includes:

[0123] a) The birth situation of the recommendation result and the recommendation explanation. For the second recommendation result, the birth situation of the second recommendation result is displayed for each intermediate result (i.e. the first recommendation result, user type, recommendation strategy, recommendation algorithm, application scenario and other information), that is, the application scenario and the source of the recommendation algorithm of the second recommendation result are traced, and the explanation of the second recommendation result is generated. For example, the second recommendation result is generated by the similarity, popularity or friends' likes of the item.

[0124] b) User feedback mark: According to the positive or negative feedback generated by the user on the second recommendation result, the user feedback mark is recorded in the corresponding second recommendation result and synchronously displayed on the display device of the computer equipment or the display interface of the second terminal.

[0125] c) Contribution analysis of the recommendation strategy. By collecting, counting and analyzing the positive or negative feedback generated by users for the second recommendation result, the sharing degree of the recommendation strategy for this recommendation method can be evaluated from multiple dimensions such as different time periods and different user groups. The contribution calculation formula of the recommendation strategy is as follows:

[0126]

[0127] Among them, C tb It is expressed as the contribution of the recommended strategy, T pi It indicates that the second recommendation result calculated corresponding to the recommendation strategy i is accepted by the user, that is, the number of results in which the user generates positive feedback, and N represents the total number of recommendation strategies.

[0128] This embodiment can effectively track the execution paths of different modules in the recommendation method by constructing a directed acyclic graph, and retain key intermediate results in different modules, thereby achieving effective monitoring of the recommendation method.

[0129] In a specific embodiment, in response to multiple operation data of the user, the recommendation method can display multiple second recommendation results on the visual interface, and at the same time, a threshold for the number of displayed information can be set on the visual interface, for example, only the second recommendation result information of the most recent 10 operations is displayed. At the same time, for the user's historical behavior data, a historical operation recommendation button can be set to retrieve the corresponding second recommendation result information in the database for instant query results display, so that the operator can analyze the performance of the recommendation method in a more macroscopic way.

[0130] Please refer to Figure 5 , as a pair Figure 2 In order to implement the recommended method shown in FIG. 1 , another embodiment of the present application provides a computer device 300. The embodiment of the computer device 300 is similar to Figure 2 Corresponding to the illustrated recommendation method embodiment, the computer device 300 can be specifically applied in a server. The computer device 300 includes an acquisition module 301, which is used to acquire user operation data from a first terminal; a recommendation module 302, which is used to call at least one recommendation algorithm to calculate at least one first recommendation result in response to the user operation data; an interaction module 303, which is used to display user information and the algorithm identifier of the called recommendation algorithm on the display device of the computer device 300 or the second terminal, and in response to the operation of the algorithm identifier, display the first recommendation result information calculated by the corresponding recommendation algorithm on the display device of the computer device 300 or the second terminal.

[0131] In a specific embodiment, the recommendation module 302 is used to respond to user operation data and call at least one recommendation algorithm to calculate at least one first recommendation result, including: responding to user operation data, determining the user type, application scenario and recommendation strategy in turn, and calling at least one recommendation algorithm according to the recommendation strategy to calculate at least one first recommendation result.

[0132] In a specific embodiment, the interaction module 303 is also used to display the type identification of the determined user type, the scenario identification of the determined application scenario, and the policy identification of the determined recommended strategy on the display device of the computer device 300 or the second terminal; in response to the operation of the type identification, scenario identification or policy identification, the corresponding user type information, application scenario information or recommended strategy information is displayed on the display device of the computer device 300 or the second terminal.

[0133] In a specific embodiment, the recommendation module 302 is further configured to perform deduplication and sorting processing on at least one first recommendation result to generate a second recommendation result.

[0134] In a specific embodiment, the interactive module 303 is also used to display the processing identifier of the deduplication and sorting processing on the display device or the second terminal of the computer device 300; in response to the operation of the processing identifier, the deduplication and sorting processing information is displayed on the display device or the second terminal of the computer device 300.

[0135] In a specific embodiment, the interaction module 303 is also used to display the result identification of the second recommendation result on the display device of the computer device 300 or the second terminal; in response to the operation of the result identification, the second recommendation result information is displayed on the display device of the computer device 300 or the second terminal.

[0136] In a specific embodiment, the computer device 300 also includes a sending module 304 and a calculating module 305; the sending module 304 is used to send the second recommendation result to the first terminal; the obtaining module 301 is also used to obtain user feedback information from the first terminal; the calculating module 305 is used to calculate the contribution value of the recommendation strategy based on the user feedback information; the interactive module 303 is also used to display the contribution value of the recommendation strategy on the display device of the computer device or the second terminal.

[0137] It should be noted that the principle and working process of the computer device provided in this embodiment are similar to those of the above-mentioned recommended method. For relevant details, please refer to the above description and will not be repeated here.

[0138] like Figure 6 As shown, a computer system suitable for implementing the recommended method provided by the above embodiment includes a central processing module (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage part into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The CPU, ROM and RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0139] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0140] In particular, according to this embodiment, the process described in the flowchart above can be implemented as a computer software program. For example, this embodiment includes a computer program product, which includes a computer program tangibly contained on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium.

[0141] The flow chart and schematic diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product of the present embodiment. In this regard, each square box in the flow chart or schematic diagram can represent a part of a module, program segment or code, and a part of the above-mentioned module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the schematic diagram and / or flow chart, and the combination of the square boxes in the schematic diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] As another aspect, this embodiment further provides a non-volatile computer storage medium, which may be the non-volatile computer storage medium included in the above-mentioned device in the above-mentioned embodiment, or may be a non-volatile computer storage medium that exists independently and is not installed in the terminal. The above-mentioned non-volatile computer storage medium stores one or more computer programs, and when the above-mentioned one or more computer programs are executed by a processor, the above-mentioned device can implement the following operation: Figure 2 The recommended method is described in the embodiment shown.

[0143] Obviously, the above embodiments of the present application are merely examples for clearly illustrating the present application, and are not limitations on the implementation methods of the present application. For ordinary technicians in this field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present application are still within the scope of protection of the present application.

Claims

1. A recommendation method, applied to a computer device, characterized in that: include: Acquiring user operation data from the first terminal; In response to the user operation data, calling at least one recommendation algorithm to calculate and obtain at least one first recommendation result; Displaying user information and an algorithm identifier of the called recommendation algorithm on a display device of the computer device or a second terminal; In response to the operation on the algorithm identifier, displaying first recommendation result information calculated by the corresponding recommendation algorithm on the display device of the computer device or the second terminal; De-duplication and sorting of the at least one first recommendation result to generate a second recommendation result; Sending a second recommendation result to the first terminal; User feedback information from the first terminal is obtained, and a contribution value of the recommended strategy is calculated according to the user feedback information and displayed on a display device of the computer device or a second terminal.

2. The method according to claim 1, characterized in that In response to the user operation data, calling at least one recommendation algorithm to calculate and obtain at least one first recommendation result includes: In response to the user operation data, the user type, application scenario and recommendation strategy are determined in sequence, and at least one recommendation algorithm is called according to the recommendation strategy to calculate and obtain at least one first recommendation result.

3. The method according to claim 2, characterized in that The method further includes: Displaying the determined type identifier of the user type, the determined scenario identifier of the application scenario, and the determined policy identifier of the recommended policy on the display device of the computer device or the second terminal; In response to the operation of the type identification, scenario identification or policy identification, the corresponding user type information, application scenario information or recommended policy information is displayed on the display device of the computer device or the second terminal.

4. The method according to claim 1, characterized in that: The method further includes: Displaying a processing identifier of the deduplication and sorting process on a display device of the computer device or a second terminal; In response to the operation on the processing identifier, deduplication and sorting processing information is displayed on the display device of the computer device or the second terminal.

5. The method according to claim 1, characterized in that The method further includes: Displaying a result identification of the second recommendation result on the display device of the computer device or the second terminal; In response to the operation on the result identifier, second recommendation result information is displayed on the display device of the computer equipment or the second terminal.

6. A computer device comprising: An acquisition module, used to acquire user operation data from the first terminal; A recommendation module, configured to, in response to the user operation data, call at least one recommendation algorithm to calculate at least one first recommendation result, perform deduplication and sorting processing on the at least one first recommendation result, and generate a second recommendation result; an interactive module, configured to display user information and an algorithm identifier of the called recommendation algorithm on a display device of the computer device or a second terminal, and in response to an operation on the algorithm identifier, display first recommendation result information calculated by the corresponding recommendation algorithm on the display device of the computer device or the second terminal; The computer device also includes a sending module and a computing module; The sending module is used to send the second recommendation result to the first terminal; The acquisition module is further used to acquire user feedback information from the first terminal; The calculation module is used to calculate the contribution value of the recommendation strategy according to the user feedback information; The interaction module is further used to display the contribution value of the recommended strategy on the display device of the computer equipment or the second terminal.

7. The computer device according to claim 6, characterized in that The recommendation module is used to respond to the user operation data and call at least one recommendation algorithm to calculate at least one first recommendation result, including: responding to the user operation data, determining the user type, application scenario and recommendation strategy in turn, and calling at least one recommendation algorithm according to the recommendation strategy to calculate at least one first recommendation result.

8. The computer device according to claim 7, characterized in that The interaction module is also used to display the type identification of the determined user type, the scenario identification of the determined application scenario and the policy identification of the determined recommended strategy on the display device of the computer device or the second terminal; in response to the operation of the type identification, scenario identification or policy identification, the corresponding user type information, application scenario information or recommended strategy information is displayed on the display device of the computer device or the second terminal.

9. The computer device according to claim 6, characterized in that The interactive module is also used to display the processing identifier of the deduplication and sorting processing on the display device of the computer device or the second terminal; in response to the operation of the processing identifier, the deduplication and sorting processing information is displayed on the display device of the computer device or the second terminal.

10. The computer device according to claim 6, characterized in that The interactive module is further used to display a result identifier of the second recommendation result on the display device of the computer device or the second terminal; in response to an operation on the result identifier, display the second recommendation result information on the display device of the computer device or the second terminal.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images