Recommendation explanation generation method and device

Abstract

The invention provides a recommendation explanation generation method and device. The method comprises the steps of obtaining target user information and recommendation object information; generating a first prompt based on the user information and the recommendation object information; and taking the first prompt as an input of an explanation generation model, and outputting a recommendation explanation of the recommendation object, the recommendation explanation indicating a recommendation reason of the recommendation object. According to the recommendation explanation generation method provided by the invention, the personalized information of the user is considered, and the personalized recommendation explanation content is generated for the user, so that the generated recommendation explanation is more targeted, and the quality of the recommendation explanation is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a method and device for generating a recommendation explanation. Background Art

[0002] Recommendation systems have become the core of modern user interaction experience. They help users discover valuable and preferred content through personalized recommendations. In this field, collaborative filtering methods based on user-item interaction have always been the mainstream. These methods, from basic matrix calculations to complex deep neural networks, have demonstrated excellent predictive capabilities in key tasks such as click-through rate prediction and score prediction, and have achieved significant recommendation results and benefits. However, these methods usually lack transparency. They only provide recommended content without providing the logic behind the recommendation. Summary of the invention

[0003] The embodiments of the present application provide a method and device for generating a recommendation explanation, which makes the generated recommendation explanation more targeted by considering the personalized information of the user, thereby improving the quality of the recommendation explanation.

[0004] In a first aspect, the present application provides a method for generating a recommendation explanation, including obtaining target user information and recommendation object information; generating a first prompt based on the user information and the recommendation object information; using the first prompt as an input to an explanation generation model, and outputting a recommendation explanation for the recommendation object, wherein the recommendation explanation indicates the reason for recommending the recommendation object.

[0005] This application constructs generation prompts through target user information and recommendation object information, fully considering the user's personalized information, making the generated recommendation explanation more targeted and improving the quality of the recommendation explanation.

[0006] In one possible implementation, the target user information includes user attribute information and historical interaction information, and the historical interaction information indicates the interaction between the target user and the historical recommended objects; the specific implementation of generating the first prompt based on the user information and the recommended object information is: based on the user attribute information, the historical interaction information and the recommended object information, a chain of thought (CoT) prompt is generated.

[0007] Through the Cot prompt design, the explanation generation model based on the large language model is guided to generate users' views and attitudes towards the recommended objects, fully considering user characteristics, providing more details, and enhancing the persuasiveness of the recommendation.

[0008] In another possible implementation, a thought chain prompt is generated based on the user attribute information, the historical interaction information and the recommended object information. A specific implementation is: fill the user attribute information, the historical interaction information and the recommended object information into a preset CoT prompt template to obtain a CoT prompt.

[0009] For example, the CoT prompt template contains a guiding sentence such as "Please analyze the views of...", and the CoT prompt template reserves positions for user attribute information, historical interaction information, and recommended object information. The user attribute information, historical interaction information, and recommended object information are filled into the corresponding reserved positions to construct a CoT prompt. The constructed CoT prompt can guide the interpretation generation model based on the large language model to generate a detailed analysis of the user's attitude and views on the current recommended object, thereby helping users to have a deeper understanding of product characteristics and assisting them in making more informed decisions.

[0010] In another possible implementation, the explanation generation model includes a target large language model (LLM), and the target large language model is fine-tuned based on a proximal policy optimization (PPO) algorithm.

[0011] PPO training is a type of reinforcement learning. The PPO algorithm is used to fine-tune the target LLM. There is no need to construct recommended explanation data for training, which saves a lot of manpower and resources and significantly reduces training costs.

[0012] In another possible implementation, the target large language model is fine-tuned based on the feedback information of the recommended explanations. That is, the recommended explanations generated by the explanation generation model in the present application can be directly connected to the recommendation system for effect verification, form feedback, and further improve the quality of the recommended explanations.

[0013] In another possible implementation, the target large language model is provided with an adapter; the feedback information of the recommended explanation includes a reward corresponding to the recommended explanation, and the reward is used to adjust parameters of the adapter to achieve fine-tuning of the target large language model.

[0014] In another possible implementation, the reward includes an LC alignment reward (LLM-CTR alignment reward); the LC alignment reward is determined based on the following steps: generating a second prompt based on the recommendation explanation; using the second prompt as an input of the large language model, outputting a first click-through rate prediction result, the first click-through rate prediction result indicating an interaction probability between the target user and the recommended object, and determining the LC alignment reward based on the first click-through rate prediction result and a first true label, the first true label indicating whether the target user interacts with the recommended object.

[0015] The LC consistency reward is used to adjust the parameters of the adapter, thereby fine-tuning the target large language model, so that the target large language model can capture the ability that actually affects user decisions and improve the quality of the recommendation explanations generated by the target large language model.

[0016] In another possible implementation, the reward includes an IC alignment reward (ID-CTR alignment reward); the IC alignment reward is determined based on the following steps: using the recommendation explanation, target user information, and recommendation object information as inputs to the click rate prediction model, and outputting a second click rate prediction result; using the target user information and recommendation object information as inputs to the click rate prediction model, and outputting a third click rate prediction result; determining the IC alignment reward based on the second click rate prediction result, the third click rate prediction result, and the second true label. The IC alignment reward is used as feedback information for the recommendation explanation to fine-tune the target large language model, further improving the quality of the recommendation explanation generated by the target large language model.

[0017] In another possible implementation, the recommendation explanation includes one or more of the target user's historical interaction behavior, the target user's preference, and the compatibility between the target user and the recommended object.

[0018] The recommendation explanation generated by the embodiment of the present application takes into account the personalized differences of users and explores the potential needs of users. The recommendation explanation is more detailed, more informative, and more convincing.

[0019] In another possible implementation, the method for generating recommended explanations provided in the present application also includes: processing the recommended explanations to obtain recommended explanation text; displaying the recommended explanation text to the user through a recommended page, the recommended page including a recommended object display area, a slidable text box is set in the recommended object display area, and the recommended explanation text is set in the slidable text box, or a button is set in the recommended object display area, and the recommended explanation text is displayed to the user when the button is triggered.

[0020] Taking into account the fact that the generated recommendation explanation text is long, this application has redesigned the recommendation UI to make full use of the interface space and use the interface space efficiently to ensure that the information of the recommendation explanation is clearly displayed.

[0021] On the second aspect, the present application also provides a device for generating recommendation explanations, including an acquisition module, a prompt module and an explanation generation module, wherein the acquisition module is used to acquire target user information and recommendation object information; the prompt module is used to generate a first prompt based on the user information and the recommendation object information; the explanation generation module is used to use the first prompt as an input of an explanation generation model, and output a recommendation explanation of the recommendation object, wherein the recommendation explanation indicates the reason for recommending the recommended object.

[0022] In a possible implementation, the target user information includes user attribute information and historical interaction information, and the historical interaction information indicates the interaction between the target user and the historical recommended objects; the prompt module is specifically used to generate a CoT prompt based on the user attribute information, the historical interaction information and the recommended object information.

[0023] In another possible implementation, a thought chain prompt is generated based on the user attribute information, the historical interaction information and the recommended object information. A specific implementation is: fill the user attribute information, the historical interaction information and the recommended object information into a preset CoT prompt template to obtain a CoT prompt.

[0024] In another possible implementation, the explanation generation model includes a target large language model, and the target large language model is fine-tuned based on the end policy optimization PPO algorithm.

[0025] In another possible implementation, the target large language model is fine-tuned based on the feedback information of the recommended explanations.

[0026] In another possible implementation, the target large language model is provided with an adapter; the feedback information of the recommended explanation includes a reward corresponding to the recommended explanation, and the reward is used to adjust parameters of the adapter to achieve fine-tuning of the target large language model.

[0027] In another possible implementation, the reward includes an LC consistency reward; the LC consistency reward is determined based on the following steps: generating a second prompt based on the recommendation explanation; using the second prompt as an input to the large language model, outputting a first click-through rate prediction result, the first click-through rate prediction result indicating the probability of interaction between the target user and the recommended object, and determining the LC consistency reward based on the first click-through rate prediction result and a first true label, the first true label indicating whether the target user interacts with the recommended object.

[0028] In another possible implementation, the reward includes an IC consistency reward; the IC consistency reward is determined based on the following steps: taking the recommendation explanation, target user information and recommended object information as inputs to a click-through rate prediction model, and outputting a second click-through rate prediction result; taking the target user information and recommended object information as inputs to a click-through rate prediction model, and outputting a third click-through rate prediction result; determining the IC consistency reward based on the second click-through rate prediction result, the third click-through rate prediction result and the second true label.

[0029] In another possible implementation, the recommendation explanation includes one or more of the target user's historical interaction behavior, the target user's preference, and the compatibility between the target user and the recommended object.

[0030] In another possible implementation, the recommended explanation generation device provided by the present application also includes a text processing module: the text processing module is used to process the recommended explanation to obtain a recommended explanation text; the recommended explanation text is displayed to the user through a recommendation page, and the recommendation page includes a recommended object display area, a slidable text box is set in the recommended object display area, and the recommended explanation text is set in the slidable text box, or a button is set in the recommended object display area, and the recommended explanation text is displayed to the user when the button is triggered.

[0031] In a third aspect, an embodiment of the present application provides a computing device, comprising a memory and a processor, wherein the memory stores instructions, and when the instructions are executed by the processor, the method described in the first aspect or any possible implementation of the first aspect is implemented.

[0032] In a fourth aspect, the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0033] In a fifth aspect, the present application provides a computer-readable storage medium, including computer program instructions, when the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect. Exemplarily, the computing device cluster may include one or more computing devices.

[0034] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to perform the method described in the first aspect or any possible implementation of the first aspect. Exemplarily, the computing device cluster may include one or more computing devices.

[0035] In a seventh aspect, an embodiment of the present application further provides a chip, comprising at least one processor and a communication interface, wherein the processor is used to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0036] It can be understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram showing the principle of a traditional recommendation system;

[0038] Figure 2 and Figure 3Schematic diagrams of recommendation pages of a group purchase APP and a shopping APP respectively showing an explainable recommendation system in the related art;

[0039] Figure 4 A schematic diagram of the implementation process of an explainable recommendation system in the related art is shown;

[0040] Figure 5 A schematic diagram of a system architecture is shown;

[0041] Figure 6 The system architecture of a terminal-cloud system to which the method for generating a recommendation explanation provided in an embodiment of the present application can be applied is shown;

[0042] Figure 7 A schematic diagram of an implementation architecture of a method for generating a recommended explanation provided in an embodiment of the present application is shown;

[0043] Figure 8 A schematic diagram of an architecture for explaining the training of a generative model provided in an embodiment of the present application is shown;

[0044] Fig. 9 A schematic diagram of constructing a CoT prompt is shown;

[0045] Fig.10 Another schematic diagram of the architecture of the training of the interpretation generation model provided by the embodiment of the present application is shown;

[0046] Fig.11 A schematic diagram of a UI interface for displaying recommendation results provided by the present application is shown;

[0047] Fig.12 A schematic diagram of another UI interface for displaying recommendation results provided in an embodiment of the present application is shown;

[0048] Fig.13 A flowchart of a method for generating a recommended explanation provided in an embodiment of the present application;

[0049] Fig.14 A schematic diagram of the structure of a device for processing code problems provided in an embodiment of the present application;

[0050] Fig.15 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0051] Fig.16 A schematic diagram of a computing device cluster provided in an embodiment of the present application;

[0052] Fig.17 yes Fig.16 A schematic diagram of an application scenario of a computing device cluster is provided. DETAILED DESCRIPTION

[0053] The term "and / or" mentioned in this article is a kind of relationship that describes the association of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The symbol " / " in this article indicates that the associated objects are in an or relationship, for example, A / B means A or B.

[0054] The terms "first" and "second" etc. in the specification and claims herein are used to distinguish different objects, rather than to describe the specific order of objects. It should be understood that the terms used in this way can be interchangeable in appropriate circumstances, which is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, systems, products or equipment.

[0055] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0056] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0057] Figure 1 Figure 2 shows a schematic diagram of the principle of a traditional recommendation system. Figure 1 As shown in the figure, traditional recommendation systems are all black box structures, and users are unaware of the recommendation principles and reasons.

[0058] As recommendation systems penetrate into every aspect of users' lives, users begin to pay more attention to privacy protection and hope to understand the logic behind the recommendation results. Explainable recommendation systems have emerged. Explainable recommendation systems provide corresponding explanations while providing recommendation results. Explainable recommendation systems have the following advantages:

[0059] 1. It is conducive to building user trust and improving satisfaction.

[0060] 2. It can enhance the persuasiveness of recommended content.

[0061] 3. Assist in debugging and optimizing the recommendation system to improve iteration and optimization efficiency.

[0062] 4. In key industries such as finance and healthcare, users have a stronger demand for explanations of recommendations. Reasonable explanations of recommendations are one of the key factors in converting users to consumption.

[0063] The recommendation explanations given by products that apply the explainable recommendation system in related technologies are too general and simple, and do not take into account the personalization and differentiation of users.

[0064] Figure 2 and Figure 3 Schematic diagrams of recommendation pages of a group purchase APP and a shopping APP using an explainable recommendation system in related technologies are shown respectively. Figure 2 and 3 As shown, a space is reserved on the product side of the recommendation page to provide users with a reason for the recommendation through short text, such as Figure 2 The recommendation page of a group-buying app shown in the picture only displays "genuine goods and fair prices" under the product name. Figure 3 The recommendation page of a shopping app shown in the figure only displays "good sound quality" as the reason for recommendation under the product name.

[0065] Figure 4 FIG. 1 shows a schematic diagram of the implementation process of an explainable recommendation system in the related art. Figure 4 As shown in the figure, the common methods of explainable recommendation in related technologies rely on a posterior paradigm. The explanation generation model and the recommendation model are two independent models. The recommendation model is trained using business data first, and then the explanation generation model is built. The current explanation generation model requires high-quality data. The common method is to use the user's comment data as the original data, and invest a lot of labor costs to clean, process, rewrite and other operations on the comment data to obtain "pseudo explanations" and use them as training data sets for training the explanation generation model.

[0066] There are some obvious problems with the business model of the explainable recommendation system in related technologies, mainly including:

[0067] Insufficient personalization: Current recommendation reasons are too general, such as using product descriptive language such as "genuine goods at a reasonable price" to show the same recommendation reason to all users. However, different users may like the same product for different reasons. The current recommendation system does not take into account the personalized differences between users, and therefore cannot accurately tap into the potential needs of each user.

[0068] Recommendation content lacks depth: The reasons for recommendation are too brief and general, fail to fully demonstrate the advantages and features of the product, and lack necessary details and persuasiveness, which may affect user decision-making.

[0069] The system is not flexible: Even if the user refreshes the homepage multiple times, the recommendation explanation content of the same product remains unchanged. This shows that the recommendation explanation is not adaptable to changes in user behavior and iterative updates of the recommendation algorithm, and the update frequency is low.

[0070] In addition, most mainstream explainable recommendation techniques use posterior methods, which brings some additional problems:

[0071] The construction cost of training datasets is high: Constructing training datasets for explanation generation models requires a lot of manpower and resources. The current mainstream method is to use user reviews as potential "pseudo-explanation" data, and then transform them into effective training samples through steps such as screening, extraction, and reorganization.

[0072] Lack of feedback mechanism: Current posterior methods usually train a recommendation model first and pass the prediction results to the subsequent explanation generation model. However, this one-way process lacks an evaluation mechanism for the quality of the generated explanations, and does not integrate user feedback back into the recommendation system.

[0073] In view of this, an embodiment of the present application provides a method and device for generating a recommendation explanation, which considers the user's personalized information and generates personalized recommendation explanation content for the user, so that the generated recommendation explanation is more targeted and the quality of the recommendation explanation is improved.

[0074] The specific implementation of the method and device for generating the recommended interpretation provided by the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0075] Figure 5 Schematic diagram of a system architecture is shown. The system includes a terminal 510, a server 530, and a network 520 that connects the terminal 510 and the server 530. The server 530 may include one or more servers ( Figure 5 Only one server is used as an example for explanation), the server 530 can provide a recommendation explanation generation service for one or more terminals 510, and provide high-quality recommendation explanations.

[0076] In some embodiments, server 530 can also provide other services or software applications including non-virtual environments and virtual environments. In some embodiments, these services can be provided as web-based services or cloud services, such as provided to users of terminals under a software as a service (SaaS) model.

[0077] exist Figure 5In the system shown, the server 530 may include one or more components that implement the functions performed by the server 530. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. A user of the operating terminal 510 may in turn use one or more client applications to interact with the server 530 to utilize the services provided by these components. It should be understood that Figure 5 This is only an example of a system for implementing the various methods described in the embodiments of the present application. There may be other different configurations, and the embodiments of the present application are not limited to this.

[0078] Various types of applications can be installed on the terminal 510, such as shopping applications, short video applications, group purchasing applications, etc. The user can trigger a recommendation request in any application. The terminal 510 sends the recommendation request to the server 530. The server 530 can generate a recommendation result based on the recommendation business data and the recommendation explanation generation method provided in the embodiment of the present application. Each recommended object or the topk recommended objects in the recommendation result have a recommendation explanation, and the recommendation result with the recommendation explanation is returned to the terminal 510 to be displayed to the user.

[0079] Terminal 510 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers or laptop computers), workstation computers, wearable devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple IOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or various mobile operating systems, such as Microsoft Windows Mobile OS, Windows phone, Android, iPhone OS, etc. Portable handheld devices may include cellular phones, smart phones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays and smart watches, etc.

[0080] The network 520 may be any type of network known to those skilled in the art, and may support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, 3G, 4G, 5G, etc. Exemplarily, the one or more networks 420 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0081] Server 530 may include one or more general purpose computers, dedicated server computers (e.g., personal computer servers, UNIX servers, terminal servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement or combination. Server 330 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization, such as one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device. In various embodiments, server 430 may run one or more services or software applications that provide the functionality described below.

[0082] In some embodiments, server 530 may be a server of a distributed system or a server combined with a blockchain. Server 530 may also be a cloud server or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in a cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0083] It should be noted that, in some other examples, the terminal 510 can also complete the generation method of the recommended interpretation provided in the embodiment of the present application by itself without the need for the cooperation of the server, and the present application does not make any specific limitations on this.

[0084] Figure 6 The system architecture of a terminal cloud system to which the method for generating a recommendation explanation provided in an embodiment of the present application can be applied is shown. Figure 6 As shown, the cloud side includes a cloud platform and basic resources. The cloud platform includes a cloud platform manager. The basic resources may include multiple servers. Each server may be a working node, or each server may include multiple working nodes.

[0085] The working node may be a computing device card or a virtual machine (VM). The computing device card may be at least one of a central processing unit (CPU), a graphics processing unit (GPU) and a neural network processing unit (NPU).

[0086] The cloud platform manager will maintain or periodically collect information about each work node in the basic resources, such as the usage of resources on each work node, etc. This information can be used as decision-making information when allocating the generation task of the recommendation explanation.

[0087] The cloud platform manager can receive recommendation requests from the end-side device, and then the cloud platform manager will execute the corresponding recommendation task. The cloud platform manager can also assign the recommendation request to one or more working nodes in the cloud system, and one or more working nodes will execute the corresponding recommendation interpretation generation task.

[0088] After the working node completes the task of generating the recommendation explanation, the cloud platform manager can return the recommendation result with the recommendation explanation to the user.

[0089] Figure 7 A schematic diagram of an implementation architecture of a method for generating a recommended explanation provided in an embodiment of the present application is shown. Figure 7 As shown in the figure, first, a CoT prompt is constructed based on the data of the current recommendation business (including target user information and recommended object information), guiding the LLM to gradually infer the reasons why the user may like or dislike the current recommended object. After the CoT prompt is input into the LLM, the LLM is guided to infer and output a detailed recommendation explanation text. Subsequently, the recommendation explanation text is processed as necessary, such as sentence extraction, and then the processed recommendation explanation text is input into the recommendation model for verification. The verification results are used as feedback to fine-tune the generation ability of the large language model, so that it can understand the recommendation scenario more accurately, adapt to the recommendation task more effectively, and improve the recommendation effect. This process only relies on common user behavior feedback signals in the recommendation system, such as clicks and purchases, without the need to construct additional explanation data. The construction of CoT fully considers the user's personalized information, ensuring that the large language model can generate richer and more targeted explanation content.

[0090] The model training process is completed on the server. Once the training is completed, the model will be deployed to the cloud server and updated and iterated regularly to maintain its accuracy and timeliness.

[0091] When a user initiates a recommendation request on their terminal device, the request will carry relevant information about the user. The user sends a request to the cloud service through the terminal device, and after receiving the request, the cloud service generates a recommendation explanation and recommendation results based on the user's personalized data. These recommendations are then sent back to the user's terminal device and displayed on the user's screen.

[0092] Figure 8 FIG. 1 shows a schematic diagram of an architecture for explaining the training of a generative model provided in an embodiment of the present application. Figure 8As shown in the figure, the structure of the explanation generation model is shown in the right box, and the explanation generation model includes LLM and adapter. The structure of the recommendation model is shown in the left box, and the recommendation model includes LLM. First, an explanation generation prompt is constructed based on user information and recommended object information, and the explanation generation prompt is used as the input of the explanation generation model to output the generated recommended explanation. Then, a CTR prediction prompt is constructed based on the generated recommended explanation, and the CTR prediction prompt is used as the input of the recommendation model to output the click-through rate (CTR) prediction result. The LC consistency reward is determined based on the CTR prediction result and the true label (indicating the real interaction between the user and the recommended object, such as whether the recommended object is clicked). Then, the LLM is fine-tuned using LC consistency training, that is, the parameters of the LLM in the right box are kept unchanged, and the parameters of the adapter are adjusted.

[0093] Fig. 9 A schematic diagram of constructing a CoT prompt is shown. Fig. 9 As shown, the required user information and recommended object information are extracted from the business data. Optionally, the user information includes user attribute information and historical interaction information. The user attribute information can be, for example, Fig. 9 As shown, male, 39 years old; white-collar; likes high-tech products; currently lives in Beijing; high-income group. For example, the interactive information is: has purchased a Huawei mobile phone. The recommended objects are, for example, product 1 and product 2. The information of product 1 is: product name: Xiaomi Su7, product evaluation: 1. Streamlined design and fashion; 2. Excellent acceleration ability...; the information of product 2 is: product name: Huawei M7; product evaluation: 1. Both controllable and comfortable driving experience; 2. Strong sense of technology, large interior space...; Based on the user information and the recommended object information, a CoT prompt is constructed, and the CoT prompt is input into the LLM (LLM here refers to Figure 8 The LLM in the right box in the middle guides the LLM to analyze the user's attitude and opinions towards the current recommended object step by step.

[0094] One possible implementation of the construction of CoT hint is: Design a CoT hint template: Create a CoT hint template such as Fig. 9 As shown in the figure, the template contains a guiding sentence such as "Please analyze the viewpoint of..." step by step. The template will reserve space for user information and product information, which will be replaced by actual data. Extract key information: Extract key user, product and interaction information from business data, and accurately fill this information into the CoT prompt template, such as Fig. 9Input CoT prompts to LLM: Input the constructed CoT prompts to LLM to guide LLM to generate a detailed analysis of the user's attitude and opinion on the current product, thereby helping users to understand the product characteristics more deeply and assisting them to make more informed purchase decisions.

[0095] Depend on Fig. 9 It can be seen that the recommendation explanation text generated by LLM is too detailed and too lengthy. In order to facilitate display to users, the generated recommendation explanation text is also processed, such as sentence extraction processing, to extract key information. For example, the recommendation explanation text for product 2 is processed as follows: "You have bought a Huawei mobile phone before. I guess you like and trust the Huawei brand. The Huawei M7 car is recommended to meet your preferences."

[0096] Back to Figure 8 After generating the recommended explanation, the recommended explanation is input into the recommendation model again, and the feedback of the recommended explanation is used to fine-tune the LLM of the explanation generation model. The following is a detailed introduction to the fine-tuning of the LLM of the explanation generation model based on the recommended explanation.

[0097] In one example, the PPO algorithm can be used to fine-tune the LLM of the explanation generation model. The objective function of the PPO algorithm is as follows:

[0098]

[0099] Where x represents the CoT prompt, Z represents the explanation text, and R(Z) is the reward function. express Figure 8 The LLM+adapter in the right box of πini+ represents the LLM without the adapter. The role of the reward function R(Z) is to use the real-world click-through rate prediction feedback to improve the quality of the generated recommendation explanations, and reward explanations that accurately reflect user intentions and preferences.

[0100] Optionally, R(Z) is a reward function that can be an LC consistency reward function. LLM is used to evaluate whether it is possible to infer the user's future interactive behavior based on the content of the recommendation explanation. The higher the LC consistency reward (i.e., the value of the LC consistency reward function), the more accurate the content of the recommendation explanation is in capturing and understanding the potential factors that affect user behavior. In an embodiment of the present application, this process can be constructed as a binary classification question, and LLM is used to predict whether the user will like the recommended object (for example, a book) based on the generated recommendation explanation (including attitudes and opinions towards the recommended object). The closer the predicted result of the click-through rate is to the true label, the more accurate the explanation is, which implies the ability to capture the factors that actually affect the user's decision. The LC consistency reward function is as follows:

[0101]

[0102]

[0103] Among them, R(A) represents the LC consistency reward, yC,i represents the true label, represents the CTR prediction result output by LLM, clip is the cutoff function, δ is the constraint, for example, δ=±0.5, which means The value of is constrained to be between ±0.5.

[0104] Optionally, the LLM can be lightly fine-tuned by Lora. The model of the solution provided in the embodiment of the present application can be called ExpCTR-LLM. The experimental results of the fine-tuned LLM are shown in Table 1:

[0105]

[0106] Table 1

[0107] In some other examples, the LLM-based explanation generation model can also be fine-tuned based on other methods.

[0108] Fig.10 FIG. 2 shows another schematic diagram of an architecture for explaining the training of a generative model provided in an embodiment of the present application. Fig.10 As shown, an explanation generation prompt is constructed based on user information and recommended object information, and the explanation generation prompt is used as the input of the explanation generation model to output the generated recommended explanation. Then, the input with the additional recommended explanation and the input without the additional recommended explanation are used as the input of the CTR model respectively, and the output results of the two CTR models are obtained respectively. The IC consistency reward is determined by using these two output results, and the parameters of the adapter in the explanation generation model are adjusted based on the IC consistency reward to achieve fine-tuning of the LLM-based explanation generation model.

[0109] The embodiment of the present application inputs the generated recommendation explanation as a text feature hC,i into the traditional ID-based CTR model to ensure that the recommendation explanation is consistent with the internal operating mechanism and prediction results of the CTR model.

[0110] Exemplarily, the recommended explanation text is used as the input of the explanation encoder, and the text feature vector hC,i is output. The text feature vector hC,i and the vector representation zC,i of the original input feature user and product of the CTR model are concatenated to obtain a concatenated vector, and the concatenated vector is used as the input of the CTR model to output the first CTR prediction result. As shown below:

[0111]

[0112] Optionally, the explanation encoder can use a pre-trained model such as BERT or BGE to encode the generated recommended explanation text.

[0113] Then, the text feature vector hC,i without the additional recommended explanation text, that is, zero tensors, is used as the input of the CTR model, that is, the original user and product vector representation zC,i is used as the input of the CTR model to obtain the second CTR prediction result. Based on the first CTR prediction results And the second CTR prediction result Determine IC consistency rewards.

[0114] To evaluate the impact of the semantic representation of the generated recommendation explanations on the performance of the CTR model, we compared the performance of the CTR model with and without these explanations and quantified the difference in CTR prediction results. If the performance is significantly improved when the explanations are added, it indicates that the introduced semantic features contribute positively to the model. This means that the explanations are better aligned with the CTR model and therefore can get higher rewards. The IC consistency reward is calculated as follows:

[0115]

[0116] Among them, RUA represents the IC consistency reward, yC,i represents the true label, represents the first CTR prediction result, that is, the concatenated vector of the text feature vector hC,i of the recommended explanation text and the vector representation zC,i of the user and the product is used as the input of the CTR model and the output of the CTR prediction result; Represents the second CTR prediction result, that is, the CTR prediction result output by taking only the vector representation zC,i of the user and the product as the input of the CTR model.

[0117] The embodiment of the present application designs an iterative training method, specifically, first using LLM to generate recommendation explanations for each group of user products in the training set, and after collecting all the recommendation explanations, training a CTR model that includes the recommendation explanations as text features, and then using IC consistency rewards to fine-tune LLM, and then repeating the above process, the quality of the recommendation explanations generated by LLM gradually improves. The final experimental results are shown in Table 2:

[0118]

[0119] Table 2

[0120] In another example, considering that the recommendation explanation text generated by the embodiment of the present application is more detailed and includes more content, such as one or more of the user's historical interaction behavior, the user's preferences, and the degree of fit between the user and the recommended object, the length of the recommendation explanation text is longer than that of the existing one. For example, the recommendation explanation text generated by the method for generating the recommendation explanation provided by the embodiment of the present application is "Yesterday you ordered a takeaway from ***. I guess you like the taste of Nanjing cuisine. I recommend Nanjing Dapaidang to suit your preferences", while the existing recommendation explanation is only a relatively simple and general short text such as "genuine goods at a good price". Therefore, in order to better display the recommendation explanation text generated by the embodiment of the present application to the user, the embodiment of the present application also redesigned the UI interface for displaying the recommendation results.

[0121] Fig.11 A schematic diagram of a UI interface for displaying recommendation results provided by the present application is shown. Fig.11 As shown, a slidable text box is set in the display area of ​​the recommended object, such as the lower side of the recommended object. The recommended explanation text corresponding to the recommended object can be in the slidable text box, and the user can read the detailed recommended explanation text by sliding the text box.

[0122] Fig.12 FIG. 1 is a schematic diagram showing another UI interface for displaying recommendation results provided by an embodiment of the present application. Fig.12 As shown, a button is added in the display area of ​​the recommended object, for example, at the lower side of the recommended object, and the user can click the button to view the recommended explanation text for the recommended object.

[0123] It is understandable that, since the generation of the recommendation explanation of the embodiment of the present application takes into account the personalized information of the user, even for the same recommendation object, the recommendation explanations generated by different users are different. In addition, the generation of the recommendation explanation further takes into account specific information such as the user's historical interaction information. When the user refreshes the recommendation result display page, the recommendation explanation text of the recommendation object will change dynamically.

[0124] Fig.13 A flowchart of a method for generating a recommendation explanation provided in an embodiment of the present application. The method can be executed by any device, equipment, platform or device cluster with computing capabilities. The present application embodiment does not specifically limit the specific computing device for executing the method, and a suitable computing device can be selected as needed. For example, Figure 5 The server 530 is shown as executing the implementation to provide the user with a recommended explanation generation service. For the convenience of description, the following text does not distinguish the form of the execution subject, and all are described as a recommended explanation generation device. Fig.13As shown, the method for generating the recommended explanation provided by the present application at least includes steps S1301 to S1303.

[0125] In step S1301, target user information and recommendation object information are obtained.

[0126] The method for generating a recommendation explanation provided in the embodiment of the present application can be applied to a recommendation system, so that the recommended object in the recommendation result generated by the recommendation system has a personalized recommendation explanation. The recommendation system can be applied to a variety of recommendation fields. For example, application recommendation, music playback recommendation, video playback recommendation, reading recommendation, news information recommendation, web page recommendation, product recommendation, group purchase recommendation and other recommendation fields.

[0127] Taking the end-cloud collaborative architecture as an example, when a user opens a group-buying APP on a terminal (such as a mobile phone), the recommendation system of the group-buying APP can be triggered. The recommendation system will generate a recommendation request, which will carry relevant information of the user, such as the user ID, and send the recommendation request to the cloud server. After receiving the request, the cloud server obtains the user information from the recommendation service database according to the user ID.

[0128] Optionally, the user information may include user attribute information and historical interaction information. The user attribute information may be attributes related to user preference characteristics, such as gender, age, occupation, income, residential address, hobbies, and education level. The historical interaction information may be the user's previous purchase behavior, adding to a shopping cart, clicking, commenting, watching, browsing, etc. For example, the historical interaction information may be that the user has purchased a Huawei mobile phone.

[0129] After receiving the recommendation request, the cloud server will respond to the recommendation request and predict the probability of the user group buying each candidate product based on the user's historical interaction behavior and the product characteristics in the group buying APP. Based on the predicted probability results, the recommendation system can display the products in descending order of predicted probability values, thereby increasing the user's purchase probability.

[0130] In one example, an embodiment of the present application may generate recommendation explanations only for the top k recommended objects in the recommendation results, where k can be set as needed. For example, recommendation explanations are generated only for the top five items in the recommendation results, thereby reducing computing power overhead and speeding up the generation of recommendation explanations.

[0131] In another example, the embodiment of the present application may also generate recommendation explanations for all recommended objects in the recommendation results.

[0132] The predicted probability result can be obtained by inference calculation through the recommendation model. For example, the user characteristics and product characteristics are used as the input of the recommendation model, and the recommendation model performs inference to output the predicted probability, which indicates the probability of the user group buying each candidate product. Optionally, the recommendation model can be a recommendation model based on LLM, or a non-LLM recommendation model such as a CTR model.

[0133] After the recommendation model predicts the recommended object, the recommendation system obtains the information of the recommended object for which the recommendation explanation needs to be generated. For example, if the recommended object for which the recommendation explanation needs to be generated is the top 5 recommended objects with the highest predicted probability, the information of the 5 recommended objects is obtained from the recommendation service database. Each recommended object information may include attribute information and evaluation information of the recommended object.

[0134] In step S1302, a first prompt is generated based on the user information and the recommendation object information.

[0135] In the embodiment of the present application, the explanation generation model can be an explanation generation model based on a large language model, which can construct a CoT prompt (i.e., the first prompt) based on the acquired user information and recommended object information, and use the CoT prompt to guide the LLM to gradually infer the user's attitude and opinion towards the current recommended object, and guide the LLM to infer and output a detailed recommended explanation text.

[0136] The detailed implementation of the construction of the CoT prompt can be found in the above description of the CoT prompt, which will not be repeated here for the sake of brevity.

[0137] Of course, in some other examples, other types of prompts may be generated based on user information and recommended object information. For example, the prompt may be "The user information is ***, and the recommended object information is ****. Please analyze the user's feelings about the recommended object based on the above information."

[0138] In step S1303, the first prompt is used as an input of the explanation generation model to output a recommended explanation of the recommended object.

[0139] After the CoT prompt is input into the explanation generation model, the explanation generation model is guided to gradually infer the reasons why the user may like or dislike the current recommended object, and output a detailed recommendation explanation text.

[0140] In one example, the recommendation explanation text generated by LLM is too detailed and too lengthy. In order to facilitate display to users, the generated recommendation explanation text is also processed, such as sentence extraction processing, to extract key information. For example, the recommendation explanation text for product 2 after processing is: "You have bought a Huawei mobile phone before. I guess you like and trust the Huawei brand. We recommend that the Huawei M7 car meets your preferences."

[0141] Optionally, in order to make the LLM-based recommendation explanation generation model better applicable to the recommendation explanation generation task scenario, the LLM can be fine-tuned. The specific method for fine-tuning the LLM is described above, and for the sake of brevity, it will not be repeated here.

[0142] The present application also provides a device for generating a recommendation explanation, such as Fig.14 As shown, including:

[0143] The acquisition module 1401 is used to acquire target user information and recommendation object information;

[0144] A prompt module 1402, configured to generate a first prompt based on the user information and the recommended object information;

[0145] The explanation generation module 1403 is used to use the first prompt as an input of an explanation generation model, and output a recommendation explanation of the recommended object, where the recommendation explanation indicates a reason for recommending the recommended object.

[0146] Among them, the acquisition module 1401, the prompt module 1402 and the explanation generation module 1403 can all be implemented by software, or can be implemented by hardware. Exemplarily, the following takes the acquisition module 1401 as an example to introduce the implementation of the acquisition module 1401. Similarly, the implementation of the prompt module 1402 and the explanation generation module 1403 can refer to the implementation of the acquisition module 1401.

[0147] As an example of a software functional unit, the acquisition module 1401 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the acquisition module 1401 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0148] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0149] As an example of a hardware functional unit, the acquisition module 1401 may include at least one computing device, such as a server, etc. Alternatively, the acquisition module 1401 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0150] The multiple computing devices included in the acquisition module 1401 can be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module 1401 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the acquisition module 1401 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0151] It should be noted that, in other embodiments, the acquisition module 1401 can be used to execute any step in the method for generating a recommended interpretation, the prompt module 1402 can be used to execute any step in the method for generating a recommended interpretation, and the explanation generating module 1403 can be used to execute any step in the method for generating a recommended interpretation; the steps that the acquisition module 1401, the prompt module 1402 and the explanation generating module 1403 are responsible for implementing can be specified as needed, and the full functions of the device for generating a recommended interpretation are realized by respectively implementing different steps in the method for generating a recommended interpretation through the acquisition module 1401, the prompt module 1402 and the explanation generating module 1403.

[0152] The present application embodiment also provides a computing device 1500. Fig.15 As shown, computing device 1500 includes: bus 1502, processor 1504, memory 1506 and communication interface 1508. Processor 1504, memory 1506 and communication interface 1508 communicate through bus 1502. Computing device 1500 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 1500.

[0153] The bus 1502 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 The bus 1502 may include a path for transmitting information between various components of the computing device 1500 (eg, the memory 1506, the processor 1504, and the communication interface 1508).

[0154] The processor 1504 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0155] The memory 1506 may include a volatile memory, such as a random access memory (RAM). The processor 1504 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0156] The memory 1506 stores executable program codes, and the processor 1504 executes the executable program codes to respectively implement the functions of the aforementioned acquisition module 1401, the prompt module 1402, and the explanation generation module 1403, thereby implementing the method for generating the recommended explanation. That is, the memory 1506 stores instructions for executing the method for generating the recommended explanation.

[0157] The communication interface 1508 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1500 and other devices or communication networks.

[0158] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0159] like Fig.16 As shown, the computing device cluster includes at least one computing device 1500. The memory 1506 in one or more computing devices 1500 in the computing device cluster may store the same instructions for executing the method for generating the recommended explanation.

[0160] In some possible implementations, the memory 1506 of one or more computing devices 1500 in the computing device cluster may also store partial instructions for executing the method for generating the recommended interpretation. In other words, the combination of one or more computing devices 1500 may jointly execute instructions for executing the method for generating the recommended interpretation.

[0161] It should be noted that the memory 1506 in different computing devices 1500 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the generating device for the recommended explanation. That is, the instructions stored in the memory 1506 in different computing devices 1500 may implement the functions of one or more modules among the acquisition module 1401, the prompt module 1402 and the explanation generating module 1403.

[0162] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.17 A possible implementation is shown. Fig.17 As shown, two computing devices 1500A and 1500B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1506 in the computing device 1500A stores instructions for executing the functions of the acquisition module 1401 and the prompt module 1402. At the same time, the memory 1506 in the computing device 1500B stores instructions for executing the functions of the prompt module 1402 and the interpretation generation module 1403.

[0163] It should be understood that Fig.17The functionality of the computing device 1500A shown in FIG. 1 may also be implemented by multiple computing devices 1500. Similarly, the functionality of the computing device 1500B may also be implemented by multiple computing devices 1500.

[0164] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the method for generating a recommended interpretation.

[0165] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute a method for generating a recommended interpretation.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating a recommendation explanation, characterized in that: include: Obtain target user information and recommended object information; generating a first prompt based on the user information and the recommended object information; The first prompt is used as an input of an explanation generation model, and a recommendation explanation of the recommended object is output, where the recommendation explanation indicates a reason for recommending the recommended object.

2. The method according to claim 1, characterized in that The target user information includes user attribute information and historical interaction information, wherein the historical interaction information indicates the interaction between the target user and the historical recommended object; The generating a first prompt based on the user information and the recommended object information includes: Based on the user attribute information, the historical interaction information and the recommended object information, a thought chain prompt is generated.

3. The method according to claim 2, characterized in that The generating of thought chain prompts based on the user attribute information, the historical interaction information and the recommended object information includes: The user attribute information, the historical interaction information and the recommended object information are filled into a preset position in a preset thought chain prompt template to obtain the thought chain prompt.

4. The method according to any one of claims 1 to 3, characterized in that: The interpretation generation model includes a target large language model, and the target large language model is obtained by fine-tuning based on an end strategy optimization algorithm.

5. The method according to claim 4, characterized in that The target large language model is fine-tuned based on the feedback information of the recommended explanation.

6. The method according to claim 5, characterized in that The target large language model is provided with an adapter; The feedback information of the recommended explanation includes a reward corresponding to the recommended explanation, and the reward is used to adjust the parameters of the adapter to achieve fine-tuning of the target large language model.

7. The method according to claim 6, characterized in that The rewards include LC consistency rewards; The LC consistency reward is determined based on the following steps: generating a second prompt based on the recommended interpretation; The second prompt is used as an input of the large language model, and a first click rate prediction result is output, where the first click rate prediction result indicates an interaction probability between the target user and the recommended object. The LC consistency reward is determined based on the first click rate prediction result and a first true label, where the first true label indicates whether the target user interacts with the recommended object.

8. The method according to claim 6, characterized in that The rewards include IC consistency rewards; The IC consistency reward is determined based on the following steps: Using the recommendation explanation, the target user information and the recommendation object information as inputs of a click rate prediction model, and outputting a second click rate prediction result; Using the target user information and the recommended object information as inputs of a click rate prediction model, and outputting a third click rate prediction result; The IC consistency reward is determined based on the second click rate prediction result, the third click rate prediction result and the second true label.

9. The method according to any one of claims 1 to 8, characterized in that: The recommendation explanation includes one or more of the target user's historical interaction behavior, the target user's preference, and the compatibility between the target user and the recommendation object.

10. The method according to any one of claims 1 to 9, further comprising: Processing the recommended explanation to obtain a recommended explanation text; The recommended explanation text is displayed to the user through a recommendation page, and the recommendation page includes a recommendation object display area. A slidable text box is set in the recommendation object display area, and the recommended explanation text is set in the slidable text box. Alternatively, a button is set in the recommendation object display area, and the recommended explanation text is displayed to the user when the button is triggered.

11. A device for generating a recommendation explanation, characterized in that: include: The acquisition module is used to obtain target user information and recommended object information; A prompt module, configured to generate a first prompt based on the user information and the recommended object information; An explanation generation module is used to take the first prompt as an input of an explanation generation model and output a recommendation explanation of the recommended object, wherein the recommendation explanation indicates a reason for recommending the recommended object.

12. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The method comprises computer program instructions, and when the instructions are executed by a computing device cluster, the computing device cluster enables the computing device cluster to perform the method according to any one of claims 1 to 10, wherein the computing device cluster comprises at least one computing device.

14. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 10, wherein the computing device cluster includes at least one computing device.

Images