A / B test problem response method and device based on large model agent

Abstract

The invention relates to an A / B test problem response method and device based on a large model agent, and relates to the technical field of large models, agents and computers. The method comprises the steps of displaying an intelligent interaction page associated with an intelligent agent in response to a triggering operation on an intelligent agent identifier in an A / B test platform; in response to a work order creation operation in the intelligent interaction page, intercepting a target work order request corresponding to the work order creation operation sent to a work order server, and judging whether an intelligent agent can process a target problem in a target work order corresponding to the work order creation operation; acquiring a first similar question and a target answer corresponding to the first similar question in a question and answer knowledge base through a large model under the condition that the intelligent agent can process the target question; and displaying the target answer on the intelligent interaction page. Therefore, work orders submitted by the user can be reduced, waste of platform resources is reduced, the problem response speed and efficiency can be improved, and the use experience of the user is improved.

Description

Technical Field

[0001] The present disclosure relates to the fields of large models, intelligent agents, and computer technology. Specifically, it relates to a method and device for responding to A / B test problems based on a large model intelligent agent. Background Art

[0002] To help users solve various problems encountered during the use of the platform, the platform usually provides a problem feedback function. For example, in an A / B test platform, users can feedback various problems encountered during the A / B test by submitting work orders.

[0003] However, a large number of work orders will overwhelm the customer service staff of the platform, resulting in an extended problem handling cycle and reduced efficiency. Users may need to wait for a certain period of time to receive an effective response and solution, which will not only reduce the user experience but also cause waste of platform resources. Summary of the Invention

[0004] This Summary of the Invention section is provided to introduce concepts in a concise form that will be described in detail in the subsequent Detailed Implementation section. This Summary of the Invention section is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a method for responding to A / B test problems based on a large model intelligent agent. The method for responding to A / B test problems based on a large model intelligent agent includes: In response to a trigger operation on an intelligent agent identifier in an A / B test platform, display an intelligent interaction page associated with the intelligent agent, where the intelligent agent is used to respond to problems feedback by a user in the intelligent interaction page through a large model; In response to a work order creation operation in the intelligent interaction page, intercept a target work order request corresponding to the work order creation operation sent to a work order server, and determine whether the intelligent agent can handle a target problem in the target work order corresponding to the work order creation operation, where the work order server is used to generate a work order to be processed by a customer service staff in response to a work order request; When it is determined that the intelligent agent can handle the target problem, obtain a first similar problem and a target answer corresponding to the first similar problem from a question and answer knowledge base through the large model, where the similarity between the first similar problem and the target problem is greater than a first preset threshold; Display the target answer in the intelligent interaction page.

[0006] In a second aspect, the present disclosure provides a device for responding to A / B test problems based on a large model intelligent agent. The device for responding to A / B test problems based on a large model intelligent agent includes: A first display module, configured to display an intelligent interaction page associated with the intelligent agent in response to a triggering operation on the intelligent agent identifier in the A / B test platform, where the intelligent agent is used to respond to questions feedback by the user in the intelligent interaction page through a large model; An interception module, configured to intercept a target work order request corresponding to the work order creation operation sent to the work order server in response to a work order creation operation in the intelligent interaction page, and determine whether the intelligent agent can handle a target problem in the target work order corresponding to the work order creation operation, where the work order server is used to generate a work order to be processed by a customer service staff in response to a work order request; An acquisition module, configured to, when it is determined that the intelligent agent can handle the target problem, obtain a first similar problem and a target answer corresponding to the first similar problem from a question and answer knowledge base through the large model, where the similarity between the first similar problem and the target problem is greater than a first preset threshold; A second display module, configured to display the target answer in the intelligent interaction page.

[0007] In a third aspect, the present disclosure provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processing device, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, the present disclosure provides an electronic device, including: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the method described in the first aspect.

[0009] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] Through the above technical solution, in the A / B test platform, when a user creates a work order, the target work order request corresponding to the work order creation operation sent to the work order server can be intercepted, and it can be determined whether the intelligent agent can handle the target problem in the target work order. Furthermore, when the intelligent agent can handle the target problem, a first similar problem and a target answer corresponding to the first similar problem are obtained from the question and answer knowledge base through the large model, and the target answer is displayed in the intelligent interaction page. By adopting the above method, the work order submitted by the user can be intercepted, and when the intelligent agent can handle the target problem, the user's problem can be responded to through the large model, which can not only reduce the work order submitted by the user and waste of platform resources, but also improve the problem response speed and efficiency, and further improve the user experience.

[0011] Other features and advantages of the present disclosure will be described in detail in the following detailed description section. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In combination with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the original components and elements are not necessarily drawn to scale. In the drawings: Figure 1 is a flowchart of a method for responding to A / B test questions based on a large model agent according to an exemplary embodiment of the present disclosure; Figure 2 is a schematic diagram of a process for intercepting a work order according to an exemplary embodiment of the present disclosure; Figure 3 is a schematic diagram of the display of similar questions according to an exemplary embodiment of the present disclosure; Figure 4 is a schematic diagram of the display of a communication dialogue page according to an exemplary embodiment of the present disclosure; Figure 5 is a schematic diagram of the display of a group entry prompt according to an exemplary embodiment of the present disclosure; Figure 6 is a schematic diagram of the display of creating a work order according to an exemplary embodiment of the present disclosure; Figure 7 is a schematic diagram of the display of inputting a question according to an exemplary embodiment of the present disclosure; Figure 8 is a schematic diagram of the display of a question announcement according to an exemplary embodiment of the present disclosure; Figure 9 is a block diagram of a device for responding to A / B test questions based on a large model agent according to an exemplary embodiment of the present disclosure; Figure 10 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0013] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0014] It should be understood that the various steps described in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0015] As used herein, the term "comprising" and its variations are open-ended, i.e., "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0016] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0017] It should be noted that the modifications of "one" and "plural" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that, unless clearly indicated otherwise in the context, it should be understood as "one or more".

[0018] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0019] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0020] For example, when responding to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium that performs the operations of the technical solutions of the present disclosure according to the prompt message.

[0021] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving an active request from the user may be, for example, in the form of a pop-up window, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It can be understood that the above-mentioned notice and the process of obtaining user authorization are only illustrative and do not limit the implementation manner of the present disclosure. Other manners that comply with relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0023] Meanwhile, it can be understood that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and related provisions.

[0024] An A / B testing platform is a tool or system for conducting controlled experiments, which can help users evaluate the effects of different design schemes or strategies, so as to optimize products, services or promotion activities.

[0025] In order to help users solve various problems encountered during the use of the platform, the platform usually provides a problem feedback function. For example, in an A / B testing platform, users can feedback various problems encountered during the A / B testing process by submitting work orders, or users can consult the solution to the problem by having a conversation with an intelligent agent associated with a large model.

[0026] In actual use, when users have doubts about the accuracy or reliability of the intelligent agent, they usually directly submit work orders to feedback problems. A large number of work orders will overwhelm the customer service staff of the platform, resulting in an extended problem handling cycle and reduced efficiency. Users may need to wait for a certain period of time to get an effective response and solution, which will not only reduce the user experience, but also cause waste of platform resources.

[0027] In view of this, the present disclosure provides an A / B testing problem response method and device based on a large model intelligent agent to solve the above technical problems.

[0028] The following further explains the embodiments of the present disclosure with reference to the accompanying drawings.

[0029] Figure 1 is a flowchart of an A / B testing problem response method based on a large model intelligent agent shown according to an exemplary embodiment of the present disclosure. Referring to Figure 1 this, the A / B testing problem response method based on a large model intelligent agent may include the following steps: S101: In response to a trigger operation on the intelligent agent identifier in the A / B testing platform, display an intelligent interaction page associated with the intelligent agent.

[0030] Wherein, the intelligent agent is used to respond to the problems feedback by the user in the intelligent interaction page through the large model.

[0031] Exemplarily, the agent identifier can be displayed on the platform page of the A / B test platform. When the agent identifier is triggered by an operation such as clicking, the intelligent interaction page associated with the agent can be displayed, where the agent is the Figure 2 XX intelligent assistant in. Of course, it can also be to trigger the display of the intelligent interaction page associated with the agent in other ways, and the embodiments of the present disclosure do not impose any restrictions on this.

[0032] S102: In response to a work order creation operation in the intelligent interaction page, intercept the target work order request corresponding to the work order creation operation sent to the work order server, and determine whether the agent can handle the target problem in the target work order corresponding to the work order creation operation.

[0033] Among them, the work order server is used to generate a work order to be processed by customer service personnel in response to a work order request. Customer service personnel can be those who can answer or solve user problems, including on-duty customer service, on-duty operation and maintenance, etc., and the present disclosure does not limit this.

[0034] Exemplarily, as Figure 2 shown, the intelligent interaction page displays a control 21 for creating a work order. In response to the triggering operation on the control 21, a work order editing area for editing the work order is displayed in the dialogue area of the intelligent interaction page. Further, in response to the work order editing operation in the work order editing area, the target work order corresponding to the work order editing operation is displayed. After the user finishes editing, the work order can be submitted by triggering the Figure 2 "Submit Work Order" in.

[0035] Among them, the control 21 for creating a work order can be always displayed on the intelligent interaction page, facilitating users to create work orders at any time and improving the user experience.

[0036] S103: When it is determined that the agent can handle the target problem, obtain the first similar problem and the target answer corresponding to the first similar problem from the Q&A knowledge base through the large model.

[0037] Among them, the similarity between the first similar problem and the target problem is greater than the first preset threshold.

[0038] In the embodiments of the present disclosure, the Q&A knowledge base can be obtained by summarizing and organizing historical problem solutions, function documents of the A / B test platform, requirement documents, historical intelligent conversations, online knowledge documents, etc., and can be specifically set according to requirements, and the present disclosure does not limit this. Through the Q&A knowledge base, information such as the solution methods of historical problem solutions and platform-related knowledge can be provided to users to help users solve various problems encountered in the process of using the A / B test platform.

[0039] Exemplarily, by providing the user with the solution to a similar problem that has been solved, it is possible to broaden the user's thinking for problem-solving and reduce the number of work orders submitted by the user.

[0040] S104: Display the target answer on the intelligent interaction page.

[0041] In the embodiments of the present disclosure, it is possible to first intercept the target work order request sent to the work order server, and then determine whether the intelligent agent can solve the problem feedback by the user. Furthermore, in the case where the intelligent agent can solve the problem feedback by the user, obtain the solution to a similar problem to the problem feedback by the user through the large model and display it to the user, which not only improves the problem response speed and efficiency, but also reduces the number of work orders submitted by the user.

[0042] By adopting the above method, it is possible to intercept the work orders submitted by the user, and in the case where the intelligent agent can handle the target problem, respond to the user's problem through the large model, which can not only reduce the number of work orders submitted by the user, reduce the waste of platform resources, but also improve the problem response speed and efficiency, thereby improving the user's experience.

[0043] In a possible manner, a work order submission control is displayed on the intelligent interaction page. In response to a work order creation operation on the intelligent interaction page, intercept the target work order request corresponding to the work order creation operation sent to the work order server, including: in response to a work order editing operation on the intelligent interaction page, display the target work order corresponding to the work order editing operation; in response to a first trigger operation on the work order submission control on the intelligent interaction page, intercept the target work order request corresponding to the target work order sent to the work order server. The A / B test problem response method further includes: during the process of displaying the target answer, control the work order submission control to be displayed as non-triggerable; after displaying the target answer, control the work order submission control to be displayed as triggerable; in response to a second trigger operation on the work order submission control, send a target work order request to the work order server.

[0044] Exemplarily, continue to refer to Figure 2 , in response to the user's trigger operation on the Figure 2 "Submit Work Order" control in Figure 2 , intercept the target work order request corresponding to the target work order sent to the work order server. During the process of displaying the target answer, that is, during the process of the large model searching for similar problems, control the work order submission control to be displayed as non-triggerable, and then after displaying the target answer, control the work order submission control to be displayed as triggerable. As shown in

[0045] In this way, during the process of the intelligent agent attempting to answer the user's question, the user cannot continue to submit a work order. After the intelligent agent answers the user's question, if the user believes that the answer from the intelligent agent does not solve the problem, the user can continue to submit a work order. If the user believes that the answer from the intelligent agent can solve the problem, there is no need to submit a work order. By attempting to use the large model to solve the user's problem when the user submits a work order, the waste of platform resources can be effectively reduced.

[0046] In a possible way, determining whether the intelligent agent can handle the target problem in the target work order corresponding to the work order creation operation includes: performing intent recognition on the target problem through the large model, and determining that the intelligent agent can handle the target problem in the target work order corresponding to the work order creation operation when there is a first similar problem in the Q&A knowledge base.

[0047] In the embodiments of the present disclosure, intent recognition can be performed on the target problem through the large model, and then it is queried in the Q&A knowledge base whether there is a first similar problem similar to the target problem. The first preset threshold can be set according to requirements, and the present disclosure does not limit this. When it is determined that there is a first similar problem in the Q&A knowledge base, it means that the intelligent agent can handle the target problem feedback by the user.

[0048] It should be noted that since the user needs to describe the problem when filling out the problem feedback work order, and the same problem may have different textual expressions, intent recognition can be performed on the target problem first to determine the actual problem that the user needs to feedback, improve the accuracy of obtaining similar problems subsequently, and further improve the accuracy of the answer, and further reduce the user's submission of work orders.

[0049] It is worth noting that if after performing intent recognition on the target problem through the large model, the result of the intent recognition indicates that the user has not submitted valid content related to the problem, a prompt message for prompting the user to re-enter the problem or re-edit the work order can be displayed to avoid submitting invalid problem work orders and further reduce the waste of platform resources.

[0050] In a possible way, the A / B test question response method further includes: when there is no first similar problem in the Q&A knowledge base, checking whether there is a second similar problem in the historical feedback data, and the similarity between the target problem and the second similar problem is greater than the second preset threshold; when there is a second similar problem in the historical feedback data, the second similar problem is displayed on the intelligent interaction page.

[0051] Exemplarily, when there is no first similar problem in the Q&A knowledge base, it is possible to check whether there is a second similar problem similar to the target problem in the historical feedback data, where the historical feedback data can be historical feedback work orders, historical conversation content, etc., and the second preset threshold can be set according to requirements, and the present disclosure does not limit this.

[0052] For example, when there is a second similar question in the historical feedback data, the second similar question can be shown to the user, such as Figure 3 the questions shown under "Guessing What You Want to Ask" as shown, and the detailed information of the similar questions, such as the solution method, etc., can be viewed, opening up ideas for the user to solve the problem and reducing the number of work orders submitted by the user.

[0053] It should be noted that it can be determined that the intelligent agent cannot answer the user's question when the first similar question does not exist in the Q&A knowledge base, or it can also be determined that the intelligent agent cannot answer the user's question when the first similar question does not exist in the Q&A knowledge base and the second similar question does not exist in the historical feedback data. Then, when the intelligent agent cannot answer the user's question, the work order submitted by the user can be automatically continued, reducing user operations and improving the submission efficiency of the work order.

[0054] In a possible way, showing the second similar question on the intelligent interaction page includes: showing the second similar question in a first style on the intelligent interaction page. Showing the target answer on the intelligent interaction page includes: showing the target answer in a second style on the intelligent interaction page, and the first style is different from the second style.

[0055] For example, when comparing the first similar question with the second similar question, the relevance between the first similar question and the target question is greater than the relevance between the second similar question and the target question. Therefore, the probability that the solution method of the first similar question can solve the problem feedback by the user is greater than the probability that the solution method of the second similar question can solve the problem feedback by the user.

[0056] Through different display styles, the relevance between the displayed content and the problem feedback by the user can be intuitively shown, facilitating the user to quickly view the response content and improving the problem-solving efficiency. For example Figure 2 shows the target answer in a card style, Figure 3 shows the second similar question in the form of a question list, and the specific display style can be set according to requirements, and the present disclosure does not limit this.

[0057] In other possible implementation manners, the first similar question, the target answer, and the second similar question can also be shown simultaneously to open up ideas for the user to solve the problem and reduce the number of work orders submitted by the user.

[0058] In a possible way, the A / B test question response method further includes: displaying an interaction entry for the first similar question on the intelligent interaction page; in response to a trigger operation on the interaction entry for the first similar question, displaying an interaction dialogue page; wherein, when the first similar question is in a resolved state, the target answer corresponding to the first similar question is displayed on the interaction dialogue page, and when the first similar question is in an unresolved state, problem-related information is displayed on the interaction dialogue page, and the problem-related information includes at least one of information about customer service staff, a problem description generated by a large model based on the target question, and a problem summary.

[0059] Exemplarily, an interaction entry for the first similar question can be displayed on the intelligent interaction page, such as Figure 2 the "join group" control in. Through the interaction entry, users can enter the communication group for similar questions, and the group members can include other users who encounter this problem and the customer service staff who solve this problem. It should be noted that the communication group can be for not only the first similar question but also other questions, that is, the same communication group can be used to communicate a single question or multiple different questions, and the present disclosure does not limit this.

[0060] Exemplarily, after the user triggers the "join group" control, the interaction dialogue page is displayed. As Figure 4 shown, if the first similar question has been resolved, the solution to the similar question can be displayed, so that the users who enter the group do not need to pay attention to the details of the conversation and can directly refer to the solution to the similar question to complete the problem handling, improving the efficiency of solving user problems. In addition, the solution to the similar question can also be posted as a group announcement to help other users who encounter similar questions understand the solution to the problem.

[0061] Exemplarily, continuing to refer to Figure 4 , if the first similar question is unresolved, the target work order created by the user can be obtained, the problems feedback by the user can be summarized, such as the problem description and problem summary generated by a large model based on the target question, and the corresponding customer service staff information can be provided to help the user sort out the problems to be feedback and know the information of the corresponding customer service staff, improving the user experience. In addition, an editing function for the problem-related information can also be provided so that the user can supplement and modify the problems feedback, such as more accurately describing the problems the user needs to feedback, improving the efficiency of problem solving.

[0062] In a possible way, the A / B test question response method further includes: in response to receiving multiple fourth similar questions within a preset duration, displaying interaction entries for the multiple fourth similar questions on the intelligent interaction page and / or sending an exception reminder to the customer service staff of the A / B test platform, and the similarity between each of the multiple fourth similar questions and the target question is greater than a fourth preset threshold.

[0063] Exemplarily, when multiple similar feedback problems are received within a preset duration, communication entrances for multiple similar feedback problems can be presented to the user, so that customer service staff on the platform can uniformly provide solutions for users encountering similar problems, improving the efficiency of users obtaining problem-solving methods and reducing the number of work orders submitted by users. The fourth preset threshold and the preset duration can be set according to requirements, and the present disclosure places no restrictions on this.

[0064] Exemplarily, if multiple similar feedback problems are received within the preset duration, it indicates that an anomaly may have occurred on the platform. An anomaly reminder can be sent to the customer service staff to facilitate the timely resolution of problems by the customer service staff and prevent the scope of the anomaly from expanding.

[0065] In a possible manner, the A / B test problem response method further includes: displaying, on the intelligent interaction page, a prompt message and / or a submission entrance for prompting the user to continue submitting a target work order.

[0066] Exemplarily, as Figure 5 shown, if the user has joined the communication group, a prompt message for prompting the user to continue submitting the target work order and / or a submission entrance for submitting the target work order can be displayed, so that when the user does not obtain a problem-solving method within the communication group, the problem can be reported in the form of a work order.

[0067] It should be noted that if the user has an intercepted target work order, the intercepted target work order can be continued to be submitted by triggering the "Submit Work Order" control. If the user reports a problem in the form of a conversation, triggering the "Submit Work Order" control can display a work order editing area for the user to edit the work order.

[0068] In a possible manner, the A / B test problem response method further includes: in response to the relevance of the problem sent by the user through the input box on the intelligent interaction page being lower than the third preset threshold, displaying a first answer on the intelligent interaction page, where the first answer is used to prompt the user to re-enter the problem; in response to the work order creation operation by the user when no problem is sent or the relevance of the sent problem to the A / B test platform is lower than the third preset threshold, displaying a second answer on the intelligent interaction page, where the second answer is used to prompt the user to enter a problem through the input box.

[0069] Exemplarily, referring to Figure 6 , if the content sent by the user on the intelligent interaction page has a relevance to the A / B test platform lower than the third preset threshold, such as a problem irrelevant to the platform or invalid content, a prompt message for prompting the user to re-describe the problem can be displayed to help the user correctly report the problem and improve the problem-solving efficiency.

[0070] Exemplarily, continuing to refer to Figure 6, if the user triggers the control 21 for submitting a work order when no questions are sent or the relevance of the sent questions to the A / B test platform is lower than the third preset threshold, a prompt message can be displayed to prompt the user to input questions through the input box on the intelligent interaction page, recommending that the user first use the intelligent assistant to obtain the solution to the questions, improving the question-solving efficiency and reducing the user's submission of work orders.

[0071] In a possible way, the A / B test question response method further includes: in response to the sending operation of the question input in the input box on the intelligent interaction page, determining whether the intelligent agent can handle the input question; in the case of determining that the intelligent agent can handle the input question, obtaining the third similar question and the corresponding third answer of the third similar question in the Q&A knowledge base through the large model, where the similarity between the third similar question and the input question is greater than the first preset threshold; and displaying the third answer on the intelligent interaction page.

[0072] Exemplarily, referring to Figure 7 , the user can feedback questions by means of chatting with the intelligent agent through the input box. Similar to the way of feedback questions through work orders, the intent of the sent input question can be recognized first, and whether there is a third similar question similar to it is searched in the Q&A knowledge base. If a third similar question similar to it is found, it means that the intelligent agent can handle the input question, and then the answer corresponding to the third similar question is displayed, reducing the user's submission of work orders, thereby reducing the waiting time for the user to get an answer and the waste of platform resources.

[0073] Exemplarily, if there is no third similar question similar to it in the Q&A knowledge base, it can be searched whether there is a historical feedback question similar to it in the historical feedback data. If a historical feedback question similar to it is found, the historical feedback question can also be displayed.

[0074] By adopting the above method, when the user feedbacks questions in a chatting way, the answers to similar questions can also be shown to the user and the user can be guided to communicate in the group, facilitating the user to quickly obtain the solution to the questions and reducing the user's submission of work orders. The specific process can refer to the embodiments of feedbacking questions through work orders, which will not be elaborated herein.

[0075] In a possible way, the A / B test question response method further includes: displaying a question announcement on the intelligent interaction page, where the question announcement is used to explain the abnormal questions of the A / B test platform; in response to the triggering operation of the question announcement, expanding and displaying the announcement details of the question announcement.

[0076] Exemplarily, such as Figure 8As shown in the figure, the A / B test platform provides a platform announcement function. Customer service staff can issue abnormal announcements for platform abnormalities so that users can timely understand the abnormal problems that occur on the platform, reduce the problem tickets submitted by users related to platform abnormalities, and avoid the resource pressure brought by repeated tickets in the short term. It is also possible to create a communication group for platform abnormalities so that users can enter the communication group to obtain corresponding solutions.

[0077] Exemplarily, the announcement details of the problem announcement can be expanded and displayed, such as explaining the reasons for the abnormal problems and the solutions to solve the abnormal problems, etc. The present disclosure does not limit this.

[0078] By adopting the above method, it is possible to reduce the creation of problem tickets by users through methods such as platform announcements and guiding similar problems into the group before the users create problem tickets. When the users create problem tickets, by judging the historical interaction content of the users, it is preferred to recommend that the users solve the problems in a conversational manner first, reducing the creation of problem tickets by the users. After the users create problem tickets, intercept the problem tickets. When the intelligent agent can solve the users' problems, output the solutions to the users' problems through the large model. For example, provide solutions to the problems feedback by the users through the Q&A knowledge base or historical feedback data, reducing the submission of problem tickets by the users. Thus, it helps the users obtain relevant answers to the feedback problems faster, avoids too long processing time of the problems, and at the same time reduces the input of duplicate resources on the platform.

[0079] Based on the same concept, the embodiments of the present disclosure also provide an A / B test problem response device based on a large model intelligent agent, as Figure 9 shown. The A / B test problem response device 900 based on the large model intelligent agent may include: A first display module 901, configured to display an intelligent interaction page associated with the intelligent agent in response to a triggering operation on the intelligent agent identifier in the A / B test platform, where the intelligent agent is used to respond to problems feedback by a user in the intelligent interaction page through a large model; An interception module 902, configured to intercept a target work order request corresponding to the work order creation operation sent to the work order server in response to a work order creation operation in the intelligent interaction page, and determine whether the intelligent agent can process a target problem in the target work order corresponding to the work order creation operation, where the work order server is configured to generate a work order to be processed by customer service staff in response to a work order request; An acquisition module 903, configured to obtain a first similar problem and a target answer corresponding to the first similar problem in a Q&A knowledge base through the large model when it is determined that the intelligent agent can process the target problem, where the similarity between the first similar problem and the target problem is greater than a first preset threshold; A second display module 904, configured to display the target answer in the intelligent interaction page.

[0080] Optionally, a work order submission control is displayed on the intelligent interaction page, and the interception module 902 is configured to: In response to a work order editing operation on the intelligent interaction page, display a target work order corresponding to the work order editing operation; In response to a first trigger operation on the work order submission control on the intelligent interaction page, intercept a target work order request corresponding to the target work order sent to the work order server; The A / B test question response device 900 based on the large model intelligent agent further includes a control module, and the control module is configured to: During the process of displaying the target answer, control the work order submission control to be in a non-triggerable state; After displaying the target answer, control the work order submission control to be in a triggerable state; In response to a second trigger operation on the work order submission control, send the target work order request to the work order server.

[0081] Optionally, the interception module 902 is configured to: Perform intent recognition on the target question through the large model. In the case where the first similar question exists in the Q&A knowledge base, determine that the intelligent agent can handle the target question in the target work order corresponding to the work order creation operation.

[0082] Optionally, the A / B test question response device 900 based on the large model intelligent agent further includes: A search module, configured to search whether there is a second similar question in the historical feedback data in the case where the first similar question does not exist in the Q&A knowledge base, and the similarity between the target question and the second similar question is greater than a second preset threshold; A third display module, configured to display the second similar question on the intelligent interaction page in the case where the second similar question exists in the historical feedback data.

[0083] Optionally, the third display module is configured to: Display the second similar question on the intelligent interaction page in a first style; The second display module 904 is configured to: Display the target answer on the intelligent interaction page in a second style, where the first style is different from the second style.

[0084] Optionally, the A / B test question response device 900 based on the large model intelligent agent further includes a fourth display module, and the fourth display module is configured to: Display an entry for communicating about the first similar question on the intelligent interaction page; In response to a trigger operation on the entry for communicating about the first similar question, display a communication dialogue page; Wherein, when the first similar question is in a resolved state, the communication dialogue page displays the target answer corresponding to the first similar question, and when the first similar question is in an unresolved state, the communication dialogue page displays problem-related information, and the problem-related information includes at least one of customer service staff information, a problem description generated by the large model based on the target question, and a problem summary.

[0085] Optionally, the A / B test question response device 900 based on a large model intelligent agent further includes a fifth display module, and the fifth display module is configured to: Display prompt information and / or a submission entry for prompting the user to continue submitting the target work order on the intelligent interaction page.

[0086] Optionally, the A / B test question response device 900 based on a large model intelligent agent further includes a sixth display module, and the sixth display module is configured to: In response to the problem sent by the user through the input box on the intelligent interaction page having a relevance to the A / B test platform lower than a third preset threshold, display a first answer on the intelligent interaction page, and the first answer is used to prompt the user to re-enter the problem; In response to the user's work order creation operation when no problem is sent or the sent problem has a relevance to the A / B test platform lower than the third preset threshold, display a second answer on the intelligent interaction page, and the second answer is used to prompt the user to enter a problem through the input box.

[0087] Optionally, the A / B test question response device 900 based on a large model intelligent agent further includes a seventh display module, and the seventh display module is configured to: In response to a sending operation of the problem entered in the input box on the intelligent interaction page, determine whether the intelligent agent can process the input problem; When it is determined that the intelligent agent can process the input problem, obtain a third similar question and the third answer corresponding to the third similar question in the Q&A knowledge base through the large model, wherein the similarity between the third similar question and the input problem is greater than the first preset threshold; Display the third answer on the intelligent interaction page.

[0088] Optionally, the A / B test question response device 900 based on a large model intelligent agent further includes a notice module, and the notice module is configured to: Display a problem announcement on the intelligent interaction page, where the problem announcement is used to explain the abnormal problems of the A / B test platform; In response to a trigger operation on the problem announcement, expand and display the announcement details of the problem announcement.

[0089] Optionally, the A / B test problem response device 900 based on the large model agent further includes a response module, and the response module is configured to: In response to receiving multiple fourth similar problems within a preset duration, display communication entrances of the multiple fourth similar problems on the intelligent interaction page and / or send an exception reminder to the customer service staff of the A / B test platform. The similarity between the multiple fourth similar problems and the target problem is greater than a fourth preset threshold.

[0090] Based on the same concept, an embodiment of the present disclosure further provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processing device, the steps of any of the above-mentioned A / B test problem response methods based on the large model agent are implemented.

[0091] Based on the same concept, an embodiment of the present disclosure further provides an electronic device, which may include: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of any of the above-mentioned A / B test problem response methods based on the large model agent.

[0092] Based on the same concept, an embodiment of the present disclosure further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned A / B test problem response methods based on the large model agent are implemented.

[0093] Next, refer to Figure 10 , which shows a schematic structural diagram of an electronic device 1000 suitable for implementing the embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 10 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0094] As Figure 10As shown, the electronic device 1000 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 1001, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage device 1008 into the random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. The input / output (I / O) interface 1005 is also connected to the bus 1004.

[0095] Generally, the following devices may be connected to the I / O interface 1005: an input device 1006 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1007 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1008 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 10 an electronic device 1000 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0096] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above functions defined in the method of the embodiment of the present disclosure are executed.

[0097] It should be noted that the above-mentioned computer-readable medium in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0098] In some embodiments, any currently known or future-developed network protocol, such as HTTP (HyperText Transfer Protocol), can be used for communication, and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0099] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; or it can exist separately without being assembled into the electronic device.

[0100] The above computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: display an intelligent interaction page associated with the agent in response to a trigger operation on the agent identifier in the A / B test platform, where the agent is used to respond to questions feedback by the user in the intelligent interaction page through a large model; intercept a target work order request corresponding to the work order creation operation sent to the work order server in response to a work order creation operation in the intelligent interaction page, and determine whether the agent can handle the target problem in the target work order corresponding to the work order creation operation, where the work order server is used to generate a work order to be processed by a customer service staff in response to a work order request; in the case of determining that the agent can handle the target problem, obtain a first similar problem and a target answer corresponding to the first similar problem in a question-and-answer knowledge base through the large model, where the similarity between the first similar problem and the target problem is greater than a first preset threshold; display the target answer in the intelligent interaction page.

[0101] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0103] The modules described in the embodiments of the present disclosure can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0104] The functions described above herein can be performed, at least in part, by one or more hardware logic components. By way of example, and not limitation, the types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0105] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

[0107] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, although a number of specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments may also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0108] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms for implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated herein.

Claims

1. A method for responding to A / B test questions based on large model agents, characterized in that, The A / B test question response method based on the large model agent includes: In response to a triggering operation on the agent identifier in the A / B test platform, display the intelligent interaction page associated with the agent, where the agent is used to respond to questions fed back by the user in the intelligent interaction page through the large model; In response to a work order creation operation in the intelligent interaction page, intercept the target work order request corresponding to the work order creation operation sent to the work order server, and determine whether the agent can handle the target question in the target work order corresponding to the work order creation operation. Among them, the work order server is used to generate a work order to be processed by the customer service staff in response to the work order request; In the case of determining that the agent can handle the target question, obtain the first similar question and the target answer corresponding to the first similar question in the Q&A knowledge base through the large model, where the similarity between the first similar question and the target question is greater than the first preset threshold; Display the target answer on the intelligent interaction page.

2. The A / B test problem response method based on a large model intelligent agent according to claim 1, wherein The intelligent interaction page displays a work order submission control. The intercepting the target work order request corresponding to the work order creation operation sent to the work order server in response to the work order creation operation in the intelligent interaction page includes: In response to a work order editing operation in the intelligent interaction page, display the target work order corresponding to the work order editing operation; In response to a first triggering operation on the work order submission control in the intelligent interaction page, intercept the target work order request corresponding to the target work order sent to the work order server; The A / B test question response method further includes: During the process of displaying the target answer, control the work order submission control to be in a non-triggerable state; After displaying the target answer, control the work order submission control to be in a triggerable state; In response to a second triggering operation on the work order submission control, send the target work order request to the work order server.

3. The A / B test question response method based on a large model intelligent agent according to claim 1, wherein The determining whether the agent can handle the target question in the target work order corresponding to the work order creation operation includes: Perform intent recognition on the target question through the large model. In the case where the first similar question exists in the Q&A knowledge base, determine that the agent can handle the target question in the target work order corresponding to the work order creation operation.

4. The A / B test question response method based on a large model agent according to claim 3, wherein The A / B test question response method further includes: In the case where the first similar question does not exist in the Q&A knowledge base, check whether there is a second similar question in the historical feedback data, where the similarity between the target question and the second similar question is greater than the second preset threshold; In the case where the second similar question exists in the historical feedback data, display the second similar question on the intelligent interaction page.

5. The A / B test question response method based on a large model agent according to claim 4, wherein The displaying the second similar question on the intelligent interaction page includes: Display the second similar question on the intelligent interaction page in a first style; The displaying the target answer on the intelligent interaction page includes: Display the target answer on the intelligent interaction page in a second style, where the first style is different from the second style.

6. The A / B test question response method based on a large model agent according to any one of claims 1-5, characterized in that, The A / B test question response method further includes: Displaying an interaction entry for the first similar question on the intelligent interaction page; In response to a trigger operation on the interaction entry for the first similar question, displaying an interaction dialogue page; Wherein, when the first similar question is in a resolved state, the target answer corresponding to the first similar question is displayed on the interaction dialogue page, and when the first similar question is in an unresolved state, problem-related information is displayed on the interaction dialogue page, and the problem-related information includes at least one of information such as customer service staff information, problem descriptions generated by the large model based on the target question, and problem summaries.

7. The A / B test question response method based on the large model agent according to claim 6, wherein, The A / B test question response method further includes: Displaying prompt information and / or a submission entry for prompting the user to continue submitting the target work order on the intelligent interaction page.

8. The A / B test question response method based on a large model agent according to any one of claims 1-5, characterized in that, The A / B test question response method further includes: In response to the problem sent by the user through the input box on the intelligent interaction page having a relevance lower than a third preset threshold to the A / B test platform, displaying a first answer on the intelligent interaction page, where the first answer is used to prompt the user to re-enter the problem; In response to a work order creation operation by the user when no problem is sent or the sent problem has a relevance lower than the third preset threshold to the A / B test platform, displaying a second answer on the intelligent interaction page, where the second answer is used to prompt the user to enter a problem through the input box.

9. The A / B test question response method based on a large model agent according to any one of claims 1-5, characterized in that, The A / B test question response method further includes: In response to a send operation for the problem entered in the input box on the intelligent interaction page, determining whether the intelligent agent can process the input problem; When it is determined that the intelligent agent can process the input problem, obtaining a third similar question and the third answer corresponding to the third similar question from the Q&A knowledge base through the large model, where the similarity between the third similar question and the input problem is greater than the first preset threshold; Displaying the third answer on the intelligent interaction page.

10. The A / B test question response method based on the large model agent according to any one of claims 1-5, characterized in that, The A / B test question response method further includes: Displaying a problem announcement on the intelligent interaction page, where the problem announcement is used to explain abnormal problems of the A / B test platform; In response to a trigger operation on the problem announcement, expanding and displaying the announcement details of the problem announcement.

11. The A / B test question response method based on a large model agent according to any one of claims 1-5, characterized in that, The A / B test question response method further includes: In response to receiving multiple fourth similar questions within a preset time period, displaying interaction entries for the multiple fourth similar questions on the intelligent interaction page and / or sending an exception reminder to the customer service staff of the A / B test platform, where the similarity between each of the multiple fourth similar questions and the target question is greater than a fourth preset threshold.

12. An A / B test question response device based on a large model intelligent agent, characterized in that, The A / B test question response device based on a large model intelligent agent includes: A first display module, configured to display an intelligent interaction page associated with the intelligent agent in response to a trigger operation on the intelligent agent identifier in the A / B test platform, where the intelligent agent is used to respond to problems feedback by the user on the intelligent interaction page through the large model. An interception module, configured to intercept a target work order request corresponding to the work order creation operation sent to the work order server in response to a work order creation operation in the intelligent interaction page, and determine whether the intelligent agent can handle a target problem in the target work order corresponding to the work order creation operation, where the work order server is configured to generate a work order to be processed by a customer service staff in response to a work order request; An acquisition module, configured to, when determining that the intelligent agent can handle the target problem, obtain a first similar problem and a target answer corresponding to the first similar problem in a question and answer knowledge base through the large model, where the similarity between the first similar problem and the target problem is greater than a first preset threshold; A second display module, configured to display the target answer on the intelligent interaction page.

13. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processing device, it implements the steps of the method according to any one of claims 1-11.

14. An electronic device, characterized in that, Comprising: A storage device, on which a computer program is stored; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1-11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-11.

Images