Method, device and equipment for guiding user operation based on large model and readable storage medium

Abstract

The invention relates to the field of artificial intelligence, and particularly provides a method, device and equipment for guiding user operation based on a large model and a readable storage medium, and the method comprises the steps: obtaining voice reply information of a user in a process of guiding the user to operate according to a process node; recognizing the voice reply information and the actual operation of the user through a preset large language model to obtain a voice recognition result and an actual operation state; comparing the voice recognition result with the actual operation state to obtain a comparison result; and guiding the user to perform related operation according to a comparison result. Through the method, the effect of quickly guiding the user to perform accurate operation can be achieved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence. Specifically, it relates to a method, device, equipment and readable storage medium for guiding user operations based on a large model. Background Art

[0002] In the process of robot marketing, robots usually guide users to perform operations, such as guiding users to download application programs, guiding users to open SMS connections, guiding users to follow public accounts and click on specified menus of public accounts, etc. In this process, the robot has to constantly recognize what the user says and detect the user's operations.

[0003] During the guiding process, it may be possible to misunderstand the user's intention, or the user operates relatively quickly and the robot fails to keep up with the user's rhythm, resulting in a very poor user experience.

[0004] Therefore, how to quickly guide users to perform accurate operations is a technical problem that needs to be solved. Summary of the Invention

[0005] The purpose of the embodiments of this application is to provide a method for guiding user operations based on a large model. Through the technical solutions of the embodiments of this application, the effect of quickly guiding users to perform accurate operations can be achieved.

[0006] In a first aspect, the embodiments of this application provide a method for guiding user operations based on a large model, including: during the process of guiding the user to perform operations according to process nodes, obtaining the user's voice reply information; identifying the voice reply information and the user's actual operations through a preset large language model to obtain a voice recognition result and an actual operation state; comparing the voice recognition result and the actual operation state to obtain a comparison result; guiding the user to perform relevant operations according to the comparison result.

[0007] In the above embodiments of this application, while identifying the user's reply information through the large language model, it also identifies whether the user's current operation is consistent with the reply information, and guides the corresponding operation according to the comparison result, so as to quickly guide the user to perform accurate operations.

[0008] In some embodiments, comparing the voice recognition result and the actual operation state to obtain a comparison result includes: comparing the operation completion situation in the voice recognition result with the actual operation completion situation corresponding to the actual operation state to determine whether the operation completion situation and the actual operation completion situation are consistent.

[0009] In the above embodiments of this application, it is possible to compare the operation completion situation in the voice recognition result with the actual operation completion situation corresponding to the actual operation state to determine whether the user's reply and the actual operation are consistent.

[0010] In some embodiments, according to the comparison result, the user is guided to perform relevant operations, including: if the operation completion status is consistent with the actual operation completion status, guiding the user to perform the operation of the next node of the process node; if the operation completion status is inconsistent with the actual operation completion status, guiding the user to repeat the current node operation of the process node; after guiding the user to repeat the current node operation of the process node multiple times, guiding the user to perform the operation of the next node of the process node.

[0011] In the above embodiments of the present application, according to the result of whether the user's reply is consistent with the actual operation, different guiding methods can be selected to guide the user to perform corresponding operations.

[0012] In some embodiments, before obtaining the user's voice reply information during the process of guiding the user to perform operations according to the process node, it further includes: training a basic model through the voice information of multiple users and / or the operation situation of the user to obtain a large language model.

[0013] In the above embodiments of the present application, by pre-training a large language model, the model can be made to learn to recognize the user's replied voice while recognizing the user's operation.

[0014] In a second aspect, an embodiment of the present application provides a device for guiding user operations based on a large model, including:

[0015] An acquisition module, configured to acquire the user's voice reply information during the process of guiding the user to perform operations according to the process node;

[0016] An identification module, configured to identify the voice reply information and the actual operation of the user through a preset large language model to obtain a voice recognition result and an actual operation status;

[0017] A comparison module, configured to compare the voice recognition result with the actual operation status to obtain a comparison result;

[0018] A guiding module, configured to guide the user to perform relevant operations according to the comparison result.

[0019] Optionally, the comparison module is specifically configured to:

[0020] Compare the operation completion situation in the voice recognition result with the actual operation completion situation corresponding to the actual operation status to determine whether the operation completion situation is consistent with the actual operation completion situation.

[0021] Optionally, the guiding module is specifically configured to:

[0022] If the operation completion situation is consistent with the actual operation completion situation, guide the user to perform the operation of the next node of the process node;

[0023] If the operation completion status does not match the actual operation completion status, guide the user to repeat the operation of the current node of the process node;

[0024] After guiding the user to repeat the operation of the current node of the process node multiple times, guide the user to perform the operation of the next node of the process node.

[0025] Optionally, the device further includes:

[0026] A training module, which is used to train a basic model with voice information of multiple users and / or the operation status of the user before the acquisition module obtains the voice reply information of the user during the process of guiding the user to perform operations according to the process node, so as to obtain a large language model.

[0027] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, where the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect above are run.

[0028] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the method provided in the first aspect above are run.

[0029] Other features and advantages of the present application will be described in the subsequent description, and part of them will become obvious from the description, or be understood by implementing the embodiments of the present application. The objectives and other advantages of the present application can be achieved and obtained through the structures specifically pointed out in the written description, claims, and drawings. Description of the Drawings

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0031] Figure 1 It is a flowchart of a method for guiding user operations based on a large model provided by an embodiment of the present application;

[0032] Figure 2 It is a schematic block diagram of a device for guiding user operations based on a large model provided by an embodiment of the present application;

[0033] Figure 3 It is a structural schematic diagram of a device for guiding user operations based on a large model provided by an embodiment of the present application. Detailed Embodiments

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application to be protected, but only represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0035] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of the present application, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0036] The present application is applied to the scenario where artificial intelligence guides users to perform relevant operations. The specific scenario is to first guide users to perform some operations, identify the completion status of the users based on the voice reply information, and the completion status of the actual operations, and then perform the next step of guidance according to the comparison result.

[0037] In the process of robot marketing, the robot usually guides users to perform operations, such as: guiding users to download application programs, guiding users to open SMS connections, guiding users to follow public accounts and click on the specified menus of public accounts, etc. In this process, the robot has to constantly identify what the user says and detect the user's operations. During the guidance process, it is possible to misunderstand the user's intention, or the user operates relatively quickly and the robot fails to keep up with the user's rhythm, resulting in a very poor user experience.

[0038] Therefore, in the process of guiding users to perform operations according to process nodes, the present application obtains the voice reply information of the users; identifies the voice reply information and the actual operations of the users through a preset large language model to obtain the voice recognition result and the actual operation status; compares the voice recognition result and the actual operation status to obtain a comparison result; and guides the users to perform relevant operations according to the comparison result. While identifying the user's reply information through the large language model, it also identifies whether the user's current operation is consistent with the reply information, and guides the corresponding operation according to the comparison result, so as to quickly guide the users to perform accurate operations.

[0039] In the embodiments of the present application, the execution entity may be a device for guiding a user to operate the operating system based on a large model. In practical applications, the device for guiding a user to operate based on a large model may be an electronic device such as a terminal device and a server, which is not limited herein.

[0040] Next, in combination with Figure 1 the method for guiding a user to operate based on a large model in the embodiments of the present application will be described in detail.

[0041] Please refer to Figure 1 , Figure 1 which is a flowchart of a method for guiding a user to operate based on a large model provided by the embodiments of the present application. As Figure 1 shown, the method for guiding a user to operate based on a large model includes:

[0042] Step 110: During the process of guiding the user to operate according to the process node, obtain the voice reply information of the user.

[0043] Among them, the process node may be a certain node of any process. The process may be a sales process in a robot sales scenario, an addition process in a scenario where a robot adds customer contact information, a redemption process where a robot guides a user to redeem a preferential activity, or an operation node in a related process scenario such as guiding a user to open WeChat, search for WeChat, follow a public account, enter a coupon code, view an activity, purchase a product, and finally make a payment. The voice reply information may be the completion status of the user after completing one or more operations, or suggestions put forward by the user under the condition of relevant guiding operations, or the attitude of whether the user is willing to proceed with the process, etc.

[0044] Optionally, the process may be to guide the user to follow a public account to redeem a preferential offer and purchase a product, and the user makes a payment through Alipay. Specifically, each process node includes: guiding the user to open WeChat; guiding the user to find the WeChat search box; guiding the user to click the public account button and enter the public account name; guiding the user to click to follow; guiding the user to click on the link in the public account; guiding the user to enter the user's exclusive coupon code; guiding the user to view the guiding activity; guiding the user to redeem the activity offer; guiding the user to purchase a new product; guiding the user to open the default browser; guiding the user to jump to Alipay for payment.

[0045] In some embodiments of the present application, before obtaining the voice reply information of the user during the process of guiding the user to operate according to the process node, it further includes: training a basic model through the voice information of multiple users and / or the operation situation of the users to obtain a large language model.

[0046] In the above process of the present application, the large language model can be pre-trained to enable the model to learn to recognize the user's replied voice while recognizing the user's operations.

[0047] Among them, the basic model can be an open-source basic speech recognition model and / or an operation detection model. By using the basic model to detect the operation status of users based on the speech information of different users in the past and the speech information of the user's reply, it is determined whether the speech reply information and the operation status are synchronized and consistent. By comparing the recognition result with the result pre-stored in the system, the parameters of the basic model are adjusted, and a large language model is obtained. Finally, relevant verification operations are performed on the large language model.

[0048] Step 120: Use the preset large language model to recognize the speech reply information and the actual operation of the user, and obtain the speech recognition result and the actual operation status.

[0049] Among them, the speech reply information includes the completion status of the user's guidance operation under the current node. The actual operation can be to set a monitor to monitor the user's operation status in real time. For example, click status, link jump status, and operation implementation progress, etc. The speech recognition result includes the result of whether the user has completed the current operation.

[0050] Step 130: Compare the speech recognition result with the actual operation status to obtain a comparison result.

[0051] In some embodiments of the present application, comparing the speech recognition result with the actual operation status to obtain a comparison result includes: comparing the operation completion status in the speech recognition result with the actual operation completion status corresponding to the actual operation status to determine whether the operation completion status and the actual operation completion status are consistent.

[0052] In the above process of the present application, it is possible to compare the operation completion status in the speech recognition result with the actual operation completion status corresponding to the actual operation status to determine whether the user's reply and the actual operation are consistent.

[0053] Among them, the operation completion status represents the completion status of the guidance operation considered by the user himself, and the above completion status is recognized through the speech reply information.

[0054] For example, after the user inputs information, and there will be corresponding events when clicking the front end of the button, and the corresponding events are transmitted to the server. Every time the user responds with a sentence, the large language model will query the actual operation status of the current user on the server.

[0055] Optionally, when detecting the actual operation status of the user, the current user's status can be determined through both the large language model and the buried point method. During the conversation between the system and the user, the actual operation status of the user is comprehensively judged based on the user's buried point status and the large language model status obtained by the system.

[0056] Step 140: Guide the user to perform relevant operations according to the comparison result.

[0057] Among them, the related operations include guiding the user to perform operations at the next process node and repeating the operations at the current process node, and may also include hanging up the user's operations, etc.

[0058] In some embodiments of the present application, according to the comparison result, the user is guided to perform related operations, including: if the operation completion situation is consistent with the actual operation completion situation, guiding the user to perform the operation at the next node of the process node; if the operation completion situation is inconsistent with the actual operation completion situation, guiding the user to repeat the operation at the current node of the process node; after guiding the user to repeat the operation at the current node of the process node multiple times, guiding the user to perform the operation at the next node of the process node.

[0059] In the above process, the present application can select different guiding methods according to the user's reply and the result of whether the actual operation is consistent, and guide the user to perform corresponding operations.

[0060] Among them, the number of times can be set according to requirements.

[0061] For example, when the user is at the node of entering the activation code, the activation code entered is incorrect. For example, at this time, the user says that a coupon code has been entered, and the robot starts to introduce the activity discount. In fact, the user has entered it wrong, and it is inappropriate for the robot to introduce the activity discount at this time. At this time, the background will monitor in real time whether the user has really entered the coupon code, and comprehensively guide the user to perform the operation at the next node of the process node or repeat the operation at the current node according to the user's operation status and answer.

[0062] For example, if the user indicates that the coupon code has been entered, but the actual status is that it has not been entered, this situation will stay at the current node three times (the user will be asked whether the coupon code was entered incorrectly or if there are other problems three times), and then proceed to the next node.

[0063] In the above Figure 1 In the process shown, in the process of guiding the user to perform operations according to the process node, the present application obtains the user's voice reply information; recognizes the voice reply information and the user's actual operation through a preset large language model to obtain the voice recognition result and the actual operation status; compares the voice recognition result and the actual operation status to obtain a comparison result; according to the comparison result, guides the user to perform related operations, while recognizing the user's reply information through the large language model, recognizes whether the user's operation at this time is consistent with the reply information, and guides the corresponding operation according to the comparison result, so as to quickly guide the user to perform accurate operations.

[0064] The foregoing has described the method for guiding user operations based on a large model. Next, a device for guiding user operations based on a large model will be described in conjunction with Figure 1 the following Figure 2 - Figure 3 description.

[0065] Please refer to Figure 2 , which is a schematic block diagram of a device 200 for guiding user operations based on a large model provided in an embodiment of the present application. The device 200 may be a module, a program segment, or code on an electronic device. The device 200 corresponds to the above Figure 1 method embodiment and can execute Figure 1 each step involved in the method embodiment. The specific functions of the device 200 can be seen in the following description. To avoid repetition, the detailed description is appropriately omitted here.

[0066] Optionally, the device 200 includes:

[0067] An acquisition module 210, configured to obtain the voice response information of the user during the process of guiding the user to operate according to the process node;

[0068] An identification module 220, configured to identify the voice response information and the actual operation of the user through a preset large language model to obtain a voice recognition result and an actual operation state;

[0069] A comparison module 230, configured to compare the voice recognition result with the actual operation state to obtain a comparison result;

[0070] A guidance module 240, configured to guide the user to perform relevant operations according to the comparison result.

[0071] Optionally, the comparison module is specifically configured to:

[0072] Compare the operation completion situation in the voice recognition result with the actual operation completion situation corresponding to the actual operation state to determine whether the operation completion situation is consistent with the actual operation completion situation.

[0073] Optionally, the guidance module is specifically configured to:

[0074] If the operation completion situation is consistent with the actual operation completion situation, guide the user to perform the operation of the next node of the process node; if the operation completion situation is inconsistent with the actual operation completion situation, guide the user to repeat the operation of the current node of the process node; after guiding the user to repeat the operation of the current node of the process node multiple times, guide the user to perform the operation of the next node of the process node.

[0075] Optionally, the device further includes:

[0076] A training module, configured to train a basic model through the voice information of multiple users and / or the operation situations of users to obtain a large language model before the acquisition module obtains the voice response information of the user during the process of guiding the user to operate according to the process node.

[0077] Please refer to Figure 3Schematic block diagram of a device for guiding user operations based on a large model provided in an embodiment of the present application. The device may include a memory 310 and a processor 320. Optionally, the device may further include: a communication interface 330 and a communication bus 340. The device corresponds to the above Figure 1 method embodiment and is capable of executing Figure 1 each step involved in the method embodiment. The specific functions of the device can be referred to the description below.

[0078] Specifically, the memory 310 is used to store computer-readable instructions.

[0079] The processor 320 is used to process the readable instructions stored in the memory and is capable of executing Figure 1 each step in the method.

[0080] The communication interface 330 is used for signaling or data communication with other node devices. For example: for communication with a server or a terminal, or for communication with other device nodes. The embodiments of the present application are not limited to this.

[0081] The communication bus 340 is used to implement the direct connection communication between the above components.

[0082] Among them, the communication interface 330 of the device in the embodiment of the present application is used for signaling or data communication with other node devices. The memory 310 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. Optionally, the memory 310 may also be at least one storage device located far from the aforementioned processor. The memory 310 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 320, the electronic device executes the above Figure 1 shown method process. The processor 320 may be used on the device 200 and is used to execute the functions in the present application. Exemplarily, the above-mentioned processor 320 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The embodiments of the present application are not limited to this.

[0083] The embodiment of the present application also provides a readable storage medium. When the computer program is executed by the processor, it executes the method process executed by the electronic device in the Figure 1 shown method embodiment.

[0084] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described device can refer to the corresponding process in the foregoing method, and will not be elaborated herein.

[0085] In summary, the embodiments of the present application provide a method, device, equipment, and readable storage medium for guiding user operations based on a large model. The method includes obtaining voice response information of the user during the process of guiding the user to perform operations according to process nodes; identifying the voice response information and the actual operations of the user through a preset large language model to obtain a voice recognition result and an actual operation state; comparing the voice recognition result and the actual operation state to obtain a comparison result; and guiding the user to perform relevant operations according to the comparison result, achieving the effect of quickly guiding the user to perform accurate operations.

[0086] In several embodiments provided by the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of devices, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0087] In addition, each functional module in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0088] When the above-mentioned functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0089] The above are only the embodiments of this application and are not used to limit the protection scope of this application. For those skilled in the art, this application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application. It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0090] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by this application and should be covered by the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

[0091] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including an..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

Claims

1. A method for guiding user operation based on a large model, characterized in that: include: In the process of guiding the user to perform operations according to the process nodes, obtaining the user's voice response information; Recognize the voice reply information and the actual operation of the user through a preset large language model to obtain a voice recognition result and an actual operation status; Comparing the speech recognition result with the actual operation state to obtain a comparison result; According to the comparison result, the user is guided to perform relevant operations.

2. The method according to claim 1, characterized in that The comparing the speech recognition result with the actual operation state to obtain a comparison result includes: The operation completion status in the voice recognition result is compared with the actual operation completion status corresponding to the actual operation state to determine whether the operation completion status is consistent with the actual operation completion status.

3. The method according to claim 2, characterized in that The step of guiding the user to perform relevant operations according to the comparison result includes: If the operation completion status is consistent with the actual operation completion status, guiding the user to perform the next node operation of the process node; If the operation completion status is inconsistent with the actual operation completion status, guiding the user to repeat the current node operation of the process node; After guiding the user to repeatedly perform the current node operation of the process node for multiple times, guiding the user to perform the next node operation of the process node.

4. The method according to any one of claims 1 to 3, characterized in that: In the process of guiding the user to operate according to the process node, before obtaining the voice reply information of the user, the method further includes: The large language model is obtained by training the basic model through the voice information of multiple users and / or the operation conditions of the users.

5. A device for guiding user operation based on a large model, characterized in that: include: An acquisition module, used to acquire the user's voice reply information in the process of guiding the user to operate according to the process node; A recognition module, used to recognize the voice reply information and the actual operation of the user through a preset large language model, and obtain a voice recognition result and an actual operation status; A comparison module, used for comparing the speech recognition result with the actual operation state to obtain a comparison result; A guiding module is used to guide the user to perform relevant operations according to the comparison result.

6. The device according to claim 5, characterized in that The comparison module is specifically used for: The operation completion status in the voice recognition result is compared with the actual operation completion status corresponding to the actual operation state to determine whether the operation completion status is consistent with the actual operation completion status.

7. The device according to claim 6, characterized in that The guiding module is specifically used for: If the operation completion status is consistent with the actual operation completion status, guiding the user to perform the next node operation of the process node; If the operation completion status is inconsistent with the actual operation completion status, guiding the user to repeat the current node operation of the process node; After guiding the user to repeatedly perform the current node operation of the process node for multiple times, guiding the user to perform the next node operation of the process node.

8. The device according to any one of claims 5 to 7, characterized in that: The device also includes: The training module is used to train the basic model through the voice information of multiple users and / or the user's operation conditions before obtaining the user's voice reply information in the process of guiding the user to operate according to the process node in the acquisition module, so as to obtain the large language model.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 4 are executed.

10. A computer-readable storage medium, characterized in that: include: A computer program, when the computer program is run on a computer, causes the computer to execute the method according to any one of claims 1 to 4.

Images