Multi-person session application method, device, electronic equipment and system

Abstract

The invention relates to the technical field of artificial intelligence, and particularly provides a multi-person session application method and device, electronic equipment and a system. The method comprises the following steps: a session control device determines a to-be-speaked participant device based on scene description information of a current scene, a historical session record in the current scene and a distributed role of each participant device by adopting a session distribution model, and sends a speaking instruction to the to-be-speaked participant device. Any participant device selects a session reply model and a knowledge base adaptive to the current scene and the distributed role; after a speaking instruction is received, searching background knowledge related to the speaking content of the last session in the knowledge base; and outputting speaking information based on the scene description information, the historical session record, the distributed role and the background knowledge by adopting a session reply model. In this way, the accuracy and speciality of the speaking content during the multi-person session and the fluency of the session can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and system for multi-person conversation application. Background Art

[0002] With the continuous development and progress of artificial intelligence technology, artificial intelligence (AI) has shown great potential and value in many fields, especially in natural language processing. The ability of AI to understand natural language and generate responses can provide users with a new way of communication and provide more personalized and intelligent services.

[0003] Under traditional technology, the conversation scenario is usually determined first, which may include the conversation topic and the conversation role (i.e., the conversation object), etc. The multiple roles of the conversation are usually assigned to different devices. Each device usually uses polling or random methods to determine the speaking order, and generates and outputs the speech content based on the AI ​​model according to the speaking order, thereby realizing multi-person conversation.

[0004] However, using this approach, the generated speech content is usually less accurate and professional, and is usually unable to dynamically adjust the role of the next speaker according to the current context, resulting in poor conversation fluency. Summary of the invention

[0005] To solve the above technical problems, the embodiments of the present application provide a method, device, electronic device and system for multi-person conversation application.

[0006] In a first aspect, an embodiment of the present application provides a multi-person conversation application system, including a conversation control device and a plurality of participant devices, wherein: Any participant device is used to be assigned a role corresponding to the current scenario and select a conversation response model and knowledge base adapted to the current scenario and the assigned role; The conversation control device is used to determine the participant device to be spoken based on the scene description information of the current scene, the historical conversation records in the current scene and the roles assigned to each participant device by using the conversation allocation model if it is detected that a participant device has finished speaking; and send a speaking instruction to the participant device to be spoken; Any participant device is used to, after receiving a speech instruction, if the current response mode is the model response mode, retrieve the background knowledge related to the speech content of the last conversation in the selected knowledge base according to the speech content of the last conversation in the current scene; adopt the selected conversation response model, based on the scene description information of the current scene, the historical conversation records in the current scene, the role assigned to the participant device and the background knowledge, output the speech information.

[0007] In one implementation, each participant device is further configured to: When the reply mode is the model reply mode, after a supervision event is detected, the current reply mode is controlled to switch from the model reply mode to the user reply mode, so that the participant can speak in the role assigned to him according to the speech information input by the user.

[0008] In one implementation, each participant device is further configured to: When the reply mode is the user reply mode, if the speech information input by the user is received, the speech information input by the user is broadcast to the session control device and other participant devices.

[0009] In one implementation, each participant device is further configured to: When the reply mode is the user reply mode, if the speech information input by the user is not received within the first time period after the speech instruction is received, the speech prompt information is output to guide the user to input the speech information.

[0010] In one implementation, each participant device is further configured to: When the reply mode is the user reply mode, if no speech information input by the user is received within a second time period after receiving the speech instruction, the reply mode is controlled to switch from the user reply mode to the model reply mode; the second time period is greater than the first time period.

[0011] In a second aspect, an embodiment of the present application provides a method for a multi-person conversation application, including: If it is detected that a participant device in the multi-person conversation application system has finished speaking, the conversation allocation model is used to determine the participant device to be spoken based on the scene description information of the current scene, the historical conversation records in the current scene, and the roles assigned to each participant device; Send a speaking instruction to the device of the participant who wants to speak.

[0012] In a third aspect, an embodiment of the present application provides a method for a multi-person conversation application, including: Assigned the role corresponding to the current scene; Select a conversation response model and knowledge base that is suitable for the current scenario and the assigned role; After receiving a speech instruction sent by a conversation control device in a multi-person conversation application system, if the current reply mode is a model reply mode, the background knowledge related to the speech content of the last conversation in the current scenario is retrieved from the selected knowledge base; The selected conversation response model is used to output speech information based on the scene description information of the current scene, the historical conversation records in the current scene, the assigned roles and background knowledge.

[0013] In a fourth aspect, an embodiment of the present application provides a device for a multi-person conversation application, including: A determination unit, configured to determine a participant device to be spoken based on the scene description information of the current scene, the historical conversation records in the current scene, and the roles assigned to each participant device, by using a conversation allocation model if it is detected that a participant device in the multi-person conversation application system has finished speaking; The sending unit is used to send a speaking instruction to the participant device to be spoken.

[0014] In a fifth aspect, an embodiment of the present application provides a device for a multi-person conversation application, including: An allocation unit, used to be allocated a role corresponding to the current scene; A selection unit, used to select a conversation response model and knowledge base adapted to the current scenario and the assigned role; A retrieval unit, configured to retrieve background knowledge related to the speech content of the last conversation in the selected knowledge base according to the speech content of the last conversation in the current scenario after receiving the speech instruction sent by the conversation control device in the multi-person conversation application system, if the current reply mode is the model reply mode; The output unit is used to output speech information using the selected conversation response model based on the scene description information of the current scene, the historical conversation records in the current scene, the assigned roles and background knowledge.

[0015] In a sixth aspect, an electronic device is provided in an embodiment of the present application, including: Processor; and A memory storing computer instructions for causing a processor to execute the steps of a method provided in any of the various optional implementations of the above-mentioned session.

[0016] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium storing computer instructions, which are used to enable a computer to execute the steps of the method provided in any of the various optional implementations of the above-mentioned session.

[0017] In the eighth aspect, a computer program product is provided in an embodiment of the present application, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the steps of the method provided in any of the various optional implementations of the above-mentioned session.

[0018] The method of multi-person conversation application in the embodiment of the present application includes: if the conversation control device detects that a participant device in the multi-person conversation application system has finished speaking, the conversation allocation model is used to determine the participant device to be spoken based on the scene description information of the current scene, the historical conversation records in the current scene, and the roles assigned to each participant device; and a speech instruction is sent to the participant device to be spoken. Any participant device selects a conversation reply model and a knowledge base adapted to the current scene and the assigned role; after receiving the speech instruction sent by the conversation control device in the multi-person conversation application system, if the current reply mode is the model reply mode, the background knowledge related to the speech content of the last conversation in the selected knowledge base is retrieved according to the speech content of the last conversation in the current scene; and the selected conversation reply model is used to output the speech information based on the scene description information of the current scene, the historical conversation records in the current scene, the assigned roles and the background knowledge. In this way, the conversation reply model and the knowledge base adapted to the current scene and the assigned roles can be selected to generate the speech content, which can improve the accuracy and professionalism of the speech content, and the speech order can be dynamically adjusted according to the current context, which can improve the fluency of the conversation and reduce the deviation of the conversation from reality. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is an example diagram of the architecture of a multi-person conversation application system in an embodiment of the present application.

[0020] Figure 2 It is a flowchart of a method for multi-person conversation application in an embodiment of the present application.

[0021] Figure 3 This is a structural example diagram of a session control device in an embodiment of the present application.

[0022] Figure 4 This is a structural example diagram of a participant device in an embodiment of the present application.

[0023] Figure 5 It is a structural block diagram of a device for a multi-person conversation application in an embodiment of the present application.

[0024] Figure 6 It is a structural block diagram of another device for multi-person conversation application in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described implementation methods are part of the implementation methods of the present application, rather than all of the implementation methods. Based on the implementation methods in the present application, all other implementation methods obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In addition, the technical features involved in the different implementation methods of the present application described below can be combined with each other as long as they do not conflict with each other.

[0026] In actual application scenarios, the speaking order of each role is usually determined in a round-robin or random manner. Then, the device corresponding to each role generates and outputs the speech content based on the AI ​​model according to the speaking order.

[0027] However, using this approach, the speech content generated by the AI ​​model is usually less accurate and professional, and is usually unable to dynamically adjust the role of the next speaker according to the current context, resulting in poor conversation fluency.

[0028] Based on the defects of the above-mentioned related technologies, the embodiments of the present application provide a method, device, electronic device and system for a multi-person conversation application, aiming to improve the accuracy and professionalism of the speech content and the fluency of the conversation.

[0029] See also Figure 1 As shown, an example diagram of the architecture of a multi-person conversation application system is provided in an embodiment of the present application. The multi-person conversation application system includes a conversation control device and multiple participant devices.

[0030] In one application scenario, the session control device and the participant device can be any type of electronic device, for example, the electronic device can be a terminal device and a server. In another application scenario, the multi-person session application system can also be an electronic device, and the session control device and the participant device are different software function modules in the multi-person session application system, for example, the session control device can be a server, and each participant device can be a different client.

[0031] Among them, different participant devices are respectively assigned different roles corresponding to the current scene. It should be noted that in the embodiment of the present application, each role refers to each participant in a multi-person conversation, which can also be called a conversation object, and the role corresponds to the participant one by one. For example, the roles corresponding to the multiple participant devices are: teacher, student, and parent. For another example, the roles corresponding to the multiple participant devices are: teacher A, teacher B, student C, and student D.

[0032] Furthermore, the multi-person conversation application system may also include a control terminal, which is used to control the start and end of the conversation. In the embodiment of the present application, the control terminal is described as being located in the conversation control device. In actual applications, the control terminal may also be located in an electronic device other than the conversation control device and the multiple participant devices, which is not limited here.

[0033] The session control device is mainly used to control the speaking order of each participant device. Specifically, a session allocation model can be used to dynamically determine the next participant device to speak (i.e., to speak). The session allocation model can be a large language model (LLM). The LLM model refers to a pre-trained natural language processing model with a large number of parameters and rich language knowledge.

[0034] The current scene includes scene description information of the conversation and multiple participants of the conversation. The scene description information may include a conversation topic to be discussed. Multiple participants of the conversation discuss around a conversation topic to realize a multi-person conversation.

[0035] The participant device can be pre-configured with scene description information of the current scene, and can be assigned a role corresponding to the current scene (i.e., participant), and can be mounted with one or more knowledge bases for selecting a knowledge base that is adapted to the current scene and the currently assigned role, and can be mounted with one or more LLM models for selecting a conversation response model for determining the speech content of the current role in the current scene.

[0036] The conversation reply models selected by different participant devices may be the same, or the conversation reply models selected by at least two participant devices may be different.

[0037] The session control device may be pre-configured with the scenario description information of the current scenario, the role assigned to each participant device in the current scenario, and the session allocation model for determining the speaking order.

[0038] In one implementation, the session control device is used to: If a session is detected to be open (e.g., a session opening instruction is received from the control end), the session allocation model is used to determine the first participant device to speak based on the scene description information of the current scene and the role assigned to each participant device, and a speaking instruction is sent to the first participant device to speak.

[0039] Furthermore, if it is detected that a participant device has finished speaking, the conversation allocation model is used to determine the participant device to be spoken based on the scene description information of the current scene, the historical conversation records under the current scene, and the roles assigned to each participant device; and a speech instruction is sent to the participant device to be spoken.

[0040] And, when the session termination is detected (eg, a session termination instruction sent by the control end is received), the process of determining the participant device to be spoken is stopped, thereby terminating the current session.

[0041] In one implementation, any participant device is used to: After receiving the speech instruction, if the current response mode is the model response mode, the background knowledge related to the speech content of the last conversation in the current scenario is retrieved from the selected knowledge base; the selected conversation response model is used to output the speech information based on the scenario description information of the current scenario, the historical conversation records in the current scenario, the role assigned to the participant's device, and the background knowledge.

[0042] When outputting the speech information, the following steps may be adopted: broadcasting the speech information to the session control device and other participant devices.

[0043] Furthermore, it is also possible to switch to a user reply mode so that the model reply can be replaced by the user reply.

[0044] In one embodiment, each participant device is also used for: when the response mode is a model response mode, upon detecting a supervision event, controlling the current response mode to switch from the model response mode to the user response mode, so that the participant can speak in the role assigned to the participant according to the speech information input by the user.

[0045] In this way, the role's speech can be made directly by the real user.

[0046] It should be noted that in the user reply mode, the user can speak at any time, that is, the user can speak in any order other than the speaking order indicated by the session control device.

[0047] In one implementation, when the reply mode is the user reply mode, if speech information input by the user is received, the speech information input by the user is broadcast to the session control device and other participant devices.

[0048] Furthermore, if the user does not speak in time, a speaking reminder may be provided to the user.

[0049] In one implementation, each participant device is further configured to: when the reply mode is the user reply mode, if no user input speech information is received within a first time period after receiving a speech instruction, output speech prompt information to guide the user to input speech information.

[0050] Furthermore, if the user does not speak for a long time, the user can switch to the model reply mode to avoid conversation interruption.

[0051] In one embodiment, each participant device is also used for: when the reply mode is the user reply mode, if no speech information input by the user is received within a second time period after receiving the speech instruction, then controlling the reply mode to switch from the user reply mode to the model reply mode; the second time period is greater than the first time period.

[0052] For example, the first time period may be 30 seconds, and the second time period may be 60 seconds. In actual applications, both the first time period and the second time period may be set according to actual application scenarios, and are not limited here.

[0053] Optionally, it is not necessary to output the speech prompt information to the user, and the second time period may not be restricted by the first time period.

[0054] In this way, the speaking order of each participant device can be dynamically controlled by the session control device. When it is the turn of each participant device to speak, if it is in the model reply mode, it can speak based on the current context through the session reply model; and if it is in the user reply mode, it can also speak through the real user.

[0055] See also Figure 2 FIG. 1 is a flowchart of a method for multi-person conversation application in an embodiment of the present application, which can be applied to Figure 1 In the multi-person conversation application system, the following is combined Figure 1 and Figure 2 The method is described below. The specific implementation process of the method is as follows: Step 201: If the session control device detects that a participant device in the multi-person session application system has finished speaking, the session allocation model is used to determine the participant device to speak based on the scene description information of the current scene, the historical session records in the current scene, and the roles assigned to each participant device.

[0056] Furthermore, before executing step 201, the session control device and each participant device may be configured respectively according to the current scenario.

[0057] The conversation control device may be pre-configured with scene description information of the current scene, multiple roles in the current scene and their corresponding role information, and a conversation allocation model. The conversation allocation model is used to determine the participant device corresponding to the target role to be spoken. The role information may include the name, identity, and description of the role.

[0058] Combine the following Figure 3 , which explains the composition of the session control device. Figure 3 The figure shows an example structure diagram of a session control device. Figure 3In the embodiment, the conversation control device may include a scene module, a role module, a conversation recording module, a model module and a speech distribution module.

[0059] Among them, the scene module is used to be configured with the scene description information of the current scene. The scene description information can be information related to the topic of the conversation, such as a medical conversation. The role module is used to be configured with multiple roles and their corresponding role information in the current scene. The conversation record module is used to store historical conversation records in the current scene. The model module is used to set the conversation allocation model in the current scene, and can also mount multiple LLM models, and select the conversation allocation model adapted to the current scene from each LLM model. The speech allocation module is used to use the conversation allocation model to determine the participant device to be spoken when the conversation allocation conditions are met, and send a speech instruction to it.

[0060] The session allocation conditions may include the following items: detecting that a session is started, and detecting that a participant device has finished speaking.

[0061] Each participant device may be pre-configured with the scene description information of the current scene, the assigned role in the current scene and its corresponding role information, may be mounted with one or more large language models and one or more knowledge bases, and may select a conversation response model and knowledge base adapted to the current scene and the assigned role. The conversation response models selected by different participant devices are the same, or at least two participant devices select different conversation response models. The conversation control device and each participant device may store historical conversation records in the current scene.

[0062] Furthermore, the session control device may also determine the first participant device to speak when the session is started.

[0063] In one implementation, in the current scenario, if a session is detected to be started, a session allocation model is used to determine the participant device to be spoken based on the scenario description information of the current scenario and the role assigned to each participant device; and a speaking instruction is sent to the participant device to be spoken.

[0064] For example, initial prompt information can be generated based on the scene description information of the current scene and the role information corresponding to the role assigned to each participant device, and the initial prompt information can be input into the conversation allocation model to obtain the participant device corresponding to the target role to be spoken, and send a speaking instruction to it.

[0065] For example, if a session start instruction is received from the user through voice or keystroke, it is determined that the session start is detected.

[0066] Furthermore, each historical speech content may be screened to obtain the screened historical speech content, and a historical conversation record may be generated based on the screened historical speech content.

[0067] For example, the historical conversation record only contains the latest one or n speeches, where n is a positive integer, such as 3.

[0068] In one implementation, when executing step 201, the following steps may be used: In the current scenario, if a speech information broadcast by a participant device is received, the session allocation model is used to determine the participant device to speak based on the scenario description information of the current scenario and the roles assigned to each participant device.

[0069] Based on the scene description information of the current scene, the historical conversation records in the current scene, and the role information corresponding to the roles assigned to the devices of each participant, the first prompt information can be generated, and the first prompt information can be input into the conversation allocation model to obtain the participant device corresponding to the target role to be spoken.

[0070] For example, the first prompt information may be: You are a host. I will give you a conversation scenario and several roles. Based on the conversation history and role identity, you will be asked to determine which role should answer the current conversation. You will follow the following constraints: 1. Your reply can only contain the role name, and the return format is <role> Role Name< / role> ; 2. You should try to give all characters a chance to speak, don’t let a character reply multiple times, and don’t let the same character speak continuously; 3. Don’t go back to the reasoning process, just go back to the conclusion; 4. Please ignore the examples I gave when replying, and only refer to the conversation content and scenario description I gave; Let me give you an example of a multi-round conversation: <role> teacher< / role> The child's latest grades have dropped a bit." <role> Mother< / role> Teacher, thank you for your concern. In which subject has your child's grades declined most? Now I will give you a scenario. Here is the scenario description: Communication about students' grades; the characters are teacher and mother, and their conversation content is: <role> teacher< / role> The child's latest grades have dropped a bit." <role> Mother< / role> Thank you for your concern, teacher. In which subject has the child's grades declined the most? Please judge which role should continue the next conversation based on the current conversation.

[0071] In this way, the participant device corresponding to the target role of the next speech can be determined according to the current context.

[0072] Step 202: The session control device sends a speaking instruction to the participant device to be spoken.

[0073] Step 203: After any participant device receives a speech instruction sent by the session control device in the multi-person session application system, if the current reply mode is the model reply mode, the background knowledge related to the speech content of the last session in the current scenario is retrieved from the selected knowledge base.

[0074] Optionally, retrieval-augmented generation (RAG) can be used to retrieve the target knowledge base. RAG is a model that combines retrieval and generation techniques. It generates answers or content by referencing information from an external knowledge base. It has strong interpretability and customization capabilities and is suitable for multiple natural language processing tasks such as question-answering systems, document generation, and intelligent assistants.

[0075] Combine the following Figure 4 For a description of the composition of the participant equipment, see Figure 4 The figure shows an example structure diagram of a participant device. Figure 4 In the example, each participant device may include the following modules: a scenario module, a role module, a session record module, a knowledge base module, a model module, and a speech module.

[0076] Among them, the scene module is used to be configured with the scene description information of the current scene. The role module is used to be assigned the role corresponding to the current scene and its corresponding role information. The session record module is used to store historical session records. The knowledge base module is mounted with one or more knowledge bases that can be selected for use. The model module is mounted with one or more large language models that can be selected for use. The speech module is used to select a conversation response model that is suitable for the current scene and the assigned role from the major language models, and select a knowledge base that is suitable for the current scene and the assigned role from the mounted knowledge bases. If in the model response mode, after receiving the speech instruction, the conversation response model and the knowledge base can be used to generate and output the speech information. If in the user response mode, the speech information input by the user can be output in real time when the speech information input by the user is received.

[0077] For example, taking a scene and a knowledge base corresponding to a role in the scene as an example, the generation of the knowledge base is explained. The professional knowledge of the scene and the role can be obtained, encoded, and multiple text vectors can be obtained, and each text vector is stored in the corresponding knowledge base. In actual applications, different knowledge bases can be obtained for different scenes and different roles.

[0078] In one implementation, when executing step 203, the following steps may be used: S2031: Obtain the scene description information of the current scene, the assigned role corresponding to the current scene and its corresponding role information, and the historical session record.

[0079] S2032: According to the speech content of the last conversation in the current scenario, background knowledge related to the speech content of the last conversation is retrieved from the selected knowledge base.

[0080] In one implementation, the speech content of the last conversation is vectorized to obtain a conversation vector, and a knowledge vector matching the conversation vector is searched from a target knowledge base, and the matching knowledge vector is used as background knowledge.

[0081] Optionally, the matching knowledge vectors may be queried by using vector similarity (eg, cosine similarity).

[0082] Step 204: Any participant device uses the selected conversation reply model to output speech information based on the scene description information of the current scene, the historical conversation records in the current scene, the assigned role and background knowledge.

[0083] In one implementation, when executing step 204, the following steps may be used: Based on the scene description information of the current scene, the historical conversation records, the assigned roles and the background knowledge in the current scene, the second prompt information is generated; the second prompt information is input into the selected conversation response model to obtain the speech information.

[0084] For example, the second prompt information may be: You are an actor. I will give you a conversation scenario. Please play the role of teacher. Give a reasonable response based on the teacher's role information and historical conversation records. You will follow the following constraints: 1. You should try to keep the conversation going for more rounds, and start new topics that fit the scenario description when appropriate; 2. The content of the conversation should be polite, and try to reflect the language habits and personality of the role played; 3. The content of the conversation should be logical and not make people feel that you are talking nonsense; 4. Please ignore the examples I gave when replying, and only refer to the conversation content and scenario description I gave; 5. If the conversation deviates from the scene description, try to get it back to the scene description while ensuring the continuity of the conversation plot; 6. I will provide you with a reference information. If the reference information can answer the previous question, you can use it. Otherwise, please ignore it. The current reference information is: Let me give you an example of a multi-round conversation: <role> teacher< / role> The child's latest grades have dropped a bit." <role> Mother< / role> Teacher, thank you for your concern. In which subject has your child's grades declined most? <role> teacher< / role> My child’s English grades are not very good and he is a little distracted during class.

[0085] Now I will give you a scenario. Here is the scenario description: communication about students' grades. The role information corresponding to the role you play: teacher's occupation information. The content of their conversation: <role> teacher< / role> The child's latest grades have dropped a bit." <role> Mother< / role> Teacher, thank you for your concern. In which subject has your child's grades declined most? <role> teacher< / role> My child's English grades are not very good and he is a little distracted during class. Please respond according to your role as the teacher.

[0086] In this way, accurate speech information can be generated.

[0087] Furthermore, the participant device may also have a user takeover function, so that the reply mode may be switched to the user reply mode, so that the user may take over the role played by a participant device at any time.

[0088] In one implementation, when the reply mode is a model reply mode, upon detecting a supervision event, the current reply mode is controlled to switch from the model reply mode to the user reply mode so that the participant can speak in the role assigned to him according to the speech information input by the user.

[0089] Optionally, the supervision event may be a command for the user to take over a role, which is sent by the user through voice or keystrokes.

[0090] Furthermore, in the user reply mode, when the participant device receives the speech information input by the user, it outputs the user's speech information in real time.

[0091] In one implementation, a participant device may receive speech information input by a user in real time, and broadcast it to a session control device and other participant devices, without generating the speech information through a model.

[0092] Optionally, the user may input speech information to the participant device through any type of input method, which is not limited here.

[0093] For example, a user takeover control and a reply text box are displayed on the application page of the participant's device. When the user clicks the user takeover control, the local reply mode is switched from the model reply mode to the user reply mode, and the speech information entered by the user through the reply text box is obtained.

[0094] In this way, human-computer interaction can also be carried out during the conversation. Users can express their opinions at any time and provide more professional suggestions to guide the discussion direction of the conversation in the current scenario, which can improve the overall conversation quality and user participation.

[0095] Furthermore, the participant device may also have a session continuation function, that is, a monitoring and processing mechanism may be established to execute corresponding logical operations when the user's speech behavior meets the first continuation condition and / or the second continuation condition to ensure the continuity and validity of the session.

[0096] Optionally, the first continuation condition may be: after the session control device notifies a participant device to speak, the participant device is in user reply mode and detects that the user has not responded (i.e., has not input speech information) within the first time period. The logical operation corresponding to the first continuation condition may be: displaying speech prompt information to the user.

[0097] In one implementation, when the reply mode is the user reply mode, if the speech information input by the user is not received within a first time period after receiving the speech instruction, speech prompt information is output to guide the user to input the speech information.

[0098] For example, the first time period may be 30 seconds. In practical applications, the first time period may be set according to actual application scenarios and is not limited here.

[0099] In this way, when the conversation is difficult to start or reaches a deadlock, speech prompt information can be provided to the user to guide the user to speak, so as to help the user to better express his or her views.

[0100] Optionally, the second continuation condition may be: after the conversation control device notifies a participant device to speak, the participant device is in user reply mode and detects that the user has not responded within a second time period. The logic operation corresponding to the second continuation condition may be: switching the current reply mode from user reply mode to model reply mode.

[0101] In one implementation, when the reply mode is a user reply mode, if no speech information input by the user is received within a second time period after receiving a speech instruction, the reply mode is controlled to switch from the user reply mode to the model reply mode; the second time period is greater than the first time period.

[0102] For example, the first time period may be 60 seconds. In practical applications, the second time period may be set according to actual application scenarios, and is not limited here.

[0103] In this way, when the user is unable to continue to participate in the conversation, or is unable to participate in the conversation in time, the model response mode can be automatically switched, and accurate and coherent speech information can be generated through the conversation response model so that the conversation content can be continued, ensuring the continuity of the conversation.

[0104] Furthermore, when the conversation module model determines that speech information of any participant device is received, it can jump to step 201, so that steps 201 to 204 can be executed in a loop.

[0105] In an embodiment of the present application, on the one hand, the first prompt information can be generated in combination with the scene description information of the current scene, the role information corresponding to each role in the current scene, and the historical conversation records, so that the conversation allocation model can accurately understand and judge the current context based on the first prompt information, and then intelligently select the participant device corresponding to the target role who will speak next (i.e., to speak), thereby avoiding the problem of excessive speaking of a certain role, solving the problem of unfairness of speaking under traditional technologies and the problem of inability to intelligently schedule, ensuring that each role has the opportunity to participate in the conversation discussion, significantly improving the naturalness and fairness of the conversation, and being more in line with real dialogue scenarios.

[0106] On the other hand, each participant device can be mounted with one or more knowledge bases, and can select a knowledge base that is suitable for the current scene and the current role from each knowledge base. It can also combine RAG technology to obtain background knowledge related to the previous speech content from the target knowledge base, so that it can flexibly select a knowledge base that is suitable for the current context, and can accurately match the relevant knowledge (i.e., background knowledge) in the knowledge base, and can combine the scene description information, the role information corresponding to the assigned role, background knowledge and historical conversation records to generate a second prompt information, and then based on the second prompt information, it can generate accurate, professional and coherent speech information, especially when dealing with professional field problems, it can provide more accurate and in-depth answers, thereby improving the quality of text generation.

[0107] Furthermore, each participant's device can be equipped with one or more large language models, so that a conversation response model suitable for the current context can be flexibly selected from multiple large language models, solving the problem of insufficient adaptability when only a single model is used for conversation, and can be applied to more and more complex conversation scenarios.

[0108] Furthermore, users can take over any role at any time, directly participate in the conversation, provide personal insights or guide the conversation direction, which significantly improves the user experience and the practicality of the conversation. In addition, when a user fails to speak in time for some reason, the duration of the user's non-response can be used to determine whether to provide a prompt to speak and whether to switch to the model conversation mode, which can lower the threshold for user participation and maintain the continuity, authenticity and professionalism of the conversation.

[0109] Based on the same inventive concept, an apparatus for a multi-person conversation application is also provided in an embodiment of the present application. Since the principle of solving the problem by the above-mentioned apparatus and device is similar to that of a method for a multi-person conversation application, the implementation of the above-mentioned apparatus can refer to the implementation of the method, and the repeated parts will not be repeated. The apparatus can be applied to electronic devices. The present application does not limit the type of electronic device. It can be any type of device suitable for implementation, such as a terminal device and a server, etc. The present application will not repeat it. The apparatus embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the electronic device in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution.

[0110] See also Figure 5 FIG. 1 is a block diagram of a device for a multi-person conversation application in an embodiment of the present application. In some implementations, the device for a multi-person conversation application in the present application example includes: The determining unit 501 is used to determine the participant device to be spoken based on the scene description information of the current scene, the historical conversation records in the current scene and the roles assigned to each participant device by using the conversation allocation model if it is detected that a participant device in the multi-person conversation application system has finished speaking; The sending unit 502 is used to send a speaking instruction to a participant device that is ready to speak.

[0111] See also Figure 6 FIG. 1 is a block diagram of another device for multi-person conversation application in an embodiment of the present application. In some implementations, the device for multi-person conversation application in the present application example includes: An allocating unit 601 is used to allocate a role corresponding to the current scene; A selection unit 602, used to select a conversation reply model and knowledge base adapted to the current scenario and the assigned role; The retrieval unit 603 is configured to retrieve background knowledge related to the speech content of the last session in the selected knowledge base according to the speech content of the last session in the current scenario after receiving the speech instruction sent by the session control device in the multi-person session application system, if the current reply mode is the model reply mode; The output unit 604 is used to output speech information using the selected conversation reply model based on the scene description information of the current scene, the historical conversation records in the current scene, the assigned roles and background knowledge.

[0112] The method of multi-person conversation application in the embodiment of the present application includes: if the conversation control device detects that a participant device in the multi-person conversation application system has finished speaking, the conversation allocation model is used to determine the participant device to be spoken based on the scene description information of the current scene, the historical conversation records in the current scene, and the roles assigned to each participant device; and a speech instruction is sent to the participant device to be spoken. Any participant device selects a conversation reply model and a knowledge base adapted to the current scene and the assigned role; after receiving the speech instruction sent by the conversation control device in the multi-person conversation application system, if the current reply mode is the model reply mode, the background knowledge related to the speech content of the last conversation in the selected knowledge base is retrieved according to the speech content of the last conversation in the current scene; and the selected conversation reply model is used to output the speech information based on the scene description information of the current scene, the historical conversation records in the current scene, the assigned roles and the background knowledge. In this way, the conversation reply model and the knowledge base adapted to the current scene and the assigned roles can be selected to generate the speech content, which can improve the accuracy and professionalism of the speech content, and the speech order can be dynamically adjusted according to the current context, which can improve the fluency of the conversation and reduce the deviation of the conversation from reality.

[0113] In an embodiment of the present application, an electronic device is further provided, including: Processor; and The memory stores computer instructions, where the computer instructions are used to enable the processor to execute the method of any of the above embodiments.

[0114] In an embodiment of the present application, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a computer to execute a method in any of the above-mentioned embodiments.

[0115] An embodiment of the present application also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device implements any of the above-mentioned methods.

Claims

1. A multi-person conversation application system, characterized in that: It includes a session control device and multiple participant devices, wherein: Any participant device is used to be assigned a role corresponding to the current scenario and select a conversation response model and knowledge base adapted to the current scenario and the assigned role; The conversation control device is used to determine the participant device to be spoken based on the scene description information of the current scene, the historical conversation records in the current scene and the roles assigned to each participant device by using the conversation allocation model if it is detected that a participant device has finished speaking; and send a speaking instruction to the participant device to be spoken; Any participant device is used to, after receiving the speech instruction, if the current response mode is the model response mode, retrieve the background knowledge related to the speech content of the last conversation in the selected knowledge base according to the speech content of the last conversation in the current scene; adopt the selected conversation response model, based on the scene description information of the current scene, the historical conversation records in the current scene, the role assigned to the participant device and the background knowledge, output the speech information.

2. The system according to claim 1, characterized in that Each participant device is also used to: In the case where the reply mode is the model reply mode, when a supervision event is detected, the current reply mode is controlled to switch from the model reply mode to the user reply mode, so that the participant can speak in the role assigned to him according to the speech information input by the user.

3. The system according to claim 2, characterized in that Each participant device is also used to: In the case where the reply mode is the user reply mode, if speech information input by the user is received, the speech information input by the user is broadcast to the session control device and other participant devices.

4. The system according to claim 2 or 3, characterized in that: Each participant device is also used to: In the case where the reply mode is the user reply mode, if the speech information input by the user is not received within a first time period after the speech instruction is received, speech prompt information is output to guide the user to input speech information.

5. The system according to claim 4, characterized in that Each participant device is also used to: When the reply mode is the user reply mode, if no speech information input by the user is received within a second time period after receiving the speech instruction, the reply mode is controlled to switch from the user reply mode to the model reply mode; the second time period is greater than the first time period.

6. A method for multi-person conversation application, characterized in that: The conversation control device used in the multi-person conversation application system includes: If it is detected that a participant device in the multi-person conversation application system has finished speaking, a conversation allocation model is used to determine a participant device to speak based on the scene description information of the current scene, the historical conversation records in the current scene, and the roles assigned to each participant device; Send a speaking instruction to the participant device to be spoken.

7. A method for multi-person conversation application, characterized in that: Applicable to any participant device in a multi-person conversation application system, including: Assigned the role corresponding to the current scene; Select a conversation response model and knowledge base that is suitable for the current scenario and the assigned role; After receiving the speech instruction sent by the conversation control device in the multi-person conversation application system, if the current reply mode is the model reply mode, based on the speech content of the last conversation in the current scenario, background knowledge related to the speech content of the last conversation is retrieved from the selected knowledge base; The selected conversation response model is adopted to output speech information based on the scene description information of the current scene, the historical conversation records in the current scene, the assigned roles and the background knowledge.

8. A device for multi-person conversation application, characterized in that: The conversation control device used in the multi-person conversation application system includes: A determination unit, configured to determine a participant device to be spoken based on the scene description information of the current scene, the historical conversation records in the current scene, and the roles assigned to each participant device by using a conversation allocation model if it is detected that a participant device in the multi-person conversation application system has finished speaking; The sending unit is used to send a speaking instruction to the participant device to be spoken.

9. A device for multi-person conversation application, characterized in that: Applicable to any participant device in a multi-person conversation application system, including: An allocation unit, used to be allocated a role corresponding to the current scene; A selection unit, used to select a conversation response model and knowledge base adapted to the current scenario and the assigned role; A retrieval unit, configured to retrieve background knowledge related to the speech content of the last conversation in the selected knowledge base according to the speech content of the last conversation in the current scenario after receiving the speech instruction sent by the conversation control device in the multi-person conversation application system, if the current reply mode is the model reply mode; The output unit is used to output speech information using the selected conversation response model based on the scene description information of the current scene, the historical conversation records in the current scene, the assigned roles and the background knowledge.

10. An electronic device, characterized in that: include: processor; as well as A memory storing computer instructions, wherein the computer instructions are used to enable the processor to execute the method of claim 6 or 7.

Images