Voice dialogue method, device and equipment and readable storage medium

Abstract

The application discloses a voice dialogue method, device and equipment and a readable storage medium. The method comprises the following steps: if a state indication token is a reply state token, performing state rollback processing on dialogue context information with a second state to obtain dialogue context information with a first state; the dialogue context information with the first state contains first voice text information corresponding to first voice data of a first object; a reply token is written into the dialogue context information with the first state to obtain dialogue context information with a to-be-replied state; and based on the reply token, the dialogue context information with the to-be-replied state is subjected to dialogue reply generation processing in a dialogue model to obtain reply text information for replying to the voice text information and dialogue context information with a third state. By using the application, the context pollution problem of the dialogue model can be avoided, the computing power for generating a reply by the dialogue model is saved, and the reply quality of the dialogue model is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a voice dialogue method, apparatus, device, and readable storage medium. Background Technology

[0002] Voice dialogue is a typical scenario for artificial intelligence products, where users converse with AI assistants via voice. These assistants can perform various tasks such as question answering, translation, and casual conversation. In voice dialogue scenarios, when the user pauses, the AI ​​assistant needs to determine whether to continue listening or provide a voice response; this decision-making task is called dialogue state recognition.

[0003] Dialogue state recognition can be achieved by deploying a dialogue state recognition model. Since existing voice-based AI assistants already require a large language model (dialogue model or main model) to generate responses based on user input, deploying an additional dialogue state recognition model with high accuracy would increase system maintenance complexity. Therefore, the computational power of the existing dialogue model can be reused for dialogue state recognition. In this case, the relevant content of the dialogue state recognition will also be incorporated into the dialogue context information of the dialogue model (i.e., the input and output information since the dialogue model was used). Because the dialogue model relies on this context information when generating responses, frequent user pauses will generate a large amount of dialogue state recognition content unrelated to the response, leading to severe context pollution. This not only increases the computational power required for the dialogue model to generate responses but also reduces the quality of the responses generated. Summary of the Invention

[0004] This application provides a voice dialogue method, apparatus, device, and readable storage medium, which can avoid the context pollution problem of the dialogue model, thereby saving the computing power of the dialogue model to generate responses and improving the response quality of the dialogue model.

[0005] This application provides a voice dialogue method, including:

[0006] When it is determined that the first object switches from the first voice input state to the first voice input pause state, the dialogue context information with the first state is obtained; the dialogue context information with the first state includes the first voice text information corresponding to the first voice data of the first object; the first voice data refers to the audio data stream obtained when the first object is in the first voice input state;

[0007] Write the state recognition token into the dialogue context information with the first state to obtain the dialogue context information with the state to be recognized. In the dialogue model, perform dialogue state recognition processing on the dialogue context information with the state to be recognized based on the state recognition token to obtain the state indicator token for the voice text information. Write the state indicator token into the dialogue context information with the state to be recognized to obtain the dialogue context information with the second state.

[0008] If the state indicator token is a reply state token, then the dialog context information with the second state is processed by state rollback to obtain the dialog context information with the first state.

[0009] Write the reply token into the dialogue context information with the first state to obtain the dialogue context information with the pending reply state. In the dialogue model, perform dialogue reply generation processing on the dialogue context information with the pending reply state based on the reply token to obtain reply text information for replying voice text information. Write the reply text information into the dialogue context information with the pending reply state to obtain the dialogue context information with the third state.

[0010] Among them, voice dialogue methods also include:

[0011] The dialogue audio data stream is detected and processed; the dialogue audio data stream refers to the audio data stream collected by the dialogue recording component during a voice dialogue with the first object.

[0012] When a voice signal is first detected in the dialogue audio data stream, it is determined that the first object is in the first voice input state;

[0013] When the duration of the disappearance of the voice signal exceeds the pause duration threshold for the first time in the dialogue audio data stream, it is determined that the first object switches from the first voice input state to the first voice input pause state.

[0014] Among these, obtaining the dialogue context information with the first state includes:

[0015] Obtain the first voice data of the first object;

[0016] Perform speech recognition on the first speech data to obtain the first speech text information corresponding to the first speech data;

[0017] Dialogue context information with a first state is generated based on the first voice text information.

[0018] Specifically, speech recognition is performed on the first speech data to obtain the first speech text information corresponding to the first speech data, including:

[0019] The voice recognition token is concatenated after the first voice data to obtain the voice data to be recognized;

[0020] The speech data to be recognized is input into the dialogue model. In the dialogue model, speech recognition is performed on the first speech data based on the speech recognition token to obtain the first speech text information corresponding to the first speech data.

[0021] The acquisition of the first voice data of the first object includes:

[0022] From the dialogue audio data stream, an audio data stream whose time information matches the first input time period is obtained as the first voice data; the dialogue audio data stream refers to the audio data stream collected by the dialogue recording component during the voice dialogue with the first object; the first input time information refers to the time period corresponding to when the first object is in the first voice input state.

[0023] Among them, voice dialogue methods also include:

[0024] If the state indicator token is a listening state token, then the dialogue context information with the second state is processed by state rollback to obtain the dialogue context information with the first state.

[0025] When it is determined that the first object switches from the second voice input state to the second voice input pause state, the second voice data of the first object is obtained, the second input data is subjected to voice recognition, and the second voice text information corresponding to the second voice data is obtained.

[0026] The second voice text information and the state recognition token are written into the dialogue context information with the first state to obtain the dialogue context information with the new state to be recognized. In the dialogue model, the dialogue context information with the new state to be recognized is processed for dialogue state recognition based on the state recognition token.

[0027] Among them, voice dialogue methods also include:

[0028] During the process of the first object being in the first voice input state, the first voice data of the first object is subjected to periodic voice recognition processing to obtain the first periodic voice text information; the first periodic voice text information contains the periodic voice text information corresponding to N consecutive periodic voice data in the first voice data; a periodic voice data refers to the audio data stream in the first voice data whose voice duration reaches the periodic voice duration;

[0029] When it is determined that the first object has switched from the first voice input state to the first voice input pause state, the dialogue context information with the first state is obtained, including:

[0030] When it is determined that the first object has switched from the first voice input state to the first voice input pause state, dialogue context information with an initial state is created based on the first cycle voice text information;

[0031] Speech recognition processing is performed on the first remaining speech data in the first speech data to obtain the first remaining speech text information corresponding to the first remaining speech data; the first remaining speech data refers to the audio data stream in the first speech data excluding N consecutive periodic speech data;

[0032] Write the first remaining voice text information into the dialogue context information with the initial state to obtain the dialogue context information with the first state.

[0033] Among them, voice dialogue methods also include:

[0034] If the state indicator token is a listening state token, then the dialogue context information with the second state is processed by state rollback to obtain the dialogue context information with the initial state.

[0035] During the process of the first object being in the second voice input state, the second voice data of the first object is concatenated to the end of the first remaining voice data to obtain joint voice data; the second voice data refers to the audio data stream acquired when the first object is in the second voice input state;

[0036] Periodic speech recognition processing is performed on the joint speech data to obtain second periodic speech text information; the second periodic speech text information includes the periodic speech text information corresponding to M consecutive periodic speech data in the joint speech data;

[0037] When it is determined that the first object has switched from the second voice input state to the second voice input pause state, the second cycle voice text information is written into the dialogue context information with the initial state to obtain the dialogue context information with the fourth state.

[0038] Speech recognition processing is performed on the second residual speech data in the joint speech data to obtain the second residual speech text information corresponding to the second residual speech data; the second residual speech data refers to the audio data stream in the joint speech data excluding M continuous periodic speech data;

[0039] The second remaining speech text information and the state recognition token are written into the dialogue context information with the fourth state to obtain the dialogue context information with the new state to be recognized. In the dialogue model, the dialogue context information with the new state to be recognized is processed for dialogue state recognition based on the state recognition token.

[0040] Specifically, during the process of the first object being in the first voice input state, the first voice data of the first object is subjected to periodic voice recognition processing to obtain first periodic voice text information, including:

[0041] During the process of the first object being in the first voice input state, the first voice data of the first object is subjected to duration statistics processing to obtain the cumulative voice duration of the first voice data;

[0042] When the cumulative speech duration is detected to be k times the periodic speech duration, the audio data stream corresponding to the kth periodic speech duration is sequentially obtained from the first speech data and used as the kth periodic speech data; k is an integer less than N;

[0043] Speech recognition is performed on the k-th cycle of speech data to obtain the corresponding cycle speech text information;

[0044] The first N cycles of speech-text information are sequentially concatenated to obtain the first cycle of speech-text information.

[0045] Among them, voice dialogue methods also include:

[0046] When obtaining the dialogue context information with the first state, generate a copy of the first state data for the dialogue context information with the first state;

[0047] Perform state rollback processing on the dialogue context information with the second state to obtain the dialogue context information with the first state, including:

[0048] The data contained in the dialogue context information with the second state is replaced with the copy data of the first state to obtain the dialogue context information with the first state.

[0049] Specifically, the dialogue context information with the second state undergoes state rollback processing to obtain the dialogue context information with the first state, including:

[0050] Retrieve the operation log corresponding to the dialogue context information; the operation log contains records of each operation performed on the dialogue context information.

[0051] Retrieve the target operation record from the operation log; the target operation record refers to the operation record located between the first state time node and the second state time node; the first state time node refers to the time node at which the dialogue context information with the first state is obtained; the second state time node refers to the time node at which the dialogue context information with the second state is obtained;

[0052] Generate a reverse operation record for the target operation record, and perform operation processing on the dialogue context information with the second state based on the reverse operation record to obtain the dialogue context information with the first state; the reverse operation record refers to the operation record that has the opposite operation processing to the target operation record.

[0053] One embodiment of this application provides a voice dialogue device, including:

[0054] The acquisition module is used to acquire dialogue context information with the first state when it is determined that the first object switches from the first voice input state to the first voice input pause state; the dialogue context information with the first state includes the first voice text information corresponding to the first voice data of the first object; the first voice data refers to the audio data stream acquired when the first object is in the first voice input state;

[0055] The state recognition module is used to write the state recognition token into the dialogue context information with the first state to obtain the dialogue context information with the state to be recognized. In the dialogue model, the state recognition token is used to perform dialogue state recognition processing on the dialogue context information with the state to be recognized to obtain the state indicator token for the voice text information. The state indicator token is then written into the dialogue context information with the state to be recognized to obtain the dialogue context information with the second state.

[0056] The rollback module is used to perform state rollback processing on the dialogue context information with the second state if the state indicator token is a reply state token, so as to obtain the dialogue context information with the first state.

[0057] The response module is used to write the response token into the dialogue context information with the first state to obtain the dialogue context information with the pending response state. In the dialogue model, based on the response token, the dialogue context information with the pending response state is processed to generate a dialogue response, resulting in response text information for replying to voice text information. The response text information is then written into the dialogue context information with the pending response state to obtain the dialogue context information with the third state.

[0058] One embodiment of this application provides a computer device, including: a processor, a memory, and a network interface;

[0059] The processor is connected to the memory and the network interface. The network interface is used to provide a data communication network element, the memory is used to store a computer program, and the processor is used to call the computer program to execute the method in the embodiments of this application.

[0060] One aspect of this application provides a computer-readable storage medium storing a computer program adapted for loading by a processor and executing the methods described in this application.

[0061] One aspect of this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in this application.

[0062] In this embodiment, when it is determined that the first object switches from the first voice input state to the first voice input pause state, the dialogue context information with the first state is obtained, and a state recognition token is written into the dialogue context information with the first state to obtain dialogue context information with a state to be recognized. In the dialogue model, the dialogue context information with the state to be recognized is processed based on the state recognition token to obtain a state indicator token for the voice text information. The state indicator token is written into the dialogue context information with the state to be recognized to obtain dialogue context information with a second state. If the state indicator token is a reply state token, the dialogue context information with the second state is processed to obtain dialogue context information with the first state. The reply token is written into the dialogue context information with the first state to obtain dialogue context information with a state to be replied to. In the dialogue model, the dialogue reply generation process is performed on the dialogue context information with the state to be replied to based on the reply token to obtain reply text information for replying to voice text information. The reply text information is written into the dialogue context information with the state to be replied to to obtain dialogue context information with a third state. The method provided in this application embodiment, after the dialogue state is identified through the dialogue model, will revert the dialogue context information of the dialogue model to the state before the dialogue state identification, thereby avoiding the pollution of the dialogue context information by the relevant content of the dialogue state identification, thus saving the computing power of the dialogue model to generate responses and improving the response quality of the dialogue model. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application;

[0065] Figure 2 This is a schematic diagram illustrating an application scenario of a voice dialogue method provided in an embodiment of this application;

[0066] Figure 3 This is a flowchart illustrating a voice dialogue method provided in an embodiment of this application;

[0067] Figure 4 This is a schematic diagram illustrating the state fallback of dialogue context information provided in an embodiment of this application;

[0068] Figure 5 This is a flowchart illustrating a voice dialogue method provided in an embodiment of this application;

[0069] Figure 6 This is a schematic diagram illustrating the state fallback of dialogue context information provided in an embodiment of this application;

[0070] Figure 7 This is a flowchart illustrating a voice dialogue method provided in an embodiment of this application;

[0071] Figure 8 This is a schematic diagram of the architecture of a voice dialogue system provided in an embodiment of this application;

[0072] Figure 9 This is a schematic diagram of the overall process of a voice dialogue provided in an embodiment of this application;

[0073] Figure 10 This is a schematic diagram of the structure of a voice dialogue device provided in an embodiment of this application;

[0074] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0076] For details, please see Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application. Figure 1 As shown, this network architecture may include a server 2000 and a terminal cluster. The terminal cluster may specifically include one or more terminals; the number of terminals in the terminal cluster is not limited here. Figure 1As shown, the multiple terminals may specifically include terminal 3000a, terminal 3000b, terminal 3000c, ..., terminal 3000n; terminal 3000a, terminal 3000b, terminal 3000c, ..., terminal 3000n can communicate with server 2000 through a network (i.e., through a medium that provides a communication link via wired, wireless communication links or fiber optic cables, etc.) so that each terminal can interact with server 2000.

[0077] Each terminal in the terminal cluster can include: smartphones, tablets, laptops, desktop computers, smart voice interaction devices, smart home appliances (e.g., smart TVs), wearable devices, in-vehicle terminals, aircraft, and other smart terminals with data processing capabilities. It should be understood that, as... Figure 1 Each terminal in the terminal cluster shown can have an application client installed. When the application client runs on each terminal, it can interact with the aforementioned... Figure 1 Data exchange occurs between the 2000 servers shown.

[0078] Specifically, the application client can include in-vehicle clients, smart home clients, entertainment clients (e.g., game clients), multimedia clients (e.g., video clients), social clients, and information clients (e.g., news clients) with data processing capabilities. In this embodiment, the application client can be integrated into a client (e.g., a social client), or it can be a standalone client (e.g., a news client). This embodiment does not limit the type of application client.

[0079] Among them, such as Figure 1 The server 2000 shown can be the server corresponding to the application client. The server 2000 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0080] For ease of understanding, the embodiments of this application may be described in detail below. Figure 1 One of the multiple terminals shown is selected as the first terminal. For example, in the embodiments of this application, a terminal can be selected as the first terminal. Figure 1The terminal 3000a shown serves as the first terminal, which can be equipped with an application client capable of data processing. In this case, the first terminal can interact with the server 2000 via the application client. Furthermore, in this embodiment, the user corresponding to the terminal can be referred to as an object; for example, the user corresponding to the first terminal can be referred to as the first object.

[0081] like Figure 1 As shown, the first object can generate a voice dialogue request for the AI ​​assistant through an application client installed on the first terminal, and then transmit the voice dialogue request to the server 2000. The server 2000 can then establish a voice dialogue connection between the first object and the AI ​​assistant. The first terminal can then use an integrated dialogue recording component to collect and process audio data, obtaining a dialogue audio data stream, and send the collected dialogue audio data stream to the server 2000 through the voice dialogue connection. The server 2000 can detect the dialogue audio data stream to determine whether the first object is in a voice input state or a voice input pause state. The voice input state indicates that the first object is speaking, and the voice input pause state indicates that the first object is not speaking.

[0082] Furthermore, the server 2000 can employ the voice dialogue method provided in this application to determine whether to respond to the first object when the first object is in a voice input pause state, and generate response text information for responding to the first object when it is determined to respond. Specifically, the voice dialogue method can be as follows: when the server 2000 determines that the first object has switched from a first voice input state to a first voice input pause state, it acquires dialogue context information with a first state, which includes first voice text information corresponding to the first object's first voice data. The first voice data refers to the audio data stream acquired when the first object is in the first voice input state. Then, a state recognition token is written into the dialogue context information with the first state to obtain dialogue context information with a state to be recognized. In the dialogue model, dialogue state recognition processing is performed on the dialogue context information with the state to be recognized based on the state recognition token to obtain a state indication for the voice text information. The token is used to write a state indicator token into the dialogue context information with the state to be identified, resulting in dialogue context information with the second state. If the state indicator token is a response state token, then the dialogue context information with the second state is processed to fall back to obtain dialogue context information with the first state. The response token is written into the dialogue context information with the first state, resulting in dialogue context information with the state to be responded to. In the dialogue model, based on the response token, the dialogue context information with the state to be responded to is processed to generate a dialogue response, resulting in response text information for responding to voice text information. The response text information is written into the dialogue context information with the state to be responded to, resulting in dialogue context information with the third state.

[0083] To facilitate understanding the above voice dialogue process, please also refer to... Figure 2 , Figure 2 This is a schematic diagram illustrating an application scenario of a voice dialogue method provided in an embodiment of this application. For example... Figure 2 The server 20a shown can be the one described above. Figure 1 The server 2000 in the corresponding embodiment, such as Figure 2 The terminal 20b shown can be the one described above. Figure 1 In the corresponding embodiment, the first terminal, terminal 20b, may have an application client installed, and the user corresponding to terminal 20b can be the first object U1. For ease of understanding, this embodiment uses the voice dialogue method executed by server 20a as an example for explanation.

[0084] like Figure 2 As shown, the first object U1 can initiate a voice conversation with the smart assistant through the application client installed on the terminal 20b. After the voice conversation is established, the terminal device 20b will collect and process audio data through the conversation recording component, and send the collected conversation audio data stream 201a to the server 20a in real time (e.g., Figure 2 As shown in S211a, server 20a continuously detects the received dialogue audio data stream 201a to determine whether the first object U1 is speaking.

[0085] like Figure 2 As shown, when server 20a detects the first appearance of a voice signal in the dialogue audio data stream 201a, it can determine that object U1 is in the first voice input state (e.g., Figure 2 As shown in S212a), at this time, server 20a can record the location where the voice signal appears as the starting position of the first voice data. Then, when server 20a detects for the first time in the dialogue audio data stream 201a that the duration of the disappearance of the voice signal exceeds the pause duration threshold, it determines that the first object has switched from the first voice input state to the first voice input pause state (as shown in S212a). Figure 2 As shown in S213a), similarly, server 20a can record the position where the voice signal begins to disappear as the end position of the first voice data. Then, server 20a can determine the audio data stream in the dialogue audio data stream 201a located between the start position and the end position of the first voice data as the first voice data 202a. Then, server 20a can perform speech recognition on the first voice data 202a to obtain the first voice text information corresponding to the first voice data 202a.

[0086] At this point, server 20a needs to determine whether to continue listening to object U1 or to respond to the first voice text information. Therefore, server 20a can obtain the dialogue context information with the first state (such as...). Figure 2 As shown in 203a), the dialogue context information with a first state contains first speech text information. The dialogue context information refers to the input used each time the dialogue model 204a is invoked, typically including the input of an object in a speech dialogue and the historical output of the dialogue model 204a. It should be noted that the dialogue context information with different states described in this application is essentially the same dialogue context information containing different state data; that is, the state of the dialogue context information changes whenever new data is written to it or old data is deleted from it. Then, the server 20a writes a state recognition token to the dialogue context information with the first state, thus obtaining the dialogue context information with the state to be recognized (e.g., ...). Figure 2 As shown in 204a), server 20a then inputs the data from the dialogue context information with the state to be identified into dialogue model 205a. In dialogue model 205a, the dialogue context information with the state to be identified can be processed for dialogue state identification based on this state identification token, resulting in a state indication token for the first voice text information (such as...). Figure 2 (As shown in S214a). Both the state recognition token and the state indication token are special tokens. A special token is typically a special symbol used to instruct the system or model to perform a corresponding task in multi-task processing. In this application, the state recognition token is used to instruct the dialogue model 205a to perform a dialogue state recognition task, while the state indication token is used to indicate the current state of the system. If the state indication token is a reply state token, it indicates that the first voice text information of object U1 should be replied to. If the state indication token is a listening state token, it indicates that object U1 has not finished speaking and should continue to be listened to.

[0087] like Figure 2 As shown, assuming the state indicator token output by dialogue model 205a is a response state token, server 20a will write the response state token into the dialogue context information with the state to be identified, thus obtaining dialogue context information with a second state (such as...). Figure 2 As shown in 206a). It should be understood that the inputs and outputs of dialogue model 205a are written into the dialogue context information each time it is invoked. This is to allow the model to obtain historical response generation data to assist in completing the current response generation task. However, the inputs and outputs related to dialogue state recognition processing have no effect on the response generation task, and can even be considered to pollute the dialogue context information, disrupt the response generation of dialogue model 205a, and thus reduce the quality of the response generation of dialogue model 205a. Therefore, in this application, state rollback processing can be performed on the dialogue context information with a second state, so that the dialogue context information rolls back to the first state (e.g., ...). Figure 2 (As shown in S215a).

[0088] like Figure 2 As shown, after reverting the dialogue context information to the first state, server 20a can perform response generation processing on the dialogue context information with the first state based on the response state token. Specifically, server 20a first writes the response token into the dialogue context information with the first state to obtain the dialogue context information with the state to be responded to (e.g., ...). Figure 2 As shown in 207a), the data contained in the dialogue context information with a pending response state is then input into the dialogue model 205a. In the dialogue model 205a, the dialogue response generation process is performed on the dialogue context information with a pending response state based on the response token, resulting in response text information used to respond to the first voice text information. The response text information is then written into the dialogue context information with a pending response state, resulting in dialogue context information with a third state (such as...). Figure 2(As shown in 208a). Then, server 20a can generate a response audio data stream of the response text information and send it to terminal 20b. Terminal 20b can then play the response text information based on the response audio data stream, thereby enabling voice dialogue between object U1 and the smart assistant.

[0089] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user data. These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without user confirmation), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of related user data must comply with the relevant laws, regulations, and standards of the relevant regions.

[0090] It is understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, as those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0091] Further, please see Figure 3 , Figure 3 This is a flowchart illustrating a voice dialogue method provided in an embodiment of this application. The method can be implemented using a computer device (e.g., the one described above). Figure 1 The server 2000 in the corresponding embodiment executes the method. The following description uses the execution of this method by a computer device as an example, wherein the voice dialogue method may include at least the following steps S101-S104:

[0092] Step S101: When it is determined that the first object switches from the first voice input state to the first voice input pause state, the dialogue context information with the first state is obtained; the dialogue context information with the first state includes the first voice text information corresponding to the first voice data of the first object; the first voice data refers to the audio data stream obtained when the first object is in the first voice input state.

[0093] Specifically, dialogue context information refers to the input when invoking a dialogue model, which includes the user's input during a dialogue and the dialogue model's historical inputs and outputs. Whenever the data in the dialogue context information changes, the state of the dialogue context information changes. Therefore, the dialogue context information with different states described in this application refers to the same dialogue context information having different data at different times.

[0094] Specifically, the first target refers to the user engaging in voice dialogue with the intelligent assistant, also known as an AI assistant. This is a software application developed based on artificial intelligence technology, capable of performing various tasks such as question-and-answer, translation, and casual conversation through voice dialogue. When the first target is in a voice input state, it means they are speaking; when they are in a voice input pause state, it means they have paused speaking. The first voice input state refers to the first time the first target is in this state. At this time, the dialogue model has not yet been invoked, and the corresponding dialogue context information does not yet contain the historical inputs of the dialogue model. The first voice input pause state refers to the first time the first target is in this state. It can be understood that when the first target switches from the first voice input state to the first voice input pause state, it indicates a pause in the voice dialogue. At this point, a judgment needs to be made on the current dialogue state: whether to respond to what the first target said previously, or to continue waiting for the first target to speak.

[0095] Specifically, when the first object switches from the first voice input state to the first voice input pause state, determining the current speaking state requires combining the first voice text information. Therefore, in this application, when it is determined that the first object has switched from the first voice input state to the first voice input pause state, the dialogue context information with the first state that needs to be obtained refers to the dialogue context information after the first voice text information has been written. It can be understood that this is the first time the first object has paused speaking, and the dialogue context information after the first voice text information has been written does not contain other data. If it is not the first time the first object has paused speaking, then the dialogue context information after the voice text information written at the time of speaking will also contain the previous input and output information of the dialogue model.

[0096] Optionally, a feasible implementation of determining whether the first object switches from the first voice input state to the first voice input pause state can be: detecting and processing the dialogue audio data stream; the dialogue audio data stream refers to the audio data stream collected by the dialogue recording component during the voice dialogue with the first object; when a voice signal is detected for the first time in the dialogue audio data stream, the first object is determined to be in the first voice input state; when the duration of the disappearance of the voice signal exceeds the pause duration threshold for the first time in the dialogue audio data stream, the first object is determined to switch from the first voice input state to the first voice input pause state. By detecting the dialogue audio stream data, it is possible to determine when a voice signal appears in the dialogue audio data stream and when the voice signal disappears. When the duration of the disappearance of the voice signal exceeds a certain time, that is, the pause duration threshold, it can be considered that the first object has paused speaking, which will trigger the system to make a judgment on the current dialogue state. The pause duration threshold can be set based on the actual situation, for example, the pause duration threshold can be 30ms (milliseconds), 1s (seconds), etc., which is not limited in this application.

[0097] Specifically, detection processing can be achieved through Voice Activity Detection (VAD) technology. VAD is a technique that automatically detects the presence or absence of speech in an audio signal. Its main task is to distinguish between speech and non-speech components (such as silence and noise) in an audio stream. VAD methods can mainly include threshold-based methods (e.g., short-time energy thresholding, zero-crossing rate thresholding), statistical model-based methods (e.g., Hidden Markov Models (HMMs), Gaussian Mixture Models (GMMs)), and deep learning-based methods (e.g., Deep Neural Networks (DNNs), Convolutional Neural Networks (CNNs)), etc. This application does not impose specific limitations on VAD methods.

[0098] Step S102: Write the state recognition token into the dialogue context information with the first state to obtain the dialogue context information with the state to be recognized. In the dialogue model, perform dialogue state recognition processing on the dialogue context information with the state to be recognized based on the state recognition token to obtain the state indication token for the first voice text information. Write the state indication token into the dialogue context information with the state to be recognized to obtain the dialogue context information with the second state.

[0099] Specifically, to conserve computing power, this application does not deploy an additional dialogue state recognition model for dialogue state identification. Instead, it reuses the computing power of the large language model used for dialogue response generation for dialogue state recognition. In other words, the dialogue model is a large language model capable of multiple tasks, including dialogue response generation and dialogue state recognition. These two tasks can share the part of the large language model that understands the semantics of user input (i.e., speech and text information), thus reducing system complexity. The Large Language Model (LLM) is an artificial intelligence model based on deep learning technology, typically possessing massive parameters and powerful language understanding and generation capabilities. It learns the statistical patterns and semantic information of language through unsupervised or self-supervised learning on large-scale text data, enabling it to generate natural and fluent text, answer questions, engage in dialogue, and complete various text generation tasks.

[0100] Specifically, a state recognition token is a special token, also known as a special token, which is a special symbol or string with a specific function or meaning used to help the model perform operations such as text processing, representation, and understanding. The state recognition token is used to instruct the dialogue model to perform dialogue state recognition processing on the input dialogue context information. Therefore, after obtaining the dialogue context information with the first state, the state recognition token needs to be written into the dialogue context information with the first state to obtain the dialogue context information with the state to be recognized. Then, the data contained in the dialogue context information with the state to be recognized is input into the dialogue model. The dialogue model can then perform dialogue state recognition processing on the dialogue context information with the state to be recognized based on the state recognition token, obtaining a state indicator token for the first speech text information. The output of the dialogue model is also written into the dialogue context information. Therefore, the state indicator token is written into the dialogue context information with the state to be recognized to obtain the dialogue context information with the second state, which is the dialogue context information stored after the dialogue model completes the dialogue state recognition processing for the first speech text information.

[0101] Specifically, the status indicator token is also a special token, which can include a reply status token and a listening status token. The reply status token is used to indicate that the current dialogue state is a reply state, indicating that a reply should be given to the first voice text information at this time. The listening status token is used to indicate that the current dialogue state is a continue listening state, indicating that the listener should continue to wait for the first object to give voice input and no reply is required.

[0102] Step S103: If the status indicator token is a reply status token, then perform status rollback processing on the dialogue context information with the second status to obtain the dialogue context information with the first status.

[0103] Specifically, state rollback processing, also known as state rollback processing or context rollback processing, is used to roll back the dialogue context information from the second state to the first state. This ensures that when the dialogue model generates dialogue responses, it no longer performs meaningless recognition and understanding on content related to dialogue state recognition that is unrelated to the dialogue response. This can improve the quality of dialogue response generation by the dialogue model and save some computing power.

[0104] Specifically, when writing to the dialogue context information, a corresponding operation log can be recorded simultaneously. The log can contain the operation type (e.g., insert, update, delete), key-value pairs, etc. When state rollback is required, these operations can be executed in reverse order based on the operation records in the log to restore the dialogue context information to its previous state. A feasible implementation process for performing state rollback on dialogue context information with a second state to obtain dialogue context information with a first state can include: obtaining the operation log corresponding to the dialogue context information; the operation log contains records of each operation on the dialogue context information; obtaining the target operation record from the operation log; the target operation record refers to the operation record located between the first state time node and the second state time node; the first state time node refers to the time node when the dialogue context information with the first state is obtained; the second state time node refers to the time node when the dialogue context information with the second state is obtained; generating a reverse operation record for the target operation record, and performing operations on the dialogue context information with the second state based on the reverse operation record to obtain the dialogue context information with the first state; the reverse operation record refers to the operation record that has the opposite operation processing to the target operation record. For ease of understanding, assume the target operation record contains: 1. Insert status identification token, 2. Insert status indicator token. Then its corresponding reverse operation record would be: 1. Delete status indicator token, 2. Delete status indicator token.

[0105] Optionally, when obtaining the dialogue context information with the first state, a first state copy data for the dialogue context information with the first state can be generated. At this time, a feasible implementation process for performing state rollback processing on the dialogue context information with the second state to obtain the dialogue context information with the first state can be: replacing the data contained in the dialogue context information with the second state with the first state copy data to obtain the dialogue context information with the first state.

[0106] Step S104: Write the reply token into the dialogue context information with the first state to obtain the dialogue context information with the pending reply state. In the dialogue model, perform dialogue reply generation processing on the dialogue context information with the pending reply state based on the reply token to obtain reply text information for replying to the first voice text information. Write the reply text information into the dialogue context information with the pending reply state to obtain the dialogue context information with the third state.

[0107] Specifically, the response token is also a special token used to instruct the dialogue model to generate a dialogue response based on the input dialogue context information. Therefore, the state recognition token needs to be written into the dialogue context information with the first state to obtain the dialogue context information with the response state. Then, the data contained in the dialogue context information with the response state is input into the dialogue model. The dialogue model can then perform dialogue state recognition processing on the dialogue context information with the response state based on the response token to obtain the response text information used to respond to the first voice text information. Writing the response text information into the dialogue context information with the response state yields the dialogue context information with the third state.

[0108] Optionally, the computer device can perform speech-to-speech processing on the reply text information to obtain the reply voice data corresponding to the reply text information (i.e., the above). Figure 2 The aforementioned audio data stream sends the response voice data to the first terminal corresponding to the first object. The first terminal can then play the response text information based on the response voice data. Here, speech processing refers to TTS (Text-to-Speech) processing, a technology that converts text into speech, also known as speech synthesis technology.

[0109] To better understand the state transition process of the above dialogue context information, please also refer to... Figure 4 , Figure 4 This is a schematic diagram illustrating the state fallback of dialogue context information provided in an embodiment of this application. For example... Figure 4 As shown, when the first user pauses speaking, the dialogue context information will contain "user input 1.1". At this time, the dialogue context information can be said to have state S41 (e.g., Figure 4 As shown in 401), where the user input 1.1 refers to the aforementioned first voice text information, the computer device then determines that the first object has paused speaking, triggering dialogue state recognition. At this time, the computer device writes a state recognition token into the dialogue context information. The dialogue context information contains "user input 1.1 + state recognition token". At this point, the dialogue context information can be said to have a state to be recognized (e.g., ...). Figure 4(As shown in 402). Then, the computer device uses the dialogue model to identify the dialogue state of the user input 1.1 in the dialogue context information. The model response output by the dialogue model is also written into the dialogue context information. The dialogue context information will then contain "user input 1.1 + state recognition token + model response - response state token". At this time, the dialogue context information can be said to have state S42 (e.g., Figure 4 (As shown in 403).

[0110] like Figure 4 As shown, after the dialogue context information contains the output of the dialogue model regarding dialogue state recognition, the computer device performs a state rollback process on the dialogue context information, reverting it to the state before dialogue state recognition, that is, reverting the dialogue context information from state S42 to state S41. The reverted dialogue context information only contains "user input 1.1" (e.g., ...). Figure 4 (As shown in 404).

[0111] like Figure 4 As shown, after completing the rollback, the computer device writes a reply token into the dialog context information. The dialog context information will then contain "user input 1.1 + rollback token". At this point, the dialog context information can be said to have a pending reply state (e.g., ...). Figure 4 (As shown in 405). Then, the computer device can generate a dialogue response to the user input 1.1 through the dialogue model. The model response output by the dialogue model will also be written into the dialogue context information, which will then contain "user input 1.1 + state recognition token + model response - response 1". At this time, the dialogue context information can be said to have state S43 (e.g., Figure 4 (As shown in 406). Reply 1 refers to the aforementioned reply text information.

[0112] The method provided in this application embodiment can simultaneously perform dialogue state recognition and dialogue response generation through a dialogue model, saving the computational cost of the model. After the dialogue state is recognized through the dialogue model, the dialogue context information of the dialogue model will be reverted to the state before the dialogue state recognition was performed, thereby avoiding the pollution of the dialogue context information by the content related to the dialogue state recognition, thus saving the computational power of the dialogue model to generate responses and improving the response quality of the dialogue model.

[0113] Further, please see Figure 5 , Figure 5 This is a flowchart illustrating a voice dialogue method provided in an embodiment of this application. The method can be implemented using a computer device (e.g., the one described above). Figure 1The server 2000 in the corresponding embodiment executes the method. The following description uses the execution of this method by a computer device as an example, wherein the voice dialogue method may include at least the following steps S201-S206:

[0114] Step S201: When it is determined that the first object has switched from the first voice input state to the first voice input pause state, the first voice data of the first object is acquired.

[0115] Specifically, a feasible implementation process for obtaining the first voice data of the first object can be as follows: From the dialogue audio data stream, obtain the audio data stream whose time information matches the first input time period, as the first voice data. Here, the dialogue audio data stream refers to the audio data stream collected by the dialogue recording component during the voice dialogue with the first object; the first input time information refers to the time period corresponding to when the first object is in the first voice input state. For example, if the duration of the dialogue audio data stream is 12 seconds, and the first input time information is from the 3rd to the 8th second, then the first voice data refers to the audio data stream from the 3rd to the 8th second of the dialogue audio data stream.

[0116] Step S202: Perform speech recognition on the first speech data to obtain the first speech text information corresponding to the first speech data, and generate dialogue context information with a first state based on the first speech text information.

[0117] Specifically, speech recognition can be achieved using ASR (Automatic Speech Recognition) technology. ASR is a technology that converts human speech signals into text or machine-understandable commands. The process involves: first, denoising and filtering the speech signal to improve its quality for easier subsequent analysis and processing; then, extracting parameters that characterize speech features from the preprocessed signal, such as Mel-frequency cepstral coefficients (MFCC) and linear predictive cepstral coefficients (LPCC), which reflect the acoustic properties of speech; next, matching the extracted speech features with an acoustic model, typically trained on large amounts of speech data, which converts speech features into corresponding phoneme or syllable sequences; finally, processing the phoneme or syllable sequences output by the acoustic model using a language model, predicting and filtering possible text sequences based on grammar, vocabulary, and semantics to obtain the recognition result that best fits the language rules and context.

[0118] Specifically, speech recognition can be implemented using a separate ASR module, or it can be implemented by reusing a dialogue model, similar to dialogue state recognition. In this case, a feasible implementation process for performing speech recognition on the first speech data to obtain the corresponding first speech-text information can be as follows: A speech recognition token is concatenated to the first speech data to obtain the speech data to be recognized; the speech data to be recognized is input into the dialogue model, where the dialogue model performs speech recognition on the first speech data based on the speech recognition token to obtain the corresponding first speech-text information. Here, the speech recognition token is also a special token used to instruct the dialogue model to perform speech recognition processing on the input speech data.

[0119] Step S203: Write the state recognition token into the dialogue context information with the first state to obtain the dialogue context information with the state to be recognized. In the dialogue model, perform dialogue state recognition processing on the dialogue context information with the state to be recognized based on the state recognition token to obtain the state indication token for the voice text information. Write the state indication token into the dialogue context information with the state to be recognized to obtain the dialogue context information with the second state.

[0120] Specifically, the implementation of step S203 can be found above. Figure 3 The specific description of step S102 in the corresponding embodiment will not be repeated here.

[0121] Step S204: If the state indicator token is a listening state token, then perform state rollback processing on the dialogue context information with the second state to obtain the dialogue context information with the first state.

[0122] Specifically, the implementation of step S204 can be found above. Figure 3 The specific description of step S103 in the corresponding embodiment will not be repeated here.

[0123] Step S205: When it is determined that the first object has switched from the second voice input state to the second voice input pause state, the second voice data of the first object is obtained, and the second voice data is used for voice recognition to obtain the second voice text information corresponding to the second voice data.

[0124] Specifically, the second voice input state refers to the first object being in the voice input state for the second time, indicating that the first object started speaking for the second time after pausing speaking for the first time. The second voice input pause state refers to the first object pausing speaking for the second time. The second voice data refers to the audio data stream acquired when the first object is in the second voice input state. The acquisition process and the corresponding speech recognition process can refer to the acquisition process and speech recognition process of the first voice data, which will not be elaborated here.

[0125] Step S206: Write the second voice text information and the state recognition token into the dialogue context information with the first state to obtain dialogue context information with a new state to be recognized. In the dialogue model, perform dialogue state recognition processing on the dialogue context information with the new state to be recognized based on the state recognition token.

[0126] Specifically, the dialogue context information with a new state to be identified will include first voice text information, second voice text information, and a state identification token. The computer device performs dialogue state identification processing on the dialogue context information with a new state to be identified based on the state identification token, and outputs a state indication token for the first voice text information and the second voice text information. Then, the computer device decides to continue to acquire new voice data of the first object based on the state indication token, and then generates reply text information for replying to the first voice text information and the second voice text information.

[0127] To better understand the state transition process of the above dialogue context information, please also refer to... Figure 6 , Figure 6 This is a schematic diagram illustrating the state fallback of dialogue context information provided in an embodiment of this application. For example... Figure 6 As shown, when the first user pauses speaking, the dialogue context information will contain "user input 1.1". At this time, the dialogue context information can be said to have state S61 (e.g., Figure 6 As shown in 601), where the user input 1.1 refers to the aforementioned first voice text information, the computer device then determines that the first object has paused speaking, triggering dialogue state recognition. At this time, the computer device writes a state recognition token into the dialogue context information. The dialogue context information contains "user input 1.1 + state recognition token", at which point the dialogue context information can be said to have a state to be recognized (e.g., ...). Figure 6(As shown in 602). Then, the computer device uses the dialogue model to identify the dialogue state of the user input 1.1 in the dialogue context information. The model response output by the dialogue model is also written into the dialogue context information. Assuming the model response is a listening state token, the dialogue context information will contain "user input 1.1 + state recognition token + model response - listening state token". At this time, the dialogue context information can be said to have state S62 (e.g., ...). Figure 6 (As shown in 603).

[0128] like Figure 6 As shown, after the dialogue context information contains the output of the dialogue model regarding dialogue state recognition, the computer device performs a state rollback process on the dialogue context information, reverting it to the state before dialogue state recognition, that is, reverting the dialogue context information from state S62 to state S61. The reverted dialogue context information only contains "user input 1.1" (e.g., ...). Figure 6 (As shown in 604).

[0129] like Figure 6 As shown, after completing the state rollback, the computer device will acquire the voice data of the first object again. When the first object pauses speaking again, the computer device will write the corresponding voice-text information (assuming it's user input 1.2) into the dialogue context information with state S61, resulting in dialogue context information containing "user input 1.1 + user input 1.2". At this point, the dialogue context information can be said to have state S63 (e.g., ...). Figure 6 (As shown in 605). Then, the computer device will write the state recognition token into the dialogue context information again. The dialogue context information contains "user input 1.1 + user input 1.2 + state recognition token". At this time, the dialogue context information can be said to have a new state to be recognized (such as...). Figure 6 (As shown in 606). Then, the computer device uses the dialogue model to identify the dialogue state of user input 1.1 and user input 1.2 in the dialogue context information. The model response output by the dialogue model will continue to be written into the dialogue context information. Assuming that the current dialogue state is identified as a response state, the dialogue context information will contain "user input 1.1 + user input 1.2 + state identification token + model response - response state token". At this time, the dialogue context information can be said to have state S64 (e.g., ...). Figure 6 (As shown in 607).

[0130] like Figure 6As shown, after the dialogue context information again contains the output of the dialogue model regarding dialogue state recognition, the computer device will perform another state rollback process on the dialogue context information, reverting it to the state before dialogue state recognition, that is, reverting the dialogue context information from state S64 to state S63. The reverted dialogue context information will only contain "user input 1.1 + user input 1.2" (e.g., ...). Figure 6 (As shown in 608). Then, the computer device performs dialogue response generation processing on the dialogue context information with state S63 through the dialogue model to obtain response 1 for responding to user input 1.1 and user input 1.2.

[0131] The method provided in this application allows for simultaneous dialogue state recognition and dialogue response generation through a dialogue model, saving computational costs. Each time the user pauses, the system outputs a listening state token (representing continued listening) through the dialogue model, maintaining the listening state while removing context related to the current dialogue state recognition. This ensures that when generating a dialogue response based on the user's voice and text information, the dialogue context only contains the user's voice and text information, without any dialogue state recognition-related output (listening state token), thus avoiding any impact on the dialogue model's response generation. Furthermore, the dialogue context information used by the dialogue model is significantly shorter, reducing the computational power required for model responses.

[0132] Further, please see Figure 7 , Figure 7 This is a flowchart illustrating a voice dialogue method provided in an embodiment of this application. The method can be implemented using a computer device (e.g., the one described above). Figure 1 The server 2000 in the corresponding embodiment executes the method. The following description uses the execution of this method by a computer device as an example, wherein the voice dialogue method may include at least the following steps S301-S309:

[0133] Step S301: During the process of the first object being in the first voice input state, the first voice data of the first object is subjected to periodic voice recognition processing to obtain the first periodic voice text information.

[0134] Specifically, the first periodic speech-text information includes the periodic speech-text information corresponding to N consecutive periodic speech data in the first speech data; a periodic speech data refers to the audio data stream in the first speech data whose speech duration reaches the duration of a periodic speech.

[0135] Specifically, a feasible implementation of performing periodic speech recognition processing on the first speech data of the first object to obtain the first periodic speech text information during the first object's first speech input state can be as follows: During the first object's first speech input state, the first speech data of the first object is subjected to duration statistics processing to obtain the cumulative speech duration of the first speech data; when the cumulative speech duration is detected to be k times the periodic speech duration, the audio data stream corresponding to the kth periodic speech duration is sequentially obtained from the first speech data as the kth periodic speech data; k is an integer less than N; speech recognition is performed on the kth periodic speech data to obtain the periodic speech text information corresponding to the kth periodic speech data; the first N periodic speech text information are sequentially concatenated to obtain the first periodic speech text information. The periodic speech duration can usually be set according to the actual situation, such as 1s, 2s, etc., and this application does not impose any restrictions here.

[0136] For ease of understanding, assume the duration of a periodic speech is 2 seconds. During the first voice input state, the computer device continuously receives the first voice data from the first terminal. When the computer device detects that the cumulative speech duration of the first voice data reaches 2 seconds, it identifies the first 2 seconds of audio data stream as the first periodic speech data. Then, it performs speech recognition on the first periodic speech data to obtain the first periodic speech text information. Next, the computer device continues to detect the cumulative speech duration of the first data. When it detects that it reaches 4 seconds, it identifies the audio data stream corresponding to the second 2 seconds in the first voice data as the second periodic speech data. Then, it performs speech recognition on the second periodic speech data to obtain the second periodic speech text information. Suppose that at the 5th second, the computer device detects that the first object has switched from the first voice input state to the first voice input pause state. The computer device will then pause the periodic voice recognition processing of the first voice data. At this time, the audio data stream corresponding to the last second of the first voice data will not be converted because the periodic voice duration has not yet been reached. The computer device will then determine the audio data stream corresponding to the last second of the first voice data as the first remaining voice data and perform voice recognition on it separately.

[0137] Step S302: When it is determined that the first object has switched from the first voice input state to the first voice input pause state, dialogue context information with an initial state is created based on the first periodic voice text information.

[0138] Specifically, the computer device writes the first-cycle voice text information into the dialogue context information, thus obtaining the dialogue context information with the initial state, which is the dialogue context information containing the first-cycle voice text information.

[0139] Step S303: Perform speech recognition processing on the first remaining speech data in the first speech data to obtain the first remaining speech text information corresponding to the first remaining speech data.

[0140] Specifically, if the first speech data contains N periodic speech data, then the first remaining speech data is the audio data stream in the first speech data excluding the N periodic speech data that are continuous.

[0141] Step S304: Write the first remaining voice text information into the dialogue context information with the initial state to obtain the dialogue context information with the first state.

[0142] Specifically, this means that the dialogue context information in the first state at this time will include "first cycle voice text information + first remaining voice text information".

[0143] Step S305: Write the state recognition token into the dialogue context information with the first state to obtain the dialogue context information with the state to be recognized. In the dialogue model, perform dialogue state recognition processing on the dialogue context information with the state to be recognized based on the state recognition token to obtain the state indication token for the voice text information. Write the state indication token into the dialogue context information with the state to be recognized to obtain the dialogue context information with the second state.

[0144] For details on the implementation process of step S305, please refer to the above. Figure 3 The specific description of step S102 in the corresponding embodiment will not be repeated here.

[0145] Step S306: If the state indicator token is a listening state token, then perform state rollback processing on the dialogue context information with the second state to obtain the dialogue context information with the initial state.

[0146] Specifically, since the duration corresponding to the first remaining speech data may be short, the accuracy of the first remaining speech text information obtained by speech recognition is not very high. Therefore, when performing state rollback, the dialogue context information can be rolled back to the initial state, that is, rolled back to the state containing the first cycle speech text information. Then, while the first object is in the second speech input state, the first remaining speech data is re-recognized.

[0147] Step S307: During the process of the first object being in the second voice input state, the second voice data of the first object is concatenated to the end of the first remaining voice data to obtain joint voice data. Periodic voice recognition processing is performed on the joint voice data to obtain second periodic voice text information.

[0148] Specifically, the second speech data refers to the audio data stream acquired when the first object is in the second speech input state; the second periodic speech text information includes the periodic speech text information corresponding to M consecutive periodic speech data in the joint speech data.

[0149] Specifically, the process of performing periodic speech recognition processing on the joint speech data can be found in the detailed description of performing periodic speech recognition processing on the first speech data of the first object to obtain the first periodic speech text information, which will not be repeated here.

[0150] Step S308: When it is determined that the first object switches from the second voice input state to the second voice input pause state, the second periodic voice text information is written into the dialogue context information with the initial state to obtain the dialogue context information with the fourth state.

[0151] Specifically, the dialogue context information with the fourth state will include "first-cycle speech-text information + second-cycle speech-text information".

[0152] Step S309: Perform speech recognition processing on the second remaining speech data in the joint speech data to obtain the second remaining speech text information corresponding to the second remaining speech data. Write the second remaining speech text information and the state recognition token into the dialogue context information with the fourth state to obtain the dialogue context information with a new state to be recognized. In the dialogue model, perform dialogue state recognition processing on the dialogue context information with the new state to be recognized based on the state recognition token.

[0153] Specifically, the second remaining speech data refers to the audio data stream excluding the M consecutive periodic speech data in the joint speech data. The implementation process of step S309 can be found in the description of steps S303-S305, and will not be repeated here.

[0154] The method described in this application embodiment allows for speech recognition of the first subject's speech data when the speaking duration reaches the periodic speech duration, eliminating the need to wait for the first subject to finish speaking before speech recognition, thus reducing the time the first subject spends waiting for a response. Furthermore, for any remaining speech data generated before the periodic speech duration is reached when the first subject pauses, the remaining speech-text information is rolled back when the computer device determines the current state is listening, thereby avoiding the problem of low accuracy caused by the short duration of the corresponding remaining speech data.

[0155] Further, please see Figure 8 , Figure 8 This is a schematic diagram of the architecture of a voice dialogue system provided in an embodiment of this application. Figure 8 As shown, the entire voice dialogue system may include a voice activity detection module 81, a voice recognition module 82, a response generation module 83, and a text-to-speech module 84. The voice activity detection module 81 is responsible for determining whether the current audio contains human voice, i.e., the voice activity detection shown in step S101 above. The voice recognition module 82 is responsible for recognizing human voice content as text, i.e., recognizing voice data as voice-text information as described above. The response generation module 83 is responsible for the dialogue state recognition, dialogue response generation, and state rollback processing described in this application. The text-to-speech module 84 is responsible for converting the response text information generated by the response generation module 83 into a voice signal and outputting it to the user.

[0156] like Figure 8 As shown, the voice activity detection module 81 and the voice recognition module 82 can accept user voice input audio signals. When the voice activity detection module 81 detects that there was a voice in the previous moment but no voice in the current moment, it will trigger the response generation module 83 to determine whether to continue listening or generate a response based on the voice text information generated by the voice recognition module 82.

[0157] To better understand the processing flow of the aforementioned voice dialogue system, please refer to [link / reference needed]. Figure 9 , Figure 9 This is a schematic diagram of the overall process of a voice dialogue provided in an embodiment of this application. Figure 9 As shown, the entire voice dialogue process can include the following steps:

[0158] Step S91: Obtain the voice data of the first subject when he / she speaks.

[0159] Step S92: When the first object is detected to have paused speaking, the voice text information corresponding to the voice data is written into the dialogue context information.

[0160] Specifically, the implementation process of steps S91-S92 can be found above. Figure 5 The specific descriptions of steps S201-S202 in the corresponding embodiments will not be repeated here.

[0161] Step S93: Write the state recognition token into the dialogue context information, perform dialogue state recognition processing on the dialogue context information based on the state recognition token in the dialogue model, and write the output state indication token into the dialogue context information.

[0162] For details on the implementation process of step S93, please refer to the above. Figure 3 The specific description of step S102 in the corresponding embodiment will not be repeated here.

[0163] Step S94: Determine whether the status indication token is a reply status token. If yes, proceed to step S95; otherwise, proceed to step S96.

[0164] Step S95: Revert the dialogue context information to the state before the current dialogue state recognition process, and perform response generation processing on the reverted dialogue context information.

[0165] Specifically, the implementation process of step S95 can be found above. Figure 3 The specific descriptions of steps S103-S104 in the corresponding embodiments will not be repeated here.

[0166] Step S96: Revert the dialogue context information to the state before the current dialogue state recognition process, and continue to execute step S91.

[0167] The method provided in this application utilizes a main model (dialogue model) with semantic understanding capabilities for dialogue state recognition, thus achieving high recognition accuracy. Furthermore, it reuses the computational power consumed by the main model in generating dialogue responses, resulting in lower cost and latency. Moreover, once the state indicator token in the dialogue context information helps the system determine the current state—that is, whether the system is in a listening or responding state—it immediately rolls back the state information, avoiding contamination of the main model's context by dialogue state recognition. This leads to better response quality, and the shorter context further conserves the main model's computational power.

[0168] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a voice dialogue device provided in an embodiment of this application. The voice dialogue device can be a computer program (including program code) running on a computer device; for example, the voice dialogue device is an application software. The voice dialogue device 1 can be used to execute corresponding steps in the voice dialogue method provided in the embodiments of this application. Figure 10As shown, the voice dialogue device 1 may include: an acquisition module 110, a status recognition module 120, a rollback module 130, and a response module 140.

[0169] The acquisition module 110 is used to acquire dialogue context information with the first state when it is determined that the first object switches from the first voice input state to the first voice input pause state; the dialogue context information with the first state includes the first voice text information corresponding to the first voice data of the first object; the first voice data refers to the audio data stream acquired when the first object is in the first voice input state;

[0170] The state recognition module 120 is used to write the state recognition token into the dialogue context information with a first state to obtain the dialogue context information with a state to be recognized. In the dialogue model, the state recognition token is used to perform dialogue state recognition processing on the dialogue context information with a state to be recognized to obtain a state indicator token for the voice text information. The state indicator token is written into the dialogue context information with a state to be recognized to obtain the dialogue context information with a second state.

[0171] The rollback module 130 is used to perform state rollback processing on the dialogue context information with the second state if the state indicator token is a reply state token, so as to obtain the dialogue context information with the first state.

[0172] The response module 140 is used to write the response token into the dialogue context information with a first state to obtain the dialogue context information with a pending response state. In the dialogue model, the dialogue response generation process is performed on the dialogue context information with a pending response state based on the response token to obtain the response text information for replying to voice text information. The response text information is then written into the dialogue context information with a pending response state to obtain the dialogue context information with a third state.

[0173] The specific implementation methods of the acquisition module 110, status recognition module 120, rollback module 130, and response module 140 can be found above. Figure 3 The specific descriptions of steps S101-S104 in the corresponding embodiments will not be repeated here.

[0174] In one possible implementation, the voice dialogue device 1 further includes a voice detection module 150.

[0175] The voice detection module 150 is used to detect and process the dialogue audio data stream; the dialogue audio data stream refers to the audio data stream collected by the dialogue recording component during the voice dialogue with the first object.

[0176] The voice detection module 150 is also used to determine that the first object is in the first voice input state when a voice signal is first detected in the dialogue audio data stream;

[0177] The voice detection module 150 is also used to determine that the first object switches from the first voice input state to the first voice input pause state when the duration of the disappearance of the voice signal exceeds the pause duration threshold for the first time in the dialogue audio data stream.

[0178] The voice detection module 150 can be found in the above-mentioned... Figure 3 The optional description of step S101 in the corresponding embodiments will not be repeated here.

[0179] In one possible implementation, when the acquisition module 110 acquires the dialogue context information having a first state, it specifically performs the following operations:

[0180] Obtain the first voice data of the first object;

[0181] Perform speech recognition on the first speech data to obtain the first speech text information corresponding to the first speech data;

[0182] Dialogue context information with a first state is generated based on the first voice text information.

[0183] The specific implementation of the acquisition module 110 for acquiring dialogue context information with a first state can be found in the above description. Figure 5 The specific descriptions of steps S201-S202 in the corresponding embodiments will not be repeated here.

[0184] In one possible implementation, when the acquisition module 110 performs speech recognition on the first speech data to obtain the first speech text information corresponding to the first speech data, it is specifically used to perform the following operations:

[0185] The voice recognition token is concatenated after the first voice data to obtain the voice data to be recognized;

[0186] The speech data to be recognized is input into the dialogue model. In the dialogue model, speech recognition is performed on the first speech data based on the speech recognition token to obtain the first speech text information corresponding to the first speech data.

[0187] The acquisition module 110 is used to perform speech recognition on the first speech data. The specific implementation method for obtaining the first speech text information corresponding to the first speech data can be found in the above description. Figure 5 The specific description of step S202 in the corresponding embodiments will not be repeated here.

[0188] In one possible implementation, when the acquisition module 110 acquires the first voice data of the first object, it is specifically used to perform the following operations:

[0189] From the dialogue audio data stream, an audio data stream whose time information matches the first input time period is obtained as the first voice data; the dialogue audio data stream refers to the audio data stream collected by the dialogue recording component during the voice dialogue with the first object; the first input time information refers to the time period corresponding to when the first object is in the first voice input state.

[0190] The specific implementation of the acquisition module 110 for acquiring the first voice data of the first object can be found in the above description. Figure 5 The specific description of step S201 in the corresponding embodiments will not be repeated here.

[0191] In one possible implementation, the voice dialogue device 1 further includes a first listening module 160.

[0192] The first listening module 160 is used to perform state rollback processing on the dialogue context information with the second state if the state indicator token is a listening state token, so as to obtain the dialogue context information with the first state.

[0193] The first listening module 160 is also used to acquire the second voice data of the first object when it is determined that the first object has switched from the second voice input state to the second voice input pause state, perform voice recognition on the second input data, and obtain the second voice text information corresponding to the second voice data.

[0194] The first listening module 160 is also used to write the second voice text information and the state recognition token into the dialogue context information with the first state, to obtain the dialogue context information with the new state to be recognized, and to perform dialogue state recognition processing on the dialogue context information with the new state to be recognized based on the state recognition token in the dialogue model.

[0195] The specific implementation of the first listening module 160 can be found in the above description. Figure 5 The specific descriptions of steps S204-S206 in the corresponding embodiments will not be repeated here.

[0196] In one possible implementation, the voice dialogue device 1 further includes a period recognition module 170.

[0197] The periodic recognition module 170 is used to perform periodic speech recognition processing on the first speech data of the first object during the first speech input state to obtain first periodic speech text information; the first periodic speech text information includes periodic speech text information corresponding to N consecutive periodic speech data in the first speech data; a periodic speech data refers to an audio data stream in the first speech data whose speech duration reaches the periodic speech duration;

[0198] The acquisition module 110 is used to acquire dialogue context information with the first state when it is determined that the first object has switched from the first voice input state to the first voice input pause state. Specifically, it is used to perform the following operations:

[0199] When it is determined that the first object has switched from the first voice input state to the first voice input pause state, dialogue context information with an initial state is created based on the first cycle voice text information;

[0200] Speech recognition processing is performed on the first remaining speech data in the first speech data to obtain the first remaining speech text information corresponding to the first remaining speech data; the first remaining speech data refers to the audio data stream in the first speech data excluding N consecutive periodic speech data;

[0201] Write the first remaining voice text information into the dialogue context information with the initial state to obtain the dialogue context information with the first state.

[0202] The specific implementation of the period recognition module 170 and the acquisition module 110 for acquiring dialogue context information with the first state when it is determined that the first object has switched from the first voice input state to the first voice input pause state can be found in the above description. Figure 6 The specific descriptions of steps S301-S304 in the corresponding embodiments will not be repeated here.

[0203] In one possible implementation, the voice dialogue device 1 further includes a second listening module 180.

[0204] The second listening module 180 is used to perform state rollback processing on the dialogue context information with the second state if the state indication token is a listening state token, so as to obtain the dialogue context information with the initial state.

[0205] The second listening module 180 is also used to concatenate the second voice data of the first object to the end of the first remaining voice data during the process of the first object being in the second voice input state, so as to obtain joint voice data; the second voice data refers to the audio data stream obtained when the first object is in the second voice input state;

[0206] The second listening module 180 is also used to perform periodic speech recognition processing on the joint speech data to obtain second periodic speech text information; the second periodic speech text information includes the periodic speech text information corresponding to M consecutive periodic speech data in the joint speech data.

[0207] The second listening module 180 is also used to write the second-cycle voice text information into the dialogue context information with the initial state when it is determined that the first object has switched from the second voice input state to the second voice input pause state, so as to obtain the dialogue context information with the fourth state.

[0208] The second listening module 180 is also used to perform speech recognition processing on the second residual speech data in the joint speech data to obtain the second residual speech text information corresponding to the second residual speech data; the second residual speech data refers to the audio data stream in the joint speech data excluding the M periodic speech data that have continuity.

[0209] The second listening module 180 is also used to write the second remaining voice text information and the state recognition token into the dialogue context information with the fourth state to obtain the dialogue context information with the new state to be recognized, and to perform dialogue state recognition processing on the dialogue context information with the new state to be recognized based on the state recognition token in the dialogue model.

[0210] The specific implementation of the second listening module 180 can be found in the above description. Figure 6 The specific descriptions of steps S305-S309 in the corresponding embodiments will not be repeated here.

[0211] In one possible implementation, the periodicity recognition module 170 is used to perform periodic speech recognition processing on the first speech data of the first object during the first speech input state to obtain the first periodic speech text information, specifically for the following operations:

[0212] During the process of the first object being in the first voice input state, the first voice data of the first object is subjected to duration statistics processing to obtain the cumulative voice duration of the first voice data;

[0213] When the cumulative speech duration is detected to be k times the periodic speech duration, the audio data stream corresponding to the kth periodic speech duration is sequentially obtained from the first speech data and used as the kth periodic speech data; k is an integer less than N;

[0214] Speech recognition is performed on the k-th cycle of speech data to obtain the corresponding cycle speech text information;

[0215] The first N cycles of speech-text information are sequentially concatenated to obtain the first cycle of speech-text information.

[0216] The periodic recognition module 170 is used to perform periodic speech recognition processing on the first speech data of the first object during the first speech input state to obtain the first periodic speech text information. The specific implementation of this method can be found above. Figure 6 The specific description of step S301 in the corresponding embodiments will not be repeated here.

[0217] In one possible implementation, the acquisition module 110 is also used to perform the following operations:

[0218] When obtaining the dialogue context information with the first state, generate a copy of the first state data for the dialogue context information with the first state;

[0219] The acquisition module 110 is used to perform state rollback processing on the dialogue context information with the second state. When obtaining the dialogue context information with the first state, it is specifically used to perform the following operations:

[0220] The data contained in the dialogue context information with the second state is replaced with the copy data of the first state to obtain the dialogue context information with the first state.

[0221] In one possible implementation, the acquisition module 110 is used to perform state rollback processing on the dialogue context information with the second state to obtain the dialogue context information with the first state, specifically for performing the following operations:

[0222] Retrieve the operation log corresponding to the dialogue context information; the operation log contains records of each operation performed on the dialogue context information.

[0223] Retrieve the target operation record from the operation log; the target operation record refers to the operation record located between the first state time node and the second state time node; the first state time node refers to the time node at which the dialogue context information with the first state is obtained; the second state time node refers to the time node at which the dialogue context information with the second state is obtained;

[0224] Generate a reverse operation record for the target operation record, and perform operation processing on the dialogue context information with the second state based on the reverse operation record to obtain the dialogue context information with the first state; the reverse operation record refers to the operation record that has the opposite operation processing to the target operation record.

[0225] The apparatus provided in this application embodiment can revert the dialogue context information of the dialogue model to the state before the dialogue state recognition was performed after the dialogue state recognition was performed through the dialogue model. This avoids the content related to the dialogue state recognition from polluting the dialogue context information, thereby saving the computing power of the dialogue model to generate responses and improving the response quality of the dialogue model.

[0226] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 11 As shown above, Figure 10 The voice interaction device 1 in the corresponding embodiment can be applied to a computer device 1000, which may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. Figure 11 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0227] In such Figure 11 In the computer device 1000 shown, the network interface 1004 provides network communication elements; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0228] When it is determined that the first object switches from the first voice input state to the first voice input pause state, the dialogue context information with the first state is obtained; the dialogue context information with the first state includes the first voice text information corresponding to the first voice data of the first object; the first voice data refers to the audio data stream obtained when the first object is in the first voice input state;

[0229] Write the state recognition token into the dialogue context information with the first state to obtain the dialogue context information with the state to be recognized. In the dialogue model, perform dialogue state recognition processing on the dialogue context information with the state to be recognized based on the state recognition token to obtain the state indicator token for the voice text information. Write the state indicator token into the dialogue context information with the state to be recognized to obtain the dialogue context information with the second state.

[0230] If the state indicator token is a reply state token, then the dialog context information with the second state is processed by state rollback to obtain the dialog context information with the first state.

[0231] Write the reply token into the dialogue context information with the first state to obtain the dialogue context information with the pending reply state. In the dialogue model, perform dialogue reply generation processing on the dialogue context information with the pending reply state based on the reply token to obtain reply text information for replying voice text information. Write the reply text information into the dialogue context information with the pending reply state to obtain the dialogue context information with the third state.

[0232] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 , Figure 5 , Figure 6 The description of the voice dialogue method in any corresponding embodiment will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0233] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program executed by the aforementioned voice dialogue device 1. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figure 3 , Figure 5 , Figure 6 The description of the voice dialogue method in any corresponding embodiment is already provided, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0234] The aforementioned computer-readable storage medium can be the internal storage unit of the voice interaction device provided in any of the foregoing embodiments or the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0235] Furthermore, it should be noted that this application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned... Figure 3 , Figure 4 The method provided in any of the corresponding embodiments.

[0236] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0237] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0238] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the foregoing description as a network element. Whether these network elements are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described network elements using different methods for each specific application, but such implementation should not be considered beyond the scope of this application.

[0239] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A voice dialogue method, characterized in that, include: When it is determined that the first object has switched from the first voice input state to the first voice input pause state, the dialogue context information with the first state is obtained; The dialogue context information with a first state includes first voice text information corresponding to the first voice data of the first object; the first voice data refers to the audio data stream acquired when the first object is in a first voice input state; Write the state recognition token into the dialogue context information with the first state to obtain the dialogue context information with the state to be recognized. In the dialogue model, perform dialogue state recognition processing on the dialogue context information with the state to be recognized based on the state recognition token to obtain the state indication token for the first voice text information. Write the state indication token into the dialogue context information with the state to be recognized to obtain the dialogue context information with the second state. If the status indicator token is a reply status token, then the dialogue context information with the second state is processed by state rollback to obtain the dialogue context information with the first state. A reply token is written into the dialogue context information with the first state to obtain dialogue context information with a pending reply state. In the dialogue model, a dialogue reply generation process is performed on the dialogue context information with the pending reply state based on the reply token to obtain reply text information for replying to the first voice text information. The reply text information is written into the dialogue context information with the pending reply state to obtain dialogue context information with the third state.

2. The method according to claim 1, characterized in that, Also includes: The dialogue audio data stream is detected and processed; the dialogue audio data stream refers to the audio data stream collected by the dialogue recording component during the voice dialogue with the first object. When a voice signal is first detected in the dialogue audio data stream, it is determined that the first object is in a first voice input state; When the duration of the disappearance of the voice signal exceeds the pause duration threshold for the first time in the dialogue audio data stream, it is determined that the first object switches from the first voice input state to the first voice input pause state.

3. The method according to claim 1, characterized in that, The step of obtaining the dialogue context information with a first state includes: Obtain the first voice data of the first object; Perform speech recognition on the first speech data to obtain the first speech text information corresponding to the first speech data; Based on the first voice text information, generate dialogue context information with a first state.

4. The method according to claim 3, characterized in that, The step of performing speech recognition on the first speech data to obtain the first speech text information corresponding to the first speech data includes: The voice recognition token is concatenated to the first voice data to obtain the voice data to be recognized; The speech data to be recognized is input into a dialogue model. In the dialogue model, speech recognition is performed on the first speech data based on the speech recognition token to obtain the first speech text information corresponding to the first speech data.

5. The method according to claim 3, characterized in that, The step of obtaining the first voice data of the first object includes: From the dialogue audio data stream, an audio data stream whose time information matches the first input time period is obtained as the first voice data; the dialogue audio data stream refers to the audio data stream collected by the dialogue recording component during the voice dialogue with the first object; the first input time information refers to the time period corresponding to when the first object is in the first voice input state.

6. The method according to claim 3, characterized in that, Also includes: If the state indicator token is a listening state token, then the dialogue context information with the second state is processed by state rollback to obtain the dialogue context information with the first state. When it is determined that the first object has switched from the second voice input state to the second voice input pause state, the second voice data of the first object is obtained, and the second voice data is subjected to voice recognition to obtain the second voice text information corresponding to the second voice data. The second voice text information and the state recognition token are written into the dialogue context information with the first state to obtain dialogue context information with a new state to be recognized. In the dialogue model, the dialogue context information with the new state to be recognized is processed for dialogue state recognition based on the state recognition token.

7. The method according to claim 1, characterized in that, Also includes: During the process of the first object being in the first voice input state, the first voice data of the first object is subjected to periodic voice recognition processing to obtain the first periodic voice text information; the first periodic voice text information includes the periodic voice text information corresponding to N consecutive periodic voice data in the first voice data; a periodic voice data refers to the audio data stream in the first voice data whose voice duration reaches the periodic voice duration. When it is determined that the first object has switched from the first voice input state to the first voice input pause state, the step of obtaining dialogue context information with the first state includes: When it is determined that the first object has switched from the first voice input state to the first voice input pause state, dialogue context information with an initial state is created based on the first periodic voice text information; The first remaining speech data in the first speech data is subjected to speech recognition processing to obtain the first remaining speech text information corresponding to the first remaining speech data; the first remaining speech data refers to the audio data stream in the first speech data excluding the N consecutive periodic speech data; The first remaining voice text information is written into the dialogue context information with the initial state to obtain the dialogue context information with the first state.

8. The method according to claim 7, characterized in that, Also includes: If the state indicator token is a listening state token, then the dialogue context information with the second state is subjected to state rollback processing to obtain the dialogue context information with the initial state. During the process of the first object being in the second voice input state, the second voice data of the first object is concatenated to the end of the first remaining voice data to obtain joint voice data; the second voice data refers to the audio data stream obtained when the first object is in the second voice input state; The joint speech data is subjected to periodic speech recognition processing to obtain second periodic speech text information; the second periodic speech text information includes periodic speech text information corresponding to M consecutive periodic speech data in the joint speech data; When it is determined that the first object switches from the second voice input state to the second voice input pause state, the second periodic voice text information is written into the dialogue context information with the initial state to obtain the dialogue context information with the fourth state. The second remaining speech data in the joint speech data is subjected to speech recognition processing to obtain the second remaining speech text information corresponding to the second remaining speech data; the second remaining speech data refers to the audio data stream in the joint speech data excluding the M continuous periodic speech data; The second remaining voice text information and the state recognition token are written into the dialogue context information with the fourth state to obtain dialogue context information with a new state to be recognized. In the dialogue model, the dialogue context information with the new state to be recognized is processed for dialogue state recognition based on the state recognition token.

9. The method according to claim 7, characterized in that, The step of performing periodic speech recognition processing on the first speech data of the first object during the first speech input state to obtain first periodic speech text information includes: During the process of the first object being in the first voice input state, the first voice data of the first object is subjected to duration statistical processing to obtain the cumulative voice duration of the first voice data. When the cumulative speech duration is detected to be k times the periodic speech duration, the audio data stream corresponding to the kth periodic speech duration is sequentially obtained from the first speech data and used as the kth periodic speech data; k is an integer less than N; Speech recognition is performed on the k-th periodic speech data to obtain the periodic speech text information corresponding to the k-th periodic speech data; The first N cycles of speech-text information are sequentially concatenated to obtain the first cycle of speech-text information.

10. The method according to claim 1, characterized in that, Also includes: When acquiring dialogue context information with a first state, first state copy data is generated for the dialogue context information with the first state. The step of performing state rollback processing on the dialogue context information with the second state to obtain the dialogue context information with the first state includes: The data contained in the dialogue context information with the second state is replaced with the copy data of the first state to obtain the dialogue context information with the first state.

11. The method according to claim 1, characterized in that, The step of performing state rollback processing on the dialogue context information with the second state to obtain the dialogue context information with the first state includes: Obtain the operation log corresponding to the dialogue context information; the operation log contains records of each operation performed on the dialogue context information. Obtain the target operation record from the operation log; the target operation record refers to the operation record located between the first state time node and the second state time node; the first state time node refers to the time node at which the dialogue context information with the first state is obtained; the second state time node refers to the time node at which the dialogue context information with the second state is obtained; Generate a reverse operation record for the target operation record, and perform operation processing on the dialogue context information with the second state according to the reverse operation record to obtain the dialogue context information with the first state; the reverse operation record refers to an operation record that has the opposite operation processing to the target operation record.

12. A voice dialogue device, characterized in that, include: The acquisition module is used to acquire dialogue context information with the first state when it is determined that the first object has switched from the first voice input state to the first voice input pause state. The dialogue context information with a first state includes first voice text information corresponding to the first voice data of the first object; the first voice data refers to the audio data stream acquired when the first object is in a first voice input state; The state recognition module is used to write a state recognition token into the dialogue context information with a first state to obtain dialogue context information with a state to be recognized. In the dialogue model, the state recognition token is used to perform dialogue state recognition processing on the dialogue context information with a state to be recognized to obtain a state indication token for the voice text information. The state indication token is then written into the dialogue context information with a state to be recognized to obtain dialogue context information with a second state. The rollback module is used to perform a rollback process on the dialogue context information with the second state if the state indicator token is a reply state token, so as to obtain the dialogue context information with the first state. The response module is used to write a response token into the dialogue context information with a first state to obtain dialogue context information with a pending response state. In the dialogue model, based on the response token, the dialogue context information with a pending response state is processed to generate a dialogue response, resulting in response text information for replying to the voice text information. The response text information is then written into the dialogue context information with a pending response state to obtain dialogue context information with a third state.

13. A computer device, characterized in that, include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide data communication functions, the memory is used to store program code, and the processor is used to call the program code to execute the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and to execute the method according to any one of claims 1-11.

15. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they can perform the method described in any one of claims 1-11.

Images