A speech assistance dialogue management method, device, equipment and storage medium

Abstract

The application discloses a kind of speech assistance dialogue management method, device, equipment and storage medium, by responding to the wake-up word language monitored, record audio data to form voice file, and obtain vehicle state information;According to the vehicle state information, the voice file is carried out voice recognition using natural language understanding component, and system cognitive information is extracted;According to the system cognitive information, the corresponding logic element set is matched in pre-built logic database;The logic element set is sent to finite state machine, generates the finite state machine state set, event set and action set, and executes logic engine;Through logic engine, the function corresponding to the pre-built signal source set is called, and the execution result is output. It can solve the problem that the existing speech assistance function logic function is complex, improve development efficiency, reduce development cycle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech recognition, in particular to a speech assistance dialogue management method, device and equipment and storage medium. BACKGROUND

[0002] Current voice assistant interaction has become an important human-computer interaction mode, but in the current field of car voice assistants, the logic function is very complex, the development and customization of voice assistant functions, especially the subsequent iteration and optimization of voice assistants, the development cycle is very long, usually more than 40 people team for half a year to one year, in the aspect of dialogue logic customization, professional developers are also needed to customize the development of dialogue management, which costs a lot of enterprise investment and is very inefficient. SUMMARY

[0003] In order to solve the above problems, the present application provides a speech assistance dialogue management method, device, equipment and storage medium, which can solve the problem of complex logic function of existing speech assistance function, improve the development efficiency and reduce the development cycle.

[0004] The present application provides a speech assistance dialogue management method, which comprises:

[0005] In response to the monitored wake-up word, audio data is recorded to form a speech file, and vehicle state information is obtained;

[0006] According to the vehicle state information, a natural language understanding component is used to perform speech recognition on the speech file, and system cognitive information is extracted;

[0007] According to the system cognitive information, a corresponding set of logic elements is matched in a pre-built logic database;

[0008] The set of logic elements is sent to a finite state machine, a set of states, a set of events and a set of actions of the finite state machine are generated, and a logic engine is executed;

[0009] The functions corresponding to the pre-built set of sources are called by the logic engine, and the execution result is output.

[0010] Preferably, the vehicle state information includes driving state, software state, user state and vehicle positioning obtained by sensors and GPS.

[0011] Preferably, the system cognitive information specifically includes instruction information, intent information and word slot information.

[0012] As a preferred solution, the set of logic elements is composed of elements with different logic effects, and the elements specifically include start elements, logic judgment elements, source calling elements, reply elements and connections.

[0013] Preferably, the logic element set is sent to a finite state machine, the state set, event set and action set of the finite state machine are formed, and a logic engine is executed, specifically including:

[0014] The state set, event set and action set of the finite state machine are initialized according to the logic element set;

[0015] The instruction information, intention information and slot information in the logic element set are executed as logic inputs in a logic link;

[0016] An upper-level service in the logic link publishes a TellNext event according to the logic element set, a corresponding node action is executed by a TellNext action, and the next TellNext event is published after the node action is executed, the next node is searched by the TellNext action, and the execution is repeated until no next node is reached, a program exception occurs or a remote service request times out, and the logic link execution is completed.

[0017] As a preferred solution, the logic database stores different dialogue logic element sets required by different dialogue targets.

[0018] Preferably, the source set specifically includes a music query function, a map query function and a weather query function.

[0019] The embodiment of the application also provides a speech assistance dialogue management device, and the device includes:

[0020] An awakening module is configured to record audio data to form a voice file and acquire vehicle state information in response to a monitored awakening word.

[0021] A speech recognition module is configured to perform speech recognition on the voice file by using a natural language understanding component according to the vehicle state information, and extract system cognitive information.

[0022] A matching module is configured to match a corresponding logic element set in a pre-built logic database according to the system cognitive information.

[0023] An execution module is configured to send the logic element set to a finite state machine, generate the state set, event set and action set of the finite state machine, and execute a logic engine.

[0024] An output module is configured to call a corresponding function of a pre-built source set by using the logic engine, and output an execution result.

[0025] Preferably, the vehicle state information includes driving state, software state, user state and vehicle positioning obtained by a sensor and a GPS.

[0026] Preferably, the system cognition information specifically comprises instruction information, intention information and word slot information.

[0027] As a preferred solution, the logic element set is composed of elements with different logic effects, and the elements specifically include a start element, a logic judgment element, a signal source calling element, a reply element and a connection line.

[0028] Preferably, the execution module is specifically used for:

[0029] initializing the state set, the event set and the action set of the finite state machine according to the logic element set;

[0030] performing the instruction information, the intention information and the word slot information in the logic element set as logic input in the logic link;

[0031] the upper-layer service in the execution logic link publishes a TellNext event according to the logic element set, a TellNext action is executed to perform a corresponding node action, after the node action is executed, a next TellNext event is published, the TellNext action continues to find the next node to perform an action, and the process is repeated until no next node is reached, a program exception occurs or a remote service request times out, and the execution of the logic link is completed.

[0032] As a preferred solution, the logic database stores different dialogue logic element sets required by different dialogue targets.

[0033] Preferably, the signal source set specifically includes a music query function, a map query function and a weather query function.

[0034] The embodiment of the application further provides a terminal device, including a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, and the processor implements the speech assistance dialogue management method according to any one of the above embodiments when executing the computer program.

[0035] The embodiment of the application further provides a computer readable storage medium, including a stored computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute the speech assistance dialogue management method according to any one of the above embodiments when the computer program runs.

[0036] The application provides a speech assistance dialogue management method, device, equipment and storage medium, audio data is recorded to form a voice file in response to a monitored wake-up word, and vehicle state information is obtained; speech recognition is performed on the voice file by a natural language understanding component according to the vehicle state information, and system cognitive information is extracted; a corresponding set of logical elements is matched in a pre-built logical database according to the system cognitive information; the set of logical elements is sent to a finite state machine, a set of states, a set of events and a set of actions of the finite state machine are generated, and a logic engine is executed; the functions of a pre-built set of sources are called by the logic engine, and an execution result is output. The application can solve the problem of complex logic function of existing speech assistance function, improve development efficiency and reduce development cycle. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a flowchart of a speech assistance dialogue management method provided by an embodiment of the application;

[0038] Figure 2 is a flowchart of corpus word segmentation provided by an embodiment of the application;

[0039] Figure 3 is a structural diagram of a speech assistance dialogue management device provided by an embodiment of the application;

[0040] Figure 4 is a structural diagram of a terminal device provided by an embodiment of the application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0042] Embodiment one

[0043] Referring to Figure 1 is a flowchart of a speech assistance dialogue management method provided by an embodiment of the application, and the method comprises steps S1-S5;

[0044] S1, audio data is recorded to form a voice file in response to a monitored wake-up word, and vehicle state information is obtained;

[0045] S2, speech recognition is performed on the voice file by a natural language understanding component according to the vehicle state information, and system cognitive information is extracted;

[0046] S3, matching a corresponding logical element set in a pre-built logical database according to the system cognitive information;

[0047] S4, sending the logical element set to a finite state machine, generating the finite state machine state set, event set and action set, and executing a logic engine;

[0048] S5, calling the corresponding function of the pre-built signal source set through the logic engine, and outputting the execution result.

[0049] In the specific implementation of the embodiment, the people in the vehicle, including the driver and the copilot and the rear passengers, are the initiators of the voice dialogue, and wake up the voice assistant service of the vehicle machine through a specific wake-up word to initiate a dialogue service;

[0050] The vehicle machine receives the start instruction of the voice assistant function, completes network connection and configuration initialization;

[0051] When the vehicle machine monitors the corresponding wake-up word, the voice assistant dialogue function is started at this time, and the voice data is obtained through the recording device to form a voice file; otherwise, the vehicle machine will not respond to the subsequent voice of the user.

[0052] When obtaining the voice file, the vehicle state information needs to be obtained synchronously, and the function controlled by the user through the voice data input needs to be related to the vehicle state and control, and the vehicle state information at this time is obtained to facilitate the vehicle to respond to the voice assistant dialogue;

[0053] According to the vehicle state information, a natural language understanding component is used for voice recognition of the voice file, and the NLU component forms system cognitive information that the system can recognize through recognition of the user voice.

[0054] According to the system cognitive information, a corresponding logical element set is matched in a pre-built logical database, and the matched logical element set is sent to a FSM dialogue logic execution engine for process-free execution;

[0055] The logical element set is sent to a finite state machine, the finite state machine state set, event set and action set are generated, and a logic engine is executed;

[0056] The corresponding function of the pre-built signal source set is called through the logic engine, the vehicle machine related function or the corresponding control is obtained, and the execution result is output.

[0057] After the execution engine executes all the logic according to the logical element set, specific weather and software instructions are obtained, the engine returns the result to the vehicle machine, the vehicle machine plays a result prompt after receiving the input, and displays the weather information on the large screen of the vehicle machine.

[0058] By storing the logic element set in the database, the logic element set is directly read when needed to be used, and is injected into the FSM dialogue logic execution engine to execute. Only by configuring all dialogue logic element sets through the visual interface, the development of the session service can be completed, the problem of complex logic function of the existing speech assistance function can be solved, the development efficiency of the speech assistance dialogue function is improved, and the development period is reduced.

[0059] In another embodiment provided by the application, the vehicle state information includes driving state, software state, user state and vehicle positioning obtained through sensors and GPS.

[0060] In the embodiment, the car machine sends the request for dialogue execution to the NLU component to further analyze and execute the dialogue service by obtaining the driving state, software state, user state and vehicle positioning through various sensors and GPS on the car.

[0061] In another embodiment provided by the application, the system cognitive information specifically includes instruction information, intent information and word slot information.

[0062] In the embodiment, the NLU component, i.e., the natural language understanding component, is equivalent to the cognition of human beings to natural language, and the voice of the user is converted into instruction or intent, word slot and other information that can be cognized by the system through the NLU component.

[0063] In another embodiment provided by the application, the logic element set is composed of elements with different logic effects, and the elements specifically include a start element, a logic judgment element, a source calling element, a reply element and a connection line.

[0064] In the embodiment, referring to Figure 2 is a structural schematic diagram of the logic element set provided by the embodiment of the application; the logic element set is composed of elements with different shapes, and the elements with different shapes have different logic effects; specifically, the elements include a start element, a logic judgment element, a source calling element, a reply element and a connection line.

[0065] In Figure 2 , the elements include a start element and a logic judgment element, and contain whether to execute the user to check today's weather, the user to check tomorrow's weather or other logic judgment;

[0066] Figure 2 The source calling element in contains elements of calling a source to check today's weather, calling a source to check tomorrow's weather and replying that it is not supported.

[0067] Figure 3 The reply element in specifically includes replying to today's weather information and replying to tomorrow's weather information.

[0068] Through the logical element set of the logical unit in the dialogue logic, it contains a series of logical settings for achieving the user's goal, including listening to songs, navigation, querying food, querying the weather, etc.

[0069] In yet another embodiment provided by the application, the logical element set is sent to the finite state machine, the state set, event set and action set of the finite state machine are formed, and the logic engine is executed, specifically comprising:

[0070] The state set, event set and action set of the finite state machine are initialized according to the logical element set;

[0071] The instruction information, intent information and slot information in the logical element set are executed as logical inputs in the logic link;

[0072] The upper-level service in the logic link publishes a TellNext event according to the logical element set, and the corresponding node action is executed by the TellNext action. After the node action is executed, the next TellNext event is published, and the next node action is executed by the TellNext action. This is repeated until the end condition is reached, the program is abnormal, or the remote service request is timed out, and the logic link execution is completed.

[0073] In the specific implementation of the embodiment, the state, event and action of the finite state machine are initialized by the logical element set to form the finite state machine program that triggers different events to execute different actions in different states. The intent, slot and vehicle state information recognized by the NLU component are executed as logical inputs in the logic link.

[0074] The finite state machine is a three-tuple structure composed of state, event and action, which represents the logical link relationship of triggering different events to execute different actions in different states. For example, when the state is condition, if conditionEvent is triggered, a logical judgment action is executed.

[0075] In the current business, the state machine only needs one state. After the upper-level service initializes the finite state machine, a TellNext event is published, and then the TellNext action publishes the element-related events and executes the corresponding actions of the elements according to the element order of the dialogue flow. After the corresponding actions of the elements are executed, the TellNext event is published to let the TellNext action continue to execute the next node, and this is repeated until the end condition is reached.

[0076] The end condition specifically includes no next node, program abnormality and remote service request timeout. After any end condition is reached, the execution is stopped.

[0077] The dialog logic execution engine implemented by the finite state machine improves the execution efficiency of the voice assistance function.

[0078] In another embodiment provided by the application, the logical database stores a set of dialog logic elements required by different dialog targets.

[0079] In the embodiment, the pre-built logical database stores a set of dialog logic elements required by different dialog targets, which are usually designed by an operator through a visual interface and exist in the database for use by the identification scheduling service. The set of dialog logic elements includes functions such as listening to music, navigation, querying for food, and querying for weather.

[0080] In another embodiment provided by the application, the source set specifically includes music query function, map query function, and weather query function.

[0081] In the embodiment, the pre-built source set, i.e., the set of information sources, is used to expand the capability range of the dialog system and mainly includes functions such as music query, map query, casual query, and weather query. The source set is called by the dialog logic execution engine of the finite state machine to execute the corresponding functions.

[0082] In another embodiment provided by the application, referring to Figure 4 is a structural schematic diagram of a voice assistance dialog management device provided by an embodiment of the application. The device includes a wake-up module configured to record audio data to form a voice file and acquire vehicle state information in response to a monitored wake-up word;

[0083] a voice recognition module configured to perform voice recognition on the voice file by using a natural language understanding component according to the vehicle state information and extract system cognitive information;

[0084] a matching module configured to match a corresponding set of logic elements in a pre-built logical database according to the system cognitive information;

[0085] an execution module configured to send the set of logic elements to a finite state machine, generate a set of states, a set of events, and a set of actions of the finite state machine, and execute a logic engine;

[0086] an output module configured to call a corresponding function of a pre-built source set by using the logic engine and output an execution result.

[0087] The voice assistance dialog management device provided by the embodiment can perform all steps and functions of the voice assistance dialog management method provided by any of the above embodiments, and the specific functions of the device will not be described here.

[0088] Embodiment Nine

[0089] Referring to Figure 1is a structural schematic diagram of a terminal device provided by an embodiment of the present application. The terminal device comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, for example, a speech assistance dialogue management program. The processor implements the steps in each of the speech assistance dialogue management method embodiments described above when executing the computer program, for example, steps S1-S5 shown in the above embodiment. ​ Alternatively, the processor implements the functions of each module in the above device embodiments when executing the computer program.

[0090] For example, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present application. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the speech assistance dialogue management device. For example, the computer program can be divided into several modules, and the specific functions of each module have been described in detail in any of the speech assistance dialogue management method embodiments provided above, and the specific functions of the device will not be described here.

[0091] The speech assistance dialogue management device can be a desktop computer, a notebook, a palm computer, a cloud server, and other computing devices. The speech assistance dialogue management device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the schematic diagram is only an example of the speech assistance dialogue management device and does not constitute a limitation on the speech assistance dialogue management device, which can include more or fewer components than the diagram, or combine certain components, or different components, for example, the speech assistance dialogue management device can also include input / output devices, network access devices, buses, etc.

[0092] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The processor is the control center of the speech assistance dialogue management device, which connects each part of the speech assistance dialogue management device through various interfaces and lines.

[0093] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the speech assistance dialogue management device by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required for a function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory devices.

[0094] The modules integrated in the speech assistance dialogue management device can be stored in a computer readable storage medium if they are realized in the form of software function units and sold or used as independent products. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be realized. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0095] It should be noted that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which are also considered within the scope of protection of the present application.

Claims

1. A method for managing auxiliary dialogue, characterized in that, The method comprises: In response to the monitored wake-up word, audio data is recorded to form a voice file, and vehicle state information is obtained; According to the vehicle state information, a natural language understanding component is used to perform voice recognition on the voice file to extract system cognitive information; wherein the system cognitive information specifically includes instruction information, intent information and word slot information; According to the system cognitive information, a corresponding logical element set is matched in a pre-built logical database; Through a logical execution engine, the logical element set is sent to a finite state machine to generate a finite state machine state set, an event set and an action set, and to execute a "state-event-action" execution logic link; Through a logical execution engine, the functions corresponding to the pre-built signal source set are called to output the execution result; The logical element set is composed of elements with different logical effects, and the elements specifically include start elements, logical judgment elements, signal source calling elements, reply elements and connections; The logical element set is sent to the finite state machine to form the finite state machine state set, the event set and the action set, and to execute the "state-event-action" execution logic link, specifically comprising: According to the logical element set, the state set, the event set and the action set of the finite state machine are initialized; wherein the finite state machine is a ternary structure composed of states, events and actions, representing the logical link relationship of triggering different events to execute different actions under different states; The instruction information, the intent information and the word slot information recognized by the natural language understanding component are used as logical inputs in the execution logic link; In the state of the current business, the upper-level service in the execution logic link publishes a TellNext event according to the logical element set, and the TellNext action executes the corresponding node action, and after the node action is executed, the next TellNext event is published, and the TellNext action continues to find the next node to execute the action, and the process is repeated until there is no next node, the program is abnormal or the remote service request is timed out, and the execution of the execution logic link is completed. 2.The method of claim 1, wherein, The vehicle state information includes driving state, software state, user state and vehicle positioning obtained through sensors and GPS. 3.The method of claim 1, wherein, The logical database stores the dialogue logical element set required by different dialogue targets. 4.The method of claim 1, wherein, The signal source set specifically includes music query function, map query function and weather query function.

5. A speech assist dialogue management apparatus characterized by comprising: The device comprises: A wake-up module for recording audio data to form a voice file in response to the monitored wake-up word, and obtaining vehicle state information; A voice recognition module for performing voice recognition on the voice file using a natural language understanding component according to the vehicle state information to extract system cognitive information; wherein the system cognitive information specifically includes instruction information, intent information and word slot information; A matching module for matching a corresponding logical element set in a pre-built logical database according to the system cognitive information; Through a logical execution engine, the logical element set is sent to a finite state machine to generate a finite state machine state set, an event set and an action set, and to execute a "state-event-action" execution logic link; Through a logical execution engine, the functions corresponding to the pre-built signal source set are called to output the execution result; The logical element set is composed of elements with different logical effects, and the elements specifically include start elements, logical judgment elements, signal source calling elements, reply elements and connections; The logical element set is sent to the finite state machine to form the finite state machine state set, the event set and the action set, and to execute the "state-event-action" execution logic link, specifically comprising: According to the logical element set, the state set, the event set and the action set of the finite state machine are initialized; wherein the finite state machine is a ternary structure composed of states, events and actions, representing the logical link relationship of triggering different events to execute different actions under different states; The instruction information, the intent information and the word slot information recognized by the natural language understanding component are used as logical inputs in the execution logic link; In the state of the current business, the upper-level service in the execution logic link publishes a TellNext event according to the logical element set, and the TellNext action executes the corresponding node action, and after the node action is executed, the next TellNext event is published, and the TellNext action continues to find the next node to execute the action, and the process is repeated until there is no next node, the program is abnormal or the remote service request is timed out, and the execution of the execution logic link is completed. The vehicle state information includes driving state, software state, user state and vehicle positioning obtained through sensors and GPS. The logical database stores the dialogue logical element set required by different dialogue targets. The signal source set specifically includes music query function, map query function and weather query function. The execution module is used to send the set of logical elements to the finite state machine through the logic execution engine, generate the state set, event set and action set of the finite state machine, and execute the execution logic link of "state-event-action". The output module is used to call the functions corresponding to the pre-built information source set through the logic execution engine and output the execution results; The set of logical elements consists of elements with different logical effects. The elements specifically include start elements, logical judgment elements, source invocation elements, response elements, and connections. The execution module is specifically used for: The state set, event set, and action set of the finite state machine are initialized according to the set of logical elements; wherein, the finite state machine is specifically a triplet structure composed of state, event, and action, representing the logical link relationship of different actions being executed after different events are triggered in different states; The instruction information, intent information, and slot information identified by the natural language understanding component are used as logical inputs in the execution logic chain. In the current business state, the upper-layer service in the execution logic chain publishes a TellNext event based on the set of logical elements. The TellNext action then executes the corresponding node action. After the node action is completed, the next TellNext event is published, and the TellNext action continues to find the next node to execute the action. This process repeats until there is no next node, a program error occurs, or a remote service request times out, thus completing the execution of the execution logic chain.

6. A terminal device, characterized by comprising: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the speech aid dialogue management method as described in any one of claims 1 to 4.

7. A computer readable storage medium characterized by The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the speech aid dialogue management method as described in any one of claims 1 to 4.

Images