Bus-based voice dialogue management method, system and device, and storage medium

Abstract

The invention discloses a bus-based voice dialogue management method, system and device and a storage medium, and belongs to the technical field of voice control, and the method comprises the steps: 1, creating a bus manager; 2, starting an internal thread of the bus manager; 3, the bus manager adds nodes and starts the nodes; step 4, subscribing a corresponding theme by each node; 5, the bus manager receives and distributes the message of the corresponding theme; and step 6, each node receives and processes the corresponding message. The method has little influence on the existing computing power of the voice service, and only a code logic structure needs to be adjusted; the logic can be reused to other vehicle types in the future, such as E541, E542 and the like; engine replacement is more convenient, and modification and influence are very small; and all engine processing is switched from the main thread to the sub-thread and can be carried out in parallel, so that the UI drawing pressure of the main thread is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of voice control, and in particular relates to a bus-based voice dialogue management method, system, device and storage medium. Background Art

[0002] In the process of developing the car cockpit system, voice interaction is a very important part. The dialogue control is the most important part of voice interaction. The dialogue control in voice is the connection and coordination of the entire voice interaction process. The dialogue specifically refers to the entire process from the user saying the wake-up word, the voice system is awakened, to the user starting to speak, the user ending speaking, real-time text recognition and screen display, display of the user's text, and then understanding the semantics of the text, converting it into the corresponding voice command, sending it to each application module, and giving the corresponding feedback broadcast after the application executes the command. Of course, the dialogue will also include whether the next round of dialogue needs to continue, context synchronization and other functional points.

[0003] Traditional voice dialogue management is generally divided into multiple functional engines and a controller. The multiple functional engines include recognition engine (ASREngine), semantic engine (NLUEngine), and broadcast engine (TTSEngine). The dialogue management module connects multiple engines together through the controller (VuiEngineMgr) to form a complete voice process processing and control system. The advantage is the high degree of integration, but the disadvantage is that replacing the engine is very complicated and requires certain modifications to the controller, which is not conducive to the access of new engines and has poor flexibility.

[0004] For ease of understanding, some commonly used abbreviations are explained. ASR stands for Automatic Speech Recognition, which means speech-to-text conversion, and is generally understood as real-time text on the screen. NLU stands for Natural Language Understanding, which means natural language understanding, and is generally understood as text-to-semantics conversion. In actual programs, semantic results similar to the following will be given. The program can determine the actions to be performed based on the semantic results. Taking the following result as an example, the action is to retrieve the contacts named Zhang San and display the list cards. TTS is the abbreviation of Text To Speech, which generally means text-to-speech broadcast. VUI is the abbreviation of Voice User Interface, which generally refers to the user interface or interactive interface involved in voice.

[0005] "input":"Call Zhang San",

[0006] "task":"Phone",

[0007] "param":{

[0008] "namePinyin":"zhang san",

[0009] "name":"Zhang San",

[0010] "action":"Call",

[0011] "intent":"make a call",

[0012] "object":"Phone"

[0013] }

[0014] In addition, this traditional voice dialogue management system also needs to distinguish between offline and online modes, so the number of engines will increase from the original 3 to 6, and all arbitration and judgment logic are written in VuiEngineMgr, the code will be very large and complex. If a new ASREngine needs to be replaced, in addition to adding a new ASREngine, some interfaces of VuiEngineMgr need to be corrected. This will lead to a high degree of coupling between VuiEngineMgr and each engine, and the workload of replacing an engine alone is large, and the probability of error will be much higher. Summary of the invention

[0015] In view of the above defects in the prior art, the present invention provides a bus-based voice dialogue management method, system, device and storage medium, including: creating a bus manager, starting the internal thread of the bus manager, adding nodes to the bus manager, each node subscribing to the corresponding topic, the bus manager receiving the message of the corresponding topic and distributing it, each subsequent node receiving the corresponding message and processing it, each subscription node giving an example, real-time identification node and voice broadcast node, etc., which can effectively solve the problem of strong coupling of each engine module;

[0016] The present invention is achieved through the following technical solutions:

[0017] In a first aspect, the present invention provides a bus-based voice dialogue management method, which specifically comprises the following steps:

[0018] Step 1: Create a bus manager;

[0019] Step 2: Start the internal thread of the bus manager;

[0020] Step 3: The bus manager adds nodes and starts them;

[0021] Step 4: Each node subscribes to the corresponding topic;

[0022] Step 5: The bus manager receives the message of the corresponding topic and distributes it;

[0023] Step 6: Each node receives the corresponding message and processes it.

[0024] Furthermore, step one specifically includes the following contents:

[0025] A1. The auxiliary service of the voice application is started by the Android system;

[0026] A2. When the auxiliary service is launched, the voice application application will be launched first;

[0027] A3.When Application starts, the Android system will call back onCreate;

[0028] A4. The voice service will be started in onCreate of Application;

[0029] A5. The bus manager is created when the voice service starts.

[0030] Furthermore, step 2 specifically includes the following contents:

[0031] B1. Create an internal thread;

[0032] B2. Start the internal thread;

[0033] B3. Create a processor.

[0034] Furthermore, step three specifically includes the following contents:

[0035] Through the bus manager's additional node interface, all related nodes that need to process voice dialogue management are added to the bus manager's member variables; the member variable is a synchronous hash map type variable, and then the start interface of all nodes is called through the starter's loading interface to start all nodes.

[0036] Furthermore, step 4 specifically includes the following contents:

[0037] Each node subscribes to all relevant topics through the subscription interface of the bus client. The bus client then registers the corresponding node to the member variable topic mapping object of the bus manager. The object stores a list of topics and corresponding node names for the subsequent internal sub-thread bound processor to traverse this list, find the corresponding node, and distribute the corresponding topic.

[0038] Furthermore, step five specifically includes the following contents:

[0039] The bus client provides a way to send topic messages to the outside world by publishing or publishing sticky topic interfaces, and finally enters the publishing and publishing sticky interfaces of the bus manager. The interface searches the topic-to-topic mapping object to find the node name set corresponding to the topic. After that, it traverses all child nodes in the node name set, and traverses the currently running node list according to the node name, finds the corresponding node, and calls back the message processing callback interface of the corresponding node.

[0040] Furthermore, step six specifically includes the following contents:

[0041] When the setting value is changed, the current latest setting value will be recorded in the database or setting value sharer, and the kernel will be notified to update the latest setting value, and the corresponding module will be restarted to ensure that the setting takes effect; when distributing voice stream data, a copy of the byte array will be sent to the kernel for parsing, and the converted text will be returned to the upper layer in the form of a callback. The upper layer will then convert the result into a recognition result topic and distribute it to the corresponding node, and finally display it to the user.

[0042] In a second aspect, the present invention provides a bus-based voice dialogue management system, which is used to implement the management method as described in the first aspect, including:

[0043] Create a module for creating a bus manager;

[0044] The startup module is used to start the internal thread of the bus manager;

[0045] The append and start module is used for the bus manager to append nodes and start them;

[0046] Subscription module, used for each node to subscribe to the corresponding topic;

[0047] Receiving and distributing module, used for the bus manager to receive and distribute messages of corresponding topics;

[0048] The receiving and processing module is used for each node to receive and process the corresponding message.

[0049] In a third aspect, the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a bus-based voice dialogue management method as described in any one of the present invention is implemented.

[0050] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a bus-based voice dialogue management method as described in any one of the present inventions.

[0051] Compared with the prior art, the advantages of the present invention are as follows:

[0052] The bus-based voice dialogue management method, system, device and storage medium of the present invention have little impact on the existing computing power of the voice service, and only require adjustment of the code logic structure; the logic can be reused for other future models, such as E541, E542, etc.; engine replacement is more convenient, and the changes and impacts are very small; all engine processing is switched from the main thread to the sub-thread, which can be parallelized, reducing the UI drawing pressure of the main thread. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0054] Figure 1 A flowchart of a bus-based voice dialogue management method of the present invention;

[0055] Figure 2 It is a bus structure topology diagram of the present invention;

[0056] Figure 3 A schematic diagram of the relationship between each node and each functional engine of the present invention;

[0057] Figure 4 A structural block diagram of a bus-based voice dialogue management system of the present invention;

[0058] Figure 5 It is a structural schematic diagram of an electronic device in Example 3 of the present invention. DETAILED DESCRIPTION

[0059] In order to clearly and completely describe the technical solution and its specific working process of the present invention, the specific implementation methods of the present invention are as follows in conjunction with the accompanying drawings of the specification:

[0060] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0061] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0062] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0063] Example 1

[0064] like Figure 1 As shown, this embodiment provides a bus-based voice dialogue management method, which specifically includes the following steps:

[0065] Step 1: Create a bus manager;

[0066] A. The auxiliary service of the voice application is started by the Android system;

[0067] B. When the auxiliary service is launched, the voice application's Application will be launched first;

[0068] C. When Application starts, the Android system will call back onCreate;

[0069] D. The voice service will be started in onCreate of Application;

[0070] E. The bus manager is created when the voice service is started;

[0071] The bus manager is responsible for managing all nodes, and each node corresponds to a functional engine, such as wake-up engine, real-time recognition engine, voice broadcast engine, etc. Creating a bus manager will get an instance of the bus manager, which is the basis of the bus mode.

[0072] Step 2: Start the internal thread of the bus manager;

[0073] A. Create an internal thread;

[0074] B. Start the internal thread;

[0075] C. Create a processor;

[0076] In order to process all messages in the background and distribute them to each node, most functions in the old voice dialogue management architecture are run on the main thread, which will occupy the time slice of the main thread and affect the UI refresh performance. Therefore, the new voice dialogue management architecture introduces internal threads. Creating an internal thread will get an instance of the internal thread. Starting the internal thread will put the internal thread in a running state. During the running process, it will keep listening for new messages. After that, you need to create a processor and bind this processor to the internal thread. Once there is a new message, the processor will distribute it to each functional node and engine.

[0077] Step 3: The bus manager adds nodes and starts them;

[0078] Through the bus manager's add node interface, all related nodes that need to handle voice dialogue management are added to the bus manager's member variables. This member variable is a synchronous hash map type variable, rather than a simple mapping type object, which prevents abnormal situations caused by simultaneous access by multiple threads and ensures that the bus manager is thread-safe. Then, the start interface of all nodes is called through the starter's loading interface to start all nodes.

[0079] Step 4: Each node subscribes to the corresponding topic;

[0080] Each node subscribes to all relevant topics through the subscription interface of the bus client. The bus client then registers the corresponding node to the member variable topic mapping object of the bus manager. The object stores a list of topics and corresponding node names for subsequent internal sub-thread bound processors to traverse this list, find the corresponding node, and distribute the corresponding topic.

[0081] Step 5: The bus manager receives the message of the corresponding topic and distributes it;

[0082] The bus client provides a way to send topic messages to the outside world by publishing or publishing sticky topic interfaces, and finally enters the publishing and publishing sticky interfaces of the bus manager. The interface searches the topic-to-topic mapping object to find the node name set corresponding to the topic. After that, it traverses all child nodes in the node name set, and traverses the currently running node list according to the node name, finds the corresponding node, and calls back the message processing callback interface of the corresponding node.

[0083] Step 6: Each node receives the corresponding message and processes it.

[0084] The bus client provides a way to send topic messages to the outside world by publishing or publishing sticky topic interfaces, and finally enters the publishing and publishing sticky interfaces of the bus manager. The interface searches the topic-to-topic mapping object to find the node name set corresponding to the topic. After that, it traverses all child nodes in the node name set, and traverses the currently running node list according to the node name, finds the corresponding node, and calls back the message processing callback interface of the corresponding node.

[0085] Figure 2 This is a bus topology diagram. It can be clearly seen from the figure that each functional node is connected to the bus, which makes it convenient to add / replace / delete functional nodes; Figure 3 This is a diagram of the relationship between nodes and engines. As can be seen from the diagram, there is a layer of isolation between nodes and engines. The specific implementation of the engine is isolated in the form of an interface, which makes it more convenient and quick to replace an engine in the future.

[0086] Example 2

[0087] like Figure 4 As shown, this embodiment provides a bus-based voice dialogue management system, which is used to implement the management method as described in Example 1, including:

[0088] Create a module for creating a bus manager;

[0089] The startup module is used to start the internal thread of the bus manager;

[0090] The append and start module is used for the bus manager to append nodes and start them;

[0091] Subscription module, used for each node to subscribe to the corresponding topic;

[0092] Receiving and distributing module, used for the bus manager to receive and distribute messages of corresponding topics;

[0093] The receiving and processing module is used for each node to receive and process the corresponding message.

[0094] Example 3

[0095] Figure 5 This is a schematic diagram of the structure of a computer device in Example 3 of the present invention. Figure 5 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 5 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0096] like Figure 5 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0097] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0098] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0099] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0100] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0101] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the computer device 12 in this embodiment, the display 24 does not exist as an independent individual, but is embedded in the mirror surface, and when the display surface of the display 24 is not displayed, the display surface of the display 24 and the mirror surface are visually integrated. In addition, the computer device 12 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through a bus 18. It should be understood that although not shown in the figures, other hardware and / or software modules may be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0102] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing a bus-based voice dialogue management method provided in an embodiment of the present invention.

[0103] Example 4

[0104] Embodiment 4 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a bus-based voice dialogue management method as provided in all the embodiments of the present application.

[0105] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0106] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0107] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0108] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0109] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A bus-based voice dialogue management method, characterized in that: The specific steps include: Step 1: Create a bus manager; Step 2: Start the internal thread of the bus manager; Step 3: The bus manager adds nodes and starts them; Step 4: Each node subscribes to the corresponding topic; Step 5: The bus manager receives the message of the corresponding topic and distributes it; Step 6: Each node receives the corresponding message and processes it.

2. A bus-based voice dialogue management method as claimed in claim 1, characterized in that: Step 1 specifically includes the following: A1. The auxiliary service of the voice application is started by the Android system; A2. When the auxiliary service is launched, the voice application application will be launched first; A3.When Application starts, the Android system will call back onCreate; A4. The voice service will be started in onCreate of Application; A5. The bus manager is created when the voice service starts.

3. A bus-based voice dialogue management method as claimed in claim 1, characterized in that: Step 2 specifically includes the following: B1. Create an internal thread; B2. Start the internal thread; B3. Create a processor.

4. A bus-based voice dialogue management method as claimed in claim 1, characterized in that: Step 3 specifically includes the following: Through the bus manager's additional node interface, all related nodes that need to process voice dialogue management are added to the bus manager's member variables; the member variable is a synchronous hash map type variable, and then the start interface of all nodes is called through the starter's loading interface to start all nodes.

5. A bus-based voice dialogue management method as claimed in claim 1, characterized in that: Step 4 specifically includes the following: Each node subscribes to all relevant topics through the subscription interface of the bus client. The bus client then registers the corresponding node to the member variable topic mapping object of the bus manager. The object stores a list of topics and corresponding node names for the subsequent internal sub-thread bound processor to traverse this list, find the corresponding node, and distribute the corresponding topic.

6. A bus-based voice dialogue management method as claimed in claim 1, characterized in that: Step 5 specifically includes the following: The bus client provides a way to send topic messages to the outside world by publishing or publishing sticky topic interfaces, and finally enters the publishing and publishing sticky interfaces of the bus manager. The interface searches the topic-to-topic mapping object to find the node name set corresponding to the topic. After that, it traverses all child nodes in the node name set, and traverses the currently running node list according to the node name, finds the corresponding node, and calls back the message processing callback interface of the corresponding node.

7. A bus-based voice dialogue management method as claimed in claim 1, characterized in that: Step 6 specifically includes the following: When the setting value is changed, the current latest setting value will be recorded in the database or setting value sharer, and the kernel will be notified to update the latest setting value, and the corresponding module will be restarted to ensure that the setting takes effect; when distributing voice stream data, a copy of the byte array will be sent to the kernel for parsing, and the converted text will be returned to the upper layer in the form of a callback. The upper layer will then convert the result into a recognition result topic and distribute it to the corresponding node, and finally display it to the user.

8. A bus-based voice dialogue management system, used to implement the management method according to any one of claims 1 to 7, characterized in that: include: Create a module for creating a bus manager; The startup module is used to start the internal thread of the bus manager; The append and start module is used for the bus manager to append nodes and start them; Subscription module, used for each node to subscribe to the corresponding topic; Receiving and distributing module, used for the bus manager to receive and distribute messages of corresponding topics; The receiving and processing module is used for each node to receive and process the corresponding message.

9. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a bus-based voice dialogue management method as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, a bus-based voice dialogue management method as described in any one of claims 1-7 is implemented.

Images