Human-computer interaction method and device

Abstract

The present application discloses a human-computer interaction method and device, which relates to the field of artificial intelligence. The method includes receiving a target command issued by a user; generating a target decision of the target command using historical commands, historical decisions of historical commands and target commands, wherein the historical commands are commands of historical human-computer interaction tasks, and the target commands are commands of current human-computer interaction tasks; and outputting the target decision.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI), and in particular to a human-computer interaction method and device. Background Art

[0002] With the development of artificial intelligence (AI), electronic devices can use human-computer interaction techniques (HCI) to communicate with users, so that electronic devices can understand the user's intentions and complete the work intended by the user. At present, human-computer interaction has a wide range of applications in many fields, such as smart homes, autonomous driving, etc. However, the interaction between electronic devices and users is not very "natural" and "smart". The intention obtained by the electronic device through natural language understanding of the received voice commands is not very consistent with the user's real intention. In addition, the decision-making of electronic devices is mechanical and rigid, and it is impossible to give users the best decision. The user experience of human-computer interaction is low. Summary of the invention

[0003] The present application provides a human-computer interaction method and device, which, when performing natural language understanding of a target command issued by a user in a current human-computer interaction task, refers to the semantics of historical commands in historical human-computer interaction tasks, assists in natural language understanding of the target command, and makes the result of natural language understanding more consistent with the user's true intention; when executing system decisions, historical decisions are referred to, and target decisions can be optimized based on historical decisions, effectively improving the user experience of human-computer interaction.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, the present application provides a human-computer interaction method, which can be applied to an electronic device, or the method can be applied to a human-computer interaction device that can support the electronic device to implement the method, for example, the human-computer interaction device includes a chip system, and the method includes: after the electronic device receives a target command issued by a user, the target decision of the target command is generated by using the historical command, the historical decision of the historical command and the target command, and the target decision is output. Among them, the historical command is the command of the historical human-computer interaction task, and the target command is the command of the current human-computer interaction task. The historical command can be a command of one or more historical users and the electronic device for multiple rounds of human-computer interaction tasks. For example, the historical command can be a command of multiple historical users and the electronic device for multiple rounds of human-computer interaction tasks. The historical user can also include the user who issued the target command. In this way, when the target command issued by the user in the current human-computer interaction task is understood in natural language, the semantics of the historical commands in the historical human-computer interaction task are referred to, and the natural language understanding of the target command is assisted, so that the result of the natural language understanding is more in line with the user's true intention; when the system decision is executed, the historical decision is referred to, and the target decision can be optimized according to the historical decision, which effectively improves the user experience of human-computer interaction.

[0006] In one possible implementation, a target decision for a target command is generated using historical commands, historical decisions of historical commands, and a target command, including: an electronic device weightedly encodes historical commands based on command semantic relevance weights to obtain historical command encoding information, wherein the command semantic relevance weights represent the degree of semantic relevance between the target command and the historical command; and a target decision is generated based on the target command, the historical command encoding information, and the historical decisions of the historical command.

[0007] Among them, before weighted encoding the historical commands based on the command semantic relevance weights, the electronic device semantically encodes the target command to obtain the semantic vector of the target command; and calculates the similarity between the semantic vector of the target command and the semantic vector of the historical commands to obtain the command semantic relevance weight.

[0008] In another possible implementation, weighted encoding of historical commands based on command semantics-related weights to obtain historical command encoding information includes: weighted encoding of historical commands based on command semantics-related weights and user weights to obtain historical command encoding information, wherein the user weight represents the degree of association between the user and the historical user who issued the historical command. The electronic device can obtain the degree of association between the user and the historical user based on the user's voiceprint to obtain the user weight.

[0009] In another possible implementation, historical commands are weightedly encoded based on command semantic relevance weights and user weights to obtain historical command encoding information, including: weighted encoding historical commands based on command semantic relevance weights, user weights, and user relationship relevance weights to obtain historical command encoding information, where the user relationship relevance weights are relationship strength values ​​of multiple preset users.

[0010] In another possible implementation, a target decision is generated based on the target command, historical command encoding information, and historical decisions of historical commands, including: using an intent understanding model to perform natural language understanding on the word vector of the target command and the historical command encoding information to obtain the intent and slot of the target command; generating a target decision based on the intent and slot of the target command and the historical decision encoding vector of the historical commands.

[0011] Specifically, according to the intent and slot of the target command and the historical decision coding vector of the historical command, the target decision is generated, including: encoding the intent and slot of the target command to obtain a decision coding vector; weighted encoding the historical decision coding vector based on the weight of the historical decision coding vector to obtain historical decision coding information, the weight of the historical decision coding vector represents the degree of correlation between the decision coding vector and the historical decision coding vector; using the decision model to analyze the decision coding vector and the historical decision coding information to generate the target decision. When executing system decisions, the historical decisions of historical commands are referred to, and the decision content can be optimized according to the historical decisions of historical commands, enriching the information for electronic equipment to make decisions, and effectively improving the user experience of human-computer interaction.

[0012] Among them, before weighted encoding the historical decision coding vector based on the historical decision coding vector weight to obtain the historical decision coding information, the electronic device calculates the similarity between the decision coding vector and the historical decision coding vector to obtain the historical decision coding vector weight.

[0013] In another possible implementation, historical decision coding vectors are weighted encoded based on historical decision coding vector weights to obtain historical decision coding information, including: weighted encoding historical decision coding vectors based on historical decision coding vector weights and user weights to obtain historical decision coding information; or, weighted encoding historical decision coding vectors based on historical decision coding vector weights, user weights, and user relationship relevance weights to obtain historical decision coding information.

[0014] In another possible implementation, the electronic device encodes the intention and slot of the target command to obtain a decision encoding vector, including: the electronic device encodes the intention and slot of the target command and the occupancy state of the electronic device to obtain a decision encoding vector. The occupancy state of the electronic device is used to enhance the semantic vector of the historical command, thereby further improving the accuracy of the system decision.

[0015] In the second aspect, the present application provides a human-computer interaction device, which is applied to an electronic device; the electronic device includes a voice transceiver, which is used to receive a target command issued by a user and feedback the voice of the decision to the user. The human-computer interaction device includes: an acquisition unit, a processing unit and a feedback unit. The acquisition unit is used to receive a target command issued by a user; the processing unit is used to generate a target decision of a target command using a historical command, a historical decision of a historical command and a target command, the historical command is a command of a historical human-computer interaction task, and the target command is a command of a current human-computer interaction task; the feedback unit is used to output the target decision. In this way, when the target command issued by the user in the current human-computer interaction task is understood in natural language, the semantics of the historical commands in the historical human-computer interaction task are referred to, and the natural language understanding of the target command is assisted, so that the result of the natural language understanding is more in line with the user's true intention; when executing the system decision, the historical decision is referred to, and the target decision can be optimized according to the historical decision, which effectively improves the user experience of the human-computer interaction. These units can perform the corresponding functions in the method example of the first aspect above, and refer to the detailed description in the method example for details, which will not be repeated here.

[0016] In a third aspect, the present application provides an electronic device comprising: at least one processor, a memory and a voice transceiver, wherein the voice transceiver is used to receive a target command issued by a user and provide voice feedback of the decision to the user, the memory is used to store computer programs and instructions, and the processor is used to call the computer programs and instructions, and assist the voice transceiver in executing the human-computer interaction method described in any one of the first aspect or possible implementation methods of the first aspect.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, comprising: computer software instructions; when the computer software instructions are executed in an electronic device, the electronic device executes the first aspect or a possible implementation of the first aspect.

[0018] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the first aspect or a possible implementation of the first aspect.

[0019] In a sixth aspect, the present application provides a chip system, which is applied to an electronic device; the chip system includes an interface circuit and a processor; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive a signal from a memory of the electronic device and send a signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the chip system executes the first aspect or a possible implementation method of the first aspect.

[0020] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of features or beneficial effects means that specific technical features, technical solutions or beneficial effects are included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be realized without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in a specific embodiment that does not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the composition of an electronic device provided in one embodiment of the present application;

[0022] Figure 2 A flowchart of a human-computer interaction method provided in one embodiment of the present application;

[0023] Figure 3 A schematic diagram of a speech recognition process provided by an embodiment of the present application;

[0024] Figure 4 A flowchart of a human-computer interaction method provided in one embodiment of the present application;

[0025] Figure 5 A flowchart of a human-computer interaction method provided in one embodiment of the present application;

[0026] Figure 6 A flowchart of a human-computer interaction method provided in one embodiment of the present application;

[0027] Figure 7 A schematic diagram of an intent understanding model provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the composition of a human-computer interaction device provided in one embodiment of the present application;

[0029] Fig. 9 A schematic diagram of the composition of a human-computer interaction device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0030] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects rather than to limit a specific order.

[0031] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0032] "Multiple" means two or more, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, for elements (element) in the singular form "a", "an", and "the", unless the context clearly stipulates otherwise, they do not mean "one or only one", but "one or more than one". For example, "a device" means one or more such devices. Furthermore, at least one (at least one of)... "means one or any combination of the subsequent associated objects, for example, "at least one of A, B and C" includes A, B, C, AB, AC, BC, or ABC.

[0033] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0034] The electronic device in this embodiment is a device including a display screen and a camera. The specific form of the electronic device is not particularly limited in the present application embodiment. For example, the electronic device can be a television, a tablet computer, a projector, a mobile phone, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, and a personal digital assistant (personal digital assistant, PDA), augmented reality (augmented reality, AR), virtual reality (virtual reality, VR) device, smart speaker, smart TV and other Internet of Things (IoT) devices.

[0035] Please refer to Figure 1 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 1 As shown, the electronic device includes: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a power management module 140, an antenna, a wireless communication module 160, an audio module 170, a speaker 170A, a speaker interface 170B, a microphone 170C, a sensor module 180, a button 190, an indicator 191, a display screen 192, and a camera 193, etc. Among them, the above-mentioned sensor module 180 may include sensors such as a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, and an ambient light sensor.

[0036] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0037] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0038] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0039] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0040] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and / or a USB interface, etc.

[0041] In this embodiment, when the processor 110 is used to perform natural language understanding on the target command, it combines historical commands to perform natural language understanding on the target command to obtain the intent and slot of the target command. The historical command is a command of a historical human-computer interaction task, and the target command is a command of a current human-computer interaction task. Optionally, the historical command may be a command of multiple historical human-computer interaction tasks. Multiple historical human-computer interaction tasks may be tasks in which multiple historical users have conducted multiple rounds of dialogues with electronic devices. Furthermore, the processor 110 determines the target decision in combination with the decision of the historical command, and the intent and slot of the target command.

[0042] Among them, intent and slots together constitute "user actions". Machines cannot directly understand natural language, so the role of user actions is to map natural language into structured semantic representations that machines can understand. Slots have the ability to remember states for multiple rounds. Slots include slots. For example, in the taxi-hailing scenario, slots include the departure location slot and the destination slot.

[0043] The power management module 140 is used to connect to a power source. The power management module 140 may also be connected to the processor 110, the internal memory 121, the display screen 192, the camera 193, the wireless communication module 160, etc. The power management module 140 receives input of a power source and supplies power to the processor 110, the internal memory 121, the display screen 192, the camera 193, the wireless communication module 160, etc. In some embodiments, the power management module 140 may also be disposed in the processor 110.

[0044] The wireless communication function of the electronic device can be realized through the antenna and the wireless communication module 160. Among them, the wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc. applied to the electronic device.

[0045] The wireless communication module 160 may be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be sent from the processor 110, modulate the frequency of the signal, amplify the signal, and convert it into electromagnetic waves for radiation through the antenna. In some embodiments, the antenna of the electronic device is coupled to the wireless communication module 160, so that the electronic device can communicate with the network and other devices through wireless communication technology.

[0046] The electronic device implements the display function through a GPU, a display screen 192, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 192 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0047] The display screen 192 is used to display images, videos, etc. The display screen 192 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc.

[0048] The electronic device can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 192 and application processor, etc. ISP is used to process the data fed back by camera 193. In some embodiments, ISP can be set in camera 193.

[0049] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0050] Alternatively, the electronic device may not include a camera, that is, the camera 193 is not provided in the electronic device (such as a television). The electronic device may be connected to an external camera 193 via an interface (such as a USB interface 130). The external camera 193 may be fixed to the electronic device via an external fixing member (such as a camera bracket with a clip). For example, the external camera 193 may be fixed to the edge of the display screen 192 of the electronic device, such as the upper edge, via an external fixing member.

[0051] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals. For example, when an electronic device is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy. Video codecs are used to compress or decompress digital videos. Electronic devices can support one or more video codecs. In this way, electronic devices can play or record videos in multiple coding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0052] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0053] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data (such as audio data, etc.) created during the use of the electronic device, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0054] The electronic device can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the microphone 170C, the speaker interface 170B, and the application processor.

[0055] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. In the present application, the speaker 170A is used to output the voice of the decision. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. In the present application, the microphone 170C is used to receive the voice of the target command issued by the user or the voice of the historical command.

[0056] The speaker interface 170B is used to connect a wired speaker. The speaker interface 170B may be a USB interface 130, or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0057] The key 190 includes a power key, a volume key, etc. The key 190 can be a mechanical key or a touch key. The electronic device can receive key input and generate key signal input related to user settings and function control of the electronic device.

[0058] The indicator 191 may be an indicator light, which may be used to indicate that the electronic device is in a power-on state, a standby state, or a power-off state, etc. For example, if the indicator light is off, it may indicate that the electronic device is in a power-off state; if the indicator light is green or blue, it may indicate that the electronic device is in a power-on state; if the indicator light is red, it may indicate that the electronic device is in a standby state.

[0059] It is understandable that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device. Figure 1 More or fewer components shown in the figure may be combined with two or more components, or may have different component configurations. For example, the electronic device may also include components such as a speaker. Figure 1 The various components shown in the drawings may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing or application specific integrated circuits.

[0060] Next, combine Figure 2 , a human-computer interaction method provided in an embodiment of the present application is described in detail.

[0061] S201: The electronic device receives a target command issued by a user.

[0062] The target command is a natural language text recognizable by the user. In some embodiments, the user can input the target command to the electronic device through an input device (such as a virtual keyboard or a physical keyboard). In other embodiments, the user can speak to the electronic device. The electronic device performs voice recognition on the voice and converts the voice into the target command. Voice refers to the voice sound of the user communicating with the electronic device.

[0063] Optionally, if the user is in a noisy environment, the electronic device may receive mixed speech, which includes speech and noise from the external environment. The electronic device can separate speech from the mixed speech using the user's voiceprint features. For example, Figure 3As shown, it is a schematic diagram of speech separation and recognition provided by an embodiment of the present application. The mixed speech is analyzed by short-time Fourier transform (STFT) to obtain a mixed speech spectrum, and the mixed speech spectrum and the user voiceprint features pre-registered in the system are input into a pre-trained speech separation model, and the user's speech spectrum is separated from the mixed speech spectrum. The spectrum obtained after separation is then used to perform speech recognition on the speech spectrum of the target speech using automatic speech recognition technology to obtain the target command. Among them, the speech separation model is trained by pre-collected multi-user speech data. The speech separation model can be a multi-layer long short-term memory model (LSTM).

[0064] S202: The electronic device generates a target decision for a target command by using the historical commands, the historical decisions of the historical commands, and the target command.

[0065] Figure 4 A flowchart of another human-computer interaction method provided in this embodiment, wherein: Figure 4 The method flow described is Figure 2 The specific operation process included in S202 is described as shown in the figure. S2021. The electronic device performs weighted encoding on the historical commands based on the command semantics related weights to obtain historical command encoding information. S2022. The electronic device generates a target decision based on the target command, the historical command encoding information and the historical decisions of the historical commands.

[0066] The command semantic relevance weight indicates the semantic relevance between the target command and the historical commands. Relevance means being associated with each other. The historical commands related to the target command may be historical commands that have some connection with the intent of the target command. For example, the target command is "The temperature is a bit cold," and the historical command is "It's so hot, turn the air conditioner to 20 degrees." Both the target command and the historical commands are related to adjusting the temperature of the air conditioner. However, the target command does not explicitly indicate that it is to adjust the temperature of the air conditioner, and the historical command indicates that the air conditioner is adjusted to a specific temperature. Therefore, in this embodiment, when performing natural language understanding on the target command issued by the user in the current human-computer interaction task, the semantics of the historical commands in the historical human-computer interaction task are referenced, which assists in the natural language understanding of the target command, so that the result of the natural language understanding is more in line with the user's true intention.

[0067] In a possible implementation, the electronic device semantically encodes the target command to obtain a semantic vector of the target command, and performs similarity calculation based on the semantic vector of the target command and the semantic vectors of historical commands to obtain a command semantic relevance weight.

[0068] Specifically, the electronic device first performs Chinese word segmentation processing on the target command to obtain a word vector of the target command. Chinese word segmentation refers to dividing a continuous sequence of characters into individual words.

[0069] The electronic device inputs the word vector of the target command into the semantic encoding model for encoding to obtain the semantic vector of the target command. The semantic encoding model can be a recurrent neural network (RNN), and the most commonly used RNN model is the bidirectional long short-term memory model (BiLSTM). BiLSTM can be implemented using a network containing 3 hidden layers and 600 nodes in each layer. For example, the target command is "the temperature is a bit cold", and the target command is segmented into Chinese words to obtain the word vector of "temperature", the word vector of "a bit" and the word vector of "cold". The word vector of "temperature", the word vector of "a bit" and the word vector of "cold" are input into the semantic encoding model, and the semantic vector of the target command "the temperature is a bit cold" is obtained through reasoning of the semantic encoding model.

[0070] It should be noted that the electronic device stores the semantic vector of the target command so that the semantic vector of the target command can be used as the semantic vector of the historical command to assist the electronic device in natural language understanding of subsequent commands.

[0071] The semantic vector of the historical command can be a matrix of M columns, where each column represents the semantic vector of a command of a historical human-computer interaction task. The semantic vector of the target command is multiplied by each column in the matrix to obtain the command semantic relevance weight. The command semantic relevance weight satisfies formula (1).

[0072] p m =u T h m (1)

[0073] Among them, u T =[x1,…,x j ],u T The semantic vector representing the target command, h m Represents the semantic vector of historical commands, p m Indicates the semantic relevance weight of the command.

[0074] Optionally, the electronic device may also perform weighted encoding on the historical commands based on the command semantics-related weight and the user weight to obtain the historical command encoding information. The user weight indicates the degree of association between the user and the historical user who issued the historical command. In some embodiments, before the user interacts with the electronic device in a human-computer manner, the electronic device may prompt the user to provide the user's voiceprint, and the electronic device stores the user's voiceprint. The electronic device compares the user's voiceprint with the voiceprint of the historical user to obtain the degree of association between the user and the historical user, that is, the user weight. The historical user refers to the user who has interacted with the electronic device in a human-computer manner. For example, the electronic device obtains the degree of similarity between the user and the historical user based on the user's voiceprint to obtain the user weight. The degree of similarity may be the likelihood of the user and the historical user. Understandably, a larger user weight indicates that the user is more likely to be the historical user, and a higher weight is set; a smaller user weight indicates that the user is less likely to be the historical user, and a lower weight is set.

[0075] Optionally, the electronic device may also perform weighted encoding on the historical commands based on the command semantic relevance weight, the user weight, and the user relationship relevance weight to obtain historical command encoding information. The user relationship relevance weight is a preset relationship strength value of multiple users. For example, the electronic device is a smart home, and the users who use the smart home are usually fixed family members. Any member of the family can set the relationship strength value with other members. The relationship strength value may include high, medium, low, and no association, etc.

[0076] Specifically, the electronic device performs weighted encoding on the historical commands based on the weighted information to obtain the semantic vector of the weighted encoded historical commands, merges the semantic vector of the weighted encoded historical commands with the semantic vector of the target command, and encodes them through a fully connected network to obtain the historical command encoding information. The weighted information includes at least one of the command semantic relevance weight, the user weight, and the user relationship relevance weight. The semantic vector of the weighted encoded historical commands satisfies formula (2).

[0077] h′=∑ m p m h m S m (2)

[0078] Among them, h′ represents the semantic vector of the historical command after weighted encoding, p m represents the command semantics related weight, h m The semantic vector representing the historical commands, S m represents the user weight. Or, S m represents the user relationship relevance weight. Or, S m Represents user weight and user relationship relevance weight.

[0079] Further, Figure 5 A flowchart of another human-computer interaction method provided in this embodiment, wherein: Figure 5 The method flow described is Figure 4 The specific operation process included in S2022 is described as shown in the figure. S20221. The electronic device uses the intention understanding model to perform natural language understanding on the word vector of the target command and the historical command encoding information to obtain the intention and slot of the target command. S20222. The electronic device generates a target decision based on the intention and slot of the target command and the historical decision encoding vector of the historical command.

[0080] Specifically, Figure 6 As shown in (a), the electronic device performs Chinese word segmentation on the target command to obtain the word vector of the target command (execution S601). For example, the target command is "the temperature is a bit cold", and the word vector of the target command includes the word vector of "temperature", the word vector of "a bit" and the word vector of "cold". The electronic device encodes the word vector of the target command using the semantic coding model to obtain the semantic vector of the target command (execution S602). The electronic device performs similarity calculation based on the semantic vector of the target command and the semantic vector of the historical command to obtain the command semantic relevance weight (execution S603). Assume that the historical command is "it's so hot, the air conditioner is turned on to 20 degrees". The command semantic relevance weight includes the semantic relevance between the target command "the temperature is a bit cold" and the historical command "it's so hot, the air conditioner is turned on to 20 degrees". It is worth noting that the command semantic relevance weight includes the semantic relevance between the target command and the commands of multiple historical human-computer interaction tasks. The semantic vector of the historical command is weighted encoded based on the first weighted information to obtain the historical command encoding information (execution S604). The first weighted information includes the command semantic relevance weight. Optionally, the first weighted information also includes user weights and user relationship relevance weights. The electronic device uses the intent understanding model to perform natural language understanding on the word vectors and historical command encoding information of the target command to obtain the intent and slot of the target command (execute S605). For example, the target command is "the temperature is a bit cold", and the intent of the target command may indicate adjusting the temperature. The slots of the target command can be "temperature", "a bit" and "cold". The intent understanding model can be an RNN, and the specific case below is implemented using a BERT model based on a TRANSFORMER structure.

[0081] The intent understanding model can be trained using bidirectional encoder representations from transformers (BERT). BERT is a bidirectional transformer-based model proposed by Google. It can be pre-trained using a large amount of unsupervised text corpus. The pre-training process includes two techniques. One is to randomly mask some characters in the training sentences to predict the masked characters. The other is to train and understand the relationship between sentences and predict the next sentence given the current text. After the BERT model is trained, the intent understanding model contains a pre-trained deep structure for semantic analysis. Then, the BERT model is fine-tuned using target-related intent understanding tasks. It should be noted that in order to introduce historical semantic related information obtained from the previous text into the model, the first input at the beginning of the sentence of the BERT network uses a weighted semantic encoding vector, such as Figure 7 As shown, the word vector of the target command and the historical command encoding information are input into the intent understanding model to obtain the intent and slot of the target command.

[0082] The decision model is a classification model whose input is intention, dialogue status and system database information, and whose output is a specific decision. Figure 6As shown in (b), the electronic device encodes the intention of the target command, the dialogue state, and the information obtained in the system database to obtain a decision coding vector (execution S606). The coding network can be implemented using a multi-layer convolutional neural network (CNN). The electronic device decision coding vector and the historical decision coding vector are similarly calculated to obtain the weight of the historical decision coding vector (execution S607). The historical decision is a system action determined by the electronic device based on the historical command. For example, the historical command is "How is the weather today?" The system action is to output "The weather is cloudy, the temperature." For another example, the historical command is "It's so hot, the air conditioner is turned on to 20 degrees." The system action is to adjust the air conditioning temperature to 20 degrees. The weight of the historical decision coding vector includes the degree of correlation between the decision coding vector of the target command "The temperature is a bit cold" and the historical decision coding vector of the historical decision "The air conditioner is turned on to 20 degrees." It is worth noting that the weight of the historical decision coding vector includes the degree of correlation between the decision coding vector of the target command and the historical decision coding vectors of the decisions of multiple historical human-computer interaction tasks. The electronic device performs weighted encoding on the historical decision coding vector based on the second weighted information to obtain the historical decision coding information (execute S608). The second weighted information includes the weight of the historical decision coding vector. Optionally, the second weighted information also includes the user weight and the user relationship correlation weight. The weight of the historical decision coding vector represents the degree of correlation between the decision coding vector and the historical decision coding vector. The electronic device uses the decision model to analyze the decision coding vector and the historical decision coding information to generate a target decision (execute S609). For example, the target command is "the temperature is a bit cold", and the target decision can be "the air conditioner is turned on to 29 degrees". The decision model can be implemented using a shallow classifier, such as a support vector machine; it can also be implemented using a deep neural network (DNN), such as a multi-layer fully connected feedforward neural network (FNN).

[0083] It should be noted that the electronic device stores the decision coding vector to assist the electronic device in making decisions on target commands issued by subsequent users.

[0084] The historical decision encoding vector can be a matrix of M columns, each column representing a historical decision encoding vector of a decision of a historical human-computer interaction task. The decision encoding vector is multiplied by each column in the matrix to obtain the weight of the historical decision encoding vector. The weight of the historical decision encoding vector satisfies formula (3).

[0085] q m =w T k m (3)

[0086] Among them, w T=[y1,…,y],w T represents the decision encoding vector, k m represents the historical decision encoding vector, q m Represents the weight of the historical decision encoding vector.

[0087] The electronic device weight-encodes the historical decision coding vector based on the second weighted information to obtain a weighted historical decision coding vector, merges the weighted historical decision coding vector and the decision coding vector, and encodes them through a fully connected network to obtain historical decision coding information. The weighted historical decision coding vector satisfies formula (4).

[0088] k′=∑ m q m k m S m (4)

[0089] Where k′ represents the historical decision coding vector after weighted coding. m represents the historical decision encoding vector, q m Represents the weight of the historical decision encoding vector. m represents the user weight. Or, S m represents the user relationship relevance weight. Or, S m Represents user weight and user relationship relevance weight.

[0090] Optionally, the electronic device uses the decision model to analyze the decision coding vector, the historical decision coding information and the user portrait of the user to determine the target decision. User portrait, also known as user role, is a virtual representation of a real user. As an effective tool for outlining users and linking user demands with design directions, user portraits have been widely used in various fields.

[0091] S203: The electronic device outputs a target decision.

[0092] In order to enable the electronic device to communicate with the user, the electronic device can use natural language generation (NLG) technology to map the target decision into a natural language expression, that is, generate a target decision text based on the target decision. Natural language generation refers to converting a machine-readable decision into a natural language text. The electronic device can display the target decision text through a display screen so that the user can obtain the system dialogue statement output by the electronic device. Optionally, the electronic device can also convert the target decision text into a target decision voice and play it to the user in voice form.

[0093] In this way, when performing natural language understanding on the target command issued by the user in the current human-computer interaction task, the semantics of the historical commands in the historical human-computer interaction tasks are referenced to assist in the natural language understanding of the target command, making the result of natural language understanding more consistent with the user's true intentions; when executing system decisions, historical decisions are referenced, and the target decision can be optimized based on historical decisions, effectively improving the user experience of human-computer interaction.

[0094] It is understandable that in order to implement the functions in the above embodiments, the electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0095] Figure 8 Schematic diagram of the structure of possible human-computer interaction devices provided in the embodiments of the present application. These human-computer interaction devices can be used to implement the functions of the electronic devices in the above method embodiments, and thus can also achieve the beneficial effects of the above method embodiments. In the embodiments of the present application, the human-computer interaction device can be as follows: Figure 1 The electronic device shown may also be a module (such as a chip) applied to an electronic device.

[0096] like Figure 8 As shown, the human-computer interaction device 800 includes an acquisition unit 810, a processing unit 820 and a feedback unit 830. The human-computer interaction device 800 is used to implement the above Figure 2 , Figure 4 , Figure 5 or Figure 6 The functions of the electronic device in the method embodiment shown in FIG.

[0097] When the human-computer interaction device 800 is used to implement Figure 2 In the method embodiment shown, the functions of the electronic device are: the acquisition unit 810 is used to execute S201; the processing unit 820 is used to execute S202; and the feedback unit 830 is used to execute S203.

[0098] When the human-computer interaction device 800 is used to implement Figure 4 In the method embodiment shown, the functions of the electronic device are: the acquisition unit 810 is used to execute S201; the processing unit 820 is used to execute S2021 and S2022; and the feedback unit 830 is used to execute S203.

[0099] When the human-computer interaction device 800 is used to implement Figure 5When the functions of the electronic device in the method embodiment are shown: the acquisition unit 810 is used to execute S201; the processing unit 820 is used to execute S2021, S20221 and S20222; and the feedback unit 830 is used to execute S203.

[0100] When the human-computer interaction device 800 is used to implement Figure 6 In the method embodiment shown, the functions of the electronic device are: the processing unit 820 is used to execute S601 to S609.

[0101] For a more detailed description of the acquisition unit 810, the processing unit 820 and the feedback unit 830, please refer to Figure 2 , Figure 4 , Figure 5 or Figure 6 The functions of the acquisition unit 810, the processing unit 820 and the feedback unit 830 may include the above Figure 1 The processor 110 is implemented in the processor.

[0102] Optional, such as Fig. 9 As shown, the human-computer interaction device 900 may include a speech recognition unit 910, a language understanding unit 920, a dialogue management unit 930, a language generation unit 940 and a speech synthesis unit 950. Among them, the speech recognition unit 910 is used to implement the function of the acquisition unit 810. For example, the speech recognition unit 910 is used to recognize the speech issued by the user and obtain the target command. The language understanding unit 920 and the dialogue management unit 930 are used to implement the function of the processing unit 820 and obtain the target decision. For example, the language understanding unit 920 is used to use the intention understanding model to perform natural language understanding on the word vector and historical command encoding information of the target command to obtain the intention and slot of the target command. The dialogue management unit 930 is used to generate the target decision according to the intention and slot of the target command and the historical decision encoding vector of the historical command. The language generation unit 940 and the speech synthesis unit 950 are used to implement the function of the feedback unit 830. For example, the language generation unit 940 is used to convert the target decision into natural language. The speech synthesis unit 950 is used to feedback the language of the decision to the user.

[0103] It is understandable that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0104] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or an electronic device. Of course, the processor and the storage medium can also be present in a network device or an electronic device as discrete components.

[0105] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0106] In the various embodiments of the present application, unless otherwise specified or provided for in any logical conflict, the terms and / or descriptions between the different embodiments are consistent and may be referenced to each other, and the technical features in the different embodiments may be combined to form new embodiments according to their inherent logical relationships.

[0107] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of this application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship; in the formula of this application, the character " / " indicates that the previous and next associated objects are in a "division" relationship.

[0108] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.

Claims

1. A human-computer interaction method, characterized in that: include: Receive target commands issued by users; Generate a target decision of the target command by using historical commands, historical decisions of the historical commands and the target command, wherein the historical commands are commands of historical human-computer interaction tasks, and the target command is a command of a current human-computer interaction task; Generating a target decision of the target command by using the historical commands, the historical decisions of the historical commands, and the target command, comprises: Performing weighted encoding on the historical commands based on command semantic relevance weights to obtain historical command encoding information, wherein the command semantic relevance weights represent the semantic relevance between the target command and the historical commands; Generate the target decision according to the target command, the historical command encoding information and the historical decisions of the historical commands; The target decision is output.

2. The method according to claim 1, characterized in that: Before weighted encoding the historical commands based on command semantic relevance weights, the method further includes: Performing semantic encoding on the target command to obtain a semantic vector of the target command; The command semantic relevance weight is obtained by performing similarity calculation based on the semantic vector of the target command and the semantic vector of the historical command.

3. The method according to claim 1, characterized in that The historical commands are weighted encoded based on command semantic relevance weights to obtain historical command encoding information, including: The historical commands are weighted encoded based on the command semantic relevance weight and the user weight to obtain historical command encoding information, wherein the user weight represents the degree of association between the user and the historical user who issued the historical command.

4. The method according to claim 3, characterized in that Before weighted encoding the historical commands based on command semantic relevance weights and user weights, the method further includes: The degree of association between the user and the historical users is obtained according to the voiceprint of the user to obtain the user weight.

5. The method according to claim 3, characterized in that: The historical commands are weighted encoded based on the command semantics-related weights and the user weights to obtain historical command encoding information, including: The historical commands are weighted encoded based on the command semantic relevance weight, the user weight and the user relationship relevance weight to obtain historical command encoding information, wherein the user relationship relevance weight is a preset relationship strength value of multiple users.

6. The method according to any one of claims 1 to 5, characterized in that Generating the target decision according to the target command, the historical command encoding information and the historical decisions of the historical commands includes: Using the intent understanding model to perform natural language understanding on the word vector of the target command and the historical command encoding information, to obtain the intent and slot of the target command; The target decision is generated according to the intent and slot of the target command and the historical decision encoding vector of the historical command.

7. The method according to claim 6, characterized in that The target decision is generated according to the intent and slot of the target command and the historical decision encoding vector of the historical command, including: Encoding the intent and slot of the target command to obtain a decision encoding vector; Performing weighted encoding on the historical decision encoding vector based on a historical decision encoding vector weight to obtain historical decision encoding information, wherein the historical decision encoding vector weight represents a degree of correlation between the decision encoding vector and the historical decision encoding vector; The decision coding vector and the historical decision coding information are analyzed using a decision model to generate the target decision.

8. The method according to claim 7, characterized in that Before weighted encoding the historical decision encoding vector based on the historical decision encoding vector weight to obtain historical decision encoding information, the method further includes: A similarity calculation is performed on the decision coding vector and the historical decision coding vector to obtain a weight of the historical decision coding vector.

9. The method according to claim 7 or 8, characterized in that: The historical decision coding vector is weighted-encoded based on the historical decision coding vector weight to obtain historical decision coding information, including: Performing weighted encoding on the historical decision encoding vector based on the historical decision encoding vector weight and the user weight to obtain historical decision encoding information; Alternatively, the historical decision coding vector is weighted-encoded based on the historical decision coding vector weight, the user weight and the user relationship relevance weight to obtain historical decision coding information.

10. A human-computer interaction device, characterized in that: include: An acquisition unit, used for receiving a target command issued by a user; a processing unit, configured to generate a target decision of the target command by using historical commands, historical decisions of the historical commands, and the target command, wherein the historical commands are commands of historical human-computer interaction tasks, and the target command is a command of a current human-computer interaction task; The processing unit is specifically used to perform weighted encoding on the historical commands based on command semantic relevance weights to obtain historical command encoding information, and generate the target decision according to the target command, the historical command encoding information and the historical decisions of the historical commands; the command semantic relevance weight indicates the semantic relevance between the target command and the historical commands; A feedback unit is used to output the target decision.

11. An electronic device, characterized in that: include: At least one processor, a memory and a voice transceiver, wherein the voice transceiver is used to receive the voice of a target command or the voice of a feedback target decision, the memory is used to store computer programs and instructions, and the processor is used to call the computer programs and instructions to assist the voice transceiver in executing the human-computer interaction method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The storage medium stores a computer program or instruction. When the computer program or instruction is executed by the human-computer interaction device, the method according to any one of claims 1 to 9 is implemented.

Images