A human-computer interaction method applied to a service robot

Abstract

This invention discloses a human-computer interaction method for service robots, involving human-computer dialogue systems and mobile robot navigation control. The biggest drawback of current human-computer dialogue systems, which primarily use sequence-to-sequence models, is that these models tend to generate high-frequency, generic responses and lack sufficient useful information, making it difficult to provide practical assistance to users. This patent introduces external knowledge to enrich the information content of the model's generated responses, improve the diversity of responses, and alleviate the problem of high-frequency generic responses. Based on this model, a human-computer interaction method for service robots is designed. This method not only has dialogue capabilities but can also parse user navigation control commands, enabling fixed-point navigation control in home or office scenarios and improving the robot's human-computer interaction capabilities.

Description

Technical Field

[0001] This invention belongs to the field of human-computer interaction, and specifically relates to a human-computer interaction method applied to service robots. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, service robots suitable for home or office environments are gradually becoming a reality. How to improve the intelligence level of service robots and achieve personalized and comfortable services has become a research hotspot. However, current natural language processing models for human-computer interaction are mainly sequence-to-sequence models. The biggest drawback of this model is that it tends to generate high-frequency, generic responses and lacks sufficient useful information, such as "okay," "yes," and "correct." These responses often do not contain any useful information, but they cannot be considered incorrect responses. The main reason is that generic responses are frequently present in dialogue data. Obviously, such dialogue systems are difficult to provide users with practical help and cannot meet the intelligent service needs of robots. Furthermore, existing end-to-end dialogue models passively generate responses based on user input; the models lack the ability to actively change the direction and content of the dialogue, thus limiting the agent's ability to guide the dialogue process. Therefore, designing a more intelligent dialogue system is an important problem that needs to be solved. Summary of the Invention

[0003] The purpose of this invention is to provide a human-computer interaction method for service robots.

[0004] The objective of this invention is achieved through the following technical solution:

[0005] A human-computer interaction method for service robots includes the following steps:

[0006] Step 1: Obtain user input and feed it into the domain detection model to determine if the user input is a navigation domain requirement;

[0007] A domain classification model is established, using the TextCNN model as input text and outputting the domain corresponding to the text. The model acquires and processes the input language text information, performing character-based vectorization to obtain an initial character-based vector. Multiple convolutional kernels, multiple window pooling processes, ReLU activation function processing, and cascaded processing are then applied to obtain an optimized character-based vector. Partial element discarding and transformation are performed to obtain the target character-based vector. Finally, the target vector is calculated through a fully connected layer of a neural network to identify the text domain probability corresponding to the language text information.

[0008] Step 2: If the domain detection model in Step 1 determines that the user input is a navigation domain requirement, then match the navigation target location and send navigation instructions to the mobile robot;

[0009] The process involves constructing an environmental map and marking target points. When a mobile robot autonomously explores an unknown environment, lacking prior information, its first step is to utilize its onboard sensors to acquire information about the surrounding environment. The motion and observation equations of the SLAM model use this information to build a local map of the robot's environment and estimate its pose. The autonomous exploration method uses the established local environmental map and the robot's pose information to calculate the target location the robot should reach next. It then sends the starting point, the established local environmental map, and the target location to the DWA local path planning model. DWA performs local path planning based on the acquired information and calls the robot's motion actuators to drive the robot to the target location. When the robot reaches the target location, the above steps are repeated to ultimately build a globally consistent environmental map. After obtaining the environmental map, the coordinates of the target location are stored in a dictionary-like list for later use by the dialogue system to issue robot navigation commands.

[0010] Step 3: If the domain detection model in Step 1 determines that the user input is a non-navigation domain requirement, then the system switches to the dialogue system model, obtains the corresponding response, and feeds the result back to the user.

[0011] A knowledge-driven dialogue system model is established; the formal definition of the human-computer dialogue task after introducing external structured knowledge is as follows:

[0012] For dataset The goal of a knowledge-based end-to-end dialogue model is to estimate the generation probability p(o1, ..., o) of sentences based on a given dataset D. T |H, K), and then generate the corresponding response sequence O = (o1, ..., o2). T ); where i represents the i-th dialogue sample in the dataset, H i Let H represent the dialogue history in the i-th dialogue sample. If the dialogue history is concatenated into a sentence containing n words, then H... i =(w i,1 , ..., w i,n If we represent the dialogue history as a sequence of N sentences that make up the dialogue history, then H i =(U i，1 , ..., U i,N In this case, for each individual sentence, omitting the symbol 'i', then U j =(w j,1 , ..., wj，n K represents a sequence of n words that make up a sentence; additionally, K... i =(e i，1 , ..., e i,M ) represents a sequence of knowledge entities of length M related to the dialogue history in the i-th dialogue sample, O i =(o i，1 , ..., o i,T Let ) represent a word sequence of length T in the i-th dialogue sample; the dialogue system model needs to encode the dialogue history, encode the knowledge, and design a decoder.

[0013] Furthermore, in step 3, the dialogue history encoding first involves concatenating several sentences from the dialogue history into a long sentence for encoding; then using... <eos>Symbols are used as delimiters to connect multiple sentences in the dialogue history into a single input sequence; the vector representation of the input sequence, the Word Embedding, is fed into the Word Encoder for encoding; the output of the Word Encoder encoding the input sequence, {s1, s2, ..., s...}, is then obtained. n Afterwards, the WordAttention module is used to align the decoder state with the encoded dialogue history information during decoding, thereby obtaining the dialogue history vector representation ctx related to the decoding state. t Define the vector representation of the decoder state at time t as h. t The calculation process for WordAttention is as follows:

[0014]

[0015]

[0016]

[0017] According to decoder state h t Calculate each vector s in the encoder output i weights Finally, the weighted sum of the encoder output vectors is calculated as the vector representation ctx of the current moment in the dialogue history. t .

[0018] Furthermore, in step 3, knowledge encoding involves associating relevant entities based on the relationships between triples from existing knowledge entities in the dialogue history, and then converting them into low-dimensional continuous vectors using Word Embedding. The KB Encoder uses a linear layer to linearly encode knowledge entities e. i Encode to obtain k i The obtained target knowledge entity sequence is encoded and output as {k1, k2, ..., k}. m The KB Attention module then aligns the decoder state with the encoded information of the target knowledge entity during decoding, allowing the model to focus on different knowledge entities at different times during the decoding process, thereby decoding different knowledge entities; as shown in the equation:

[0019]

[0020]

[0021]

[0022] The calculation process of KB Attention is similar to that of Word Attention, based on the decoder state h at the current time step. t Calculate the weights corresponding to the different knowledge entity encoding outputs ki. Finally, the weighted sum of the target knowledge entity encoding vectors is calculated as the decoder state-related target knowledge entity vector representation ktx. t .

[0023] Furthermore, step 3 requires establishing a decoder, which is then decoded using the Decoder module, as shown in the following equation:

[0024] h t =GRU(h t-1 Embedding(o t-1 ))

[0025] β t =linear([h t ;ctx t ktx t ])

[0026] p t =softmax(β) t )

[0027] o t =argmax(p t )

[0028] The decoder uses a one-way gated recurrent unit (GRU) to obtain the decoder state. At each time step, it generates candidate words based on the decoder state, dialogue history, and target knowledge entities. The GRU then stores the output from the previous time step. t -1 The low-dimensional continuous vector obtained after Word Embedding and the hidden state of the GRU at the previous time step are used as input to obtain the new decoder state h. t Then, WordAttention and KBAttention are used to obtain the decoder layer, and the three are concatenated [h] t ;ctx t ktx t Mapped to a vector β with dimension equal to the vocabulary size. t Finally, the generation probability p for different words is obtained through a softmax layer. t The word with the highest probability at this moment is obtained through the argmax function, which is the output o at the current time. t .

[0029] The beneficial effects of this invention are as follows:

[0030] (1) Based on the traditional end-to-end human-computer dialogue model, by introducing external structured knowledge, the model is guided to generate responses containing rich information, which indirectly improves the diversity of dialogue content, and by indirectly guiding the direction of dialogue through the interrelationship between knowledge, the initiative of the dialogue model is improved to a certain extent.

[0031] (2) Based on the dialogue model, the dialogue task is divided into navigation task and non-navigation task, which helps the robot not only have the ability to have dialogue, but also to perform specific navigation tasks. Attached Figure Description

[0032] Figure 1 This is a flowchart of human-computer interaction methods applied to service robots;

[0033] Figure 2 This is a diagram of a dialogue system model;

[0034] Figure 3 It is a diagram for constructing an environment map architecture. Detailed Implementation

[0035] The present invention will now be further described with reference to the accompanying drawings.

[0036] The technical solution adopted in this invention is: a human-computer interaction method for service robots, comprising the following steps:

[0037] S1. Obtain user input and send it to the domain detection model to determine if the user input is a navigation domain requirement;

[0038] S2. If the domain detection model in S1 determines that the user input is a navigation domain requirement, then match the navigation target location and send navigation instructions to the mobile robot.

[0039] S3. If the domain detection model in S1 determines that the user input is a non-navigation domain requirement, then the system will switch to the dialogue system model, obtain the corresponding response, and then feed the result back to the user.

[0040] Furthermore, in step S1, a domain classification model needs to be established. The TextCNN model is used as input text, outputting the domain corresponding to the text. The input language text information is acquired and processed, and character-based vectorization is performed to obtain an initial character-based vector. Multiple convolutional kernels, multiple window pooling processes, ReLU activation function processing, and cascaded processing are then applied to obtain an optimized character-based vector. Some elements are discarded and transformed to obtain the target character-based vector. Finally, the target vector is calculated through a fully connected layer of the neural network to identify the text domain probability corresponding to the language text information.

[0041] Furthermore, in step S2, it is necessary to construct an environmental map and mark the location of the target point.

[0042] When a mobile robot autonomously explores an unknown environment, lacking prior environmental information, its first step is to utilize its onboard sensors to acquire information about the surrounding environment. This patent uses LiDAR (Light Detection and Ranging). Next, the motion and observation equations of the SLAM (Simultaneous Localization and Mapping) model use this information to create a local map of the robot's environment and estimate its own pose. Then, the autonomous exploration method uses the established local environment map and the robot's pose information to calculate the robot's next target location and sends the starting point, the established local environment map, and the target location to the DWA (Dual Path Planning) local path planning model. DWA performs local path planning based on the information and invokes the robot's motion actuators to drive the robot to the target location. When the robot reaches the target location, the above steps are repeated to ultimately construct a globally consistent environment map. After obtaining the environment map, the coordinates of the target location are stored in a dictionary-like list for later use by the dialogue system to issue robot navigation commands.

[0043] Furthermore, in step S3, it is necessary to establish a knowledge-driven dialogue system model.

[0044] The formal definition of a human-computer dialogue task after introducing external structured knowledge is as follows: For the dataset The goal of a knowledge-based end-to-end dialogue model is to estimate the generation probability p(o1, ..., o) of sentences based on a given dataset D. T |H, K), and then generate the corresponding response sequence O = (o1, ..., o2). T Where i represents the i-th dialogue sample in the dataset, H i Let H represent the dialogue history in the i-th dialogue sample. If the dialogue history is concatenated into a sentence containing n words, then H... i =(w i,1 , ..., w i,n If we represent the dialogue history as a sequence of N sentences that make up the dialogue history, then H i =(U i，1 , ..., U i,N In this case, for each individual sentence, omitting the symbol 'i', then U j =(w j，1 , ..., w j，n K represents a sequence of n words that make up a sentence. i =(e i，1 , ..., e i,M ) represents a sequence of knowledge entities of length M related to the dialogue history in the i-th dialogue sample, O i =(o i，1 , ..., o i,T Let represent a word sequence of length T in the response of the i-th dialogue sample. This patent introduces a triplet structure into the dialogue system, where each triple includes a topic, relation, and object. The hot knowledge of triplet forms is essentially discrete data and needs to be converted into word embeddings to participate in the training and testing of the dialogue model. This model requires encoding the dialogue history, encoding the knowledge, and designing a decoder.

[0045] Furthermore, dialogue history encoding. First, several sentences from the dialogue history are concatenated into a long sentence for encoding. This patent uses... <eos>Symbols are used as delimiters to concatenate multiple sentences from the dialogue history into a single input sequence. Then, the vector representation (Word Embedding) of the input sequence is fed into a Word Encoder for encoding; here, a bidirectional GRU is used to encode the dialogue history. Finally, the output {s1, s2, ..., s...} of the Word Encoder encoding the input sequence is obtained. n Afterwards, the Word Attention module is used to align the decoder state with the encoded dialogue history information during decoding, thereby obtaining the dialogue history vector representation ctx related to the decoding state. t Define the vector representation of the decoder state at time t as h. t The calculation process of Word Attention is shown in Equation 1, based on the decoder state h. t Calculate the weights of each vector si in the encoder output. Finally, the weighted sum of the encoder output vectors is calculated as the vector representation ctx of the current moment in the dialogue history. t .

[0046]

[0047] Furthermore, knowledge encoding is required. Knowledge encoding involves using existing knowledge entities from the dialogue history, associating them with related entities based on triplet relationships, and then converting them into low-dimensional continuous vectors using Word Embedding. KBEncoder uses a linear layer ((linear)) to encode knowledge entities e. i Encode to obtain k i The obtained target knowledge entity sequence is encoded and output as {k1, k2, ..., k}. m The KB Attention module then aligns the decoder state with the encoded information of the target knowledge entity during decoding, allowing the model to focus on different knowledge entities at different times during the decoding process, thus decoding different knowledge entities. As shown in Equation 2, the calculation process of KB Attention is similar to WordAttention, based on the current decoder state h. t Calculate the output k of different knowledge entity encodings. i Corresponding weights Finally, the weighted sum of the target knowledge entity encoding vectors is calculated as the decoder state-related target knowledge entity vector representation ktx. t .

[0048]

[0049] Furthermore, a decoder needs to be built.

[0050] Decoding is performed through the Decoder module, as shown in Equation 3. The decoder uses a one-way gated recurrent unit (GRU) to obtain the decoder state, and at each time step, it generates candidate words based on the decoder state, dialogue history, and target knowledge entities. The GRU then converts the output of the previous time step into the desired value. t -1 The low-dimensional continuous vector obtained after Word Embedding and the hidden state of the GRU at the previous time step are used as input to obtain the new decoder state h. t Then, through Word Attention and KB Attention, the decoder layer is obtained and the three are concatenated [h] t ;ctx t ktx t Mapped to a vector β with dimension equal to the vocabulary size. t Finally, the generation probability p for different words is obtained through a softmax layer. t The word with the highest probability at this moment is obtained through the argmax function, which is the output o at the current time. t .

[0051]

[0052] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.< / eos> < / eos>

Claims

1. A human-computer interaction method for service robots, characterized in that: The steps are as follows: Step 1: Obtain user input and feed it into the domain detection model to determine if the user input is a navigation domain requirement; A domain classification model is established, using the TextCNN model as input text and outputting the domain corresponding to the text. The model acquires and processes the input language text information, performing character-based vectorization to obtain an initial character-based vector. Multiple convolutional kernels, multiple window pooling processes, ReLU activation function processing, and cascaded processing are then applied to obtain an optimized character-based vector. Partial element discarding and transformation are performed to obtain the target character-based vector. Finally, the target vector is calculated through a fully connected layer of a neural network to identify the text domain probability corresponding to the language text information. Step 2: If the domain detection model in Step 1 determines that the user input is a navigation domain requirement, then match the navigation target location and send navigation instructions to the mobile robot; The process involves constructing an environmental map and marking target points. When a mobile robot autonomously explores an unknown environment, lacking prior information, its first step is to utilize its onboard sensors to acquire information about the surrounding environment. The motion and observation equations of the SLAM model use this information to build a local map of the robot's environment and estimate its pose. The autonomous exploration method uses the established local environmental map and the robot's pose information to calculate the target location the robot should reach next. It then sends the starting point, the established local environmental map, and the target location to the DWA local path planning model. DWA performs local path planning based on the acquired information and calls the robot's motion actuators to drive the robot to the target location. When the robot reaches the target location, the above steps are repeated to ultimately build a globally consistent environmental map. After obtaining the environmental map, the coordinates of the target location are stored in a dictionary-like list for later use by the dialogue system to issue robot navigation commands. Step 3: If the domain detection model in Step 1 determines that the user input is a non-navigation domain requirement, then the system switches to the dialogue system model, obtains the corresponding response, and feeds the result back to the user. A knowledge-driven dialogue system model is established; the formal definition of the human-computer dialogue task after introducing external structured knowledge is as follows: For dataset The goal of a knowledge-based end-to-end dialogue model is to estimate the generation probability p(O|H,K)=p((o1,…,o) based on a given dataset D. T )|H,K), and then generate the corresponding response sequence O, O=o1,…,o T );in i represents the i-th dialogue sample in the dataset, H i Let H represent the dialogue history in the i-th dialogue sample. If the dialogue history is concatenated into a sentence containing n words, then H... i =(w i,1 ,…,w i,n ); If we represent the dialogue history as a sequence of N sentences that make up the dialogue history, then H i =(U i,1 ,…,U i,N ), then for each individual sentence, omitting the symbol i, then U j =(w j,1 ,…,w j,n K represents a sequence of n words that make up a sentence; additionally, K... i =(e i,1 ,…,e i,M ) represents a sequence of knowledge entities of length M related to the dialogue history in the i-th dialogue sample, O i =(o i,1 ,…,o i,T () represents a word sequence of length T in the response of the i-th dialogue sample; Dialogue system models require encoding dialogue history, encoding knowledge, and designing decoders.

2. The human-computer interaction method for service robots according to claim 1, characterized in that: In step 3, dialogue history encoding first involves concatenating several sentences from the dialogue history into a long sentence for encoding; then using... <eos>Symbols are used as delimiters to connect multiple sentences in the dialogue history into a single input sequence; the vector representation of the input sequence, the Word Embedding, is fed into the Word Encoder for encoding; the output of the Word Encoder encoding the input sequence is obtained as {s1, s2, ..., s...}. n Afterwards, the Word Attention module is used to align the decoder state with the encoded dialogue history information during decoding, thereby obtaining the dialogue history vector representation ctx related to the decoding state. t Define the vector representation of the decoder state at time t as h. t The calculation process for Word Attention is as follows:< / eos> According to the decoder state h t Calculate each vector s in the encoder output i weights Finally, the weighted sum of the encoder output vectors is calculated as the vector representation ctx of the current moment in the dialogue history. t .

3. The human-computer interaction method for service robots according to claim 1, characterized in that: In step 3, knowledge encoding involves using existing knowledge entities from the dialogue history and associating them with related entities based on triplet relationships. This is then converted into a low-dimensional continuous vector using Word Embedding. The KB Encoder uses a linear layer to linearly encode knowledge entities e. i Encode to obtain k i The obtained target knowledge entity sequence is encoded and output as {k1,k2,…,k m The KB Attention module then aligns the decoder state with the encoded information of the target knowledge entity during decoding, allowing the model to focus on different knowledge entities at different times during the decoding process, thereby decoding different knowledge entities; as shown in the equation: The calculation process of KB Attention is similar to that of Word Attention, based on the decoder state h at the current time step. t Calculate the output k of different knowledge entity encodings. i Corresponding weights Finally, the weighted sum of the target knowledge entity encoding vectors is calculated as the decoder state-related target knowledge entity vector representation ktx. t .

4. The human-computer interaction method for service robots according to claim 1, characterized in that: Step 3 requires establishing a decoder, which is then decoded using the Decoder module, as shown in the following equation: h t =GRU(h t-1 ,Embedding(o t-1 )) β t =linear([h t ;Ctx t ;ktx t ]) p t =softmax(β t ) o t =argmax(p t ) The decoder uses a one-way gated recurrent unit (GRU) to obtain the decoder state. At each time step, it generates candidate words based on the decoder state, dialogue history, and target knowledge entities. The GRU then stores the output from the previous time step. t-1 The low-dimensional continuous vector obtained after Word Embedding and the hidden state of the GRU at the previous time step are used as inputs to obtain the new decoder state h. t Then, through Word Attention and KB Attention, the decoder layer concatenates the three [h] t ;ctx t ktx t Mapped to a vector β with dimension equal to the vocabulary size. t Finally, the generation probability p for different words is obtained through a softmax layer. t The word with the highest probability at this moment is obtained through the argmax function, which is the output o at the current time. t .

Images