Information processing apparatus, information processing method, mobile body control apparatus, mobile body control method, and storage medium

Abstract

The present disclosure relates to an information processing apparatus, an information processing method, a mobile body control apparatus, a mobile body control method, and a storage medium. The information processing apparatus of the present disclosure is an information processing apparatus capable of controlling a mobile body based on an instruction of an utterance of a user, the information processing apparatus having: an identifying unit that identifies which of a plurality of utilization scenarios when a mobile body is utilized is a utilization scenario of a target user; an acquiring unit that acquires utterance information of the target user; a selecting unit that selects different machine learning models in accordance with the identified utilization scenario of the target user; and a deducing unit that deduces an intention of the utterance of the target user using the selected machine learning model.

Description

TECHNICAL FIELD

[0001] The present application relates to an information processing apparatus, an information processing method, a mobile body control apparatus, a mobile body control method, and a storage medium. BACKGROUND

[0002] In recent years, development of a human-machine interface using a natural language has been advanced. In Non-Patent Literature 1, a technology of performing intent classification and slot filling in a speech using a language expression model called BERT is proposed. The intent classification in a speech is, for example, a technology of inferring an intent of a user in an instruction, a question (also called an inquiry) of the user, and the slot filling is a technology of identifying information provided by the user or insufficient information and performing a question or a supplement for making the information explicit.

[0003] PRIOR ART DOCUMENT

[0004] NON-PATENT LITERATURE

[0005] Non-Patent Literature 1: Qian Chen et al., BERT for Joint Intent Classification and Slot Filling, February 28, 2019, https: / / arxiv.org / pdf / 1902.10909.pdf SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] In Non-Patent Literature 1, a technology of simultaneously performing intent classification and slot filling using a single model installed by BERT is proposed, and in order to classify a speech into any one of a large number of intent categories, learning using a large amount of data is required.

[0008] However, in order to classify an intent of a user using a single classifier model, a classification problem for a large number of intent categories assuming all scenarios (situations) needs to be solved. In a case where a user controls a mobile body by a speech, a large number of intents of the user can exist, for example, a question for availability of a mobile body traveling in the vicinity, an instruction for a route of the mobile body, an instruction related to travel of a vehicle (for example, an instruction for acceleration), an instruction for returning to a mobile body after ending a ride, and the like. That is, in control of a mobile body by a speech, in order to classify various intents from a question for availability to an instruction for returning, a large model is required, and as a result, a large amount of learning data is sometimes required, or an intent classification result with desired precision is sometimes not obtained.

[0009] The present application has been achieved in view of the above problems, and aims at achieving a technology capable of providing classification of an utterance intention by a model constructed by a smaller-scale learning in control of a mobile body by an utterance.

[0010] Means for solving the problem

[0011] According to the present application,

[0012] Provided is an information processing apparatus capable of controlling a mobile body based on an instruction of an utterance of a user, characterized by

[0013] The information processing apparatus has:

[0014] an identifying unit that identifies which of a plurality of usage scenarios when a mobile body is used is a usage scenario of a target user;

[0015] an acquiring unit that acquires utterance information of the target user;

[0016] a selecting unit that selects a different machine learning model according to the usage scenario of the target user identified; and

[0017] a deducing unit that deduces an intention of an utterance of the target user using the selected machine learning model.

[0018] Further, according to the present application,

[0019] Provided is an information processing method in an information processing apparatus capable of controlling a mobile body based on an instruction of an utterance of a user, characterized by

[0020] The information processing method has:

[0021] an identifying step in which which of a plurality of usage scenarios when a mobile body is used is a usage scenario of a target user is identified;

[0022] an acquiring step in which utterance information of the target user is acquired;

[0023] a selecting step in which a different machine learning model is selected according to the usage scenario of the target user identified; and

[0024] a deducing step in which an intention of an utterance of the target user is deduced using the selected machine learning model.

[0025] Further, according to the present application,

[0026] Provided is a control apparatus of a mobile body capable of being controlled based on an instruction of an utterance of a user, characterized by

[0027] The control device has:

[0028] a recognition unit that recognizes which of a plurality of use scenarios when a mobile body is used is a use scenario of a target user;

[0029] an acquisition unit that acquires utterance information of the target user;

[0030] a selection unit that selects a different machine learning model in accordance with the recognized use scenario of the target user; and

[0031] an inference unit that infers an intent of an utterance of the target user using the selected machine learning model.

[0032] Further, according to the present application,

[0033] A control method of a mobile body that is capable of being controlled based on an instruction of an utterance of a user is provided, characterized by

[0034] The control method has:

[0035] a recognition step in which it is recognized which of a plurality of use scenarios when a mobile body is used is a use scenario of a target user;

[0036] an acquisition step in which utterance information of the target user is acquired;

[0037] a selection step in which a different machine learning model is selected in accordance with the recognized use scenario of the target user; and

[0038] an inference step in which an intent of an utterance of the target user is inferred using the selected machine learning model.

[0039] Further, according to the present application,

[0040] A storage medium that saves a program for causing a computer to function as each unit of an information processing device is provided, characterized by

[0041] The information processing device that is capable of controlling a mobile body based on an instruction of an utterance of a user has:

[0042] a recognition unit that recognizes which of a plurality of use scenarios when a mobile body is used is a use scenario of a target user;

[0043] an acquisition unit that acquires utterance information of the target user;

[0044] a selection unit that selects a different machine learning model in accordance with the recognized use scenario of the target user; and

[0045] an inference unit that infers an intention of the utterance of the target user using the selected machine learning model.

[0046] In addition, according to the present application,

[0047] Provided is a storage medium that stores a program for causing a computer to function as each unit of a control device, characterized by

[0048] The control device of the mobile body that is capable of being controlled based on an instruction of an utterance of a user has:

[0049] an identification unit that identifies which of a plurality of use scenarios when the target user uses the mobile body is the use scenario;

[0050] an acquisition unit that acquires utterance information of the target user;

[0051] a selection unit that selects a different machine learning model according to the identified use scenario of the target user; and

[0052] an inference unit that infers an intention of the utterance of the target user using the selected machine learning model.

[0053] Effects of the Invention

[0054] According to the present application, in control of a mobile body by an utterance, it is possible to provide classification of an utterance intention by a model constructed by a smaller scale of learning. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a diagram that illustrates one example of an information processing system according to an embodiment of the present application.

[0056] Figure 2A , Figure 2B is a block diagram that illustrates a configuration example of hardware of a vehicle according to the present embodiment.

[0057] Figure 3 is a block diagram that illustrates a functional configuration example of a vehicle according to the present embodiment.

[0058] Figure 4 is a block diagram that illustrates a functional configuration example of a server according to the present embodiment.

[0059] Figure 5A is a diagram that illustrates one example of a use scenario when a vehicle is used, an intention category of an utterance associated with the use scenario, and an utterance corresponding to the intention category according to the present embodiment.

[0060] Figure 5Bis a diagram for explaining one example of a case where a Hidden Markov Model is applied in the utilization scenario before boarding in the embodiment.

[0061] Figure 5C is a diagram for explaining a case where the speech intention is respectively estimated in the continuous utilization scenario in the embodiment.

[0062] Figure 6 is a flowchart for showing a series of actions of the speech intention estimation processing involved in the embodiment.

[0063] Figure 7 is a diagram for showing one example of an information processing system involved in another embodiment.

[0064] Explanation of Reference Numerals

[0065] 100: vehicle; 110: server; 120: communication device; 404: control unit; 413: user data acquisition section; 414: scenario recognition section; 415: model selection section; 416: speech intention estimation section; 417: sound information processing section; 418: vehicle control section. DETAILED DESCRIPTION

[0066] Hereinafter, the embodiments will be described in detail with reference to the drawings. Furthermore, the following embodiments do not limit the technical scope of the invention described in the technical solution, and all combinations of features described in the embodiments are not necessarily essential to the invention. Two or more of the plurality of features described in the embodiments can be arbitrarily combined. In addition, the same reference numerals are assigned to the same or similar constituent elements, and repeated description is omitted.

[0067] Configuration of Information Processing System

[0068] Reference Figure 1 The configuration of the information processing system 1 involved in the embodiment will be described. The information processing system 1 includes a vehicle 100 as one example of a mobile body, a server 110 as one example of an information processing device, and a communication device 120.

[0069] In the information processing system 1, the user 130 is able to dialogue with the vehicle 100 or control the action of the vehicle 100 by utterance of a natural language. When the communication device 120 accepts the utterance of the user 130 to the vehicle 100, the utterance information is transmitted to the server 110. The server 110 infers the utterance intention of the user from the utterance information of the user. The server 110 performs slot filling (identifies the information that the user has provided or the insufficient information, and provides the user 130 with a question for clarification, if necessary, on the basis of the inferred utterance intention). When the server 110 identifies the utterance intention of the user, the server 110 determines the information necessary for determining the specific instruction content from the utterance information, and thereby accepts the instruction of the user. In a case where the utterance of the user 130 is an instruction to the vehicle 100 (for example, "pick up immediately at the current place"), the server 110 transmits a control instruction corresponding to the instruction to the vehicle 100.

[0070] The vehicle 100 is one example of a mobile body, for example, a super small mobile device that is mounted with a battery and moves mainly with the power of a motor. The super small mobile device refers to a super small vehicle that is more compact than a general motor vehicle and has a rated passenger of one person or two persons or the like. In the present embodiment, the vehicle 100 is, for example, a four-wheeled vehicle. In addition, in the following embodiments, the mobile body is not limited to a vehicle, and can include a small mobile device that transports goods in parallel with a walking user or guides a person, and in addition, can include other mobile bodies that are able to move autonomously (for example, a walking robot or the like).

[0071] The vehicle 100 is connected to the network 140 via wireless communication such as Wi-Fi, fifth generation mobile communication, or the like. The vehicle 100 measures the state inside and outside the vehicle (the position of the vehicle, the state of travel, the target object of the surrounding object, and the like) by various sensors, and is able to transmit the measured data to the server 110. The data thus collected and transmitted is generally referred to as floating data, probe data, traffic information, or the like. The information related to the vehicle is transmitted to the server 110 at a certain interval or according to the occurrence of a specific event. Even in a case where the user 130 is not on board, the vehicle 100 is able to travel by autonomous driving. The vehicle 100 receives information such as a control instruction provided by the server 110, or controls the action of the vehicle using the data measured by the vehicle.

[0072] The server 110 is one example of an information processing apparatus. The server 110 is configured by one or more server apparatuses, can acquire vehicle-related information transmitted from the vehicle 100, utterance information transmitted from the communication apparatus 120, and each position information via the network 140, and can control travel of the vehicle 100. The server 110 executes the user's intention estimation process described later based on the user's utterance information, and thereby estimates the user's intention in the utterance.

[0073] The communication apparatus 120 is, for example, a smartphone, but is not limited thereto, and can be a headset-type communication terminal, and can also be a personal computer, a tablet terminal, a game machine, or the like. The communication apparatus 120 is connected to the network 140 via wireless communication such as Wi-Fi, fifth-generation mobile communication, or the like, for example. The communication apparatus 120 accepts the user's 130 utterance, and transmits the accepted utterance information (sound information) to the server 110.

[0074] The network 140 includes, for example, a communication network such as the Internet, a mobile phone network, and the like, and transmits information between the server 110 and the vehicle 100, and information between the server 110 and the communication apparatus 120.

[0075] (Configuration of Vehicle)

[0076] Next, the configuration of the vehicle 100 as one example of the vehicle to which the present embodiment is applied will be described with reference to Figures 2A-2B to FIG. 1.

[0077] Figure 2A The side surface of the vehicle 100 to which the present embodiment is applied is indicated in FIG. 2. Figure 2B The internal configuration of the vehicle 100 is indicated in FIG. 3. The arrow X in the figure indicates the front-rear direction of the vehicle 100, F indicates the front, and R indicates the rear. The arrows Y and Z indicate the width direction (left-right direction) and the up-down direction of the vehicle 100.

[0078] The vehicle 100 is an electrically self-sufficient vehicle that is provided with a travel unit 12 and uses a battery 13 as a main power source. The battery 13 is, for example, a rechargeable battery such as a lithium-ion battery, and the vehicle 100 travels autonomously by power supplied from the battery 13 and by the travel unit 12. The travel unit 12 is a four-wheel vehicle that is provided with a pair of front wheels 20 and a pair of rear wheels 21 on the left and right. The travel unit 12 can also be in other forms such as a three-wheel vehicle. The vehicle 100 is provided with a seat 14 for one person or two persons. The seat 14 transmits whether or not an occupant is in the vehicle to the control unit 30 by a pressure sensor or the like, for example.

[0079] The traveling unit 12 is provided with a steering mechanism 22. The steering mechanism 22 is a mechanism that changes the turning angle of the pair of front wheels 20 with the motor 22a as a driving source. The traveling direction of the vehicle 100 can be changed by changing the turning angle of the pair of front wheels 20. The traveling unit 12 is also provided with a driving mechanism 23. The driving mechanism 23 is a mechanism that rotates the pair of rear wheels 21 with the motor 23a as a driving source. The vehicle 100 can be made to advance or retreat by rotating the pair of rear wheels 21.

[0080] The vehicle 100 is provided with detection units 15 to 17 that detect the surrounding target objects of the vehicle 100. The detection units 15 to 17 are outside sensors that monitor the surroundings of the vehicle 100, and in the case of the present embodiment, are all photographing devices that photograph the images of the surroundings of the vehicle 100, for example, are provided with optical systems such as lenses and image sensors. However, instead of the photographing devices or in addition to the photographing devices, radars, optical radars (Light Detection and Ranging) can also be employed.

[0081] The two detection units 15 are disposed in the front portion of the vehicle 100 in a manner spaced apart in the Y direction, and mainly detect target objects in front of the vehicle 100. The detection units 16 are disposed in the left side portion and the right side portion of the vehicle 100, respectively, and mainly detect target objects in the side of the vehicle 100. The detection unit 17 is disposed in the rear portion of the vehicle 100, and mainly detects target objects in the rear of the vehicle 100.

[0082] Figure 3 is a block diagram of the control system of the vehicle 100. The vehicle 100 is provided with a control unit (ECU) 30. The control unit 30 includes a processor typified by a CPU, a storage device such as a semiconductor memory, an interface with an external device, and the like. In the storage device, a program executed by the processor, data used by the processor for processing, and the like are stored. The processor, the storage device, and the interface can also be configured to be provided in a plurality of groups according to the functional categories of the vehicle 100 and be able to communicate with each other. Sound recognition of inputted sound, image processing of images photographed by the detection units can also be performed.

[0083] The control unit 30 performs corresponding processing in accordance with the detection results of the detection units 15 to 17, the input information of the operation panel 31, the sound information inputted by the sound input device 33, control instructions from the server 110, and the like. The control unit 30 performs control of the motors 22a, 23a (traveling control of the traveling unit 12), display control of the operation panel 31, reporting to the occupant of the vehicle 100 by sound, output of information.

[0084] The sound input device 33 is capable of picking up the voice of the occupant of the vehicle 100. The control unit 30 is capable of recognizing the input voice to perform the corresponding processing. The GNSS (Global Navigation Satellite system) sensor 34 receives the GNSS signal to detect the current position of the vehicle 100.

[0085] The storage device 35 is a large-capacity storage device that stores map data and the like including information on the travel road on which the vehicle 100 is capable of traveling, landmarks such as buildings, stores, and the like. The storage device 35 can also hold programs executed by the processor, data used for processing by the processor, and the like. The storage device 35 can also hold various parameters of the machine learning model used for voice recognition and image recognition performed by the control unit 30 (for example, learned parameters of a deep neural network, and the like).

[0086] The communication device 36 is a communication device capable of connecting to the network 140 via wireless communication such as Wi-Fi, fifth-generation mobile communication, and the like.

[0087] (Configuration of Server)

[0088] Next, the configuration of the server 110 as one example of the information processing device according to the present embodiment will be described with reference to Figure 4

[0089] The control unit 404 includes a processor typified by a CPU, a storage device such as a semiconductor memory, an interface with an external device, and the like. The storage device holds programs executed by the processor, data used for processing by the processor, and the like. The processor, the storage device, and the interface can be configured to be provided in multiple groups according to the functional categories of the server 110 and be capable of communicating with each other. The control unit 404 performs various actions of the server 110, the intention estimation processing of the user described later, the control of the vehicle 100, and the like by executing the programs. The control unit 404 includes, on the basis of the CPU, a GPU or a dedicated hardware suitable for processing of a machine learning model such as a neural network.

[0090] The user data acquisition unit 413 acquires the utterance information of the user 130 transmitted from the vehicle 120. In addition, the user data acquisition unit 413 acquires the floating data (for example, vehicle position, presence or absence of an occupant, and the like) information transmitted from the communication device 100. The user data acquisition unit 413 can also save the acquired image, position information in the storage unit 403. The utterance information acquired by the user data acquisition unit 413 is input to the learned model of the inference stage, but can also be used as learning data for causing the machine learning model executed by the server 110 to learn.

[0091] ​The situation recognition unit 414 recognizes a current situation (situation) in which the user is located. The situation recognition unit 414, for example, recognizes which of before getting on a vehicle, while getting on a vehicle, and after getting off a vehicle is the situation of the user. An example of a method for recognizing a situation is described later.

[0092] The model selection unit 415 selects a machine learning model in the situation recognized by the situation recognition unit 414. As described later, there are a plurality of machine learning models, each of which is associated with any one of before getting on a vehicle, while getting on a vehicle, and after getting off a vehicle, for example. That is, each machine learning model is configured in such a manner that a different estimated intention category is output for each associated use situation, and the likelihood of the intention category of any situation is output.

[0093] The speech intention estimation unit 416 estimates the intention of the speech of the user using the machine learning model selected by the model selection unit 415. A method for estimating the intention of the speech is described later.

[0094] The sound information processing unit 417 determines necessary information from the speech information in order to determine the specific instruction content based on the recognized intention of the speech. For example, in a case where the intention of the speech information of the user is to ask to come to meet, information of where to go, when to go to meet, and the like is determined. The sound information processing unit 417 receives the instruction of the user when the necessary information is determined. The sound information processing unit 417 can also include processing of slot filling. The sound information processing unit 417 can include a plurality of machine learning models different from the machine learning model used for the estimation of the intention of the speech, each of which can be configured by a deep neural network (DNN), for example. The DNN can be brought to a state where learning is completed by performing processing in a learning phase, and can perform processing (processing in an inference phase) on new speech information by inputting the new speech information to the DNN where learning is completed.

[0095] The vehicle control unit 418 controls the action of the vehicle 100 based on the speech content recognized by the sound information processing unit 417. For example, in a case where information of where to go, when to go to meet, and the like is determined from the speech information of the user, a route is determined based on the current position of the user and the vehicle, map information, and the like, and the vehicle is caused to travel on the route.

[0096] In addition, the server 110 can generally use rich computing resources compared to the vehicle 100 or the like. The server 110 can provide a result of computation more quickly compared to a case where each vehicle 100 is equipped with a computing resource for executing a machine learning model, and in addition, can contribute to cost reduction of the vehicle. In addition, the server 110 can collect learning data including a wide variety of speech information by receiving and accumulating speech information of various users, and can make a more powerful inference processing possible.

[0097] The communication unit 401 is, for example, a communication device including a communication circuit or the like, and communicates with the vehicle 100, the communication device 120, and the like. The communication unit 401 receives the position information and the presence or absence of the occupant from the vehicle 100, the utterance information and the position information from the communication device 120, and further transmits the control instruction to the vehicle 100, the utterance information to the communication device 120.

[0098] The power supply unit 402 supplies power to each unit in the server 110. The storage unit 403 is a hard disk, a semiconductor memory, or the like, which is a nonvolatile memory.

[0099] (Summary of the user's intention estimation process)

[0100] As described above, in a case where it is assumed that the user controls the mobile body by utterance, there can be a large number of user's intentions, for example, a query for the availability of calling the mobile body that is traveling nearby, an instruction to the route of the mobile body, an instruction related to the travel of the vehicle (for example, an instruction to accelerate), an instruction to the mobile body to return after the end of the ride, and the like.

[0101] However, before the use of the mobile body, it is possible to make an utterance of the intention to query the availability of the mobile body or to call the mobile body, while on the other hand, it is less likely to make an utterance of the intention to return after the use. In other words, the intention of the conversation made in each of the scenes before the ride, during the ride, and after the ride is somewhat not present in the other scenes.

[0102] Therefore, in the present embodiment, when the mobile body is controlled by utterance, the intention categories are summarized for each of the assumed scenes (use scenes), and the machine learning models are associated for each of the scenes. Each of the machine learning models only estimates the intention category of the associated use scene. By so doing, it is possible to use the model appropriate for each of the scenes, each of the models can be smaller in scale compared to a case where a single model classifies a large number of intention categories, and it is possible to construct by smaller-scale learning. In addition, an improvement in recognition accuracy can be expected.

[0103] Hereinafter, the relationship between the use scene, the intention category of the utterance, and the algorithm of the intention estimation related to the present embodiment will be described with reference to Figures 5A-5C The use scene, the relationship between the intention category of the utterance associated with the use scene, and the algorithm of the intention estimation related to the present embodiment will be described.

[0104] Figure 5A An example of the use scene when the vehicle is used, the intention category of the utterance associated with the use scene, and the utterance corresponding to the intention category is shown. As shown in Figure 5A As one example, the use scene 501 is classified into a state before the ride on the vehicle 100 (before-ride state), a state during the ride (during-ride state), and a state after the ride (after-ride state). Further,Figure 5A The term "normal" does not refer to a specific usage scenario, but rather means that the three intent categories belonging to "normal" are included in any usage scenario.

[0105] The intent category 502 represents the user's intent in their speech. The use case "Before boarding" is associated with seven intent categories, such as asking, requesting a greeting, saying hello, indicating destination, agreeing, disagreeing, and repeating a question. Similarly, the use case "While on the ride" is associated with seven intent categories, such as route instructions, stop instructions, acceleration instructions, deceleration instructions, agreeing, disagreeing, and repeating a question. Likewise, the use case "After getting off the vehicle" is also associated with... Figure 5A The seven intent categories shown are related.

[0106] Speech example 503 illustrates speech examples corresponding to various intent categories. For example, the speech "Can I take the bus now?" corresponds to the intent of "asking".

[0107] In this way, a predetermined number of scenarios are defined as assumed continuous scenarios when using a vehicle, and each scenario is associated only with a subset of all intent categories that can be associated with multiple scenarios. In this way, the machine learning model can output inferences only for a subset of all intent categories that can be associated with multiple scenarios. Therefore, the machine learning model for inferring intent categories can be miniaturized and can be trained using a small amount of training data.

[0108] In addition, Figure 5A In the examples shown, examples associated with the same number of intent categories are represented for all use cases, but each use case may be associated with a different number of intent categories. Furthermore, intent categories are not limited to the examples above; they may include other intent categories or none at all. Figure 5A This illustrates a subset of intent categories. For example, a pre-ride use case could include a "catch-up request," which requires the vehicle to catch up with the user's destination. Examples of phrases related to catch-up requests include "catch up."

[0109] In addition, the example given is a speech instance where user 130 speaks to vehicle 100. However, it is not limited to the example where user 130 speaks to vehicle 100; a speech instance where user 130 speaks to a human host (coordinating vehicle control) is also possible.

[0110] Next, refer to Figure 5BAn example of applying a hidden Markov model to the use scenario before getting on a vehicle will be described. A Markov model is a probability model in which the probability distribution of a state at an arbitrary time is a probability process that depends only on the previous state. In this embodiment, a hidden Markov model (also referred to as HMM) is applied to solve the problem of estimating a state (utterance intention) hidden behind a state (utterance information) that can be observed.

[0111] Figure 5B 510 to 513 shown in the drawing represent hidden states in the hidden Markov model and correspond to the intention categories. Further, in the example shown in the drawing, only four intention categories are represented to prevent the drawing from becoming too complex. The numerical values written in the circles of the intention categories represent initial state probabilities. That is, the probabilities (likelihoods) of the intention categories that can occur after the use scenario becomes "before getting on a vehicle" are represented. In addition, each arrow represents a transition between intention categories (states), and the numerical values (for example, "0.aa") added to the arrows represent state transition probabilities. The distribution of the initial state probabilities and the distribution of the state transition probabilities can be determined in advance, for example, by finding the transition probabilities between the intention categories and the initial state probabilities of the correct answer data included in the learning data, and these probabilities can be used. Figure 5B In the example shown in the drawing, only four intention categories are represented to prevent the drawing from becoming too complex. The numerical values written in the circles of the intention categories represent initial state probabilities. That is, the probabilities (likelihoods) of the intention categories that can occur after the use scenario becomes "before getting on a vehicle" are represented. In addition, each arrow represents a transition between intention categories (states), and the numerical values (for example, "0.aa") added to the arrows represent state transition probabilities. The distribution of the initial state probabilities and the distribution of the state transition probabilities can be determined in advance, for example, by finding the transition probabilities between the intention categories and the initial state probabilities of the correct answer data included in the learning data, and these probabilities can be used.

[0112] Further, reference will be made to Figure 5C An example of the intention estimation processing according to this embodiment will be described. In the example shown in FIG. 5, the estimation of the utterance intention for each use scenario is represented with "before getting on a vehicle" as the first use scenario and "while getting on a vehicle" as the second use scenario. 510 to 513 in the drawing correspond to the intention categories shown in FIG. 5, and the bar graphs in the probability distributions represent the probabilities (likelihoods) of the intention categories. Figure 5B The intention categories shown in the drawing correspond to 510 to 513 in the drawing, and the bar graphs in the probability distributions represent the probabilities (likelihoods) of the intention categories. 520 to 523 in the drawing correspond to the intention categories (route instruction, stop instruction, acceleration instruction, and deceleration instruction) for "while getting on a vehicle", and the bar graphs in the probability distributions represent the probabilities (likelihoods) of these intention categories.

[0113] In the initial state probability distribution 530 for the first use scenario, as shown in Figure 5BAs illustrated, the probability of the inquiry becomes higher than the probability of the greeting. After the first utilization scenario starts, the user makes a speech (for example, "Can I take a ride?"). In this way, the server 110 calculates the probability (likelihood) of the intent category using the machine learning model associated with the first utilization scenario and adds the initial state probability distribution 530, thereby calculating the probability (likelihood) of the intent category (probability distribution 540). The probability distribution 540 of the first utilization scenario indicates that the probability (likelihood) of the intent related to the inquiry is high. Further, when the user makes the next speech, the server 110 calculates the probability (likelihood) of the intent category using the same machine learning model and adds the state transition probability, thereby calculating the probability (likelihood) of the intent category. As such, the likelihood of the intent is calculated by the machine learning model, on the basis of which the final speech intent is estimated by adding the probability of the state transition from the state of a certain intent to the state of the next intent and considering that the likelihood of the intent and the probability distribution are easily transitioned.

[0114] Thereafter, when the utilization scenario changes, the server 110 calculates the probability (likelihood) of the intent category using the machine learning model associated with the second utilization scenario, the initial state probability distribution of the second utilization scenario, and the state transition probability distribution of the second utilization scenario.

[0115] In the present embodiment, the server 110 calculates the likelihood of the intent category in accordance with the following equation. The following calculation of the intent category is calculated for each utilization scenario as described above.

[0116] [Equation 1]

[0117]

[0118] Output intent category

[0119] In the above equation, b(c t ) is a discrete probability distribution of the intent category, x t is a case where the speech text is vectorized, C is a set of available intent categories (in the case where it is vectorized), c is a probability variable indicating the intent category, and the subscript t (t > 1) indicates the time. The operation result of the likelihood function P(x t | c t ) is obtained by the operation of the machine learning model (which is different for each utilization scenario). b(c t＝0 ) is the initial state probability distribution (which is different for each utilization scenario). P(c t | c t-1) indicates the state transition probability (which differs for each utilization scenario). Through this calculation, when the intention of the utterance at time t is inferred, the inference result inferred for the utterance once before the utterance at time t can be recursively added.

[0120] (A series of actions of the user's intention inference processing)

[0121] Next, a series of actions of the user's intention inference processing in the server 110 will be described with reference to Figure 6 to the server 110. This processing is realized by the control unit 404 executing a program. The machine learning model executed through this series of actions is in a state of being learned (inference phase) using the learning data. In the following description, the control unit 404 will be described as executing each processing for the sake of simplifying the description, but the corresponding processing is executed by each part of the control unit 404 described above (in the server 110). Figure 4

[0122] In S601, the control unit 404 receives a start trigger from the communication device 120. The start trigger indicates, for example, the start of utilization of a service that controls a vehicle based on the user's utterance. The start trigger is transmitted from the communication device 120, for example, in accordance with the user 130 starting an application for utilizing the service in the communication device 120, or uttering a predetermined phrase indicating the start of utilization of the service.

[0123] In S602, the control unit 404 determines a vehicle associated with the user 130. The control unit 404 determines the vehicle 100 closest to the user 130 based on, for example, the current positions of various vehicles grasped from time to time from the floating data transmitted from the vehicles, and the current position of the user 130. The method is not limited to this, and the vehicle designated by the user on the communication device 120 can be determined as the associated vehicle.

[0124] In S603, the control unit 404 acquires information for determining a utilization scenario from the determined vehicle 100. The information for determining a utilization scenario includes, for example, information on whether an occupant is in the vehicle, and information on whether the user 130 has gotten in the vehicle within a predetermined time. The information on whether an occupant is in the vehicle is obtained, for example, from the seat of the vehicle. Information on the occupant recognized by a camera provided in the vehicle can also be included.

[0125] In addition, in a case where this information is included in the floating data transmitted from the vehicle 100 to the server 110, this step can be omitted. In this case, the control unit 404 can acquire the information of the determined vehicle 100 from the floating data. Although Figure 6 ​Not illustrated, but in a case where other users are riding on the vehicle 100, the control unit 404 returns the processing to S602 and determines other vehicles.

[0126] In S604, the control unit 404 identifies a utilization scenario of the vehicle 100 by the user. The utilization scenario is identified according to the above {before getting on, while getting on, after getting off}. In a case where the occupant is not getting on and the user 130 has not gotten on within a predetermined time, the control unit 404 identifies the current utilization scenario as before getting on. In a case where the occupant is getting on on the vehicle and the occupant is the user 130, the current utilization scenario is determined as while getting on. In addition, in a case where the occupant is not getting on and the user 130 has gotten on within a predetermined time, the control unit 404 identifies the current utilization scenario as after getting off.

[0127] In S605, the control unit 404 selects a machine learning model according to the identified utilization scenario. The machine learning model is learned using learning data that is different for each corresponding utilization scenario. The learning data is, for example, utterance information of the user, a label of an intention of an utterance that becomes a correct answer, and further, a label indicating the corresponding utilization scenario. That is, the control unit 404 can learn the machine learning model for each utilization scenario by inputting only the learning data of the corresponding utilization scenario to the machine learning model when learning the machine learning model for each utilization scenario.

[0128] In S606, the control unit 404 determines whether or not the utterance information of the user is acquired. In a case where the control unit 404 acquires the utterance information of the user from the communication device 120, the processing proceeds to S607, and in a case where the control unit 404 does not acquire the utterance information of the user from the communication device 120, the processing returns to S606 and waits for the acquisition of the utterance information of the user.

[0129] In S607, the control unit 404 determines the intention of the utterance using the machine learning model selected in S605. Specifically, the control unit 404 performs an operation with respect to the above formula, calculates the output intention category argmax b(c t At this time, in a case where the utterance information after the new utilization scenario is identified, the user's utterance information performs the operation in the case of t = 1, and in a case where the utterance information after the new utilization scenario is not identified, the user's utterance information performs the operation in the case of t ≥ 2.

[0130] In S608, the control unit 404 transmits a control instruction corresponding to the intention of the utterance to the vehicle. The control unit 40 determines necessary information from the utterance information in order to determine the specific instruction content, for example, as described above, based on the inferred intention of the utterance. For example, in a case where the intention of the utterance information of the user is to ask to come to meet, information of where to go, when to meet, and the like is determined. The sound information processing section 417 can further include processing of slot filling. In addition, the control unit 404 transmits a control instruction to control the action of the vehicle 100 to the vehicle 100 based on the recognized utterance content. For example, in a case where information of where to go, when to meet, and the like is determined from the utterance information of the user, a route is determined based on the current position of the user and the vehicle, map information, and the like, and a control instruction to travel on the route is transmitted to the vehicle 100.

[0131] In S609, the control unit 404 determines whether or not the user operation has ended. The control unit 404 determines, for example, whether or not information indicating the end has been received from the communication device 120. For example, in the communication device 120, information indicating the end is transmitted from the communication device 120 in accordance with the user 130 uttering a predetermined phrase indicating the end of the use of the service or the like. In a case where the control unit 404 determines that the user operation has ended, the series of processing is ended, and in a case where the control unit 404 determines that the user operation has not ended, the processing is caused to return to S603 and the processing after S603 is repeated.

[0132] Further, in the above-described embodiment, a case where the server 110 recognizes the utilization scenario based on the information from the vehicle 100 is exemplified. However, the server 110 can recognize the utilization scenario based on other information. For example, the server 110 can recognize the utilization scenario based on the information from the communication device 120. For example, the communication device 120 can receive the start trigger transmitted from the communication device 120, the information indicating the occurrence of the approach to the vehicle, to recognize the utilization scenario. As described above, for example, in the communication device 120, the utilization trigger is transmitted from the communication device 120 in accordance with the user 130 starting the application for utilizing the above-described service, or uttering a predetermined phrase indicating the start of the utilization of the service, and the like. Further, for example, the user gets on the vehicle 100 and gets off the vehicle 100, causes the vehicle 100 to approach the communication device 120, and when the communication device 120 detects the approach to the vehicle by the proximity wireless communication or the like, transmits the information indicating the occurrence of the approach to the vehicle. For example, the server 110 recognizes the utilization scenario as before the getting on in a case where the information indicating the occurrence of the approach is not received after the reception of the start trigger, and after that, when the information indicating the occurrence of the approach is received, can recognize the utilization scenario as in the getting on. Further, when the information indicating the occurrence of the approach is received, can recognize the utilization scenario as after the getting off. Further, the utilization scenario can be recognized in the communication device 120 instead of the server 110 recognizing the utilization scenario, and the recognized utilization scenario can be transmitted to the server 110 in accordance with the switching of the utilization scenario.

[0133] As described above, in the above-described embodiment, in the information processing device capable of controlling the vehicle based on the indication of the utterance of the user, first, which one of a plurality of utilization scenarios when the object user utilizes the mobile body is recognized. After the recognition of the utilization scenario, a different machine learning model is selected in accordance with the recognized utilization scenario of the object user, and the selected machine learning model is used to estimate the intention of the utterance of the object user. By so doing, in the control of the mobile body by the utterance, the classification of the intention of the utterance is provided by the model constructed by the smaller scale learning.

[0134] (Modified Example)

[0135] Hereinafter, a modified example related to the present application will be described. In the above-described embodiment, an example in which the estimation processing of the intention of the utterance is executed in the server 110 is described. However, the above-described estimation processing of the intention of the utterance can be executed on the vehicle side. In this case, as in the above-described embodiment, the server 110 can recognize the utilization scenario of the object user in accordance with the information from the vehicle 100, and the vehicle 100 can execute the estimation processing of the intention of the utterance in accordance with the recognized utilization scenario. Figure 7As illustrated, the information processing system 700 is constituted by the vehicle 710 and the communication device 120. The utterance information of the user is transmitted from the communication device 120 to the vehicle 710. The vehicle 710 can have the same configuration as the vehicle 100 except that the control unit 30 can perform the inference processing of the utterance intention. The control unit 30 of the vehicle 710 functions as a control device in the vehicle 710 and performs the above-described inference processing of the utterance intention by executing the stored program. Figure 6 The interaction between the server and the vehicle in the series of actions illustrated can be performed inside the vehicle (for example, inside the control unit 30). The same can be performed with respect to other processing as well.

[0136] As such, in a control device capable of controlling a vehicle based on an indication of an utterance of a user, first, which one of a plurality of utilization scenarios of the object user is a utilization scenario when utilizing a mobile body is identified. After the utilization scenario is identified, a different machine learning model is selected according to the identified utilization scenario of the object user, and the intention of the utterance of the object user is inferred using the selected machine learning model. By so doing, in the control of the mobile body by the utterance, it is possible to provide classification of the utterance intention by a model constructed by a smaller scale of learning.

[0137] <Summary of Embodiments>

[0138] 1. The information processing device (for example, 110) of the above-described embodiment is an information processing device capable of controlling a mobile body (for example, 100) based on an indication of an utterance of a user,

[0139] The information processing device has:

[0140] an identification unit (for example, 414) that identifies which one of a plurality of utilization scenarios of an object user is a utilization scenario when utilizing a mobile body;

[0141] an acquisition unit (for example, 413) that acquires utterance information of the object user;

[0142] a selection unit (for example, 415) that selects a different machine learning model according to the identified utilization scenario of the object user; and

[0143] an inference unit (for example, 416) that infers an intention of the utterance of the object user using the selected machine learning model.

[0144] According to this embodiment, in the control of the mobile body by the utterance, it is possible to provide classification of the utterance intention by a model constructed by a smaller scale of learning.

[0145] 2. In the information processing device of the above-described embodiment,

[0146] The machine learning model estimates an intention category (e.g., 501, 502) different for each utilization scenario associated with the machine learning model.

[0147] According to this embodiment, it is possible to use a machine learning model suitable for each scenario.

[0148] 3. The information processing apparatus according to the above embodiment,

[0149] The estimation unit estimates the intention of the object user using a machine learning model that outputs a likelihood degree that involves only a part of all intention categories that can be associated with a plurality of utilization scenarios.

[0150] According to this embodiment, it is possible to use a model that outputs a small number of intention categories suitable for each scenario. That is, it is possible to make learning of the model easy.

[0151] 4. The information processing apparatus according to the above embodiment,

[0152] The estimation unit estimates the intention of the utterance of the object user by adding, in the output of the selected machine learning model, an operation using an initial state probability distribution set in the intention categories as a prior distribution.

[0153] According to this embodiment, it is possible to reflect common sense (the first voice of communication is a negative intention, or the possibility of a greeting intention during a conversation is low) related to an utterance depending on a scenario context.

[0154] 5. The information processing apparatus according to the above embodiment,

[0155] The initial state probability distribution set as a prior distribution is determined individually for each utilization scenario.

[0156] According to this embodiment, it is possible to reflect common sense related to an utterance for each scenario.

[0157] 6. The information processing apparatus according to the above embodiment,

[0158] The estimation unit estimates the intention of the utterance of the object user by adding, in the output of the selected machine learning model, an operation using a state transition probability distribution between intention categories.

[0159] According to this embodiment, it is possible to perform intention estimation that takes into account the order of transition of intentions in an actual dialogue.

[0160] 7. The information processing apparatus according to the above embodiment,

[0161] The state transition probability distribution is determined individually for each scenario.

[0162] According to this implementation, the state transition probabilities between intent categories in a scene can be determined individually.

[0163] 8. In the information processing apparatus of the above embodiments,

[0164] When the estimation unit estimates the intent of a speech at time t, it adds an estimation result to the output of the selected machine learning model to estimate the intent of the target user's speech in order to place the speech earlier than the speech at time t.

[0165] According to this implementation, the probability distribution of the speech up to time t-1 can be recursively considered.

[0166] 9. In the information processing apparatus of the above embodiments,

[0167] The machine learning model learns using different learning data for each corresponding application scenario. The learning data includes labels representing the application scenario.

[0168] According to this implementation method, machine learning models can learn from small-scale learning data for each use case, and the learning data can be easily classified according to the use cases for learning.

[0169] 10. In the information processing apparatus of the above embodiments,

[0170] Based on information from the mobile device associated with the target user, identify the scenario in which the target user is using the device.

[0171] According to this implementation, the utilization scenario is determined based on information provided by the moving body, i.e., the utilization object, thereby enabling the determination of the utilization scenario with high accuracy.

[0172] 11. The control device (e.g., 30) for the mobile body (e.g., 710) in the above embodiments is a control device for a mobile body capable of being controlled based on instructions given by the user.

[0173] The control device has:

[0174] The identification unit (e.g., 30, S604) identifies which of the multiple usage scenarios the user is using when using a mobile device;

[0175] The acquisition unit (e.g., 30, S606) acquires the speech information of the target user;

[0176] The selection unit (e.g., 30, S605) selects different machine learning models based on the identified user's usage scenario; and

[0177] An estimation unit (e.g., 30, S607) estimates the intention of the utterance of the subject user using the selected machine learning model.

[0178] According to this embodiment, in the control of the mobile body by the utterance, it is possible to provide classification of the utterance intention by a model constructed by a smaller scale of learning.

[0179] The present application is not limited to the above-described embodiments, and various modifications and changes can be made within the scope of the gist of the present application.

Claims

1. An information processing device capable of controlling a moving body based on user-spoken instructions, characterized in that, The information processing device has: The identification unit identifies which of the multiple usage scenarios of the user using the mobile device, and each of the multiple usage scenarios includes the user's state before boarding, during boarding, or after disembarking. The acquisition unit acquires the speech information of the target user; The selection unit selects different machine learning models from multiple machine learning models based on the identified usage scenario of the target user, and each of the multiple machine learning models is associated with any one of the pre-ride state, ride state, and post-disembarkation state of the mobile body. as well as The estimation unit uses a selected machine learning model to estimate the intent of the target user's speech.

2. The information processing device according to claim 1, characterized in that, For each use case associated with the machine learning model, the inferred intent category is different.

3. The information processing device according to claim 2, characterized in that, The estimation unit uses the machine learning model that outputs likelihood to estimate the intent of the target user, the likelihood referring only to a subset of all intent categories that can be associated with the multiple use cases.

4. The information processing apparatus according to claim 1, characterized in that, The estimation unit incorporates calculations using an initial state probability distribution pre-distributed within the intent category into the output of the selected machine learning model to estimate the intent of the target user's speech.

5. The information processing apparatus according to claim 4, characterized in that, The initial state probability distribution, which is set as a pre-distribution, is determined individually for each utilization scenario.

6. The information processing apparatus according to claim 1, characterized in that, The estimation unit incorporates calculations using the state transition probability distribution between intent categories into the output of the selected machine learning model to estimate the intent of the target user's speech.

7. The information processing apparatus according to claim 6, characterized in that, The state transition probability distribution is determined individually for each scenario.

8. The information processing apparatus according to claim 1, characterized in that, When the estimation unit estimates the intent of a speech at time t, it adds an estimation result to the output of the selected machine learning model to estimate the intent of the target user's speech in order to place the speech earlier than the speech at time t.

9. The information processing apparatus according to claim 1, characterized in that, The machine learning model learns using different learning data for each corresponding application scenario, and the learning data includes labels representing the application scenario.

10. The information processing apparatus according to claim 1, characterized in that, The identification unit identifies the scenario in which the target user is using the device based on information from the mobile body associated with the target user.

11. An information processing method, which is an information processing method in an information processing device capable of controlling a mobile body based on user speech instructions, characterized in that, The information processing method has the following characteristics: The identification step identifies which of the multiple usage scenarios the target user is using when using a mobile device, and each of the multiple usage scenarios includes the user's state before boarding, during boarding, or after disembarking from the mobile device. The acquisition step involves acquiring the speech information of the target user. In the selection step, different machine learning models are selected from multiple machine learning models based on the identified usage scenario of the target user, and each of the multiple machine learning models is associated with any one of the pre-ride state, ride state, and post-disembarkation state of the mobile body. as well as An inference step, in which a selected machine learning model is used to infer the intent of the subject user's speech.

12. A control device for a mobile body, the mobile body being controllable based on instructions spoken by a user, characterized in that, The control device has: The identification unit identifies which of the multiple usage scenarios of the user using the mobile device, and each of the multiple usage scenarios includes the user's state before boarding, during boarding, or after disembarking. The acquisition unit acquires the speech information of the target user; The selection unit selects different machine learning models from multiple machine learning models based on the identified usage scenario of the target user, and each of the multiple machine learning models is associated with any one of the pre-ride state, ride state, and post-disembarkation state of the mobile body. as well as The estimation unit uses a selected machine learning model to estimate the intent of the target user's speech.

13. A method for controlling a mobile body, the mobile body being controllable based on user-spoken instructions, characterized in that, The control method has the following characteristics: The identification step identifies which of the multiple usage scenarios the target user is using when using a mobile device, and each of the multiple usage scenarios includes the user's state before boarding, during boarding, or after disembarking from the mobile device. The acquisition step involves acquiring the speech information of the target user. In the selection step, different machine learning models are selected from multiple machine learning models based on the identified usage scenario of the target user, and each of the multiple machine learning models is associated with any one of the pre-ride state, ride state, and post-disembarkation state of the mobile body. as well as An inference step, in which a selected machine learning model is used to infer the intent of the subject user's speech.

14. A storage medium storing programs for enabling a computer to function as various units of an information processing device, characterized in that, The information processing device capable of controlling a moving body based on user-spoken instructions has the following features: The identification unit identifies which of the multiple usage scenarios of the user using the mobile device, and each of the multiple usage scenarios includes the user's state before boarding, during boarding, or after disembarking. The acquisition unit acquires the speech information of the target user; The selection unit selects different machine learning models from multiple machine learning models based on the identified usage scenario of the target user, and each of the multiple machine learning models is associated with any one of the pre-ride state, ride state, and post-disembarkation state of the mobile body. as well as The estimation unit uses a selected machine learning model to estimate the intent of the target user's speech.

15. A storage medium storing programs for enabling a computer to function as a control device, characterized in that, The control device for a mobile body capable of being controlled based on user-spoken instructions has: The identification unit identifies which of the multiple usage scenarios of the user using the mobile device, and each of the multiple usage scenarios includes the user's state before boarding, during boarding, or after disembarking. The acquisition unit acquires the speech information of the target user; The selection unit selects different machine learning models from multiple machine learning models based on the identified usage scenario of the target user, and each of the multiple machine learning models is associated with any one of the pre-ride state, ride state, and post-disembarkation state of the mobile body. as well as The estimation unit uses a selected machine learning model to estimate the intent of the target user's speech.

Images