Model generation method, model generation device, inference program and inference device

Abstract

To provide a technique that reduces the cost of generating a trained model that has obtained the ability to infer an action intention of a target.SOLUTION: A model generation device according to one aspect of the present invention acquires a training data set including a first training sample and a second training sample of observation data of a subject. The first training sample is labeled to indicate that the subject has an intention to take action. The second training sample is not labeled. The model generation device controls machine learning of an inference model by learning from positive examples using the acquired training data set and unlabeled data. The machine learning is configured by training the inference model to obtain the ability to infer from the observation data a degree of the intention of the subject to take action.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a model generation method, a model generation device, an inference program, and an inference device. [Background technology]

[0002] In recent years, various tools for remote communication, such as video conferencing and dialogue in virtual spaces, have become available. However, in remote communication via these tools, it is difficult to obtain the reactions (especially nonverbal reactions) of the other party, such as the audience, compared to real-life communication. This makes it difficult to understand the intention of the other party, which can result in communication stagnating, such as a one-sided conversation.

[0003] Therefore, development of technologies for visualizing the speech intention of the other party is progressing. For example, Non-Patent Document 1 proposes a method for generating a trained model that has acquired the ability to infer the speech intention from speech, text, and images. According to this method, the obtained trained model can be used to visualize the speech intention of the other party. As a result, it is expected that smoother communication can be achieved, such as encouraging the other party to take turns speaking at an appropriate time. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Ryo Ishii, et al. "Trimodal prediction of speaking and listening willingness to help improve turn-changing modeling", [online], [Retrieved March 5, 2024], Internet<URL:https: / / www.frontiersin.org / articles / 10.3389 / fpsyg.2022.774547 / full> Summary of the Invention [Problem to be solved by the invention]

[0005] The present inventors have found that the above-mentioned conventional methods have the following problems. In the conventional method, labels (teacher signals, ground truth data) are manually assigned to both positive and negative samples among the training samples. Then, by performing PN learning (supervised learning) using the obtained labeled training samples, a trained model capable of inferring speech intention is generated.

[0006] Possible methods for obtaining labels indicating the true value of this speech intention include self-reporting by the subject and assignment by an annotator other than the subject. However, the former method requires subjects to self-report answers while obtaining training samples, making it difficult to obtain labels retroactively. The annotation costs are high because of the answers obtained from each subject. Extraneous actions, including self-reporting answers, may prevent the acquisition of training samples that naturally express the speech intention. Furthermore, the latter method also requires the annotator to make judgments, which increases the annotation costs. Because speech intentions are difficult to express, there are limitations to the annotator's (other person's) ability to accurately determine the true value of a subject's speech intention (especially when there is no speech intention). Therefore, conventional methods may be expensive to generate trained models capable of inferring speech intentions.

[0007] It should be noted that situations where such problems arise are not limited to situations where speech intentions are inferred. The expression of intentions in actions other than speech can have the same properties as the expression of speech intentions. Therefore, similar problems can arise in situations where behavioral intentions other than speech intentions are inferred. Furthermore, in the sense that it is difficult to judge the degree of behavioral intentions externally, the behavioral intentions of any object, such as a device operated by a human or an automatically controlled robotic device, can have the same properties as the behavioral intentions of a person. Therefore, it is difficult to judge the degree of behavioral intentions externally. Similar problems can arise in any situation where behavioral intentions (intentions to take action) are inferred.

[0008] In one aspect, the present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique for reducing the cost of generating a trained model that has the ability to infer the behavioral intention of a subject, or for using the generated trained model. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems, the present invention employs the following configurations. Note that the following configurations of the invention can be combined as appropriate.

[0010] A model generation method according to one aspect of the present invention is an information processing method that executes the steps of: acquiring a training dataset including first and second training samples of observation data of a subject, wherein the first training sample is assigned a label indicating that the subject has an intention to take action, and the second training sample is not assigned the label; controlling machine learning of an inference model by learning from positive examples and unlabeled data using the acquired training dataset, wherein the machine learning is configured by training the inference model to acquire the ability to infer from the observation data the degree to which the subject has the intention to take the action; and outputting the results of the machine learning.

[0011] In this configuration, learning from positive examples and unlabeled data (PU learning) is adopted as a machine learning method for an inference model that infers a target's behavioral intention. Positive samples are obtained from scenes where there is an intention to take action. Negative samples are obtained from scenes where there is no intention to take action. Having an intention to take action may include having a high degree of intention to take action, and not having an intention to take action may include having a low degree of intention to take action. In this configuration, the first training sample corresponds to a sample labeled as positive (positive sample). The second training sample corresponds to an unlabeled sample. Because no label is assigned, it is unknown whether the second training sample is positive or negative.

[0012] A subject performs a behavior as a result of having an intention to perform that behavior. In other words, it is clear that the subject had an intention to perform the behavior immediately before the subject performed the behavior, so positive samples can be collected. On the other hand, at other times when the behavior does not occur, it is unclear whether the subject had an intention to perform the behavior. In other words, it is unclear whether the subject did not perform the behavior due to a lack of intention to perform the behavior or an intention to perform the behavior but a failure to act in accordance with that intention. Therefore, it is difficult to label samples obtained at that time. As a result, it is more difficult to collect negative samples (samples labeled as negative) than positive samples. In contrast, this configuration employs learning from the positive examples and unlabeled data, making it possible to generate a trained inference model without using such difficult-to-collect negative samples. This eliminates the need to collect negative samples, thereby reducing the cost of generating a trained model capable of inferring a subject's behavioral intention.

[0013] In the model generation method according to the above aspect, the target may be a person, and the behavior may be an utterance. This configuration can reduce costs when generating a trained model that has acquired the ability to infer the speech intention of the person.

[0014] In the model generation method according to the above aspect, the target may be a target person, and the behavior may be a behavior toward a robot device. This can reduce costs when generating trained models that have the ability to infer the behavioral intentions of subjects.

[0015] In the model generation method according to the above aspect, the target may be a mobile robot existing around a mobile object, and the behavior may be cutting in on the path of the mobile object. This configuration can reduce costs when generating a trained model that has acquired the ability to infer an intention of the mobile object to cut in on the path of the mobile object.

[0016] In the model generation method according to the above aspect, the first training sample may be extracted from a section of the observation data immediately preceding an occurrence section in which the subject is performing the behavior. This configuration allows for appropriate collection of first training samples (positive examples). As a result, appropriate generation of a trained inference model can be expected.

[0017] In the model generation method according to the above aspect, the second training sample may be extracted from a section of the observation data that precedes an occurrence section in which the subject is performing the behavior, excluding a section immediately preceding the occurrence section. This configuration makes it possible to appropriately collect the second training sample (unlabeled data). As a result, it is possible to expect appropriate generation of a trained inference model.

[0018] In the model generation method according to the above aspect, the target may be a human subject, and the observation data may be image data, sound data, text data, biometric data, device operation data, motion capture data, position data, or a combination thereof. According to this configuration, by using these types of data as observation data, it is possible to accurately grasp the state of the subject whose behavioral intention is to be inferred. As a result, it is expected that a trained inference model that appropriately infers behavioral intentions can be generated.

[0019] In the model generation method according to the above aspect, the target may be a robotic device, and the observation data may be composed of image data, sound data, text data, motion status data, position data, or a combination thereof. By using these types of data as the observation data, it is possible to accurately grasp the state of the robotic device whose behavioral intention is to be inferred. As a result, it is expected that a trained inference model that can appropriately infer behavioral intentions can be generated.

[0020] The present invention is not limited to the above-described model generation. One aspect of the present invention may be an inference program that uses a generated trained inference model. For example, the inference program according to one aspect of the present invention may be a program that causes a computer to execute the steps of: acquiring samples of observation data of a target; inferring, from the acquired samples of observation data, a degree to which the target intends to take action using the trained inference model; and outputting the inference results.

[0021] In the inference program according to the above aspect, the trained inference model may be generated by learning from positive examples and unlabeled data. With this configuration, the degree to which a target has a behavioral intention can be inferred using the trained inference model generated at low cost.

[0022] In the inference program according to the above aspect, the object may be a person, and the behavior may be an utterance. With this configuration, it is possible to infer the degree to which the person has an intention to speak.

[0023] In the inference program according to the above aspect, outputting the result of the inference The present invention may also include outputting an effect that visualizes the result of inferring the degree to which the target person has an intention to speak. With this configuration, visualization of the degree to which the target person has an intention to speak can be expected to facilitate communication.

[0024] In the inference program according to the above aspect, the observation data may include image data showing the face of the target person in a video conference. Outputting the effect may include outputting an effect of enlarging the target person's face to a terminal of a conversation partner of the target person in the video conference when the target person intends to speak. For example, explicit visualization effects such as changing the color of a frame impose a high cognitive load. Furthermore, it is difficult to infer the degree of speech intention without error. If an explicit visualization effect is used to indicate a speech intention in a situation where there is no actual speech intention, an error in the inference is explicitly revealed. This can easily undermine the reliability of the inference of the speech intention. In contrast, this configuration employs an implicit visualization effect of enlarging the target person's face, thereby suppressing an increase in cognitive load. Furthermore, it is possible to prevent an error in the inference from being explicitly revealed, which is expected to make it less likely that the reliability of the inference of the speech intention will be undermined.

[0025] In the inference program according to the above aspect, the target may be a target person, and the behavior may be a behavior toward a robotic device. With this configuration, it is possible to infer the degree to which the target person has a behavioral intention toward the robotic device.

[0026] In the inference program according to the above aspect, outputting the result of the inference may include selecting an action of the robotic device in accordance with the result of inferring the degree of intention to take action toward the robotic device, and outputting a command to the robotic device to instruct the robotic device to perform the selected action. With this configuration, the action of the robotic device can be controlled in accordance with the result of inferring the intention to take action toward the robotic device.

[0027] In the inference program according to the above aspect, the target may be a mobile robot present around the moving object, and the action may be cutting in on the path of the moving object. With this configuration, it is possible to infer the degree to which the mobile robot intends to cut in on the path of the moving object.

[0028] In the inference program according to the above aspect, outputting the inference result may include outputting an instruction to the moving object to change its path when the mobile robot has an intention to cut in. With this configuration, it is possible to control the operation of the moving object according to the result of inferring the intention of the moving object to cut in.

[0029] Note that the present invention is not limited to the above-described model generation method (information processing method). As another aspect of the model generation method according to each of the above aspects, one aspect of the present invention may be an information processing device (model generation device) that realizes all or part of the above-described configurations, a program, or a storage medium readable by a machine such as a computer on which such a program is stored. A storage medium readable by a machine such as a computer is a medium that stores information such as a program by electrical, magnetic, optical, mechanical, or chemical action. Similarly, the present invention is not limited to the above-described inference program. As another aspect of the inference program according to each of the above aspects, one aspect of the present invention may be an information processing device (inference device) that realizes all or part of the above-described configurations, an information processing method (inference method), or a storage medium readable by a machine such as a computer on which a program is stored.

[0030] For example, the model generating device according to one aspect of the present invention may include a control unit. The system may be configured to execute the steps of: acquiring a training dataset including first and second training samples of observation data of a subject, wherein the first training sample is assigned a label indicating that the subject has an intention to take the action, and the second training sample is not assigned the label; controlling machine learning of an inference model by learning from positive examples and unlabeled data using the acquired training dataset, wherein the machine learning is configured by training the inference model to acquire the ability to infer from the observation data the degree to which the subject has the intention to take the action; and outputting the results of the machine learning.

[0031] Also, for example, an inference device according to one aspect of the present invention may include a control unit. The control unit may be configured to execute the steps of acquiring samples of observation data of a subject, inferring the degree to which the subject intends to take action from the acquired samples of observation data using a trained inference model, and outputting the inference result. The trained inference model may be generated by learning from positive examples and unlabeled data. [Effects of the Invention]

[0032] According to the present invention, it is possible to provide a technique for reducing the cost of generating a trained model that has the ability to infer the behavioral intention of a subject, or for using a generated trained model. [Brief explanation of the drawings]

[0033] [Figure 1] FIG. 1 shows a schematic diagram of an example of a situation in which the present invention is applied. [Figure 2A] FIG. 2A schematically illustrates an example of a scene in which a behavioral intention is inferred according to the embodiment. [Figure 2B] FIG. 2B schematically illustrates an example of a scene in which a behavioral intention is inferred according to the embodiment. [Figure 2C]FIG. 2C schematically illustrates an example of a scene in which a behavioral intention is inferred according to the embodiment. [Figure 3] FIG. 3 is a schematic diagram showing a specific example of a scene in which a behavioral intention (utterance intention) is inferred according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of an effect for visualizing the inference result of the utterance intention according to the embodiment. [Figure 5] FIG. 5 is a schematic diagram showing a specific example of a scene in which a behavioral intention (a behavioral intention toward a robot device) according to the embodiment is inferred. [Figure 6] FIG. 6 is a schematic diagram showing a specific example of a situation in which a behavioral intention (intention to cut in on a route of a moving object) is inferred according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a method for acquiring training samples according to an embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a hardware configuration of a model generating device according to an embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a hardware configuration of an inference device according to an embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a software configuration of a model generating device according to an embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of the software configuration of an inference device according to an embodiment. [Figure 12] FIG. 12 is a flowchart illustrating an example of a processing procedure of the model generating device according to the embodiment. [Figure 13] FIG. 13 is a flowchart illustrating an example of a processing procedure of the inference device according to the embodiment. [Figure 14] Figure 14 shows the structure of the inference model employed in the experiment. [Figure 15] FIG. 15 shows the configuration of the prototype used in the examples. [Figure 16A] FIG. 16A shows the calculation results of the speech count ratio (RSTurn) in the first embodiment, the second embodiment, and the comparative example. [Figure 16B]FIG. 16B shows the calculation results of the speech time ratio (RSTime) in the first embodiment, the second embodiment, and the comparative example. [Figure 16C] FIG. 16C shows the calculation results of the difference between the speech count ratio (RSTurn) and speech time ratio (RSTime) and the ideal value (0.25) in the first embodiment, the second embodiment, and the comparative example. DETAILED DESCRIPTION OF THE INVENTION

[0034] An embodiment according to one aspect of the present invention (hereinafter also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present invention in all respects. Various improvements or modifications may be made without departing from the scope of the present invention. In implementing the present invention, a specific configuration according to the embodiment may be appropriately adopted. Note that while data appearing in the present embodiment is described in natural language, more specifically, it is specified using computer-recognizable pseudo-language, commands, parameters, machine language, etc.

[0035] §1 Application Examples 1 is a diagram showing an example of a situation in which the present invention is applied. The system according to this embodiment comprises a model generation device 1 and an inference device 2.

[0036] The model generation device 1 according to this embodiment is one or more computers configured to generate a trained inference model 50 by controlling the implementation of machine learning. First, the model generation device 1 acquires a training dataset 30 including a first training sample 31 and a second training sample 35 of observation data of a target ST. A label 32 indicating that the target ST has an intention to take action (behavioral intention) is assigned to the first training sample 31. On the other hand, no label is assigned to the second training sample 35.

[0037] The first training sample 31 corresponds to a positive sample with a label (label 32), and the second training sample 35 corresponds to an unlabeled sample. Because no label is assigned, it is unclear whether the second training sample 35 is positive or negative. The second training sample 35 includes negative samples and unobserved positive samples. Note that the positive samples are samples obtained from scenes where there is an intention to take action, and the negative samples are samples obtained from scenes where there is no intention to take action. Having an intention to take action may include having a high degree of intention to take action, and not having an intention to take action may include having a low degree of intention to take action.

[0038] The model generation device 1 generates an inference model 50 by learning from positive examples and unlabeled data (PU learning: Positive-Unlabeled Learning) using the acquired training dataset 30. The model generation device 1 controls machine learning. The machine learning is configured by training the inference model 50 to acquire the ability to infer the degree to which the target ST intends to take action from observed data. By implementing this machine learning, a trained inference model 50 can be generated. The model generation device 1 outputs the results of the machine learning.

[0039] On the other hand, the inference device 2 according to this embodiment is one or more computers configured to infer the degree to which a target has a behavioral intention using a trained inference model 60. First, the inference device 2 acquires a sample 41 of observation data of the target TT (target sample). The inference device 2 then uses the trained inference model 60 to infer the degree to which a target has a behavioral intention from the acquired sample 41 of observation data. The inference device 2 infers the degree to which the elephant TT intends to take action. The inference device 2 outputs the inference results. This series of inference processes may be executed in real time or may be executed after the fact. Note that the name of the inference device 2 may be changed as appropriate depending on the content of the inference.

[0040] As mentioned above, it is possible to collect positive samples just before a subject takes action. On the other hand, at other times when no action is taking place, it is unclear whether the subject intends to take action. Therefore, it is difficult to label samples obtained at those times. As a result, it is more difficult to collect negative samples than positive samples.

[0041] In contrast, the model generation device 1 according to this embodiment employs PU learning, which uses unlabeled data (second training samples 35) along with positive samples (first training samples 31 and labels 32), as a machine learning method for the inference model 50. By employing this PU learning, a trained inference model 50 can be generated without using negative samples, which are difficult to collect. This allows the collection of negative samples to be at least partially omitted. This reduces the cost of generating a trained inference model 50 that has acquired the ability to infer the behavioral intention of a target (target ST).

[0042] Furthermore, the inference device 2 according to this embodiment can infer the behavioral intention of the target TT by using a trained inference model 60. In one example, the trained inference model 60 may be generated by learning from positive examples and unlabeled data. This allows the trained inference model 60 generated at low cost to be used to infer the behavioral intention of the target TT. Note that, in one example, the trained inference model 60 may be the trained inference model 50 generated by the model generation device 1. In this case, the trained inference model 50 may be provided from the model generation device 1 to the inference device 2 at any timing and in any manner.

[0043] (subject) The targets (ST, TT) may be selected appropriately depending on the embodiment. The targets (ST, TT) may be at least one of a living organism and a non-living organism. In one example, the targets (ST, TT) may be at least one of a subject and an object. The targets may include a robotic device. The type of robotic device is not particularly limited and may be selected appropriately depending on the embodiment. The robotic device may be, for example, an industrial robot in a production line, an autonomous robot configured to be able to operate autonomously, or a mobile robot configured to be able to move. The industrial robot may be, for example, a vertical articulated robot, a horizontal articulated robot (SCARA robot), a parallel link robot, or an orthogonal robot. The autonomous robot may be, for example, a humanoid robot, a guide robot, an agricultural robot, a nursing care robot, a security robot, or a transport robot. The content of the autonomous processing may be selected appropriately depending on the embodiment. The mobile robot may include, for example, a cleaning robot, the above-mentioned autonomous robot (including a mobile robot) configured to be movable, a vehicle configured to be self-operated, an air vehicle capable of self-flying (such as a drone), a ship capable of self-navigation (such as a ship or submarine), etc. The robot device may be operated by a person. For example, the mobile robot may include a vehicle configured to be manually operated, an air vehicle capable of manual flight, a ship capable of manual navigation, etc. The living thing may be a human or a living thing other than a human. The type of living thing may be selected appropriately depending on the embodiment.

[0044] The individual of the target TT at the time of inference may or may not be included in the individual of the target ST at the time of learning. In other words, one individual of the target TT may or may not match one individual of the target ST. Basically, the type of the target TT at the time of inference may match the type of the target ST at the time of learning. However, the type of the target TT at the time of inference and the type of the target ST at the time of learning may be different. The relationship between T does not need to be limited to this example. When using the trained inference model 50 generated by the model generation device 1 as the trained inference model 60 in the inference stage, the type of target TT at the time of inference does not necessarily have to match the type of target ST at the time of learning, as long as the trained inference model 50 is applicable to the inference stage. Applicability to the inference stage may mean that the inference accuracy does not deteriorate to the extent that the trained inference model 50 becomes unusable.

[0045] (Observation data) The type of observation data may be selected appropriately depending on the content of inference. In one example, the observation data may be image data, sound data, text data, numerical data, or a combination thereof. Samples of the observation data (first training sample 31, second training sample 35, sample 41) may be acquired by one or more sensors S. The type of sensor S may be selected appropriately depending on the embodiment. The sensor S may include, for example, an image sensor, a microphone, an acceleration sensor, an angular acceleration sensor, a gyro sensor, a motion capture device, a blood pressure monitor, a pulse rate monitor, a heart rate monitor, an electrocardiograph, an electromyograph, a thermometer, a skin electrodermal response monitor, an electroencephalograph, a magnetoencephalograph, an activity monitor, a blood glucose monitor, an electrooculography sensor, an eye movement monitor, a wearable device (such as a smart watch), a position sensor, a microwave sensor, etc. The image sensor may include any sensor that acquires data in the form of an image or an image representation, such as, for example, an RGB camera, a depth sensor, an infrared sensor, a radar, a LiDAR (light detection and ranging), etc. The position sensor may include, for example, a Global Positioning System (GPS) sensor, a Global Navigation Satellite System (GNSS) sensor, and the like.

[0046] When a sensor S is used to acquire at least a portion of the observation data, at least a portion of the observation data (samples) may consist of sensing data obtained by the sensor S, or may consist of analysis data (analysis results) obtained by analyzing the sensing data. The analysis may include, for example, extraction of data features, calculation of feature amounts, etc. The feature extraction may include inference (regression or identification) such as voice analysis or facial expression identification. The feature amounts may include, for example, statistics such as maximum value, minimum value, median, mean value, variance, standard deviation, and n-th percentile value. The model generation device 1 and the inference device 2 may acquire samples of observation data directly from the sensor S, or indirectly via another computer.

[0047] The location of the sensor S is not particularly limited and may be selected appropriately depending on the embodiment. For example, in a situation where the behavioral intention of a subject is inferred, at least a part of the sensor S may be attached to the subject, or the subject may be observed from outside. Furthermore, for example, in a situation where the behavioral intention of a robot device is inferred, at least a part of the sensor S may be built into the robot device, attached to the robot device, or the robot device may be observed from outside.

[0048] (action) The behavior may be at least one of an observable action and a reaction. The type of behavior to be inferred is not particularly limited and may be appropriately selected depending on the embodiment. Note that a state in which there is an intention to perform a behavior (having an intention to perform a behavior) may correspond to a state in which the behavior has not been performed but is about to be performed. When the degree of behavioral intention becomes high, the probability of performing the behavior increases, but the behavior does not necessarily occur. Even when the degree of behavioral intention is high, the behavior may not be performed. Therefore, as described above, when a behavior is performed, it can be inferred that the degree of the target's behavioral intention was high immediately before the behavior. In other words, it is easy to label (annotate) samples obtained immediately before the behavior. On the other hand, accurate labeling (annotation) is difficult for samples obtained at other times because there are no such clues.

[0049] 2A to 2C schematically illustrate an example of a scene in which a behavioral intention is inferred according to the present embodiment. In the example of FIG. 2A, a scene is assumed in which an interaction occurs between one or more persons T1 and one or more persons T2. The type of interaction between the persons (T1, T2) may be selected arbitrarily. For example, remote communication, face-to-face conversation, or other communication is an example of the interaction. In this scene, at least one of the persons T1 and T2 may be a target person whose behavioral intention is to be inferred. That is, at least one of the persons T1 and T2 may be a target (ST, TT). In one example, the state of the target person (at least one of the persons T1 and T2) may be observed by a sensor S. For example, the system according to the present embodiment may be used to infer the behavioral intention of one of the persons T1 and T2 (the target person) and provide information about the inference result to the other person.

[0050] In the example of FIG. 2B , a scene is assumed in which one or more humans T3 and one or more robotic devices T4 interact with each other. The type of interaction between the humans T3 and the robotic devices T4 may be selected arbitrarily. For example, the robotic device T4 may be a mobile robot such as an autonomously or manually driven vehicle. In one example, the human T3 may be a passenger of the mobile robot. In this case, the interaction may relate to the movement of the mobile robot, such as automatic or manual control. In another example, the human T3 may be a passerby walking around the mobile robot. In this case, the interaction may relate to the movement of the mobile robot and the passerby, such as cutting into a path. In another example, the robotic device T4 may be configured to be interactive, and the human T3 may be a user of the robotic device T4. In this case, the interaction may be a dialogue between the robotic device T4 and the human T3. The dialogue may include a question to the robotic device T4. Furthermore, for example, the robotic device T4 may be an industrial robot, and the person T3 may be a worker who collaborates with the industrial robot. In this case, the interaction may be related to collaborative work on a production line or the like.

[0051] In this scenario, at least one of the human T3 and the robotic device T4 may be the target (ST, TT) whose behavioral intention is to be inferred. In one example, the human T3 may be the target whose behavioral intention is to be inferred. That is, the human T3 may be the target (ST, TT). In one example of this case, the state of the target (human T3) may be observed by a sensor S. For example, the system according to this embodiment may be used to infer the behavioral intention of the human T3 (subject) and reflect the inference result in the control of the robotic device T4. In another example, the robotic device T4 may be the target (ST, TT). In one example of this case, the state of the robotic device T4 may be observed by a sensor S. For example, the system according to this embodiment may be used to infer the behavioral intention of the robotic device T4 and provide information about the inference result to the human T3.

[0052] In the example of FIG. 2C , a scenario is assumed in which one or more robotic devices T5 and one or more robotic devices T6 interact with each other. The type of interaction between the robotic devices (T5, T6) may be selected arbitrarily. The interaction may include, for example, collaborative work on a production line, interruption of a route, etc. In this scenario, at least one of the robotic devices T5 and T6 may be a target (ST, TT) whose behavioral intention is inferred. In one example, the state of the robotic device (at least one of the robotic devices T5 and T6) may be observed by a sensor S. For example, the system according to this embodiment may be used to infer the behavioral intention of one of the robotic devices T5 and T6 and reflect the inference result in the control of the other.

[0053] In one example, as shown in parts of Figures 2A and 2B, the target (ST, TT) may be a human subject. In this case, the observation data may be image data, sound data, text data, biometric data, device operation data, motion capture data, position data, or a combination thereof. The biometric data may be a combination of various data. The biometric data may include, for example, physiological data such as respiration, blood pressure, pulse rate, heart rate, electrocardiogram, electromyogram, body temperature, electrodermal response, electroencephalogram, magnetoencephalogram, activity level, blood glucose level, electrooculography, eye movement, gaze, and other vital signs. The biometric data may also include, for example, physical movements, facial expressions, emotions, and the like. Physical movements may include, for example, posture, gestures, and the like. The type of operation data may be determined appropriately depending on the device. The type of device may be selected arbitrarily. In one example, the device may include a user terminal. In this case, the operation data may include, for example, information on an application being operated on the user terminal, operation of an input device, the result of the operation, device switching, and the like. The input device may include, for example, a mouse, keyboard, touch panel, operator, and the like. Device switching is performed between multiple user terminals, for example, switching from a laptop (personal computer) to a smartphone, and the like. In this embodiment, multiple user terminals usable by the same user may be considered as one user terminal. In one example, one or more user terminals may include an inference device 2. In this case, at least a portion of the operation data may be acquired in response to an operation on the inference device 2. In another example, the user terminal may be a computer other than the inference device 2. In this case, the operation data may be acquired as appropriate from the other computer. The configuration of the position data is not particularly limited as long as it includes information regarding the position of the subject, and may be selected as appropriate depending on the embodiment. For example, the position data may be configured to indicate a location (e.g., coordinates) where the subject is located, or may be configured to indicate whether the subject is located within a predetermined range.

[0054] According to one example of this embodiment, by employing these types of data as observation data, it is possible to accurately grasp the state of the subject whose behavioral intention is to be inferred. As a result, the model generation device 1 can be expected to generate a trained inference model 50 that appropriately infers behavioral intentions. The inference device 2 can be expected to appropriately infer the behavioral intention of the subject (subject TT). Note that the number of subjects is not particularly limited and may be selected appropriately depending on the embodiment. In one example, the behavioral intention of a single subject may be inferred. In another example, the behavioral intention of a group consisting of multiple subjects may be inferred.

[0055] In one example, as shown in part of FIG. 2B and FIG. 2C, the target (ST, TT) may be a robotic device. In this case, the observation data may be composed of image data, sound data, text data, motion status data, position data, or a combination thereof. The motion status data may have any configuration, as long as it includes information about the motion status of the robotic device, and may be appropriately selected depending on the embodiment. In one example, the motion status data may include information about communication volume, power consumption, internal status (joint angles, temperature, etc.), etc. In the case where an operator of the robotic device is present, the motion status data may include image data, sound data, text data, biometric data, device operation data, motion capture data, position data, or a combination thereof, related to the operator.

[0056] According to one example of the present embodiment, by employing these types of data as observation data, it is possible to accurately grasp the state of the robot device whose behavioral intention is to be inferred. As a result, the model generation device 1 can be expected to generate a trained inference model 50 that appropriately infers behavioral intentions. The inference device 2 can be expected to appropriately infer the behavioral intention of the target robot device (target TT). The number of robot devices is not particularly limited and may be selected appropriately depending on the embodiment. In one example, the behavioral intention of a single robot device may be inferred. In another example, the behavioral intention of a robot group consisting of multiple robot devices may be inferred. Furthermore, the type of robot device is not particularly limited and may be selected appropriately depending on the embodiment.

[0057] The above-mentioned method of inferring the behavioral intention of a robot device is applicable, for example, to a situation where a hierarchical command system based on a subsumption architecture exists in the robot device, or when an unanalyzable robot device is used. The present invention may be adopted in situations where the behavioral intention of a robot device following manual operation is to be inferred. Assume a situation where the behavioral intention to be inferred is that of either the operator (subject) or the robot device. For example, when the behavioral intention is inferred solely from the observation data of the operator, the inferred behavioral intention may be considered to be that of the operator (subject) (part of FIG. 2B). On the other hand, when the behavioral intention is inferred solely from the observation data of the robot device, the inferred behavioral intention may be considered to be that of the robot device (part of FIG. 2B and FIG. 2C). Furthermore, the situation where the behavioral intention is inferred is not limited to the examples of FIGS. 2A to 2C. The target and the behavior whose behavioral intention is to be inferred may each be selected appropriately depending on the embodiment.

[0058] (Example scene) The system according to this embodiment may be applied to various situations in which the behavioral intention of an arbitrary target is inferred. As an example, the system according to this embodiment may be applied to at least one of the following three situations. However, the application of this embodiment is not limited to the following three examples, and may be selected appropriately depending on the embodiment.

[0059] (1) Speech FIG. 3 schematically illustrates a specific example of a scene in which a behavioral intention (utterance intention) is inferred according to this embodiment. The example of FIG. 3 assumes a scene in which a speaker PU is speaking and a target person T10 is listening to the speaker PU's speech during remote communication via video conference. The video conference may include a video chat, a video call, etc. The video conference may also be referred to as a web conference or a television conference. The video conference may be used in any scene in which remote dialogue is conducted, such as a general meeting or telemedicine. The speaker PU is a user speaking in the video conference, and the target person T10 is a user listening to the speech. The speaker PU and the target person T10 may be interchanged as appropriate between multiple users. The scene in FIG. 3 is an example of the scene in FIG. 2A above.

[0060] In the learning stage, the target ST may be a subject T10. Each training sample (31, 35) may be appropriately collected from a subject of the subject T10. The behavior to be inferred may be an utterance. That is, the intention to perform an action may be an intention to speak. According to one example of this embodiment, it is possible to reduce the cost of generating a trained inference model 50 that has acquired the ability to infer the speech intention of a subject (subject T10).

[0061] Similarly, in the inference stage, the target TT may be the subject T10. The sample 41 may be appropriately acquired from a relevant individual (relevant user) of the subject T10. The behavior to be inferred may be an utterance. According to one example of the present embodiment, the trained inference model 60 can be used to infer the degree to which the subject (the subject T10) has a speech intention.

[0062] As shown in FIG. 3 , in the inference stage, speaker PU may participate in the video conference using user terminal 201, and target person T10 may participate in the video conference using user terminal 203. Video conference communication may be conducted directly between user terminals (201, 203) or indirectly via server device 202. When video conference communication is conducted directly between user terminals (201, 203), at least one of user terminals (201, 203) may be an example of inference device 2. When video conference communication is conducted via server device 202, at least one of user terminals (201, 203) and server device 202 may be an example of inference device 2.

[0063] For example, the user terminal 201 may acquire a sample 41 of observation data of the subject T10, and execute a series of inference processes using the trained inference model 60 to infer the degree to which the subject T10 has a speech intention from the acquired sample 41. The user terminal 201 may provide the inference results to at least one of the speaker PU and the user terminal 203 (subject T10). Also, for example, the server device 202 may execute the series of inference processes. The server device 202 The inference result may be provided to at least one of the user terminal 201 of the speaker PU and the user terminal 203 of the subject T10. Also, for example, the user terminal 203 may execute a series of inference processes. The user terminal 203 may provide the inference result to at least one of the subject T10 and the user terminal 201 (speaker PU).

[0064] At least one of the user terminal 203 and the sensor S1 may be used to acquire the observation data of the subject T10. In one example, as shown in FIG. 3, the sensor S1 may be provided in the user terminal 203 or connected to the user terminal 203. In another example, the sensor S1 may be provided in a computer other than the user terminal 203 or connected to the other computer. In this case, at least a portion of the observation data obtained by the sensor S1 may be provided via the other computer. The sensor S1 is an example of the above-mentioned sensor S. The type of the sensor S1 may be the same as that of the above-mentioned sensor S.

[0065] The observation data may be appropriately selected to enable inference of the speech intention of the subject T10. In one example, similar to some of the cases in FIGS. 2A and 2B above, the observation data may be composed of image data, sound data, text data, biometric data, device operation data, motion capture data, position data, or a combination thereof related to the subject T10. When image data is used as at least a portion of the observation data, the body parts depicted in the image data may be appropriately selected depending on the embodiment. In one example, the image data may be acquired to depict at least one of the face, hands, the entire body, and other body parts. For example, the image data may be acquired to depict facial states that may be related to the speech intention, such as gaze, facial expression, facial direction, facial position, and nodding. For example, the image data may be acquired to depict hand gestures that may be related to the speech intention, such as bringing a hand to the mouth or around the face. For example, the image data may be acquired to depict body movements that may be related to the speech intention, such as hand movements when sitting, body swaying, mirroring, crossing legs, and walking. When sound data is used as at least a part of the observation data, the sound data may be acquired to include at least one of the voice of the subject T10 and environmental sounds. The voice may include, for example, interjections, laughter, etc. For example, the sound data may reflect the voice style of the subject T10, such as variations in pitch and intonation. When text data is used as at least a part of the observation data, the configuration of the text data may be determined appropriately depending on the embodiment. For example, the text data may include the content of speech by the subject T10. The text of the content of speech may be acquired by any method, such as voice analysis. When location data is used as at least a part of the observation data, the location data may be configured appropriately to include information regarding the location of the subject T10. For example, the location data may be configured to indicate the distance to a video conference screen (e.g., a display of the user terminal 203, etc.). The location data may be acquired by analyzing image data acquired by an image sensor, or may be acquired by a sensor other than an image sensor (e.g., a microwave sensor, etc.).The observation data may be composed of the various data themselves, or may be composed of features obtained by converting the various data, such as facial features.

[0066] In the example of Fig. 3, outputting the inference result may be configured by outputting the result of inferring the degree to which the subject T10 has a speech intention. The form of the output may be determined arbitrarily.

[0067] In one example, outputting the inference result may include outputting an effect that visualizes the inference result regarding the degree to which the subject T10 has an intention to speak. According to this example of the present embodiment, visualizing the degree to which the subject T10 has an intention to speak can be expected to facilitate remote communication.

[0068] The output destination of the effect may be selected appropriately depending on the embodiment. The effect may be output to at least one of the user terminal 201 of the speaker PU and the user terminal 203 of the target person T10. By outputting the effect to the user terminal 201 of the speaker PU, the degree of the target person T10's speech intention can be notified to the speaker PU, thereby encouraging the target person T10 to take a turn.

[0069] Furthermore, the output timing of the effect may be determined appropriately depending on the embodiment. In one example, when it is estimated that the subject T10 has a speech intention, an effect visualizing the degree to which the subject T10 has a speech intention may be output. On the other hand, when it is estimated that the subject T10 does not have a speech intention, the output of the effect visualizing the degree to which the subject T10 has a speech intention may be omitted. Having a speech intention may include the degree to which the estimated speech intention is higher than a threshold, and not having a speech intention may include the degree to which the estimated speech intention is lower than the threshold. The degree to which the estimated speech intention is equal to the threshold may be determined to be either having a speech intention or not having a speech intention. In another example, an effect visualizing the degree to which the subject T10 has a speech intention may be output without distinguishing the degree to which the subject T10 has a speech intention.

[0070] Furthermore, as long as the degree of speech intention is visualized, the configuration of the effect is not particularly limited and may be determined appropriately depending on the embodiment. In one example, the effect may be configured to implicitly visualize the result of inferring the degree of speech intention of the subject T10. Implicit visualization may be visualization using indirect expressions such as metaphors, for example.

[0071] FIG. 4 schematically illustrates an example of an effect for implicitly visualizing the inference result of the speech intention according to this embodiment. In the example of FIG. 4, the observation data may include image data showing the face of the target person T10 in the video conference. Outputting the effect may include outputting an effect for enlarging the face of the target person T10 to a terminal of a conversation partner of the target person T10 in the video conference when the target person T10 intends to speak. The speaker PU is an example of the conversation partner of the target person T10, and the user terminal 201 is an example of the conversation partner's terminal. In the example of FIG. 4, the face enlarging effect is output to the display 2015 of the user terminal 201. Note that, in one example, the face enlarging effect may also be output to the user terminal 203 (i.e., may also be fed back to the target person T10).

[0072] The process of enlarging the face may be executed by at least one of the user terminals (201, 203) and the server device 202. When at least one of the server device 202 and the user terminal 203 is an example of the inference device 2, outputting a face enlarging effect may include enlarging, by image processing, an image of the face of the subject T10 appearing in image data acquired by the user terminal 203, and sending the image data of the enlarged face to the user terminal 201, or sending an instruction to enlarge the face to the user terminal 201. When the user terminal 201 is an example of the inference device 2, outputting a face enlarging effect by the user terminal 201 may include enlarging, by image processing, an image of the face of the subject T10 appearing in image data received directly from the user terminal 203 or indirectly via the server device 202, and outputting the image data of the enlarged face to the display 2015. The image processing of enlarging the face image may be realized by any method. For example, the image processing for enlarging the face image may be realized by a general image processing method such as edge extraction, or may be realized by a method using a trained machine learning model such as a generative model. The degree of enlarging the face may be variable or constant depending on the degree of speech intention.

[0073] For example, explicit visualization effects such as changing the frame color impose a high cognitive load. In addition, it is difficult to infer the degree of speech intention without error. If an explicit visualization effect is used to indicate speech intention in a scene where there is no actual speech intention, the inference becomes difficult. The error will be explicitly revealed. This will likely undermine the reliability of the inference of the utterance intention. In contrast, according to one example of the present embodiment, by adopting an implicit visualization effect of enlarging the face of the subject T10, it is possible to suppress an increase in cognitive load. Furthermore, it is possible to prevent the error in the inference from being explicitly revealed, which is expected to make it difficult for the reliability of the inference of the utterance intention to be undermined.

[0074] The implicit visualization effect does not have to be limited to the effect of enlarging the face. For example, signs of speech intention are known to include (A) leaning forward, (B) uncrossing one's arms, (C) turning toward the speaker (speaker PU), (D) directing one's gaze toward the speaker (speaker PU), and (E) nodding conspicuously. The face enlargement effect corresponds to (A) leaning forward. In another example, the implicit visualization effect may be composed of at least one of the following effects: crossing the subject T10's arms and then uncrossing them; turning the subject T10's face toward the speaker PU; directing the subject T10's gaze toward the speaker PU; and nodding. Each effect corresponds to the above signs (B) to (E). Each effect may be generated appropriately by image synthesis. This form of adopting implicit visualization effects can be applied not only when a trained inference model (such as trained inference model 50) generated by learning from positive examples and unlabeled data is used as the trained inference model 60, but also when other trained inference models are used.

[0075] Furthermore, the application of the above speech form need not be limited to video conference situations. In another example, the above speech form may be applied to other remote communications, such as communications via an avatar in a virtual space. In this case, the visualization effect (e.g., enlarging the face of the avatar of the subject T10) may be applied to the avatar of the subject T10. In another example, the speech form may be applied to communication situations other than remote communications, such as conversations in a real space.

[0076] (2) Actions toward robotic devices FIG. 5 schematically shows a specific example of an inferred scene of a behavioral intention (behavioral intention toward a robotic device) according to this embodiment. The example of FIG. 5 assumes a scene in which an interaction takes place between a target person T20 and a robotic device R20. The target person T20 is a user who interacts with the robotic device R20. The scene in FIG. 5 is an example of the scene in FIG. 2B above. As described above, the interaction may be related to a movement operation for a mobile robot, a route interruption, a dialogue, a collaborative work, etc.

[0077] In the learning stage, the subject ST may be the subject T20. Each training sample (31, 35) may be appropriately collected from a test subject of the subject T20. The behavior to be inferred may be behavior toward the robotic device R20. The behavior toward the robotic device R20 may include a reaction to an operation of the robotic device R20. According to one example of the present embodiment, it is possible to reduce the cost of generating a trained inference model 50 that has acquired the ability to infer the behavioral intention of a subject (subject T20) toward a robotic device (robot device R20).

[0078] Similarly, in the inference stage, the target TT may be the subject T20. The sample 41 may be appropriately acquired from a relevant individual (relevant user) of the subject T20. The behavior to be inferred may be behavior toward the robotic device R20. According to one example of the present embodiment, the trained inference model 60 can be used to infer the degree to which the subject (subject T20) has a behavioral intention toward the robotic device (robot device R20).

[0079] As shown in FIG. 5, in the inference stage, the operation of the robot device R20 may be controlled by a control device 205. In this situation, the control device 205 is an example of the inference device 2. In response to this, the control device 205 may acquire a sample 41 of observation data of the subject T20 and use the trained inference model 60 to infer the degree to which the subject T20 has a behavioral intention from the acquired sample 41. The control device 205 may then control the operation of the robot device R20 in accordance with the obtained inference result.

[0080] 5, the form of the inference device 2 need not be limited to this example. A computer other than the control device 205 may be an example of the inference device 2, and the other computer may provide the control device 205 with an inference result about the behavioral intention of the subject T20, thereby instructing the control device 205 to control the behavior of the robot device R20 in accordance with the inference result. In addition, if the inference result about the behavioral intention of the subject T20 is not reflected in the behavioral control of the robot device R20, another computer unrelated to the control device 205 may be an example of the inference device 2.

[0081] At least one of a sensor S2, a control device 205, and an external computer may be used to acquire observation data of the subject T20. In one example, as shown in FIG. 5, the sensor S2 may be provided in the control device 205, may be connected to the control device 205, or may be provided in the robot device R20. In another example, the sensor S2 may be provided in a computer other than the control device 205, or may be connected to the other computer. In this case, at least a portion of the observation data acquired by the sensor S2 may be provided via the other computer. The sensor S2 is an example of the sensor S described above. The type of the sensor S2 may be the same as that of the sensor S described above.

[0082] The observation data may be appropriately selected so as to enable inference of the behavioral intention of the subject T20 with respect to the robotic device R20. In one example, similar to the example of FIG. 3 above, the observation data may be composed of image data, sound data, text data, biometric data, device operation data, motion capture data, position data, or a combination thereof related to the subject T20. In addition to the example of FIG. 3 above, the device operation data may include input operations on the robotic device R20, etc. Note that the observation data may be composed of the various types of data themselves, or may be composed of feature quantities obtained by converting the various types of data.

[0083] In the example of Fig. 5, outputting the inference result may be configured by outputting the inference result regarding the degree to which the subject T20 has a behavioral intention toward the robotic device R20. The form of the output may be determined arbitrarily.

[0084] In one example, outputting the inference result may include selecting an action for the robotic device R20 in accordance with the inference result regarding the degree to which the subject T20 intends to take an action toward the robotic device R20, and outputting a command to the robotic device R20 to execute the selected action. When the control device 205 is an example of the inference device 2, the control device 205 may control the action of the robotic device R20 by directly issuing a command to the robotic device R20. Alternatively, the control device 205 may indirectly control the action of the robotic device R20 by issuing a command to a controller of the robotic device R20. In the latter case, the control device 205 may be referred to as a higher-level controller, and the controller of the robotic device R20 may be referred to as a lower-level controller. According to one example of the present embodiment, the action of the robotic device R20 can be controlled in accordance with the inference result regarding the subject T20's behavioral intention toward the robotic device R20. This allows for expected action control of the robotic device R20 that is suited to the subject T20.

[0085] The correspondence between the inference result of the behavioral intention and the behavior of the robot device R20 may be determined appropriately depending on the embodiment. For example, the behavioral intention of the subject T20 to be inferred may be an intention to take negative behavior toward the robot device R20, such as making a voice to avoid the robot device R20 or making a negative facial expression. In this case, the behavior of the robot device R20 may be selected depending on the inference result. Selecting an action of the robotic device R20 in accordance with the inference result may include selecting to stop the action currently being performed by the robotic device R20 or to perform an action other than the action currently being performed, when it is estimated that the subject T20 has no intention to perform a negative action. Selecting an action of the robotic device R20 in accordance with the inference result may include selecting to continue the action currently being performed by the robotic device R20, when it is estimated that the subject T20 does not have the intention to perform a negative action.

[0086] Furthermore, for example, the behavioral intention of the subject T20 to be inferred may be an intention to perform positive behavior such as making a positive expression (e.g., smiling), giving praise, or engaging in physical contact. In this case, selecting an action of the robotic device R20 in accordance with the inference result may include selecting to continue the action currently being performed by the robotic device R20 when it is estimated that the subject T20 has the intention to perform positive behavior. Selecting an action of the robotic device R20 in accordance with the inference result may include selecting to stop the action currently being performed by the robotic device R20 or to perform an action other than the action currently being performed when it is estimated that the subject T20 does not have the intention to perform positive behavior.

[0087] Furthermore, for example, the behavioral intention of the subject T20 to be inferred may be an intention to speak to the robotic device R20. In this case, selecting an action of the robotic device R20 in accordance with the inference result may include selecting the robotic device R20 to stop speaking and to perform an action to listen to the speech of the subject T20 when it is estimated that the subject T20 has the intention to speak. Selecting an action of the robotic device R20 in accordance with the inference result may include selecting the robotic device R20 to continue the action it is currently performing when it is estimated that the subject T20 does not have the intention to speak.

[0088] Furthermore, for example, the behavioral intention of the subject T20 to be inferred may be an intention to cut in on the path (course) of the robotic device R20. In this case, selecting an action of the robotic device R20 in accordance with the inference result may include selecting at least one of changing the path, slowing down the moving speed, and issuing a warning when it is estimated that the subject T20 intends to cut in. The path change may be realized by any method, such as a known path planning method. The slowing down of the moving speed may include stopping. The warning may be, for example, a beep, a message, or the like. Selecting an action of the robotic device R20 in accordance with the inference result may include selecting to continue moving along the path being adopted when it is estimated that the subject T20 does not intend to cut in.

[0089] As in the example of Figure 3, being presumed to have an intention to take action (an intention to speak, an intention to interrupt) may include the case where the degree of the presumed intention to take action is higher than a threshold. Being presumed not to have an intention to take action may include the case where the degree of the presumed intention to take action is lower than a threshold. The case where the degree of the presumed intention to take action is equal to a threshold may be determined to be either having or not having an intention to take action.

[0090] (3) Interrupting the route of a moving object FIG. 6 is a schematic diagram showing a specific example of an inferred scene of a behavioral intention (intention of a moving object to cut in on a path) according to this embodiment. In the example of FIG. 6, a scene is assumed in which a moving robot T30 is present around the moving object. The moving object may be at least one of a moving robot and a living thing (including a human). In FIG. 6, the moving robot M30 and a human M31 are examples of the moving object. When the moving object is the human M31, the scene in FIG. 6 is an example of the scene in FIG. 2B above. The human M31 is, for example, a pedestrian. When the moving object is the moving robot M30, the scene in FIG. 6 is an example of the scene in FIG. 2C above. In this case, the moving object may be referred to as a first moving robot, and the moving robot T30 may be referred to as a second moving robot. The movement of each moving robot (M30, T30) may be manually operated or automatically controlled. Note that in FIG. 6, a vehicle is used as an example of each moving robot (M30, T30). However, when each moving robot is The types of the mobile robots (M30, T30) are not limited to these examples. Each of the mobile robots (M30, T30) may be a mobile robot other than a vehicle.

[0091] In the learning phase, the target ST may be a mobile robot T30 existing around the moving body. Each training sample (31, 35) may be appropriately collected from a training individual of the mobile robot T30. Being in the vicinity may mean being within a range where the robot can cut into the path of the moving body. The behavior to be inferred may be cutting into the path of the moving body. In other words, the intention to perform the behavior may be an intention to cut into the path of the moving body. According to this example of the present embodiment, it is possible to reduce the cost of generating a trained inference model 50 that has acquired the ability to infer the intention to cut into the path of a moving body.

[0092] Similarly, in the inference stage, the target TT may be a mobile robot T30 existing around the moving object. The sample 41 may be appropriately acquired from the corresponding individual mobile robot T30. The behavior to be inferred may be interrupting the path of the moving object. According to one example of the present embodiment, the trained inference model 60 can be used to infer the degree to which the mobile robot T30 intends to interrupt the path of the moving object.

[0093] As shown in Fig. 6, in the inference stage, an on-board device 207 may be placed on the mobile robot M30. If the mobile robot M30 is a vehicle, the on-board device 207 may be an in-vehicle device. The on-board device 207 may also be a control device for the mobile robot M30. If a passenger is present on the mobile robot M30, the on-board device 207 may also be a user terminal of the passenger. The user terminal 208 may also be held by a person M31.

[0094] In this scenario, at least one of the onboard device 207 and the user terminal 208 may be an example of the inference device 2. Accordingly, at least one of the onboard device 207 and the user terminal 208 may acquire a sample 41 of observation data of the mobile robot T30 and use the trained inference model 60 to infer the extent to which the mobile robot T30 has an intention to cut in from the acquired sample 41. However, in the scenario of FIG. 6 , the form of the inference device 2 is not limited to this example. A computer other than the onboard device 207 and the user terminal 208 may be an example of the inference device 2. Note that the inference of the mobile robot T30's intention to cut in may be performed at any timing when the mobile robot T30 may be present around the moving object. In one example, if the mobile robot T30 is a vehicle, the inference of the intention to cut in may be performed at a point where cutting in may occur, such as a crosswalk, a lane change, or a road merging point.

[0095] When the moving object is the mobile robot M30, at least one of the sensor S3 and the onboard device T301 of the mobile robot T30 may be used to acquire observation data of the mobile robot M30. In one example, as shown in FIG. 6, the sensor S3 may be provided on the onboard device 207, connected to the onboard device 207, or provided on the mobile robot M30. In another example, the sensor S3 may be provided on a computer other than the onboard device 207, or connected to the other computer. In this case, at least a portion of the observation data acquired by the sensor S3 may be provided via the other computer. Also, like the onboard device 207, when the mobile robot T30 is a vehicle, the onboard device T301 may be an in-vehicle device. The onboard device T301 may be a control device for the mobile robot T30. When a passenger is present on the mobile robot T30, the onboard device T301 may be the passenger's user terminal. The onboard device T301 may transmit at least a portion of the observation data to the onboard device 207 via any data communication.

[0096] In addition, when the moving body is a person M31, at least one of the sensor S4 and the on-board device T301 of the moving robot T30 may be used to acquire observation data of the moving robot T30. In one example, as shown in FIG. 6, the sensor S4 is provided in the user terminal 208. The on-board device T301 may be mounted on the user terminal 208, connected to the user terminal 208, or attached to the person M31. In another example, the sensor S4 may be provided on a computer other than the user terminal 208, or connected to the other computer. In this case, at least a portion of the observation data obtained by the sensor S4 may be provided via the other computer. Furthermore, the on-board device T301 may transmit at least a portion of the observation data to the user terminal 208 by any data communication.

[0097] The sensors (S3, S4) are an example of the sensor S. The type of the sensors (S3, S4) may be the same as that of the sensor S. The method of data communication between the on-board device T301 and the on-board device 207 is not particularly limited and may be selected appropriately depending on the embodiment. In one example, when each mobile robot (T30, M30) is a vehicle, methods such as vehicle-to-vehicle communication or road-to-vehicle communication may be used for data communication between the on-board device T301 and the on-board device 207. The method of data communication between the on-board device T301 and the user terminal 208 is also not particularly limited and may be selected appropriately depending on the embodiment.

[0098] The observation data may be appropriately selected so as to enable inference of the mobile robot T30's intention to cut in on the path of the moving object. In one example, similar to the case of part of FIG. 2B and FIG. 2C above, the observation data may be composed of image data, sound data, text data, motion status data, position data, or a combination of these related to the mobile robot T30. The image data may be acquired so as to show the mobile robot T30. The sound data may be acquired so as to include sounds related to the mobile robot T30. Note that the observation data may be composed of the above various data themselves, or may be composed of feature quantities obtained by converting the above various data.

[0099] In the example of Fig. 6, outputting the inference result may be configured by outputting the inferred result of the degree to which the mobile robot T30 intends to cut into the path of the moving object. The form of the output may be determined arbitrarily.

[0100] In one example, outputting the inference result may include outputting an instruction to the mobile robot T30 to change its path when the mobile robot T30 intends to cut in. Similar to the example of FIG. 3 , being presumed to have an intention to cut in may include the degree of the presumed intention to cut in being higher than a threshold. Being presumed not to have an intention to cut in may include the degree of the presumed intention to cut in being lower than a threshold. The degree of the presumed intention to cut in equal to a threshold may be determined to be either an intention to cut in or not. The path change may be realized by any method, such as known path planning. According to one example of this embodiment, the movement of the mobile robot T30 can be controlled according to the result of inferring the intention of the mobile robot T30 to cut in. Furthermore, it is expected that the path change of the mobile robot T30 will be performed early, before the mobile robot T30 cuts in.

[0101] Outputting an instruction to change the route may include at least one of giving a command to the moving body to change the route and outputting instruction information instructing the user to change the route. The command to change the route may be given directly or indirectly to a control device of the moving body. For example, if the mobile robot M30 is configured to be able to move automatically, such as a vehicle configured to be able to drive automatically, the instruction to change the route may include at least one of a command to the mobile robot M30 to change the route and instruction information to an operator (driver, etc.) to change the route. On the other hand, if the mobile robot M30 is configured to move by manual operation, such as a vehicle configured to be able to drive manually, the instruction to change the route may include instruction information to the operator to change the route. Similarly, if the moving body is a person M31, the instruction to change the route may include instruction information to the person M31 to change the route. The instruction information may be displayed on a screen. The information may be output in any manner, such as by image display or audio output.

[0102] Note that in the example of Figure 6, the form of output of the inference result is not limited to this example. Outputting the inference result, together with or instead of outputting an instruction to change the route, may include at least one of outputting an instruction to the moving body to slow down its moving speed and outputting a warning to the mobile robot T30 when the mobile robot T30 intends to cut into the path of the moving body. Similar to the instruction to change the route, outputting an instruction to slow down its moving speed may include at least one of issuing a command to the moving body to slow down and outputting instruction information instructing the user to slow down. Slowing down the moving speed may include stopping. The warning may be, for example, a beep, a message, etc.

[0103] (How to obtain training samples) The training samples (31, 35) may be collected as appropriate depending on the embodiment. The first training sample 31 may be collected as appropriate from a section of the observed data that is highly likely to indicate an intention to take action. The label 32 may be generated automatically or assigned manually. Meanwhile, the second training sample 35 may be collected as appropriate from another section of the observed data.

[0104] FIG. 7 schematically illustrates an example of a method for acquiring each training sample (31, 35) according to this embodiment. As described above, a subject performs a behavior as a result of having an intention to perform that behavior. In other words, it is highly likely that the subject had a high degree of intention to perform the behavior immediately before the time when the subject performed the behavior. Therefore, in one example, the first training sample 31 may be extracted from a section ST10 in the observed data immediately before the occurrence section in which the subject performs the behavior. The length of the predetermined section ST10 (i.e., the immediately preceding index) may be determined appropriately depending on the embodiment. According to this example of this embodiment, the first training sample 31 (positive sample) can be appropriately collected. As a result, the model generation device 1 can be expected to appropriately generate a trained inference model 50.

[0105] In one example, the second training sample 35 may be extracted from a section ST11 in the observation data that precedes the occurrence section in which the subject is performing an action, excluding the section ST10 immediately preceding the occurrence section. According to this example of the present embodiment, the second training sample 35 (unlabeled sample) can be appropriately collected. As a result, the model generation device 1 can be expected to appropriately generate a trained inference model 50.

[0106] In another example, the second training samples 35 may be extracted from a section ST20 in the observation data that follows the occurrence section in which the subject is performing an action. According to this example of the present embodiment, the second training samples 35 (unlabeled samples) can be appropriately collected. As a result, the model generation device 1 can be expected to appropriately generate a trained inference model 50.

[0107] In the example of FIG. 7, the end point of the interval ST10 (the immediately preceding interval) from which the first training sample 31 is extracted coincides with the start point of the occurrence interval (the start time of the occurrence of the action). However, the interval ST10 is not limited to this example. In another example, taking into consideration that an action appears before and after the start point of the occurrence interval (for example, in the case of speech, the mouth moves), the end point of the interval ST10 may be set to a point a buffer length before the start point of the occurrence interval. The length of the buffer may be determined as appropriate.

[0108] (inference model) The inference model (50, 60) uses a machine learning model to predict the given input data (observations) The machine learning model is configured to infer the degree to which a person intends to take action from a set of data (e.g., a data sample). The machine learning model is configured to have one or more calculation parameters that can be adjusted by machine learning. The one or more calculation parameters are used to calculate the desired inference (inference of behavioral intention in this embodiment). The machine learning model may be configured, for example, by a neural network, a regression model, a decision tree model, a support vector machine, or other functional formula (calculation model). The machine learning method may be appropriately selected depending on the machine learning model employed (e.g., backpropagation, etc.).

[0109] In one example, the inference model (50, 60) may include a neural network. The structure of the neural network is not particularly limited and may be determined appropriately depending on the embodiment. The structure of the neural network may be specified, for example, by the number of layers from the input layer to the output layer, the type of each layer, the number of nodes (neurons) included in each layer, the connection relationships between the nodes in each layer, etc. In one example, the neural network may include any mechanism such as a recurrent structure, a self-attention mechanism, or an autoregressive model. Furthermore, the neural network may include any layer such as a fully connected layer, a convolutional layer, a pooling layer, a deconvolutional layer, an unpooling layer, a normalization layer, a dropout layer, or a long short-term memory (LSTM). The neural network may include any type of model such as a diffusion model, a transformer model, or a generative model. The connection weights between each node included in the neural network and the threshold value of each node are examples of calculation parameters.

[0110] In this embodiment, the inference model 50 is trained by machine learning using learning from positive examples and unlabeled data (PU learning) using the training dataset 30. Training the inference model 50 means adjusting (optimizing) the values ​​of the calculation parameters of the inference model 50 using the training dataset 30. The PU learning method may be selected appropriately depending on the embodiment. In one example, a known method may be adopted as the PU learning method. For example, the PU learning method is described in Reference 1 (Marthinus Christoffel du Plessis, et al. "Analysis of Learning from Positive and Unlabeled Data", [online], [searched March 5, 2024], Internet)<URL:https: / / papers.nips.cc / paper / 2014 / file / 35051070e572e47d2c26c241ab88307f-Paper.pdf> ), Reference 2 (Marthinus Christoffel du Plessis, et al. "Convex Formulation for Learning from Positive and Unlabeled Data", [online], [ Searched on March 5, 2024], Internet<URL:https: / / proceedings.mlr.press / v37 / plessis15.pdf> ), and Reference 3 (Ryuichi Kiryo, et al. "Positive-Unlabeled Learning with Non-Negative Risk Estimator", [online], [Retrieved March 5, 2024], Internet<URL:https: / / arxiv.org / abs / 1703.00593> ) may be adopted. The fact that the trained inference model 50 has been adapted to a target individual by unsupervised learning may be one piece of evidence that PU learning has been performed in the trained inference model 50. Note that if PU learning has been performed in the inference model 50, the use of negative samples does not need to be completely avoided. In machine learning, negative samples may not be used completely, or may be used partially.

[0111] Inferring the degree of intention to take action may include at least one of identifying whether or not there is an intention to take action and regressing the probability of taking action. The type of action to be inferred is not limited to one, and may be multiple. When multiple types of actions are set as the inference target, inferring the degree of intention to take action may include at least one of identifying whether or not there is an intention to take action for each type and regressing the probability of taking action for each type.

[0112] The input / output format of the inference model (50, 60) may be determined appropriately depending on the embodiment. In one example, the input data may be provided to the inference model (50, 60) as is, or may be provided as a pre-defined data. The inference model (50, 60) may be provided with samples of data other than the above-mentioned observation data samples. The other data may include, for example, attribute information of the subject, related information (speaker information, etc.), etc. In one example, the output of the inference model (50, 60) may be configured to directly or indirectly indicate the inference result. When the output of the inference model (50, 60) is configured to indirectly indicate the inference result, the inference result may be obtained by performing any information processing (interpretation processing, etc.) on the output of the inference model (50, 60).

[0113] (Controlling Machine Learning) Controlling machine learning may include executing machine learning within the model generation device 1 and issuing instructions to an external computer to execute machine learning. That is, in one example, the model generation device 1 may execute machine learning of the inference model 50. In another example, the model generation device 1 may issue instructions to an external computer to execute machine learning of the inference model 50. The external computer may execute machine learning of the inference model 50 in response to instructions from the model generation device 1. In another example, the model generation device 1 may execute part of the machine learning processing of the inference model 50, and the external computer may execute the remaining processing in response to instructions from the model generation device 1. Note that the external computer may be composed of one or more computers. For example, the external computer is a server device. The external computer may be connected to the model generation device 1 via a network or may be directly connected to the model generation device 1. The type of network is not particularly limited and may be selected appropriately depending on the embodiment.

[0114] (Outputs the results of machine learning) Outputting the machine learning result may include outputting any information related to the machine learning result. In one example, outputting the machine learning result may include generating learning result data indicating the machine learning result and storing the generated learning result data in any storage area. In another example, outputting the machine learning result may include outputting a convergence result of training in machine learning. The convergence result may be indicated by, for example, a loss, a number of time steps, a learning curve, etc.

[0115] (System Configuration) In one example, as shown in Figure 1, a model generation device 1 and an inference device 2 may be connected to each other via a network. The type of network may be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, etc. However, the method of exchanging data between each device is not limited to this example and may be appropriately selected depending on the embodiment. In another example, data may be exchanged using a storage medium.

[0116] 1, the model generation device 1 and the inference device 2 are separate computers. However, the configuration of the system according to this embodiment is not limited to this example and may be determined appropriately depending on the embodiment. In another example, the model generation device 1 and the inference device 2 may be configured as a single computer. In yet another example, at least one of the model generation device 1 and the inference device 2 may be configured as multiple computers.

[0117] §2 Configuration example [Hardware configuration] (Model generation device) FIG. 8 is a schematic diagram illustrating an example of the hardware configuration of the model generation device 1 according to this embodiment. The model generation device 1 according to this embodiment includes a control unit 11, a storage unit 12, a communication interface, and a The computer is electrically connected to a source 13, an input device 14, an output device 15, and a drive 16.

[0118] The control unit 11 includes a CPU (Central Processing Unit) which is a hardware processor, The system includes RAM (Random Access Memory), ROM (Read Only Memory), etc., and is configured to execute information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 may be configured, for example, with a hard disk drive, a solid state drive, etc. The storage unit 12, RAM, and ROM are examples of memory resources. In this embodiment, the storage unit 12 stores various information such as the generation program 81, the training dataset 30, and the learning result data 500.

[0119] The generation program 81 is a program for causing the model generation device 1 to execute information processing (see FIG. 12 , described later) related to machine learning of the inference model 50. The generation program 81 includes a series of instructions for the information processing. The learning result data 500 is configured to indicate information related to the generated trained inference model 50. As long as the learning result data 500 can hold information for executing the arithmetic processing of the trained inference model 50, the configuration of the learning result data 500 is not particularly limited and may be determined appropriately depending on the embodiment. In one example, the learning result data 500 may be configured to include information indicating values ​​of arithmetic parameters adjusted by machine learning. In some cases, the learning result data 500 may be configured to further include information indicating the configuration of the inference model 50 (e.g., the structure of a neural network, etc.). In this embodiment, the learning result data 500 may be generated as a result of executing the generation program 81.

[0120] The communication interface 13 is configured to perform wired or wireless communication via a network. The communication interface 13 may be configured, for example, by a wired LAN (Local Area Network) module, a wireless LAN module, etc. The model generation device 1 may perform data communication with another computer (e.g., the inference device 2, etc.) via the communication interface 13.

[0121] The input device 14 is a device for inputting, for example, a mouse, a keyboard, etc. The output device 15 is a device for outputting, for example, a display, a speaker, etc. An operator can operate the model generation device 1 by using the input device 14 and the output device 15. The input device 14 and the output device 15 may be connected via an external interface. The external interface may be appropriately configured to connect to an external device by wire or wirelessly, for example, via a USB (Universal Serial Bus) port, a dedicated port, etc. The input device 14 and the output device 15 may be integrated into one device, for example, a touch panel display, etc.

[0122] The drive 16 is a device for reading various information such as programs stored in a storage medium 91. At least one of the generation program 81 and the training dataset 30 may be stored in the storage medium 91 instead of or together with the storage unit 12. The storage medium 91 is configured to store various information (such as the stored programs) by electrical, magnetic, optical, mechanical, or chemical action so that a machine such as a computer can read the information. The model generation device 1 may acquire at least one of the generation program 81 and the training dataset 30 from the storage medium 91. The storage medium 91 may be a disk-type storage medium such as a CD or a DVD, or may be a non-disk-type storage medium such as a semiconductor memory (e.g., a flash memory). The type of the drive 16 may be selected appropriately depending on the type of the storage medium 91. The drive 16 may be connected via an external interface.

[0123] The specific hardware configuration of the model generating device 1 may be changed depending on the embodiment. For example, the control unit 11 may include a plurality of hardware processors. The hardware processors may be a microprocessor, a field-programmable gate array (FPGA), a digital signal processor (DSP), a GPU (Gateway Processor), a GPIO (Gateway Controller ... The communication interface 13, the input device 14, and the output device 15 may be configured by a PU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), etc. , and at least one of the drive 16 may be omitted. The training dataset 30 may be held in an external storage device such as a network-attached storage (NAS). The learning result data 500 may be saved in an external storage device. The model generation device 1 may be composed of multiple computers. In this case, the hardware configurations of the computers may or may not be the same. Furthermore, the model generation device 1 may be an information processing device designed specifically for the service to be provided, as well as a general-purpose server device, a general-purpose PC, a tablet PC, a terminal device, etc.

[0124] (Inference device) 9 shows a schematic diagram of an example of the hardware configuration of the inference device 2 according to this embodiment. The inference device 2 according to this embodiment is a computer in which a control unit 21, a storage unit 22, a communication interface 23, an input device 24, an output device 25, a drive 26, and an external interface 27 are electrically connected.

[0125] The control unit 21 to the drive 26 and the storage medium 92 of the inference device 2 may be configured similarly to the control unit 11 to the drive 16 and the storage medium 91 of the model generation device 1, respectively. The control unit 21 (CPU) is an example of a processor resource of the inference device 2. The storage unit 22 (and RAM, ROM) is an example of a memory resource of the inference device 2. In this embodiment, the storage unit 22 stores various information such as an inference program 82, learning result data 600, etc.

[0126] The inference program 82 is a program for causing the inference device 2 to execute information processing (see FIG. 13 described below) related to inferring the behavioral intention of a target. The inference program 82 includes a series of instructions for the information processing. The learning result data 600 is configured to indicate information related to the trained inference model 60. The learning result data 600 may be configured in the same manner as the learning result data 500. When the trained inference model 50 is used as the trained inference model 60, the learning result data 600 may be the learning result data 500. The learning result data 600 may be incorporated into the inference program 82. At least one of the inference program 82 and the learning result data 600 may be stored in a storage medium 92 instead of or together with the storage unit 22. The inference device 2 may acquire at least one of the inference program 82 and the learning result data 600 from the storage medium 92.

[0127] The inference device 2 may communicate data with other computers (e.g., the model generation device 1) via the communication interface 23. An operator can operate the inference device 2 by using the input device 24 and the output device 25. If the user terminal 201 is an example of the inference device 2, the display 2015 is an example of the output device 25.

[0128] The external interface 27 is configured to connect to an external device via a wired or wireless connection. The external interface 27 may be, for example, a USB (Universal Serial Bus) port, a dedicated port, or the like. The type and number of external interfaces 27 may be determined appropriately depending on the embodiment. In this embodiment, the inference device 2 may be connected to the sensor S via at least one of the communication interface 23 and the external interface 27.

[0129] Regarding the specific hardware configuration of the inference device 2, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 21 may be configured with a plurality of hardware The inference device 2 may include a processor. The hardware processor may be configured with a microprocessor, FPGA, DSP, GPU, ASIC, etc. At least one of the communication interface 23, the input device 24, the output device 25, the drive 26, and the external interface 27 may be omitted. The inference device 2 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. The inference device 2 may be an information processing device designed specifically for the service to be provided, as well as a general-purpose server device, a general-purpose PC, a tablet PC, a terminal device, etc.

[0130] [Software configuration] (Model generation device) 10 schematically shows an example of the software configuration of the model generation device 1 according to this embodiment. The control unit 11 of the model generation device 1 executes instructions included in the generation program 81 stored in the storage unit 12 using the CPU. As a result, the model generation device 1 operates as a computer including an acquisition unit 111, a learning processing unit 112, and an output processing unit 113 as software modules. That is, in this embodiment, each software module of the model generation device 1 is realized by the control unit 11 (CPU).

[0131] The acquisition unit 111 is configured to acquire a training dataset 30 including a first training sample 31 and a second training sample 35 of observation data of the target ST. The first training sample 31 is assigned a label 32 indicating that the target ST has an intention to take action. The second training sample 35 is not assigned a label. The learning processing unit 112 is configured to control machine learning of the inference model 50 by learning from positive examples and unlabeled data using the acquired training dataset 30. The machine learning is configured by training the inference model 50 to acquire the ability to infer the degree to which the target ST has an intention to take action from the observation data. The output processing unit 113 is configured to output the results of the machine learning.

[0132] (reasoning device) 11 schematically shows an example of the software configuration of the inference device 2 according to this embodiment. The control unit 21 of the inference device 2 executes instructions included in the inference program 82 stored in the storage unit 22 using the CPU. This causes the inference device 2 to operate as a computer equipped with an acquisition unit 211, an inference unit 212, and an output processing unit 213 as software modules. That is, in this embodiment, similar to the model generation device 1, each software module of the inference device 2 is also realized by the control unit 21 (CPU).

[0133] The acquisition unit 211 is configured to acquire observation data samples 41 of the target TT. The inference unit 212 has a trained inference model 60 by holding learning result data 600. The inference unit 212 is configured to infer the degree to which the target TT has an intention to take action from the acquired observation data samples 41 using the trained inference model 60. The output processing unit 213 is configured to output the inference result.

[0134] (others) In this embodiment, an example is described in which each software module of the model generation device 1 and the inference device 2 is implemented by a general-purpose CPU. However, some or all of the above software modules may be implemented by one or more dedicated processors or chipsets. Each of the above modules may be implemented as a hardware module. With regard to the software configuration of the model generation device 1 and the inference device 2, modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0135] §3 Example of operation [Model generation device] 12 is a flowchart showing an example of the processing procedure of the model generation device 1 according to this embodiment. The following processing procedure is an example of a model generation method executed by a computer. However, the following processing procedure of the model generation device 1 is merely an example, and each step may be changed as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0136] (Step S101) In step S101, the control unit 11 operates as the acquisition unit 111 and acquires a training data set 30 including a first training sample 31 and a second training sample 35 of observation data of the target ST. The first training sample 31 is assigned a label 32 indicating that the target ST has an intention to take a predetermined action, and the second training sample 35 is not assigned a label.

[0137] In one example, the target ST may be a human subject, and the observation data may be image data, sound data, text data, biometric data, device operation data, motion capture data, position data, or a combination thereof. In one example, the target ST may be a robotic device, and the observation data may be image data, sound data, text data, motion status data, position data, or a combination thereof.

[0138] As an example, this embodiment may be applied to the scene in FIG. 3 above. In this case, the target ST may be a target person T10, and the behavior of the inference target may be an utterance. As an example, this embodiment may be applied to the scene in FIG. 5 above. In this case, the target ST may be a target person T20, and the behavior of the inference target may be a behavior toward the robot device R20. As an example, this embodiment may be applied to the scene in FIG. 6 above. In this case, the target ST may be a mobile robot T30 present around the moving object, and the behavior of the inference target may be interrupting the path of the moving object.

[0139] In one example, the first training sample 31 may be extracted from a section ST10 in the observation data that immediately precedes the occurrence section in which the subject is performing the behavior. In one example, the second training sample 35 may be extracted from a section ST11 in the observation data that precedes the occurrence section in which the subject is performing the behavior, excluding the section ST10 immediately preceding the occurrence section. In one example, the second training sample 35 may be extracted from a section ST20 in the observation data that follows the occurrence section in which the subject is performing the behavior.

[0140] The training dataset 30 may be generated in the model generation device 1, or may be generated in another computer other than the model generation device 1. When at least a portion of the training dataset 30 is generated in another computer, the control unit 11 may acquire at least a portion of the training dataset 30 from the other computer via a network, a storage medium 91, or the like. The number of each training sample (31, 35) included in the training dataset 30 is not particularly limited and may be determined appropriately depending on the embodiment. After acquiring the training dataset 30, the control unit 11 proceeds to the next step S102.

[0141] (Step S102) In step S102, the control unit 11 operates as the learning processing unit 112 and controls the machine learning of the inference model 50 by PU learning using the acquired training dataset 30.

[0142] In one example, the control unit 11 may execute machine learning of the inference model 50 within the model generation device 1. In one example, the control unit 11 may issue an instruction to an external computer to execute machine learning of the inference model 50. In response to the instruction from the model generation device 1, the external computer Machine learning of the inference model 50 may be performed. In PU learning, the inference model 50 is trained to acquire the ability to infer from observation data the degree to which the target ST intends to take action within the scope of the training dataset 30. Completion of this machine learning generates a trained inference model 50 that has acquired the ability to infer the behavioral intention of the target ST. When the machine learning process is completed, the control unit 11 proceeds to the next step S103.

[0143] (Step S103) In step S103, the control unit 11 operates as the output processing unit 113 to output the results of the machine learning. The output destination and the content of the information to be output may be selected appropriately depending on the embodiment.

[0144] In one example, the control unit 11 may, as the output process, generate learning result data 500 indicating the results of the machine learning (i.e., the trained inference model 50) and store the generated learning result data 500 in a predetermined storage area. The predetermined storage area may be, for example, RAM in the control unit 11, the storage unit 12, an external storage device, a storage medium, or a combination thereof. The storage medium may be, for example, a CD, a DVD, a semiconductor memory, or the like. The external storage device may be, for example, a data server such as a NAS. The external storage device may be, for example, an external storage device. When the machine learning calculation process is performed on an external computer, the learning result data 500 may be generated by the external computer. In another example, the control unit 11 may, as the output process, output a convergence result of the machine learning training. The output destination may be, for example, RAM in the control unit 11, the output device 15, an external computer, an external storage device, a storage medium, or a combination thereof. In yet another example, the control unit 11 may, as the output process, output the progress of the machine learning training. In this case, the process of step S103 may be executed together with step S102.

[0145] When the output of the machine learning results is completed, the control unit 11 ends the processing procedure of the model generation device 1 according to this operation example.

[0146] When the inference device 2 uses the trained inference model 50 as the trained inference model 60, the generated learning result data 500 may be provided to the inference device 2 from the model generation device 1 or an external computer at any timing and in any manner. In one example, the learning result data 500 may be provided to the inference device 2 via a network, a storage medium, etc. In another example, the learning result data 500 may be pre-installed in the inference device 2.

[0147] Furthermore, the control unit 11 may update or generate new learning result data 500 by periodically or irregularly repeating the processes of steps S101 to S103. During this repetition, at least a portion of the training dataset 30 may be changed, modified, added, deleted, or the like as appropriate. Then, the control unit 11 may provide the updated or newly generated learning result data 500 to the inference device 2 as appropriate. In this way, the learning result data 600 (learning result data 500) held by the inference device 2 may be updated.

[0148] [Inference device] 13 is a flowchart showing an example of a processing procedure of the inference device 2 according to this embodiment. The following processing procedure is an example of an inference method executed by a computer. However, the following processing procedure of the inference device 2 is merely an example, and each step may be modified as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0149] (Step S201) In step S201, the control unit 21 operates as the acquisition unit 211 and acquires the observation data of the target TT. Obtain a sample of data.

[0150] In one example, the target TT may be a subject, and the observation data may be composed of image data, sound data, text data, biometric data, device operation data, motion capture data, position data, or a combination thereof. In one example, the target TT may be a robotic device, and the observation data may be composed of image data, sound data, text data, motion status data, position data, or a combination thereof. After acquiring the sample 41, the control unit 21 proceeds to the next step S202.

[0151] (Step S202) In step S202, the control unit 21 operates as the inference unit 212 and uses the trained inference model 60 to infer the degree to which the target TT intends to take action from the acquired observation data sample 41.

[0152] In one example, the control unit 21 may provide the acquired sample 41 to the trained inference model 60 and execute calculation processing of the trained inference model 60. As a result of this calculation, the control unit 21 may acquire from the trained inference model 60 a result of inferring the degree to which the target TT has a behavioral intention. In one example, the trained inference model 60 may be generated by PU learning. In one example, the trained inference model 50 obtained by the model generation device 1 may be used as the trained inference model 60.

[0153] In one example, this embodiment may be applied to the scene in FIG. 3 above. In this case, the target TT may be a target person T10, and the behavior of the inference target may be an utterance. In one example, this embodiment may be applied to the scene in FIG. 5 above. In this case, the target TT may be a target person T20, and the behavior of the inference target may be behavior toward the robot device R20. In one example, this embodiment may be applied to the scene in FIG. 6 above. In this case, the target TT may be a mobile robot T30 existing around the moving object, and the behavior of the inference target may be interrupting the path of the moving object. After deriving the inference result, the control unit 21 proceeds to the next step S203.

[0154] (Step S203) In step S203, the control unit 21 operates as the output processing unit 213 and outputs the result of the inference.

[0155] The output destination and the content of the output information may be selected as appropriate depending on the embodiment. In the example of the scene in FIG. 3 above, the control unit 21 may output an effect that visualizes the result of inferring the degree to which the subject T10 intends to speak, as an output process of the inference result. In one example, the effect may be configured to implicitly visualize the result of inferring the degree to which the subject T10 intends to speak. In one example, the observation data may include image data showing the face of the subject T10 in the video conference. In the output process of the inference result, when the subject T10 intends to speak, the control unit 21 may output an effect that enlarges the face of the subject T10 to the terminal of the other party in the video conference with whom the subject T10 is conversing.

[0156] In the example of the scene in Fig. 5, the control unit 21 may select an action for the robot device R20 based on the result of inferring the degree to which the robot device R20 intends to take an action. Then, the control unit 21 may output a command to the robot device R20 to instruct the robot device R20 to perform the selected action. In addition, in the example of the scene in Fig. 6, the control unit 21 may output a command to the moving object to change its path when the mobile robot T30 intends to cut in on the path of the moving object.

[0157] When the output of information is completed, the control unit 21 ends the processing procedure of the inference device 2 according to this operation example. The control unit 21 may execute a series of processes from step S201 to step S203 at any timing such as a user operation or when a condition is satisfied.

[0158] [Features] In this embodiment, in step S102, PU learning is adopted as the machine learning method for the inference model 50, which uses unlabeled data (second training samples 35) along with positive samples (first training samples 31 and labels 32). By adopting this PU learning, the trained inference model 50 can be generated without using negative samples, which are difficult to collect. This makes it possible to at least partially omit the collection of negative samples. This reduces the cost of generating a trained inference model 50 that has acquired the ability to infer the behavioral intention of a target (target ST). Furthermore, in step S202, the trained inference model 60 can be used to infer the behavioral intention of the target TT.

[0159] §4 Variations Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. The processes and means described in this disclosure can be freely combined and implemented as long as no technical contradiction occurs. Furthermore, various improvements or modifications may be made to the above embodiments as appropriate.

[0160] §5 Working Examples The following experiments were carried out to verify the effectiveness of the above-described embodiment, but the present invention is not limited to the following examples.

[0161] [First experiment] In the first experiment, we verified whether PU learning could generate a trained inference model capable of inferring behavioral intentions. Utterances were used as the behavior to be inferred. First, video conference recordings were collected. The number of participants in the recordings ranged from three to nine. In most of the recordings, each participant was displayed in a gallery view format. The face size of each participant in the view was approximately the same. A total of 121 recordings were collected. Of these, 112 recordings (135 hours 49 minutes) were used to extract training samples, 3 recordings (2 hours 47 minutes) were used for validation, and 6 recordings (4 hours 13 minutes) were used for testing.

[0162] Next, in the collected video data, reference 4 (Ruijie Tao, et al. "Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection", [online], [Retrieved March 5, 2024], Internet<URL:https: / / arxiv.org / abs / 2107.06592> We used the active speaker detection method proposed in

[2014] to determine the speaking status of each participant for each timestamp. At the same time, we extracted the facial features of each participant using OpenFace. Then, at the start of data processing, we combined the results of the speaking status determination and the facial feature extraction to adjust the obtained data based on the timestamp.

[0163] Positive samples and unlabeled samples were extracted from 112 recordings used for training. The method used to extract positive and unlabeled samples was shown in Figure 7. Specifically, in the clips of each participant's view in the recording data, the speech section of each participant was identified based on the results of the speech status judgment. The section from 10 seconds before the start of the speech in the identified speech section to a point a buffer before the start of the speech was set as the immediately preceding section, and positive samples were extracted from the immediately preceding section. Buffer The length of the sample was appropriately set so as not to exceed 10 seconds. On the other hand, unlabeled samples were extracted from the other sections excluding the immediately preceding section and the speech section (i.e., non-speech sections other than the section from which positive samples were extracted).

[0164] For the nine recordings used for verification and testing, clips containing speech were excluded based on the speech status judgment results, and then three annotators labeled (annotated) each of the obtained samples. (1) "No intention to speak," (2) "No intention to speak spontaneously, but intention to speak when requested," (3) "Intention to speak spontaneously," We set four types of labels: (1) "will be spoken," and (4) "already spoken." The label (1) is negative, the label (2) is probably positive, the label (3) is positive, and the label (4) corresponded to outliers. The true label of each sample was determined by the consistency (majority vote) of the labels assigned by the three annotators. Samples with inconsistent labels assigned by the three annotators were excluded.

[0165] Figure 14 shows the structure of the inference model used in the first experiment. As shown in Figure 14, an inference model (classifier) ​​was prepared that includes two convolutional layers (first convolutional layer, second convolutional layer), an LSTM, and a fully connected layer. The inputs C and T correspond to the facial feature axis and time axis extracted by OpenFace. 17-dimensional features (features with Facial Action Unit numbers 1, 2, 4, 5, 6, 7, 9, 10, 12, 14, 15, 17, 20, 23, 25, 26, and 45) were selected from the facial features extracted by OpenFace. The input subset consisted of feature information and two-dimensional gaze direction. That is, C was set to 19 dimensions. On the other hand, T was 25 (a 4-second video was input as 25 frames). Each convolution layer was applied in the time direction. Each convolution layer was configured to perform one-dimensional convolution. The stride of each convolution layer was set to 2. Batch normalization (BN) was applied to the input of each convolution layer. The activation function of each convolution layer was We adopted PReLU (Parametric Rectified Linear Unit) for the first convolutional layer. The dropout ratio of the first convolutional layer was set to 0.2, and the dropout ratio of the second convolutional layer was set to 0.5. The output of the fully connected layer was set to binary.

[0166] For an inference model with this structure, we performed machine learning using PU learning with positive samples and unlabeled samples obtained from the training video data. For the PU learning method, we adopted the non-negative PU learning proposed in Reference 3 above. Furthermore, we adjusted the calculation parameters of the inference model using labeled samples obtained from the validation video data. This resulted in the generation of a trained inference model.

[0167] Using the generated trained inference model, we inferred the intention of the speech for samples obtained from the test recording data. The inference results were compared with the labels from three annotators. Then, we calculated the Matthews Correlation Coefficient (MCC), Area Under the Curve (AUC), F-score, accuracy, precision, and recall from the comparison results. The results were MCC 0.175, AUC 0.602, F-score 0.392, accuracy 0.648, and precision 0.315. , and the recall rate was 0.519. These results indicate that we have acquired the ability to infer behavioral intentions. It was found that a trained inference model can be generated by PU learning.

[0168] [Second experiment] In the second experiment, we used the trained inference model obtained in the first experiment to create a prototype of a system that visualizes the degree of speech intention during video conferencing, and verified the impact of visualizing speech intention in remote communication.

[0169] (prototype) Figure 15 shows the configuration of the prototype used in the first example. (i) Feature extraction, (ii) Classifier We have prepared a prototype with three components: (i) facial image processing, and (ii) feature extraction. The system was configured to use OpenFace to extract facial features in real time from images captured by a webcam. The trained inference model obtained in the first experiment was used as the classifier. For facial image processing, when the classifier determined that the subject had an intention to speak, the facial features obtained by OpenFace were used to perform affine transformation for each frame, applying the face enlargement effect shown in Figure 4 to the facial image. The obtained images were streamed using Open Broadcaster Software.

[0170] Examples and Comparative Examples In the first example (implict), the face enlargement effect was performed using the prototype. By outputting the facial image, the speech intention was implicitly visualized in the video conference. In the second example (explicit), the facial image processing of the above prototype was omitted, and a component was added that applies an effect to the facial image that places a yellow frame on the clip in which the interlocutor appears when the classifier identifies that the subject has a speech intention. As a result, under the conditions of the second example, the speech intention was explicitly visualized in the video conference by outputting the effect of placing a yellow frame. On the other hand, under the conditions of the comparative example (baseline), the above prototype was not used, and the captured image was streamed as is, without visualizing the speech intention, as in a normal video conference.

[0171] (Experimental conditions) In the second experiment, 47 participants (25 men, 22 women, mean age 24.6 years, standard deviation 4.27) participated. The 47 participants were divided into 12 groups (11 groups of 4 people, 1 group of 3 people). Each group was assigned to one of the conditions of the first example, the second example, or the comparative example. The group of 3 people was assigned to the comparative example condition. Each participant was instructed to sit in front of a PC and participate in a group video conference consisting of one practice session and one main session.

[0172] In the practice session, participants were instructed to hold a 10-minute mock discussion on the topic, "Which is better to keep at home, a dog or a cat?" After the practice session and a 5-minute break, participants were instructed to begin the main discussion session. The main session was set to last 15 minutes. The discussion topics were selected in a balanced manner from two themes ("When traveling abroad, would you choose a tour package or an individual plan?" and "Which do you prefer, books or e-books?").

[0173] The video conference during the main session was recorded for each group, and the recorded data was analyzed to calculate the ratio of speech counts (RSTurn) and speech time ratio (RSTime) for each group. The ratio of speech counts (RSTurn) was calculated by dividing the number of times each participant spoke by the total number of times all participants spoke during the discussion. The ratio of speech time (RSTime) was calculated by dividing the total speaking time of each participant by the total speaking time of all participants during the discussion. Assuming that speaking is ideally evenly distributed in a group of four, the ideal values ​​for the ratio of speech counts (RSTurn) and speech time ratio (RSTime) are both 0.25. The difference between the calculated results of the ratio of speech counts (RSTurn) and speech time ratio (RSTime) and the ideal value (0.25) was then calculated to calculate the distribution of the difference from the ideal value. Since the ideal values ​​for the group of three people differ from those for the group of four people, the records of the group of three people were excluded from the calculation of the speech count ratio (RSTurn) and speech time ratio (RSTime).

[0174] (Experimental results) 16A and 16B show the results of calculating the speech count ratio (RSTurn) and speech time ratio (RSTime) for each group for each participant in the first example, the second example, and the comparative example. FIG. 16C shows the results of calculating the distribution of the difference between the calculation results and the ideal values ​​for the speech count ratio (RSTurn) and speech time ratio (RSTime) for each participant in the first example, the second example, and the comparative example. 16A to 16C, a comparison between the first embodiment and the comparative example shows that the speech frequency ratio (RSTurn) and speech time ratio (RSTime) approach ideal values ​​by visualizing implicit speech intentions using the face enlargement effect. This result shows that visualizing implicit speech intentions using the face enlargement effect can smooth remote communication by making speech opportunities more equal.

[0175] This specification includes the following disclosure. [Appendix 1] The computer (1) A step of acquiring a training data set (30) including a first training sample (31) and a second training sample (35) of observation data of a target (ST), wherein the first training sample (31) is assigned a label (32) indicating that the target (ST) has an intention to take action, and the second training sample (35) is not assigned the label (32); a step of controlling machine learning of an inference model (50) by learning from positive examples and unlabeled data using the acquired training dataset (30), the machine learning being configured by training the inference model (50) to acquire the ability to infer from the observation data the degree to which the subject (ST) has the intention to perform the behavior; and outputting the results of the machine learning; To execute Model generation method. [Appendix 2] The subject (ST) is a subject (T10), The action is a speech. 10. The method of generating a model according to claim 1. [Appendix 3] The subject (ST) is a subject (T20), The behavior is an behavior for a robot device (R20). 10. The method of generating a model according to claim 1. [Appendix 4] The target (ST) is a mobile robot (T30) present around the mobile bodies (M30, M31), The action is to interrupt the route of the moving body (M30, M31). 10. The method of generating a model according to claim 1. [Appendix 5] The first training sample (31) is extracted from a section (ST10) in the observation data immediately before an occurrence section in which the target (ST) is performing the behavior. 5. The model generation method of any one of appendices 1 to 4. [Appendix 6] The second training sample (35) is extracted from a section (ST11) in the observation data that is before the occurrence section in which the target (ST) is performing the behavior, excluding a section (ST10) immediately before the occurrence section. 6. The model generation method of any one of appendices 1 to 5. [Appendix 7] The subject (ST) is a subject (T1, T2, T3), The observation data may be image data, sound data, text data, biometric data, device operation data, motion capture data, position data, or a combination thereof. 7. The model generation method of any one of appendixes 1, 2, 3, 5, and 6. [Appendix 8] The target (ST) is a robotic device (T4, T5, T6), The observation data is composed of image data, sound data, text data, motion status data, position data, or a combination thereof. 10. The model generation method of any one of appendixes 1, 4, 5, and 6. [Appendix 9] On the computer (2), obtaining a sample (41) of observed data of the target (TT); using a trained inference model (60) to infer from the acquired sample of observational data (41) the degree to which the subject (TT) intends to take an action; and outputting the result of the inference; In order to execute Inference program (82). [Appendix 10] The trained inference model (60) is generated by learning from positive examples and unlabeled data. The inference program (82) described in Appendix 9. [Appendix 11] The subject (TT) is a subject (T10), The action is a speech. An inference program (82) according to appendix 9 or 10. [Appendix 12] Outputting the result of the inference includes outputting an effect that visualizes the result of inferring the degree to which the subject (T10) has the intention to speak. The inference program (82) described in Appendix 11. [Appendix 13] The observation data includes image data showing the face of the target person (T10) in the video conference, Outputting the effect includes outputting an effect of enlarging the face of the target person (T10) to a terminal of a conversation partner of the target person (T10) in the video conference when the target person (T10) intends to speak. The inference program (82) described in Appendix 12. [Appendix 14] The subject (TT) is a subject (T20), The behavior is an behavior for a robot device (R20). An inference program (82) according to appendix 9 or 10. [Appendix 15] outputting a result of the inference Selecting an action of the robot device (R20) according to a result of inferring the degree to which the user has an intention to take an action on the robot device (R20); and outputting a command to the robot device (R20) to instruct the robot device (R20) to execute the selected operation; Including, The inference program (82) described in Appendix 14. [Appendix 16] The target (TT) is a mobile robot (T30) present around the mobile bodies (M30, M31), The action is to interrupt the route of the moving body (M30, M31). An inference program (82) according to appendix 9 or 10. [Appendix 17] outputting the result of the inference includes outputting an instruction to the moving body (M30, M31) to change the path when the mobile robot (T30) has an intention to cut into the path. The inference program (82) described in Appendix 16. [Appendix 18] A model generation device (1) including a control unit (11), The control unit (11) A step of acquiring a training data set (30) including a first training sample (31) and a second training sample (35) of observation data of a target (ST), wherein the first training sample (31) is assigned a label (32) indicating that the target (ST) has an intention to take action, and the second training sample (35) is not assigned the label (32); a step of controlling machine learning of an inference model (50) by learning from positive examples and unlabeled data using the acquired training dataset (30), the machine learning being configured by training the inference model (50) to acquire the ability to infer from the observation data the degree to which the subject (ST) has the intention to perform the behavior; and outputting the results of the machine learning; configured to perform Model generation device. [Appendix 19] An inference device (2) having a control unit (21), The control unit (21) obtaining a sample (41) of observed data of the target (TT); using a trained inference model (60) to infer from the acquired sample of observational data (41) the degree to which the subject (TT) intends to take an action; and outputting the result of the inference; configured to perform Reasoning device (2). [Explanation of symbols]

[0176] 1...Model generation device, 11...control unit, 12...storage unit, 13...communication interface, 14...input device, 15...output device, 16...drive, 81...generation program, 91...storage medium, 111...acquisition unit, 112...learning processing unit, 113...output processing unit, 2... Reasoning device, 21...control unit, 22...storage unit, 23...communication interface, 24...input device, 25...output device, 26...drive, 27...External interface, 82...inference program, 92...storage medium, 30...training dataset, 31...first training sample, 32...label, 35...second training sample, 41...sample, 50...inference models, 60...trained inference models, ST·TT...Target, S...Sensor

Claims

1. The computer obtaining a training dataset including a first training sample and a second training sample of observation data of a subject, the first training sample being labeled with a label indicating that the subject has an intention to perform an action, and the second training sample being unlabeled; Controlling machine learning of an inference model by learning from positive examples and unlabeled data using the acquired training dataset, wherein the machine learning is configured by training the inference model to acquire the ability to infer, from the observation data, the degree to which the subject intends to perform the behavior; and outputting the results of the machine learning; To execute Model generation method.

2. The subject is a human subject, The action is a speech. The model generation method of claim 1 .

3. The subject is a human subject, The behavior is an behavior for a robotic device. The model generation method of claim 1 .

4. the target is a mobile robot present around the moving body, the action is to interrupt the route of the moving object; The model generation method of claim 1 .

5. The first training sample is extracted from a section immediately preceding an occurrence section in which the subject is performing the behavior in the observed data. The model generation method of claim 1 .

6. the second training sample is extracted from a section in the observation data that precedes an occurrence section in which the subject is performing the behavior, excluding a section immediately before the occurrence section; The model generation method of claim 1 .

7. The subject is a human subject, The observation data may be image data, sound data, text data, biometric data, device operation data, motion capture data, position data, or a combination thereof. The model generation method of claim 1 .

8. the target is a robotic device, The observation data is composed of image data, sound data, text data, motion status data, position data, or a combination thereof. The model generation method of claim 1 .

9. On the computer, obtaining a sample of observed data of interest; Using a trained inference model, the object is identified from the acquired samples of observation data. Inferring the degree of intent to take the action; and outputting the result of the inference; In order to execute Inference program.

10. The trained inference model is generated by learning from positive examples and unlabeled data. The inference program according to claim 9.

11. The subject is a human subject, The action is a speech. The inference program according to claim 9.

12. and outputting the inference result includes outputting an effect that visualizes the inference result regarding the degree to which the subject has an intention to speak. The inference program according to claim 11.

13. the observation data includes image data showing the faces of the participants in the video conference; outputting the effect includes, when the target person has an intention to speak, outputting an effect of enlarging the target person's face to a terminal of a conversation partner of the target person in the video conference. The inference program according to claim 12.

14. The subject is a human subject, The behavior is an behavior for a robotic device. The inference program according to claim 9.

15. outputting a result of the inference selecting an action of the robotic device in response to a result of inferring a degree of intention to take an action on the robotic device; and outputting a command to the robot device to instruct the robot device to perform the selected action; Including, The inference program according to claim 14.

16. the target is a mobile robot present around the moving body, the action is to interrupt the route of the moving object; The inference program according to claim 9.

17. and outputting the result of the inference includes outputting an instruction to the mobile body to change its path when the mobile robot has an intention to cut into the path.

17. The inference program according to claim 16.

18. A model generation device including a control unit, The control unit obtaining a training dataset including a first training sample and a second training sample of observation data of a subject, the first training sample being labeled with a label indicating that the subject has an intention to perform an action, and the second training sample being unlabeled; Learning from positive examples and unlabeled data using the acquired training dataset controlling machine learning of an inference model to be used in the analysis, the machine learning being configured to train the inference model to acquire the ability to infer, from the observation data, the degree to which the subject intends to perform the behavior; and outputting the results of the machine learning; configured to perform Model generation device.

19. An inference device including a control unit, The control unit obtaining a sample of observed data of interest; using a trained inference model to infer from the sample of observational data the degree to which the subject intends to take an action; and outputting the result of the inference; configured to perform Reasoning device.

Images