Information processing apparatus, information processing method, and computer storage medium

Abstract

Provided is an information processing apparatus, an information processing method, and a computer storage medium, which include an identification unit that performs an identification process for determining a motion of an autonomous operating body based on collected sensor information. The identification unit includes a feedback identifier that identifies feedback from a user in response to a behavior exhibited by the autonomous operating body. The feedback identifier identifies a degree of such feedback based on identifying results of contact actions and non-contact actions performed by the user on the autonomous operating body.

Description

[0001] This application is a divisional application of the PCT application for International Application No. PCT / JP2019 / 040162, filed on October 11, 2019, entitled “Information Processing Apparatus, Information Processing Method, and Program”, which entered the national phase in China on May 14, 2021, and has an application number of 201980075385.9, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to an information processing apparatus, an information processing method, and a program. BACKGROUND

[0003] In recent years, devices that autonomously operate based on various types of recognition processing have been developed. For example, in Patent Literature 1, a robot device that recognizes an environment and a state of a user and performs an action in accordance with the state is disclosed.

[0004] LIST OF CITATIONS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: JP 2003-340760 A SUMMARY

[0007] TECHNICAL PROBLEM

[0008] Here, in an autonomous operating body such as the robot device described in Patent Literature 1, it is important to recognize feedback from a user on a performed action with high precision and correctly reflect the result of the recognition into a subsequent action.

[0009] SOLUTION TO PROBLEM

[0010] According to the present disclosure, there is provided an information processing apparatus including: a recognition unit that performs recognition processing for determining an action of an autonomous operating body based on collected sensor information, wherein the recognition unit includes a feedback recognizer that recognizes feedback from a user on a behavior performed by the autonomous operating body, and the feedback recognizer recognizes at least one of positive feedback and negative feedback based on recognition results of contact actions and non-contact actions of the user with the autonomous operating body; and an action planning unit that determines a behavior to be performed by the autonomous operating body based on a result of the recognition processing of the recognition unit.

[0011] wherein the recognition unit recognizes the non-contact actions based on utterance information of the user or image information in which the user is imaged.

[0012] wherein the recognition unit determines a degree of at least one of the positive feedback and the negative feedback based on the recognition results of the contact actions and the non-contact actions.

[0013] wherein, in a case where a type of feedback based on the recognition result of the contact action and a type of feedback based on the recognition result of the non-contact action are different from each other, the feedback recognizer identifies a final type and degree of feedback by assigning a weight to the recognition result of the contact action.

[0014] wherein, in a case where the recognition result of the contact action and the recognition result of the non-contact action are not obtained within a predetermined time, the feedback recognizer identifies at least one of positive feedback and negative feedback and a degree of the identified feedback based on the recognition result of any one of the contact action and the non-contact action obtained.

[0015] wherein the recognition unit further includes a physical contact recognizer that recognizes the contact action.

[0016] wherein the physical contact recognizer recognizes the contact action based on sensor information collected by a contact sensor or an inertial sensor included in the autonomous operating body.

[0017] wherein the action planning unit corrects a score related to the behavior based on at least one of the positive feedback and the negative feedback and the degree of the identified feedback recognized by the feedback recognizer, and determines the behavior to be performed by the autonomous operating body based on the score.

[0018] wherein the action planning unit reflects at least one of the positive feedback and the negative feedback and the degree of the identified feedback into an emotion of the autonomous operating body.

[0019] wherein the action planning unit causes the autonomous operating body to preferentially perform a behavior for which the score is not calculated.

[0020] According to the present disclosure, there is provided an information processing method including: performing, by a processor, a recognition process for determining an action of an autonomous operating body based on collected sensor information, wherein performing the recognition process further includes: using a feedback recognizer that recognizes feedback from a user to a behavior performed by the autonomous operating body, and identifying at least one of positive feedback and negative feedback based on recognition results of a contact action and a non-contact action of the user to the autonomous operating body; and determining, by an action planning unit, a behavior to be performed by the autonomous operating body based on a result of the recognition process of the processor.

[0021] According to the present disclosure, there is provided a computer storage medium storing a program that, when executed, causes a computer to function as an information processing apparatus including: an identification unit that performs an identification process for determining an action of an autonomous operating body based on collected sensor information, wherein the identification unit includes a feedback identifier that identifies feedback from a user on a behavior performed by the autonomous operating body, and the feedback identifier identifies at least one of positive feedback and negative feedback based on a result of identification of a contact action and a non-contact action of the user on the autonomous operating body; and an action planning unit that determines a behavior to be performed by the autonomous operating body based on a result of the identification process of the identification unit.

[0022] According to the present disclosure, there is provided an information processing apparatus including: an identification unit that performs an identification process for determining an action of an autonomous operating body based on collected sensor information, wherein the identification unit includes a feedback identifier that identifies feedback from a user on a behavior performed by the autonomous operating body, and the feedback identifier identifies a degree of feedback based on a result of identification of a contact action and a non-contact action of the user on the autonomous operating body.

[0023] Further, according to the present disclosure, there is provided an information processing method including: performing, by a processor, an identification process for determining an action of an autonomous operating body based on collected sensor information, wherein performing the identification process further includes: using a feedback identifier that identifies feedback from a user on a behavior performed by the autonomous operating body, and identifying a degree of feedback based on a result of identification of a contact action and a non-contact action of the user on the autonomous operating body.

[0024] Further, according to the present disclosure, there is provided a program for causing a computer to function as an information processing apparatus including: an identification unit that performs an identification process for determining an action of an autonomous operating body based on collected sensor information, wherein the identification unit includes a feedback identifier that identifies feedback from a user on a behavior performed by the autonomous operating body, and the feedback identifier identifies a degree of feedback based on a result of identification of a contact action and a non-contact action of the user on the autonomous operating body. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a diagram showing a hardware configuration example of an autonomous mobile body according to a first embodiment of the present disclosure.

[0026] Figure 2 is a configuration example of an actuator included in an autonomous mobile body according to an embodiment.

[0027] Figure 3 is a diagram for explaining an operation of an actuator included in an autonomous mobile body according to an embodiment.

[0028] Figure 4 is a diagram for explaining an operation of an actuator included in an autonomous mobile body according to an embodiment.

[0029] Figure 5 is a diagram for explaining a function of a display included in an autonomous mobile body according to an embodiment.

[0030] Figure 6 is a diagram showing an operation example of an autonomous mobile body according to an embodiment.

[0031] Figure 7 is a diagram for explaining an example of a case where an autonomous mobile body as a comparative object erroneously recognizes feedback from a user according to an embodiment.

[0032] Figure 8 is a diagram for explaining an example of a case where an autonomous mobile body as a comparative object erroneously recognizes feedback from a user according to an embodiment.

[0033] Figure 9 is a diagram for explaining an overview of feedback recognition of an autonomous mobile body according to an embodiment.

[0034] Figure 10 is a diagram showing an example of a contact portion determination based on a slight vibration according to an embodiment.

[0035] Figure 11 is a diagram showing a configuration example of an information processing system according to an embodiment.

[0036] Figure 12 is a block diagram showing a functional configuration example of an autonomous mobile body according to an embodiment.

[0037] Figure 13 is a block diagram showing a detailed functional configuration example of an autonomous mobile body according to an embodiment.

[0038] Figure 14 is a diagram for explaining recognition of a contact action using an inertial sensor according to an embodiment.

[0039] Figure 15 is a diagram showing an example of classification of a contact action by a physical contact recognizer according to an embodiment.

[0040] Figure 16 is a diagram showing an example of classification of a non-contact action by a voice recognizer according to an embodiment.

[0041] Figure 17 is a flowchart showing a flow of feedback recognition according to an embodiment.

[0042] Figure 18is a diagram illustrating an example of feedback recognition based on recognition of contact actions and non-contact actions according to an embodiment.

[0043] Figure 19 is a flowchart illustrating a processing flow of a behavior selector according to an embodiment.

[0044] Figure 20 is a diagram for explaining learning of feedback aspects according to an embodiment. DETAILED DESCRIPTION

[0045] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that, in this specification and the drawings, components having substantially the same function are denoted with the same reference numerals, in order to omit repeated description.

[0046] Note that the description will be given in the following order.

[0047] 1. Embodiment

[0048] 1.1. Overview of Autonomous Operator 10

[0049] 1.2. Example of Hardware Configuration of Autonomous Operator 10

[0050] 1.3. Functional Overview

[0051] 1.4. Example of System Configuration

[0052] 1.5. Example of Functional Configuration of Autonomous Operator 10

[0053] 1.6. Functional Details

[0054] 2. Conclusion

[0055] <1. Embodiment>

[0056] <<1.1. Overview of Autonomous Operator 10>>

[0057] First, a description of an overview of the autonomous operator 10 according to an embodiment of the present disclosure will be given. The autonomous operator 10 according to an embodiment of the present disclosure is an information processing device that performs situation estimation based on collected sensor information, and autonomously selects and performs various operations according to the situation. One of the features of the autonomous operator 10 is that, unlike a robot that performs operations only according to a user's instruction command, the autonomous operator 10 autonomously performs operations assumed to be optimal for each situation.

[0058] The autonomous operator 10 according to the embodiment of the present disclosure can perform user recognition or object recognition, for example, based on an imaged image, and perform various autonomous actions according to the recognized user or object. Further, the autonomous operator 10 according to the present embodiment can also perform, for example, speech recognition based on a user's utterance, and perform actions based on the user's instructions and the like.

[0059] As described above, similarly to an animal including a human, the autonomous operator 10 according to the embodiment of the present disclosure determines and performs autonomous operation by comprehensively determining desires, emotions, surrounding environments, and the like. In the above aspect, the autonomous operator 10 is significantly different from a passive device that performs a corresponding operation or process based on an instruction.

[0060] The autonomous operator 10 according to the embodiment of the present disclosure can be an autonomous mobile robot that autonomously moves in a space and performs various operations. The autonomous operator 10 can be, for example, an autonomous mobile robot having a shape and an operation ability that imitate an animal such as a human or a dog. Further, the autonomous operator 10 can be, for example, a vehicle or another device having an ability to communicate with a user. The shape, ability, and desire level and the like of the autonomous operator 10 according to the embodiment of the present disclosure can be appropriately designed according to the object and the role.

[0061] <<1.2. Example of hardware configuration of autonomous operator 10>>

[0062] Next, a description of an example of a hardware configuration of the autonomous operator 10 according to the embodiment of the present disclosure will be given. Note that, hereinafter, a case where the autonomous operator 10 is a dog-shaped quadruped walking robot will be described as an example.

[0063] Figure 1 is a diagram showing an example of a hardware configuration of the autonomous operator 10 according to the embodiment of the present disclosure. As shown in Figure 1 , the autonomous operator 10 is a dog-shaped quadruped walking robot having a head, a torso, four legs, and a tail. Further, the autonomous operator 10 includes two displays 510 in the head.

[0064] Further, the autonomous operator 10 includes various sensors. The autonomous operator 10 includes, for example, a microphone 515, a camera 520, a time-of-flight (ToF) sensor 525, a human body sensor 530, a distance measuring sensor 535, a touch sensor 540, an illuminance sensor 545, a sole button 550, and an inertial sensor 555.

[0065] (Microphone 515)

[0066] The microphone 515 has a function of collecting surrounding sounds. The above sounds include, for example, utterances of the user and surrounding environmental sounds. The autonomous operator 10 can include, for example, four microphones on the head. By including a plurality of microphones 515, it is possible to collect sounds generated around with high sensitivity, and to achieve positioning of a sound source.

[0067] (Camera 520)

[0068] The camera 520 has a function of capturing images of the user and the surrounding environment. The autonomous operator 10 can include, for example, two wide-angle cameras located at the tip of the nose and the waist. In this case, the wide-angle camera arranged at the tip of the nose captures an image corresponding to the front field of view of the autonomous operator (i.e., the field of view of the dog), and the wide-angle camera at the waist captures an image of the surrounding area centered on the above side. The autonomous operator 10 can extract feature points of the ceiling based on, for example, an image captured by the wide-angle camera arranged at the waist, and achieve simultaneous localization and mapping (SLAM).

[0069] (ToF sensor 525)

[0070] The ToF sensor 525 has a function of detecting the distance to an object present in front of the head. The ToF sensor 525 is included at the tip of the nose of the head. With the ToF sensor 525, it is possible to detect the distance to various objects with high accuracy, and it is possible to perform operations depending on the relative position to obstacles, objects including the user, and the like.

[0071] (Human body sensor 530)

[0072] The human body sensor 530 has a function of detecting the position of the user or a pet raised by the user. The human body sensor 530 is arranged, for example, at the chest. With the human body sensor 530, by detecting a moving object present in front, it is possible to implement various operations on the moving object, for example, operations depending on emotions such as interest, fear, and surprise.

[0073] (Distance measuring sensor 535)

[0074] The distance measuring sensor 535 has a function of acquiring the state of the front floor surface of the autonomous operator 10. The distance measuring sensor 535 is arranged, for example, at the chest. With the distance measuring sensor 535, it is possible to detect the distance to an object present on the front floor surface of the autonomous operator 10 with high accuracy, and it is possible to perform operations depending on the relative position to the object.

[0075] (Touch sensor 540)

[0076] The touch sensor 540 has a function of detecting contact of a user. The touch sensor 540 is arranged, for example, in portions of the autonomous operation body 10 that the user is likely to touch, such as the top of the head, under the chin, and the back. The touch sensor 540 can be, for example, a capacitive type or a pressure-sensitive type touch sensor. With the touch sensor 540, a contact action of the user, such as a touch, a pat, a tap, or a press, can be detected, and an operation can be performed in accordance with the contact action.

[0077] (Illuminance sensor 545)

[0078] The illuminance sensor 545 detects illuminance in a space in which the autonomous operation body 10 is located. The illuminance sensor 545 can be arranged, for example, at the root of the tail in the back of the head. With the illuminance sensor 545, ambient brightness can be detected, and an operation can be performed in accordance with the brightness.

[0079] (Sole button 550)

[0080] The sole button 550 has a function of detecting whether the bottom surface of the leg of the autonomous operation body 10 is in contact with the floor. To this end, the sole button 550 is arranged at each portion of the pad corresponding to the four legs. With the sole button 550, contact or non-contact between the autonomous operation body 10 and the floor surface can be detected, and it can be grasped, for example, that the autonomous operation body 10 is lifted by the user.

[0081] (Inertial sensor 555)

[0082] The inertial sensor 555 is a 6-axis sensor that detects physical quantities such as the speed, acceleration, and rotation of the head and the torso. That is, the inertial sensor 555 detects acceleration and angular velocity on the X-axis, the Y-axis, and the Z-axis. The inertial sensor 555 is arranged on each of the head and the torso. With the inertial sensor 555, the motion of the head and the torso of the autonomous operation body 10 can be detected with high precision, and operation control can be implemented in accordance with the situation.

[0083] In the foregoing, an example of the sensor included in the autonomous operation body 10 according to the embodiment of the present disclosure has been described. Note that the description is made with reference to Figure 1 The above-described configuration is merely an example, and the configuration of the sensor that can be included in the autonomous operation body 10 is not limited to such an example. In addition to the above configuration, the autonomous operation body 10 can include, for example, a temperature sensor, a geomagnetic sensor, various communication devices including a global navigation satellite system (GNSS) signal receiver, and the like. The configuration of the sensor included in the autonomous operation body 10 can be modified flexibly in accordance with the specifications and operations.

[0084] Subsequently, a description will be given of an example of the configuration of the joint of the autonomous operation body 10 according to the embodiment of the present disclosure. Figure 2is a configuration example of the actuator 570 included in the autonomous operator 10 according to an embodiment of the present disclosure. In addition to Figure 2 The autonomous operator 10 according to an embodiment of the present disclosure has a total of 22 rotational degrees of freedom, two at each of the ears and the tail, and one at the mouth, in addition to the rotation point shown.

[0085] For example, the autonomous operator 10 has three degrees of freedom at the head, enabling both nodding and tilting the neck. In addition, the autonomous operator 10 can realize a natural and flexible motion closer to a real dog by reproducing a swinging motion of the waist by the actuator 570 included in the waist.

[0086] Note that the autonomous operator 10 according to an embodiment of the present disclosure can realize the above 22 rotational degrees of freedom by, for example, combining a 1-axis actuator and a 2-axis actuator. For example, the 1-axis actuator can be used for the elbow or the knee of the leg, and the 2-axis actuator can be used for the shoulder and the thigh root.

[0087] Figure 3 and Figure 4 is a diagram for explaining the operation of the actuator 570 included in the autonomous operator 10 according to an embodiment of the present disclosure. Referring to Figure 3 , the actuator 570 can drive the movable arm 590 at an arbitrary rotational position and rotational speed by rotating the output gear by the motor 575.

[0088] Referring to Figure 4 , the actuator 570 according to an embodiment of the present disclosure includes a back cover 571, a gear case cover 572, a control board 573, a gear case base 574, a motor 575, a first gear 576, a second gear 577, an output gear 578, a detection magnet 579, and two bearings 580.

[0089] The actuator 570 according to an embodiment of the present disclosure can be, for example, a magnetic spin valve giant magnetoresistance (svGMR). The control board 573 rotates the motor 575 based on the control of the main processor, whereby power is transmitted to the output gear 578 via the first gear 576 and the second gear 577, and the movable arm 590 can be driven.

[0090] In addition, the position sensor included in the control board 573 detects the rotational angle of the detection magnet 579 that rotates in synchronization with the output gear 578, whereby the rotational angle, i.e., the rotational position, of the movable arm 590 can be detected with high accuracy.

[0091] Note that since the magnetic svGMR is non-contact type, the magnetic svGMR has excellent durability, and by using in the GMR saturation region, has the advantage of being less affected by signal fluctuation due to distance fluctuation of the detection magnet 579 and the position sensor.

[0092] In the foregoing, a description has been given of an example of a configuration of the actuator 570 included in the autonomous operator 10 according to the embodiment of the present disclosure. With the above configuration, bending and stretching movements of the joints included in the autonomous operator 10 can be controlled with high precision, and the rotational positions of the joints can be detected accurately.

[0093] Subsequently, a description will be given of the functions of the display 510 included in the autonomous operator 10 according to the embodiment of the present disclosure, with reference to Figure 5 Figure 5 is a diagram for illustrating the functions of the display 510 included in the autonomous operator 10 according to the embodiment of the present disclosure.

[0094] (Display 510)

[0095] The display 510 has a function of visually expressing the eye movements and emotions of the autonomous operator 10. As Figure 5 indicated, the display 510 can express the movements of the eyeball, the pupil, and the eyelid in accordance with the emotions and the movements. The display 510 is intended not to display characters, symbols, or images unrelated to the eyeball movements, thereby producing a natural movement closer to a real animal such as a dog.

[0096] Further, as Figure 5 indicated, the autonomous operator 10 includes two displays 510r and 510l corresponding to the right eye and the left eye, respectively. The displays 510r and 510l are implemented by, for example, two independent organic light-emitting diodes (OLEDs). With the OLEDs, the curved surface of the eyeball can be reproduced, and a more natural appearance can be achieved as compared to a case where a pair of eyeballs is expressed with one flat panel display or a case where two eyeballs are expressed with two independent flat panel displays, respectively.

[0097] As described above, with the displays 510r and 510l, the line of sight and the emotions of the autonomous operator 10 as Figure 5 indicated can be expressed with high precision and flexibility. Further, the user can intuitively grasp the state of the autonomous operator 10 from the movements of the eyeballs displayed on the display 510.

[0098] In the foregoing, a description has been given of an example of a hardware configuration of the autonomous operator 10 according to the embodiment of the present disclosure. With the above configuration, as Figure 6 indicated, by controlling the movements of the joints and the eyeballs of the autonomous operator 10 with high precision and flexibility, a movement and an emotion expression closer to a real living being can be achieved. Note that Figure 6 is a diagram illustrating an example of the operation of the autonomous operator 10 according to the embodiment of the present disclosure, and in Figure 6 ​In the following description, a description focusing on the motion of the joints and the eyeball of the autonomous operator 10 is given, and thus the external structure of the autonomous operator 10 is shown in a simplified manner. Similarly, in the following description, the external structure of the autonomous operator 10 can be shown in a simplified manner, but the hardware configuration and the exterior of the autonomous operator 10 according to the embodiment of the present disclosure are not limited to the example shown in the drawings, and can be designed as appropriate.

[0099] <<1.3. Functional Overview>>

[0100] Next, a description of an overview of the function of the autonomous operator 10 according to the embodiment of the present disclosure will be given. As described above, in a device that performs autonomous operation, it is important to recognize feedback from a user on the performed action with high precision, and to correctly reflect the result of the recognition into a subsequent action.

[0101] Here, the above feedback means, for example, that the user shows a reaction such as praise or scolding to the action (behavior) performed by the autonomous operator 10. By recognizing the feedback as described above, the autonomous operator 10 according to the present embodiment will more often perform a behavior to be praised by the user, and will gradually stop performing a behavior to be scolded by the user.

[0102] However, in a case where the above feedback is incorrectly recognized, there is a possibility that a behavior suitable for the user's taste cannot be correctly performed. Figure 7 and Figure 8 is a diagram for illustrating an example of a case where the autonomous operator 90 according to the present embodiment incorrectly recognizes feedback from a user.

[0103] For example, in the case of the example shown in Figure 7 , the autonomous operator 90 recognizes that the user U1 praises the autonomous operator 90 by recognizing the utterance U01 of the user U1 saying "good child". However, in reality, the utterance U01 of the user U1 is directed to the user U2 in the same space, and it can be said that the above feedback recognition of the autonomous operator 90 is incorrect.

[0104] In this case, the autonomous operator 90 has a possibility of incorrectly recognizing that the user U1 prefers a behavior performed at the time of recognizing the utterance U01 or immediately before recognizing the utterance U01, and thereafter more frequently performs the behavior. However, here, for example, in a case where the user U1 has a low original evaluation of the behavior, a result that damages the user experience can be obtained.

[0105] Further, in the case of the example shown in Figure 8 , the autonomous operator 90 recognizes that the user U1 pats the head of the autonomous operator 90, thereby recognizing that the user U1 praises the autonomous operator 90. However, in Figure 8In the example shown, as shown in utterance UO2, the user U1 only pats the head of the autonomous operator 90 as a substitute for a greeting when going out.

[0106] Also in this case, similarly to Figure 7 In the case of the example shown, the autonomous operator 90 has a possibility of erroneously recognizing that the user U1 prefers a behavior performed at the time when the action of patting the head is recognized or immediately before the action of patting the head is recognized, and thereafter more frequently performs the behavior.

[0107] As described above, in feedback recognition using one of voice and contact, feedback from the user can be erroneously recognized, and reliability is insufficient, and this can also be a cause of frequent occurrence of an action that does not conform to the user's intention.

[0108] The technical concept according to the present disclosure is conceived by focusing on the above points, and makes it possible to recognize feedback from the user to the behavior of the autonomous operator with high accuracy. To this end, one of the features of the autonomous operator 10 according to the embodiment of the present disclosure is that the autonomous operator 10 comprehensively recognizes feedback from the user, for example, based on a plurality of pieces of sensor information collected. Further, at this time, the autonomous operator 10 according to the embodiment of the present disclosure can recognize the type and degree of feedback from the user, and reflect it in a later action plan.

[0109] Figure 9 is a diagram for explaining an overview of feedback recognition of the autonomous operator 10 according to the present embodiment. In Figure 9 In the case of the example shown, the user U1 pats the head, and in response to the behavior performed by the autonomous operator 10, utters the utterance UO3 of "Good boy, John".

[0110] At this time, the autonomous operator 10 according to the present embodiment recognizes the patting of the head by the touch sensor 540 arranged on the head, and also recognizes that the user U1 utters the words of praise by performing voice recognition on the utterance UO3 collected by the microphone 515.

[0111] In this case, the autonomous operator 10 according to the present embodiment can determine that the user U1 praises the autonomous operator 10 because both the recognition result of the contact action of patting the head and the non-contact action by the utterance UO3 indicate positive feedback.

[0112] Further, the autonomous operator 10 according to the present embodiment can comprehensively recognize feedback from the user by using various types of information collected and recognized, but is not limited thereto. In Figure 9 In the case of the example shown, the autonomous operator 10 can strengthen the belief that the feedback of the user U1 is positive by, for example, recognizing the smiling face of the user U1.

[0113] Further, for example, the autonomous operation body 10 can determine that the likelihood that the utterance UO3 (i.e., positive feedback) is directed to the autonomous operation body 10 is extremely high by recognizing that the name of the autonomous operation body 10 is included in the utterance UO3 with praise words.

[0114] As described above, the autonomous operation body 10 according to the present embodiment can highly accurately recognize the type and degree of feedback from a user by performing various types of recognition processing based on collected sensor information, and use the feedback for subsequent action planning.

[0115] Further, in general devices, an electrostatic or pressure-sensitive touch sensor is used to recognize a contact action such as a user's pat or tap. Here, for example, in order to recognize a contact regardless of where on the skin the contact is made as in actual living beings, it is necessary to arrange the touch sensor on the entire outside of the device. However, such an arrangement of the touch sensor is unrealistic because it increases development costs. For this reason, conventional devices have difficulty in recognizing a contact action of a user to a portion for which a touch sensor is not arranged.

[0116] On the other hand, in the autonomous operation body 10 according to the present embodiment, even in a standing state, each joint is provided with a gap that disperses force. For this reason, in the autonomous operation body 10 according to the present embodiment, a slight vibration is generated regardless of which portion is subjected to a contact action.

[0117] Therefore, the autonomous operation body 10 according to the present embodiment can recognize a characteristic vibration generated at the time of contact with each portion by detecting the above slight vibration by the inertial sensor 555, and determine the portion on which the contact action is performed.

[0118] Figure 10 is a diagram showing an example of contact portion determination based on a slight vibration according to the present embodiment. In the example shown in Figure 10 In the case of the example shown, the user Ul pats the abdomen of the autonomous operation body 10, and utters the utterance UO4 of "Good child!". However, since the touch sensor 540 according to the present embodiment is arranged only on the head, chin, and back of the autonomous operation body 10, it is generally difficult to recognize patting of the abdomen only from the detection information of the touch sensor 540.

[0119] However, the autonomous operation body 10 according to the present embodiment can recognize that the user Ul pats the abdomen by detecting a slight vibration generated by the contact action of the inertial sensor 555, and recognizing that the slight vibration is a vibration characteristic of when the abdomen is patted.

[0120] As described above, with the above function of the autonomous operation body 10 according to the present embodiment, a contact action of a user can be recognized even for a portion for which the touch sensor 540 is not arranged, and the recognition accuracy of feedback can be more effectively improved.

[0121] <<1.4. System configuration example>>

[0122] Next, a description will be given of a configuration example of the information processing system according to the present embodiment. Figure 11 is a diagram illustrating a configuration example of the information processing system according to the present embodiment. As shown in Figure 11 , it includes the autonomous operator 10 and the information processing server 20 according to the present embodiment. Further, the autonomous operator 10 and the information processing server 20 are connected via the network 30 to be able to communicate with each other.

[0123] (Autonomous operator 10)

[0124] The autonomous operator 10 according to the present embodiment is an information processing device that performs various types of recognition processing based on collected sensor information, and autonomously operates based on the results of the various types of recognition processing. As described above, the autonomous operator 10 according to the present embodiment can recognize feedback from a user to a performed behavior with high precision by combining various types of recognition processing.

[0125] (Information processing server 20)

[0126] The information processing server 20 according to the present embodiment is an information processing device that collects various types of sensor information collected by the autonomous operator 10 and learning results of the autonomous operator 10 and stores them as collective wisdom. Further, the information processing server 20 according to the present embodiment can have a function of performing recognition processing and learning based on the above sensor information.

[0127] (Network 30)

[0128] The network 30 has a function of connecting the autonomous operator 10 and the information processing server 20 to each other. The network 30 can include a public line network such as the Internet, a telephone line network, and a satellite communication network, various local area networks (LANs) or wide area networks (WANs) including Ethernet (registered trademark), or the like. Further, the network 30 can include a private line network such as an Internet protocol virtual private network (IP-VPN). Further, the network 30 can include a wireless communication network such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

[0129] In the foregoing, a configuration example of the information processing system according to the present embodiment has been described. Note that the above configuration described with reference to Figure 11 is merely an example, and the configuration of the information processing system according to the present embodiment is not limited to such an example. The configuration of the information processing system according to the present embodiment can be modified flexibly according to specifications and operations.

[0130] <<1.5. Function configuration example of autonomous operator 10>>

[0131] Next, a functional configuration example of the autonomous operator 10 according to the present embodiment will be described. Figure 12 is a block diagram illustrating a functional configuration example of the autonomous operator 10 according to the present embodiment. As shown in Figure 12 the autonomous operator 10 according to the present embodiment includes an input unit 110, an identification unit 120, a learning unit 130, an action planning unit 140, an operation control unit 150, a driving unit 160, an output unit 170, and a server communication unit 180.

[0132] (input unit 110)

[0133] The input unit 110 has a function of collecting various types of information related to the user and the surrounding environment. The input unit 110 collects, for example, the user's utterance, environmental sound generated in the surroundings, image information related to the user and the surrounding environment, and various types of sensor information. To this end, the input unit 110 includes various sensors as shown in Figure 1

[0134] (identification unit 120)

[0135] The identification unit 120 has a function of performing various types of identification related to the user, the surrounding environment, and the state of the autonomous operator 10 based on various types of information collected by the input unit 110. As an example, the identification unit 120 can perform speech recognition, contact action recognition, person recognition, facial expression and gaze recognition, object recognition, motion recognition, spatial region recognition, color recognition, shape recognition, marker recognition, obstacle recognition, step recognition, brightness recognition, and the like.

[0136] Further, one of the features of the identification unit 120 according to the present embodiment is that the identification unit 120 identifies feedback from the user's behavior based on the identification processing as described above. Details of the function of the identification unit 120 according to the present embodiment will be described separately later.

[0137] (learning unit 130)

[0138] The learning unit 130 has a function of performing various types of learning based on sensor information and the like collected by the input unit 110. For example, the learning unit 130 learns the relationship between the contact action at each part of the autonomous operator 10 and the slight vibration generated by the contact action by using a machine learning algorithm such as deep learning.

[0139] (action planning unit 140)

[0140] ​The action planning unit 140 has a function of planning an action of the autonomous mobile body 10 based on various recognition results output by the recognition unit 120 and knowledge learned by the learning unit 130. Details of the function of the action planning unit 140 according to the present embodiment will be described separately later.

[0141] (Operation control unit 150)

[0142] The operation control unit 150 has a function of controlling operations of the driving unit 160 and the output unit 170 based on the action plan of the action planning unit 140. The operation control unit 150, for example, performs rotation control of the actuator 570, display control of the display 510, voice output control of the speaker, and the like based on the above action plan.

[0143] (Driving unit 160)

[0144] The driving unit 160 has a function of bending and stretching a plurality of joints included in the autonomous mobile body 10 based on the control of the operation control unit 150. More specifically, the driving unit 160 drives the actuator 570 included in each joint based on the control of the operation control unit 150.

[0145] (Output unit 170)

[0146] The output unit 170 has a function of outputting visual information and sound information based on the control of the operation control unit 150. To this end, the output unit 170 includes the display 510 and the speaker.

[0147] (Server communication unit 180)

[0148] The server communication unit 180 according to the present embodiment performs information communication with the information processing server 20 via the network 30.

[0149] In the foregoing, an outline of the functional configuration example of the autonomous mobile body 10 according to the present embodiment has been described. Subsequently, with reference to Figure 13 the functional configuration of the autonomous mobile body 10 according to the present embodiment will be described in more detail. Figure 13 is a block diagram illustrating a detailed functional configuration example of the autonomous mobile body 10 according to the present embodiment. Note that, in Figure 13 , a configuration related to feedback recognition according to the present embodiment is mainly illustrated.

[0150] As illustrated, the input unit 110 according to the present embodiment includes a touch sensor 540, an inertial sensor 555, a sole button 550, a microphone 515, and the like. Information about contact of the head, the chin, and the back detected by the touch sensor 540, posture information and vibration information detected by the inertial sensor 555, and a ground contact state detected by the sole button 550 are input to the physical contact recognizer 122 included in the recognition unit 120. Further, sound information collected by the microphone 515 is input to the speech recognizer 124 included in the recognition unit 120.

[0151] As described above, the recognition unit 120 according to the present embodiment performs various types of recognition processing for determining the motion of the autonomous operator 10 based on various types of sensor information collected by the input unit 110. The recognition unit 120 according to the present embodiment includes, for example, the physical contact recognizer 122, the speech recognizer 124, and the feedback recognizer 126.

[0152] The physical contact recognizer 122 according to the present embodiment has a function of recognizing a contact motion of the user to the autonomous operator 10. Here, the above contact motion includes, for example, a stroking motion and a tapping motion. At this time, the physical contact recognizer 122 according to the present embodiment can recognize the contact motion as described above based on sensor information collected by the touch sensor 540 and the inertial sensor 555. Details of the function of the physical contact recognizer 122 according to the present embodiment will be described separately later.

[0153] Further, the speech recognizer 124 according to the present embodiment recognizes an utterance of the user as one non-contact motion according to the present embodiment. Details of the function of the speech recognizer 124 according to the present embodiment will be described separately later.

[0154] Further, the feedback recognizer 126 according to the present embodiment has a function of recognizing feedback of the user to the behavior performed by the autonomous operator 10. Further, the feedback recognizer 126 according to the present embodiment has a function of recognizing the type and the degree of the feedback based on the recognition result of the contact motion and the non-contact motion of the user to the autonomous operator 10.

[0155] Here, the above contact action refers to an action such as stroking or tapping. Further, the above non-contact action can be a speech, a gesture, a facial expression, or the like as a reaction to the behavior performed by the autonomous operator 10. Such a non-contact action is recognized by components such as a speech recognizer 124, a gesture recognizer (not shown), and a facial expression recognizer (not shown) included in the recognition unit 120. For example, the speech recognizer 124 can recognize a speech based on speech information collected by the microphone 515, and the gesture recognizer and the facial expression recognizer can recognize a gesture and a facial expression, respectively, based on image information that images the user. Details of the function of the feedback recognizer 126 according to the present embodiment will be described separately later.

[0156] Further, the action planning unit 140 according to the present embodiment includes a behavior selector 142 and a spinal cord reflex 144. The behavior selector 142 according to the present embodiment determines a behavior to be performed by the autonomous operator 10 based on the results of various types of recognition processing by the recognition unit 120, such as feedback from the user recognized by the feedback recognizer 126. Details of the function of the behavior selector 142 according to the present embodiment will be described separately later.

[0157] Further, the spinal cord reflex 144 according to the present embodiment determines a reflex action to be performed by the autonomous operator 10 based on the contact action or the non-contact action recognized by the recognition unit 120. This action can be performed simultaneously with the behavior selected by the behavior selector 142. For example, in a case where a speech of the user is recognized, the spinal cord reflex 144 can determine an action of moving the ears in a twitching manner. Further, in a case where a contact action is recognized, an action of moving the eyes or the tail can be determined. Further, in a case where both the contact action and the non-contact action are recognized, the spinal cord reflex 144 can determine an action of moving the ears, the eyes, and the tail simultaneously.

[0158] Further, the operation control unit 150 according to the present embodiment controls the actuators 570, the display 510, and the speaker 512 based on the behavior selected by the behavior selector 142 and the action determined by the spinal cord reflex 144.

[0159] In the foregoing, an example of the functional configuration of the autonomous operator 10 according to the present embodiment has been described. Note that the functional configuration of the autonomous operator 10 according to the present embodiment is not limited to such an example. Figure 13 The above configuration described is merely an example, and the functional configuration of the autonomous operator 10 according to the present embodiment is not limited to such an example. The functional configuration of the autonomous operator 10 according to the present embodiment can be modified flexibly according to the specifications and the operation.

[0160] <<1.6. Functional Details >>

[0161] Next, each function of the autonomous operator 10 according to the present embodiment will be described in detail. First, the function of the physical contact recognizer 122 according to the present embodiment will be described.

[0162] For example, the physical contact recognizer 122 according to the present embodiment recognizes the contact action of the user on the head, the chin, and the back based on the touch information detected by the touch sensor 540.

[0163] Further, as described above, the physical contact recognizer 122 according to the present embodiment can also recognize the contact action of the user on the portion on which the touch sensor 540 is not arranged.

[0164] Figure 14 is a diagram for illustrating the recognition of the contact action using the inertial sensor 555 according to the present embodiment. As Figure 14 indicated, the autonomous operator 10 according to the present embodiment includes the inertial sensors 555a and 555b located on the head and the torso, respectively.

[0165] As described above, each joint of the autonomous operator 10 according to the present embodiment has a gap, and the slight vibration generated when the contact action is performed varies depending on each portion due to the difference in the structure of the inside of the device.

[0166] For example, as Figure 14 indicated, a case where the abdomen is stroked is assumed. Here, a detachable cover is installed on the abdomen of the autonomous operator 10 according to the present embodiment, and the battery is arranged inside the cover. Further, the actuator 570 capable of moving the neck joint is arranged between the abdomen and the head.

[0167] Further, since the distance between the abdomen performing the contact action and the inertial sensor 555a and the distance between the abdomen and the inertial sensor 555b are different from each other, different features as illustrated in the diagram can appear in the slight vibration detected by the inertial sensors 555a and 555b, respectively.

[0168] To this end, the physical contact recognizer 122 according to the present embodiment can recognize the portion performing the contact action based on the learning result of the sensor information collected by the inertial sensor 555 due to the contact action on each portion of the autonomous operator 10 and the currently collected sensor information. For example, the above learning is performed by using supervised learning of a neural network.

[0169] With the above function of the physical contact recognizer 122 according to the present embodiment, the contact action can be recognized even in the case where the contact action is performed on the portion on which the touch sensor 540 is not arranged.

[0170] Note that when the autonomous operating body 10 is in a specific posture, the physical contact identifier 122 can also more effectively detect contact actions by limiting the candidates to locations where the contact action is predicted to be performed. For example, when the autonomous operating body 10 is in a posture showing its abdomen to the user, it is predicted that the user is very likely to perform a contact action on the abdomen. In this case, the physical contact identifier 122 can recognize that the contact action is performed on the abdomen based on the fact that a small vibration is detected while the autonomous operating body 10 is in the above posture.

[0171] Furthermore, the physical contact recognizer 122 according to the present embodiment is able to classify whether each contact action is positive feedback or negative feedback and the degree thereof based on a learning result using sensor information collected by the touch sensor 540 and the inertial sensor 555 .

[0172] Figure 15 1 is a diagram illustrating an example of classifying contact actions according to the present embodiment by the physical contact recognizer 122. For example, as shown in the figure, in the case where the abdomen, back, or head is strongly hit, the physical contact recognizer 122 may classify the contact action as negative feedback (very bad).

[0173] Furthermore, for example, in the case where stroking is performed while the abdomen is shown, the physical contact recognizer 122 may classify the contact action as mild positive feedback (good).

[0174] Hereinabove, the function of the physical contact recognizer 122 according to the present embodiment has been described in detail. Next, a description will be given of the function of the voice recognizer 124 according to the present embodiment.

[0175] The speech recognizer 124 according to this embodiment performs speech recognition and semantic analysis on the user's speech. Furthermore, the speech recognizer 124 according to this embodiment may include a sound source localization estimation function for estimating the direction of the user's speech. For example, if the name of the autonomous operating object 10 is recognized as included in the speech as a result of the above processing, the speech recognizer 124 may also determine that the speech is directed at the autonomous operating object 10. As described above, the recognition unit 120 according to this embodiment can determine whether the user's non-contact action is directed at the autonomous operating object 10 based on the sensor information collected by the input unit 110.

[0176] Furthermore, the speech recognizer 124 according to the present embodiment is capable of classifying whether the user's utterance is positive feedback or negative feedback and the degree thereof based on the results of speech recognition and semantic analysis.

[0177] Figure 16is a diagram illustrating an example in which the voice recognizer 124 according to the present embodiment classifies non-contact actions. For example, as illustrated, the voice recognizer 124 performs classification on each utterance according to the results of voice recognition and semantic analysis. For example, in a case where the recognized utterance is "You are the best", the voice recognizer 124 can classify the utterance as strong positive feedback (very good). Further, for example, in a case where the recognized utterance is "Bad child", the voice recognizer 124 can classify the utterance as strong negative feedback (very bad).

[0178] Further, the voice recognizer 124 can perform classification taking into account, for example, the user's emotion when the utterance was made in addition to the meaning of the words included in the utterance. For example, in a case where the word "idiot" with an angry emotion is input at a sound pressure greater than or equal to a predetermined value, the word "idiot" which is normally classified as "bad" can be classified as "very bad".

[0179] Further, the voice recognizer 124 can perform classification using sound source positioning information. For example, in a case where the utterance of "Good child" is recognized from the front of the autonomous operator 10, the level can be increased by one level from the normal time. This is because, similarly to the case of scolding a human child, the utterance from the front is effective in attempting to clearly convey the intention.

[0180] In the foregoing, the function of the voice recognizer 124 according to the present embodiment has been described in detail. Next, a description will be given of the function of the feedback recognizer 126 according to the present embodiment. As described above, the feedback recognizer 126 according to the present embodiment has a function of recognizing feedback from a user on behavior based on the contact action recognized by the physical contact recognizer 122 and the non-contact action recognized by the voice recognizer 124 or the like.

[0181] Figure 17 is a flowchart illustrating a flow of feedback recognition according to the present embodiment. In Figure 17 In the example illustrated, a case is illustrated in which the utterance of the user is regarded as a non-contact action according to the present embodiment.

[0182] Referring to Figure 17 , first, the input unit 110 collects various types of sensor information (S1101). The sensor information according to the present embodiment includes various types of information such as sound, image, acceleration, and angular velocity.

[0183] Next, voice recognition by the voice recognizer 124 (S1102) and recognition of contact actions by the physical contact recognizer 122 (S1103) are performed.

[0184] Next, the feedback recognizer 126 according to the present embodiment determines whether the speech recognition result and the contact action recognition result are acquired within a predetermined time (S1104).

[0185] Here, in a case where only one recognition result is acquired (S1104: No), the feedback recognizer 126 performs feedback recognition using only the acquired speech recognition result or contact action recognition result (S1106). That is, in a case where neither the contact action recognition result nor the non-contact action recognition result is acquired within the predetermined time, the feedback recognizer 126 can recognize the type and degree of feedback based on the acquired recognition result of the contact action or non-contact action.

[0186] On the other hand, in a case where both the speech recognition result and the contact action recognition result are acquired (S1104: Yes), the feedback recognizer 126 performs feedback recognition by combining the speech recognition result and the contact action recognition result (S1105).

[0187] Figure 18 is a diagram illustrating an example of feedback recognition based on recognition of a contact action and a non-contact action according to the present embodiment. For example, in a case where both the contact action and the non-contact action are recognized as positive feedback (good), the feedback recognizer 126 can recognize the final classification of feedback as "very good".

[0188] Further, for example, in a case where both the contact action and the non-contact action are recognized as negative feedback (bad), the feedback recognizer 126 can recognize the final classification of feedback as "very bad".

[0189] On the other hand, in a case where the type of feedback based on the recognition result of the contact action and the type of feedback based on the recognition result of the non-contact action are different from each other, the feedback recognizer 126 recognizes the final type and degree of feedback by assigning a weight to the recognition result of the contact action.

[0190] For example, in a case where the contact action is recognized as positive feedback (good) and the non-contact action is recognized as negative feedback (bad), as an example shown in Figure 18 , the feedback recognizer 126 can recognize the final feedback as "good".

[0191] In contrast, in a case where the contact action is recognized as negative feedback (bad) and the non-contact action is recognized as positive feedback (good), the feedback recognizer 126 can recognize the final feedback as "bad".

[0192] With the above functions of the feedback recognizer 126 according to the present embodiment, it is possible to give priority to the contact action that more directly feeds back, and it is expected to improve the accuracy of feedback recognition.

[0193] In the foregoing, a detailed description of the function of the feedback recognizer 126 according to the present embodiment has been given. Note that, in the foregoing, a case where the type of feedback includes both positive and negative types and is classified into two degrees has been exemplified, but the type and degree of feedback according to the present embodiment are not limited to such an example.

[0194] Next, the function of the behavior selector 142 according to the present embodiment will be described in detail. Based on the type and degree of feedback recognized by the feedback recognizer 126, the behavior selector 142 according to the present embodiment increases or decreases the score related to the corresponding behavior to correct the score. Furthermore, the behavior selector 142 according to the present embodiment determines the behavior to be executed by the autonomous operator 10 based on the score acquired as described above.

[0195] The behavior selector 142 can, for example, perform subtraction on the score related to the corresponding behavior in the case where the recognized feedback is negative, and perform addition on the score related to the corresponding behavior in the case where the feedback is positive. Furthermore, in this case, the behavior selector 142 can preferentially select a behavior having a high score.

[0196] With the above function of the behavior selector 142 according to the present embodiment, an action plan in which a praising behavior is more likely to be executed and a scolding behavior is less likely to be executed can be realized. Therefore, the number of behaviors matching the user's taste increases, so that the user's satisfaction can be improved. Furthermore, it is also possible to attract the user's interest more by providing a special behavior or the like that is selected only in the case where the score exceeds a predetermined threshold value due to the increase or decrease in the score based on the continuous feedback.

[0197] Furthermore, the feedback according to the present embodiment can be reflected in the emotion of the autonomous operator 10. The autonomous operator 10 according to the present embodiment has an emotion alone, and is designed so that the emotion changes according to the recognized situation. To this end, in the case where the behavior selector 142 according to the present embodiment reflects the type and degree of feedback in the emotion of the autonomous operator 10 and thus the emotion of the autonomous operator 10 tends to be pleasant through the feedback from the user, it is possible to make the subsequent action express the pleasantness, and a praised behavior is repeatedly executed.

[0198] For example, it is possible to execute the control as described above in the flowchart shown in Figure 19 The control as described above can be executed in the flowchart shown in Figure 19 is a flowchart showing the processing flow of the behavior selector 142 according to the present embodiment.

[0199] Referring to Figure 19 First, based on the feedback to the executed behavior, the behavior selector 142 calculates the score related to the behavior (S1201).

[0200] Next, the behavior selector 142 reflects the feedback into the emotion (S1202).

[0201] Next, the behavior selector 142 performs selection of a behavior based on the calculated score and the emotion (S1203).

[0202] In the above, a description of the processing flow of the behavior selector 142 according to the present embodiment has been given. Note that the behavior selector 142 according to the present embodiment can perform various behavior selections based on the score and the like in addition to the above processing.

[0203] For example, the behavior selector 142 according to the present embodiment can preferentially select a behavior for which no score is calculated. With the above control of the behavior selector 142 according to the present embodiment, by performing a behavior for which no feedback from the user has been obtained, for example, it is possible to prevent similar behaviors from being repeatedly performed, and it is possible to expect an effect that does not make the user feel bored.

[0204] Further, the behavior selector 142 according to the present embodiment can aim to select a behavior for which the user is expected to give negative feedback, and cause the autonomous operator 10 to perform the behavior. With this control, it is possible to observe which feedback is given when the user scolds the autonomous operator 10.

[0205] Figure 20 is a diagram for illustrating learning of aspects of feedback according to the present embodiment. In Figure 20 In the case of the example shown, the behavior selector 142 selects the behavior of barking loudly as a behavior for which the user U1 is expected to give negative feedback, that is, to scold the autonomous operator 10, and causes the autonomous operator 10 to perform the behavior.

[0206] Further, the user U1 utters the utterance UO5 of "Stop!" while frowning at the above behavior of the autonomous operator 10, and further hits the head of the autonomous operator 10.

[0207] As described above, with the behavior selector 142 according to the present embodiment, by causing the autonomous operator 10 to aim to perform a behavior for which the user is likely to scold, it is possible to collect information related to aspects of negative feedback from each user without any discomfort.

[0208] At this time, the learning unit 130 according to the present embodiment learns aspects of negative feedback from the user based on feedback to the behavior as described above. Therefore, it is possible to learn aspects of feedback from each user, and it is possible to achieve more accurate feedback recognition.

[0209] Note that the behavior selector 142 according to the present embodiment can also collect information related to aspects of positive feedback from each user by causing the autonomous operator 10 to aim to perform a behavior for which the user is likely to praise.

[0210] <2. Conclusion>

[0211] As described above, the autonomous operator 10 that implements the information processing method according to the embodiment of the present disclosure includes an identification unit 120 that performs an identification process for determining an action of the autonomous operator 10 based on collected sensor information. Further, the identification unit 120 according to the embodiment of the present disclosure includes a feedback identifier 126 that identifies feedback from a user to a behavior performed by the autonomous operator 10. Further, one of the features of the feedback identifier 126 according to the embodiment of the present disclosure is that the feedback identifier 126 identifies a degree of feedback based on identification results of contact actions and non-contact actions of the user to the autonomous operator 10. With this configuration, it is possible to identify feedback from a user to a behavior of the autonomous operator with high precision.

[0212] In the foregoing, the preferred embodiments of the present disclosure have been described in detail with reference to the attached drawings; however, the technical scope of the present disclosure is not limited to such examples. It is apparent that various changes in or modifications to the embodiments described in the scope of the technical ideas described in the claims can be made by those skilled in the art to which the present disclosure pertains, and it is understood that such examples belong to the technical scope of the present disclosure.

[0213] Further, the effects described in the present specification are merely illustrative or exemplary, and are not limited. That is, in addition to or in place of the above effects, the technology according to the present disclosure can exhibit other effects apparent from the descriptions in the present specification to those skilled in the art to which the present disclosure pertains.

[0214] Further, a program for incorporation into hardware such as a CPU, a ROM, and a RAM of a computer can also be created to function equivalent to the configuration of the autonomous operator 10, and a computer-readable non-transitory recording medium in which the program is recorded can also be provided.

[0215] Further, the steps related to the processing of the autonomous operator 10 in the present specification do not necessarily have to be processed in a time series in the order described in the flowchart. For example, the steps related to the processing of the autonomous operator 10 can be processed in an order different from that described in the flowchart, or can be processed in parallel.

[0216] Note that the following configurations also belong to the technical scope of the present disclosure. (1)

[0218] An information processing apparatus comprising:

[0219] an identification unit that performs an identification process for determining an action of the autonomous operator based on collected sensor information, wherein

[0220] The recognition unit includes a feedback recognizer that recognizes feedback from the user to the behavior performed by the autonomous operator, and

[0221] The feedback recognizer recognizes the degree of feedback based on the recognition result of the contact action and the non-contact action of the user to the autonomous operator. (2)

[0223] The information processing apparatus according to (1), in which

[0224] The recognition unit recognizes the non-contact action based on utterance information of the user or image information in which the user is imaged. (3)

[0226] The information processing apparatus according to (1) or (2), in which

[0227] The recognition unit determines the type of feedback based on the recognition result of the contact action and the non-contact action. (4)

[0229] The information processing apparatus according to any one of (1) to (3), in which

[0230] The recognition unit determines whether the non-contact action of the user is directed to the autonomous operator based on the sensor information. (5)

[0232] The information processing apparatus according to any one of (1) to (4), in which

[0233] In a case where the type of feedback based on the recognition result of the contact action and the type of feedback based on the recognition result of the non-contact action are different from each other, the feedback recognizer recognizes the final type and degree of feedback by assigning a weight to the recognition result of the contact action. (6)

[0235] The information processing apparatus according to any one of (1) to (5), in which

[0236] In a case where the recognition result of the contact action and the recognition result of the non-contact action are not both obtained within a predetermined time, the feedback recognizer recognizes the type and degree of feedback based on the recognition result of any one of the contact action and the non-contact action that is obtained. (7)

[0238] The information processing apparatus according to any one of (1) to (6), in which

[0239] The recognition unit further includes a physical contact recognizer that recognizes the contact action. (8)

[0241] The information processing apparatus according to (7), in which

[0242] The physical contact recognizer recognizes the contact action based on sensor information collected by the contact sensor or the inertial sensor included in the autonomous operating body. (9)

[0244] The information processing apparatus according to (8), in which

[0245] The physical contact recognizer recognizes a part of the autonomous operating body that performs the contact action based on a learning result of sensor information collected by the inertial sensor due to the contact action to each part of the autonomous operating body. (10)

[0247] The information processing apparatus according to (9), in which

[0248] The autonomous operating body includes at least two inertial sensors, and

[0249] The physical contact recognizer recognizes the contact action to the part to which the contact sensor is not arranged based on sensor information collected by the two inertial sensors. (11)

[0251] The information processing apparatus according to any one of (1) to (10), further including:

[0252] An action planning unit determines a behavior to be performed by the autonomous operating body based on a result of the recognition processing of the recognition unit. (12)

[0254] The information processing apparatus according to (11), in which

[0255] The action planning unit corrects a score related to the behavior based on a type and a degree of the feedback recognized by the feedback recognizer, and determines the behavior to be performed by the autonomous operating body based on the score. (13)

[0257] The information processing apparatus according to (12), in which

[0258] The action planning unit reflects the type and the degree of the feedback into an emotion of the autonomous operating body. (14)

[0260] The information processing apparatus according to (12) or (13), in which

[0261] The action planning unit causes the autonomous operating body to preferentially perform a behavior for which the score is not calculated. (15)

[0263] The information processing apparatus according to any one of (11) to (14), further including:

[0264] a learning unit that learns an aspect of feedback of each user. (16)

[0266] The information processing apparatus according to (15), in which

[0267] The action planning unit causes the autonomous operator to perform a behavior in which the user is predicted to give negative feedback, and

[0268] The learning unit learns an aspect of negative feedback from the user based on the feedback about the behavior. (17)

[0270] An information processing method including:

[0271] performing, by a processor, recognition processing for determining an action of an autonomous operator based on collected sensor information, in which

[0272] The performing of the recognition processing further includes:

[0273] using a feedback recognizer that recognizes feedback from a user to a behavior performed by the autonomous operator, and (18)

[0275] A program for causing a computer to function as an information processing apparatus including:

[0276] a recognition unit that performs recognition processing for determining an action of an autonomous operator based on collected sensor information, in which

[0277] The recognition unit includes a feedback recognizer that recognizes feedback from a user to a behavior performed by the autonomous operator, and

[0278] The feedback recognizer identifies a degree of feedback based on recognition results of contact actions and non-contact actions of the user to the autonomous operator.

[0279] Reference mark list

[0280] 10 autonomous operator

[0281] 110 input unit

[0282] 120 recognition unit

[0283] 122 physical contact recognizer

[0284] 124 voice recognizer

[0285] 126 feedback recognizer

[0286] 130 learning unit

[0287] 140 action planning unit

[0288] 142 behavior selector

[0289] 144 spinal reflex arrangement

[0290] 150 operation control unit.

Claims

1.An information processing apparatus comprising: an identification unit that performs an identification process for determining an action of an autonomous operating body based on collected sensor information, wherein the identification unit includes a feedback identifier that identifies feedback from a user on a behavior performed by the autonomous operating body, and the feedback identifier identifies at least one of positive feedback and negative feedback based on identification results of contact actions and non-contact actions of the user on the autonomous operating body, and in a case where the identification result of the contact action and the identification result of the non-contact action indicate feedback types different from each other, the feedback identifier determines a final type and degree of the feedback by weighting the identification result of the contact action; and an action planning unit that determines a behavior to be performed by the autonomous operating body based on a result of the identification process of the identification unit. 2.The information processing apparatus according to claim 1, wherein the identification unit identifies the non-contact action based on utterance information of the user or image information in which the user is imaged. 3.The information processing apparatus according to claim 1, wherein the identification unit determines a degree of at least one of the positive feedback and the negative feedback based on the identification results of the contact action and the non-contact action. 4.The information processing apparatus according to claim 3, wherein the feedback identifier determines a final degree of the feedback by combining both of the identification results obtained when the identification result of the contact action and the identification result of the non-contact action, and enhances the degree of the feedback when both of the contact action and the non-contact action indicate the same type of feedback. 5.The information processing apparatus according to claim 1, wherein in a case where both of the identification result of the contact action and the identification result of the non-contact action are not obtained within a predetermined time, the feedback identifier identifies at least one of the positive feedback and the negative feedback and a degree of the identified feedback based on the identification result of either one of the contact action and the non-contact action obtained. 6.The information processing apparatus according to claim 1, wherein the identification unit further includes a physical contact identifier that identifies the contact action. 7.The information processing apparatus according to claim 6, wherein the physical contact identifier identifies the contact action based on the sensor information collected by a contact sensor or an inertial sensor included in the autonomous operating body. 8.The information processing apparatus according to claim 7, wherein the physical contact identifier identifies a portion of the autonomous operating body that performs the contact action based on a learning result of the sensor information collected by the inertial sensor through the contact action of different portions of the autonomous operating body. 9.The information processing apparatus according to claim 8, wherein the autonomous operating body includes at least two inertial sensors, and the physical contact identifier identifies the contact action on a portion in which the contact sensor is not arranged based on the sensor information collected by the two inertial sensors. 10.The information processing apparatus according to claim 1, wherein the action planning unit corrects a score related to the behavior based on at least one of the positive feedback and the negative feedback identified by the feedback identifier and a degree of the identified feedback, and determines the behavior to be performed by the autonomous operator based on the score. 11.The information processing apparatus according to claim 10, wherein the action planning unit reflects at least one of the positive feedback and the negative feedback and the degree of the identified feedback into an emotion of the autonomous operator. 12.The information processing apparatus according to claim 10, wherein the action planning unit causes the autonomous operator to preferentially perform a behavior for which the score is not calculated. 13.The information processing apparatus according to claim 1, further comprising: a learning unit that learns a feedback pattern of each user. 14.The information processing apparatus according to claim 13, wherein the action planning unit causes the autonomous operator to perform a behavior predicted to receive negative feedback from the user, and the learning unit learns a pattern of negative feedback from the user based on feedback to the behavior. 15.An information processing method comprising: performing, by a processor, an identification process for determining an action of an autonomous operator based on collected sensor information, wherein performing the identification process further includes: using a feedback identifier that identifies feedback from a user to a behavior performed by the autonomous operator, and identifying at least one of positive feedback and negative feedback based on identification results of contact actions and non-contact actions of the autonomous operator by the user; in a case where the identification results of the contact actions and the identification results of the non-contact actions indicate feedback types different from each other, determining a final type and a degree of the feedback by weighting the identification results of the contact actions; and determining, by an action planning unit, a behavior to be performed by the autonomous operator based on a result of the identification process of the processor. 16.A computer storage medium storing a program that, when executed, causes a computer to function as an information processing apparatus, the information processing apparatus comprising: an identification unit that performs an identification process for determining an action of an autonomous operator based on collected sensor information, wherein the identification unit includes a feedback identifier that identifies feedback from a user to a behavior performed by the autonomous operator, and the feedback identifier identifies at least one of positive feedback and negative feedback based on identification results of contact actions and non-contact actions of the autonomous operator by the user, and in a case where the identification results of the contact actions and the identification results of the non-contact actions indicate feedback types different from each other, the feedback identifier determines a final type and a degree of the feedback by weighting the identification results of the contact actions; and an action planning unit that determines a behavior to be performed by the autonomous operator based on a result of the identification process of the identification unit.

Images