robot

Abstract

To provide a projection-type robot for communicating with a user, which avoids unnecessary power usage when the robot is not in use, and makes its expressions during communication more understandable for the user.SOLUTION: A robot (100) includes a body surface member (1), a projector (2), a sensor (3), and a controller (4). The controller (4) executes an existence determination process to determine whether a user exists or not within a predetermined region on the basis of external information acquired through the sensor (3), and a power control process to control power supply to a backlight (21) of the projector (2) if it has been determined that the user does not exist within the predetermined region.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a robot that communicates with a user.

Background Art

[0002] Robots that communicate with a user through facial expressions and conversations have been conventionally known. For example, Patent Document 1 discloses a projection-type robot device that includes a projector that projects facial expressions and changes the facial expressions by changing the video given to the projector. Further, Patent Document 2 discloses an autonomous action-type robot that can change its behavior characteristics and shift to a power-saving mode by selecting an action according to the remaining battery level by an operation control unit, thereby suppressing power consumption.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the robot described in Patent Document 1, since there is no description regarding the control of the backlight, the face is projected with a constant brightness regardless of the surrounding environment. Therefore, the power consumption when not in use is wasted. Further, even if the expression is changed, it is difficult for the change to be conveyed to the user. On the other hand, in the robot described in Patent Document 2, there is a description regarding the power-saving mode. However, there is no disclosure of changing the power supply to the liquid crystal elements that constitute the eyes when projecting the face with the projector and in the power-saving mode. Therefore, even if emotions are expressed with the eyes, since the brightness of the eyes does not change, it is difficult for the expression to be conveyed to the user.

[0005] In light of the circumstances described above, this disclosure aims to prevent wasted power consumption when a projection-type robot that communicates with users is not in use, and to make the expressions used during communication easier for users to understand. [Means for solving the problem]

[0006] To solve the above problems, a robot according to one aspect of the present disclosure comprises a body surface member, a projector that projects an image onto at least a part of the body surface member, a sensor provided on at least a part of the body surface member for acquiring information from the outside world, and a control unit that controls at least the projector, wherein the control unit performs an existence determination process that determines whether or not a user is present in a predetermined area based on the information from the outside world acquired by the sensor, and a power adjustment process that adjusts the power supply to the backlight of the projector if the existence determination process determines that the user is not present in the predetermined area. [Effects of the Invention]

[0007] According to one aspect of this disclosure, in a projection-type robot that communicates with a user, it is possible to prevent wasted power consumption when not in use and to make the expressions used during communication easier for the user to understand. [Brief explanation of the drawing]

[0008] [Figure 1] This is a perspective view showing the appearance of a robot according to an embodiment of this disclosure. [Figure 2] This is a block diagram showing the electrical configuration of the robot. [Figure 3] This is a schematic diagram illustrating how the robot displays images. [Figure 4] This flowchart shows some of the processes performed by the robot's processor. [Figure 5] A flowchart shows some of the processes performed by the processor. [Figure 6] This figure shows an example of an image projected by the robot's projector. [Figure 7] A flowchart shows some of the processes performed by the processor. [Figure 8] A flowchart shows some of the processes performed by the processor. [Modes for carrying out the invention]

[0009] <Embodiment 1> One embodiment of this disclosure will be described in detail below.

[0010] [Schematic configuration] First, let's describe the general configuration of robot 100. Figure 1 is a perspective view showing the exterior of robot 100.

[0011] The robot 100 is configured to communicate with the user. As shown in Figure 1, the robot 100 includes a body surface member 1, a projector 2, a sensor 3, and a control unit 4. The robot 100 according to this embodiment further includes a base 5 and a speaker 6. The robot 100 according to this embodiment also includes a plurality of microphones 32.

[0012] (pedestal) The base 5 is a component that forms the lower part of the robot 100. In this embodiment, the base 5 is shaped like a rectangular parallelepiped. However, the base 5 may have a shape other than a rectangular parallelepiped. Also, the robot 100 does not necessarily have to include the base 5.

[0013] (Body surface material) Body surface member 1 is a component that forms the body surface of the robot 100. The body surface member 1 according to this embodiment is mounted on a base 5. Furthermore, the body surface member 1 according to this embodiment is divided into two members: a torso member 11 and a head member 12.

[0014] The body member 11 is a member that forms the body of the robot 100 (the lower part of the body surface member 1). The body member 11 according to the present embodiment has a hollow hemispherical shape with an open upper part. The body member 11 according to the present embodiment includes a body main body 11a and an annular neck part 11b that surrounds the opening of the body main body 11a. Note that the body member 11 may have a shape other than hemispherical (for example, box-shaped, cylindrical, etc.).

[0015] The head member 12 is a member that forms the head of the robot 100 (the upper part of the body surface member 1). The head member 12 according to the present embodiment is attached on the neck part 11b of the body member 11. The head member 12 according to the present embodiment has a hollow spherical shape with an open lower part. Also, the head member 12 according to the present embodiment is formed of a translucent (partially transmitting light) material (for example, resin, glass, etc.). Note that the head member 12 may have a shape other than spherical (for example, box-shaped, cylindrical, etc.).

[0016] In the body surface member 1 according to the present embodiment, when the head member 12 is attached to the body member 11, the space inside the body member 11 and the space inside the head member 12 are continuous. Note that the body surface member 1 may be divided into three or more members, or may be constituted by one member. Also, the body surface member 1 may have a shape in which the head and the body cannot be clearly distinguished.

[0017] (Projector) The projector 2 according to the present embodiment is disposed inside the body member 11. Details of this projector 2 will be described later.

[0018] (Sensor) The sensor 3 acquires information about the outside world. The sensor 3 is provided on at least a part of the body surface member. The sensor 3 according to the present embodiment includes an image sensor 31a and a microphone 32 (voice sensor). Note that the sensor 3 may include other sensors such as a temperature sensor instead of / along with the image sensor 31a and the microphone 32.

[0019] (Image Sensor) The image sensor 31a according to this embodiment is mounted on the front surface of the body surface member 1 and is configured as part of a camera 31 capable of capturing a predetermined area and generating image data. The brightness of the captured predetermined area can be determined from the image data. In other words, the sensor 3 according to this embodiment is configured to acquire ambient brightness. The camera 31 may be provided on the neck 11b, head member 12, base 5, etc. Ambient brightness may also be acquired by a dedicated light sensor or the like. Furthermore, the camera 31 may be provided at a location away from the main body of the robot 100 (for example, on the wall or ceiling of the room where the robot is installed).

[0020] (microphone) Each of the multiple microphones 32 generates audio data from the user's voice. In this embodiment, eight microphones 32 are provided on the neck portion 11b. The eight microphones 32 are arranged at equal intervals along the circumference of the annular neck portion 11b. That is, the eight microphones 32 are provided on the front, rear, left and right sides, right front, left front, right rear, and left rear of the neck portion 11b, respectively. The microphones 32 may also be provided on the torso body 11a, head member 12, base 5, etc. Furthermore, there may be seven or fewer microphones 32, or nine or more microphones 32.

[0021] (Speaker) Speaker 6 outputs sound according to the control of processor 41. In this embodiment, speaker 6 is provided on the front of base 5. Speaker 6 may also be provided on surfaces other than the front of base 5 (e.g., sides, back, etc.). Speaker 6 may also be provided on the body member 11, head member 12, etc.

[0022] (Control Unit) The control unit 4 according to this embodiment is located inside the body member 11, at a position other than between the projector 2 and the head member 12. Details of the control unit 4 will be described later.

[0023] [Electrical configuration] Next, we will explain the electrical configuration of robot 100. Figure 2 is a block diagram showing the electrical configuration of robot 100, and Figure 3 is a schematic diagram showing how robot 100 displays its face.

[0024] In addition to the projector 2, sensor 3, control unit 4, and speaker 6, the robot 100 includes a power supply circuit 7 as shown in Figure 2. These are electrically connected. The robot 100 operates by receiving power from a power supply P. The power supply P may be a built-in battery located inside the body member 11, base 5, etc., or it may be a household power supply.

[0025] (Control Unit) The control unit 4 includes a processor 41, a primary memory 42, a secondary memory 43, and an input / output interface (hereinafter referred to as I / O IF 44). The processor 41, primary memory 42, secondary memory 43, and I / O IF 44 are interconnected via a bus 45 or the like.

[0026] The processor 41 controls at least the projector 2. In this embodiment, the processor 41 also controls the camera 31, microphone 32, speaker 6, and power supply circuit 7. Examples of devices that can be used as the processor 41 include a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), microcontroller, or a combination thereof. Details of the various processes performed by the processor 41 will be described later.

[0027] (Primary memory) The primary memory 42 stores the control program. It also stores audio data for output from the speaker 6. Furthermore, the primary memory 42 can store external information acquired by the sensor 3. Examples of devices that can be used as the primary memory 42 include flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), ODD (Optical Disk Drive), FDD (Floppy® Disk Drive), or combinations thereof.

[0028] Furthermore, the primary memory 42 stores thresholds used when making decisions in various processes (details of which will be described later). These thresholds include, for example, thresholds for determining Yes / No in each process described later, thresholds for determining whether to brighten or darken the face image I projected onto the head member 12, and thresholds for determining whether to make the color of the face image I a bright color (e.g., a warm color) or a dark color (e.g., a cool color).

[0029] (Secondary memory) The secondary memory 43 temporarily stores the control program read by the processor 41 from the primary memory 42. An example of a device that can be used as the secondary memory 43 is semiconductor RAM (Random Access Memory).

[0030] In this embodiment, the robot 100's memory is implemented using two memories (primary memory 42 and secondary memory 43), but it is not limited to this. That is, the robot 100's memory may be implemented using a single memory. In this case, for example, one memory area of ​​that memory may be used as the primary memory 42, and another memory area of ​​that memory may be used as the secondary memory 43.

[0031] (Input / Output Interface) The input / output interface (IF44) connects the processor (41) to each component (sensor (3), speaker (6), power supply circuit (7), and the LCD panel (22) of the projector (2)). Examples of input / output interfaces (IF44) include USB (Universal Serial Bus), ATA (Advanced Technology Attachment), SCSI (Small Computer System Interface), and PCI (Peripheral Component Interconnect).

[0032] (Projector) The projector 2 projects an image onto at least a portion of the body surface member 1. In this embodiment, the projector 2 projects a face-like image (hereinafter referred to as face image I) onto the head member 12. As described above, the projector 2 in this embodiment is positioned inside the torso member 11. Furthermore, as shown in Figure 3, the projector 2 in this embodiment includes a backlight 21 that irradiates light L upward and a liquid crystal panel 22 interposed between the backlight 21 and the head member. The backlight 21 is positioned to irradiate light L upward (in the direction where the head member 12 is located). Note that the projector 2 may be configured to project an image onto the torso member 11. Also, the projector 2 may be configured to project an image other than face image I (for example, the time, which will be described later).

[0033] In this embodiment, the projector 2, with the configuration described above, projects a face image I onto the inner surface of the head member 12, as shown in Figure 3. As described above, the head member 12 in this embodiment is made of a material that partially transmits light L. Therefore, a user facing the robot 100 can see the face image I projected onto the head member 12 as the face of the robot 100 from outside the head member 12, as shown in Figure 1. In other words, the robot 100 is a projection-type robot that displays a face by projection from the projector 2. The projector 2 may be positioned outside the body surface member 1 and project the face image I from outside the head member 12 using a method such as projection mapping. In that case, the head member 12 may be made of a material that does not transmit light.

[0034] (power circuit) The power supply circuit 7 adjusts the power supplied from the power supply P to the backlight 21 of the projector 2 in accordance with the control of the processor 41. The power supply circuit 7 may also adjust the power supplied to components other than the projector 2.

[0035] (Electrical configuration and other aspects) The robot 100 may also include a second processor. In that case, the second processor may control components other than the projector 2. The robot 100 may also have a communication interface and communicate with the outside world. Furthermore, the robot 100 may have a clock for obtaining the time.

[0036] [Operation] Next, the operation of robot 100 will be described. Figures 4, 5, 7, and 8 are flowcharts showing some of the processes performed by the processor 41 of robot 100. Figure 6 shows an example of a face image I projected by the projector 2 of robot 100.

[0037] The robot 100 according to this embodiment starts up when predetermined startup conditions are met. These startup conditions include, for example, the power switch being turned ON. After startup, the processor 41 according to this embodiment sets the mode variable to "1" (step S1), as shown in Figure 4. The mode variable indicates the operating mode. There are four types of mode variables according to this embodiment: "1" indicating communication mode, "2" indicating power saving mode, "3" indicating sleep mode, and "4" indicating lighting mode. After startup, the processor 41 according to this embodiment performs acquisition processing, as shown in Figure 4 (step S2). In the acquisition processing, the processor 41 acquires information about the outside world from the sensor 3. Specifically, if the sensor 3 includes an image sensor 31a, the processor 41 acquires image data (including ambient brightness) from the camera 31. If the sensor 3 includes a microphone 32, the processor 41 acquires audio data from the microphone 32. If the robot 100 is equipped with a clock, the processor 41 also acquires the current time as external information during this acquisition process.

[0038] (Image recognition processing and speech recognition processing) If the acquired external information includes image data, the processor 41 performs image recognition processing. In the image recognition processing, the processor 41 recognizes the user's facial expression based on the image data generated by the camera (step S3). Also, if the acquired external information includes audio data, the processor 41 performs speech recognition processing (step S3). In the speech recognition processing, the processor 41 recognizes the user's speech content (for example, "make it darker," "make it brighter," etc.) based on the audio data generated by the microphone.

[0039] After recognizing the user's facial expression and speech content, the processor 41 determines whether a predetermined lighting switching condition has been met (step S4). The lighting switching condition includes, for example, a predetermined switch operation being performed, the ambient brightness being below a predetermined level, and the current time being evening or later. If it is determined that the lighting switching condition has been met (step S4: Yes), the processor 41 switches the mode variable to "4" which indicates the lighting mode (step S5). On the other hand, if it is determined that the lighting switching condition has not been met (step S4: No), the processor 41 proceeds to the process in step S6.

[0040] After switching the mode variable to "4", or after determining that the lighting switching condition is not met, the processor 41 determines whether the current mode variable is "1", which indicates communication mode (step S6). If it determines that the mode variable is "1" (step S6: Yes), the processor 41 transitions its operating mode to communication mode (1). The operation in this communication mode (1) will be described later.

[0041] On the other hand, if the mode variable is determined not to be "1" (step S6: No), the processor 41 determines whether the mode variable is "2", which indicates power saving mode (step S7). If the mode variable is determined to be "2" (step S7: Yes), the processor 41 transitions its operating mode to power saving mode (2). The operation in this power saving mode (2) will be described later.

[0042] On the other hand, if the mode variable is determined not to be "2" (step S7: No), the processor 41 determines whether the mode variable is "3", which indicates sleep mode (step S8). If the mode variable is determined to be "3" (step S8: Yes), the processor 41 transitions its operating mode to sleep mode (3). The operation in this sleep mode (3) will be described later.

[0043] On the other hand, if the mode variable is not "3" (i.e., a predetermined condition is met) (step S8: No), the processor 41 transitions its operating mode to the illumination mode (4). The operation in this illumination mode (4) will be described later.

[0044] (Power regulation processing) Furthermore, when the robot 100 starts up, the processor 41 begins power adjustment processing. Power adjustment processing is the process of controlling the power supply circuit 7 to adjust the power supply to the backlight 21 of the projector 2. This power adjustment processing is performed continuously while the robot is running.

[0045] [Communication Mode] When the processor 41 transitions to communication mode (1), as shown in Figure 5, it first supplies power to the backlight 21 according to the acquired external information in the power adjustment process (step A1). If the sensor 3 is configured to acquire ambient brightness, the processor 41 performs control in the power adjustment process to supply power to the backlight 21 according to the ambient brightness acquired by the sensor. For example, the processor 41 controls the backlight 21 so that it becomes brighter when the surroundings are brighter and dimmer when the surroundings are darker. By changing the brightness of the body surface according to the ambient brightness, it is possible to prevent the robot from appearing relatively dark or appearing too bright, making communication difficult. In addition, while power according to the ambient brightness is supplied, the processor 41 adjusts the color output of the liquid crystal panel 22 according to the ambient brightness. For example, when the surroundings are bright (daytime), the processor 41 controls the liquid crystal panel 22 so that the body surface member 1 becomes cool-colored. On the other hand, if the surroundings are dark (nighttime), the processor 41 controls the liquid crystal panel 22 so that the body surface material 1 becomes warm in color.

[0046] Furthermore, if the robot 100 is equipped with a clock, the processor 41 may, in its power adjustment processing, perform control to supply power to the backlight 21 according to the time acquired by the clock. Specifically, it controls the backlight to be brighter in the morning and daytime, and dimmer in the evening and at night. This changes the brightness of the body surface according to the current time. As a result, it is possible to prevent the robot from appearing relatively dark or appearing too bright, making communication difficult.

[0047] (Existence determination process) While supplying power to the backlight 21 according to the acquired external information, the processor 41 performs an existence determination process (step A2). In the existence determination process, the processor 41 determines whether or not a user is present within a predetermined area based on the acquired external information. The predetermined area refers to, for example, the area that the camera 31 can photograph (the area radiating from the lens) and within a predetermined distance from the camera 31. As described above, the camera 31 according to this embodiment is provided on the front of the torso member 11 of the body surface member 1. Therefore, the predetermined area according to this embodiment is the front side of the robot 100. In the existence determination process according to this embodiment, the processor 41 determines that a user is present if, for example, the image data generated by the camera 31 detects at least the entire upper body of the user at a size greater than a predetermined size.

[0048] If the above existence determination process determines that no user exists within the predetermined area (Step A2: No), the processor 41 switches the operating mode to "2", which indicates power saving mode (Step A3), and then executes the process from Step S2 onwards (5) as shown in Figure 4 again.

[0049] (Possibility assessment process) If the presence determination process determines that a user exists within a predetermined area (Step A2: Yes), the processor 41 further executes a possibility determination process (Step A4). In the possibility determination process, the processor 41 determines the possibility of the user communicating with the robot. In the possibility determination process according to this embodiment, the processor 41 determines the possibility of communication based on at least one of the image data acquired from the camera 31 or the audio data acquired from the microphone 32. Specifically, the processor 41 determines that the possibility is low if it detects a predetermined event from at least one of the image data or the audio data. The predetermined events include, for example, the user not appearing in the image, the user not facing the robot 100, the user looking away, two or more users appearing in the image and facing each other (people communicating with each other), no sound being input to the microphone 32 for a predetermined time (the user not speaking), the volume of the sound being below a predetermined level, and the volume of the sound input from the microphone 32 located at the front being lower than the volume of the sound input from the microphone 32 located at other locations (such as someone other than a user speaking or noise).

[0050] If the above possibility determination process determines that the likelihood of the user communicating with the robot is low (Step A4: YES), the processor 41 switches the operating mode to "2", which indicates power saving mode (Step A3), and executes the process from Step S2 onwards (5) as shown in Figure 4 again. In power saving mode (2), the power supply to the backlight 21 is stopped or reduced. Therefore, if it is determined that the likelihood of communication is low, unnecessary power consumption can be reduced by, for example, stopping or reducing the power supply. On the other hand, by increasing the power supply and displaying the image brightly, it is possible to appeal to the user that the robot wants to communicate.

[0051] (Awakening judgment process) On the other hand, if the possibility determination process determines that there is a high probability that the user will communicate (Step A4: NO), the processor 41 executes an awakening determination process (Step A5). In the awakening determination process, the processor 41 determines whether or not the user is awake. In the awakening determination process according to this embodiment, the processor 41 makes the determination based on image data of the user's face generated by the camera 31. Specifically, for example, from the image data, the processor 41 measures the degree to which the user's eyelids are lowered and the frequency of eyelid movement, in addition to the up-and-down movement of the chest and body movements associated with breathing, and determines that the user is awake if the measurement result is higher than a threshold stored in the primary memory 42 for determining whether or not the user is awake. In addition, in the awakening determination process, the processor 41 may also determine whether or not the user is awake based on voice data generated by the microphone 32, detection results from other means (heart rate / respiratory sensor, sleep monitor placed under bedding, smartphone sleep measurement app, etc.).

[0052] In the wakefulness determination process described above, if it is determined that the user is not awake (i.e., asleep or semi-awake) (Step A5: NO), the processor 41 switches the operating mode to "3", which indicates sleep mode (Step A6), and then executes the process from Step S2 onwards (5) shown in Figure 4 again.

[0053] On the other hand, if the wakefulness determination process determines that the user is awake (Step A5: YES), the processor 41 adjusts, in the power adjustment process, the power supply to the backlight 21 and the color rendering of the liquid crystal panel 22 according to at least one of the facial expression recognized in the image recognition process and the speech content recognized in the speech recognition process (Step A7). As described above, the wakefulness determination process (Step A5) is executed when the possibility determination process (Step A4) determines that there is a high probability that the user will communicate. For this reason, the power adjustment process (Step A7) can also be said to be executed when the possibility determination process determines that there is a high probability that the user will communicate (Step A4: NO). Furthermore, in the power adjustment process according to this embodiment, the processor 41 also adjusts the color rendering of the liquid crystal according to the speech content recognized in the speech recognition process. For example, if the user has a calm facial expression, the processor 41 controls the liquid crystal panel 22 so that the body surface member 1 becomes warm-colored. On the other hand, if the user has a tense facial expression, the processor 41 controls the liquid crystal panel 22 so that the body surface member 1 becomes cool-colored.

[0054] Furthermore, for example, if at least one of the user's facial expression and / or speech content is bright, the processor 41 projects the face image I at high brightness, as shown on the left side of Figure 6. In this case, the processor 41 may change the color of the face image I to a brighter color. On the other hand, if at least one of the user's facial expression and / or speech content is dark, the processor 41 projects the face image I at a low brightness, which is darker than the high brightness mentioned above, as shown on the right side of Figure 6. In this case, the processor 41 may change the color of the face image I to a darker color. Also, if at least one of the user's facial expression and / or speech content is anything other than the above, the processor 41 projects the face image I at a normal brightness, which is darker than the high brightness mentioned above and brighter than the low brightness mentioned above, as shown in the center of Figure 6. As a result, the body surface member 1, when communicating with the user, will have a brightness and color that matches the user's facial expression and / or speech content (emotions). As a result, the user will develop a deeper attachment to the robot.

[0055] After completing the process in step A7, the processor 41 executes the process from step S2 onwards (5) as shown in Figure 4 again.

[0056] [Power saving mode] When the processor 41 transitions to power saving mode (2), as shown in Figure 7, it performs control to stop or reduce the power supply to the backlight 21 during power adjustment processing (step B1). As a result, the body surface member 1 becomes dimmer than when it was operating in communication mode (1). Therefore, power consumption can be reduced when it is not in use (when there is no user in front of it, or when there is a low probability of communication even if there is a user).

[0057] Furthermore, in this power adjustment process, the processor 41 may reduce the power supply to the backlight 21 without stopping it. This ensures that when the power adjustment process is performed (when there is no user present or when the likelihood of the user communicating is low), the power supply to the backlight 21 is reduced but maintained, so that the face continues to be projected onto the head member 12 (making it clear that the robot 100 is running). This prevents the user from mistakenly believing that the robot 100 is not running.

[0058] (Second possibility determination process) While the power supply to the backlight 21 is stopped or reduced, the processor 41 performs a second possibility determination process (step B2). In the second possibility determination process, the processor 41 determines the possibility of the user communicating with the robot. In the second possibility determination process according to this embodiment, the processor 41 determines that the possibility is high, for example, when it detects the presence of a user or a predetermined action of the user from image data generated by the camera 31, or when it detects voice input from the microphone 32 on the front of the neck 11b.

[0059] In the second possibility determination process described above, if it is determined that there is a high probability that the user will communicate with the robot (step B2: Yes), the processor 41 switches the operating mode to "1" which indicates communication mode (step B3), and executes the process from step S2 onwards (5) shown in Figure 4 again. Then, the processor 41, having transitioned to communication mode (1), executes control in the power adjustment process to restart or increase the power supply to the backlight 21. Therefore, if there is a possibility that the user will initiate communication while the power supply to the backlight 21 is reduced or stopped, the system can automatically return to a state where the image is brightly projected onto the body surface member 1.

[0060] On the other hand, if the second possibility determination process determines that the likelihood of the user communicating with the robot is low (step B2: No), the processor 41 repeats the process (5) from step S2 onwards, as shown in Figure 4.

[0061] [Sleep Mode] (Second Awakening Judgment Process) When the processor 41 transitions to sleep mode (3), it executes a second awakening determination process (step C1), as shown in Figure 8. In the second awakening determination process, the processor 41 determines whether or not the user is awake. The specific determination operation is the same as in the awakening determination process (step A5) described above.

[0062] In the second awakening determination process described above, if it is determined that the user is awake (step C1: YES), the processor 41 switches the operating mode to "1" which indicates communication mode (step C2), and then executes the process from step S2 onwards (5) as shown in Figure 4 again.

[0063] (Sleep judgment processing) In the second awakening determination process described above, if it is determined that the user is not awake (i.e., asleep or semi-awake) (step C1: NO), the processor 41 executes the sleep determination process (step C3). In the sleep determination process, the processor 41 determines whether the user is asleep or semi-awake. Specifically, it measures the up-and-down movement of the chest and body movements associated with breathing, and determines that the user is semi-awake if the measurement result is higher than the threshold stored in the primary memory 42 for determining whether or not the user is asleep.

[0064] (Read-aloud judgment processing) If the sleep determination process determines that the user is in a semi-awake state (step C3: NO), the processor 41 executes the read-aloud determination process (step C4). In the read-aloud determination process, the processor 41 determines, based on information from the outside world, whether or not a third party is reading a book to the user. Specifically, the processor 41 determines that a book is being read aloud if, for example, it detects a third party along with the semi-awake user from the image data, detects a book, picture book, etc., or detects a voice below a predetermined volume from the audio data.

[0065] If the above read-aloud judgment process determines that read-aloud is taking place (Step C4: Yes), the power adjustment process executes a process to set the power supplied to the backlight 21 to the first power (Step C5). The first power is less than the normal power supply. The normal power supply is the power corresponding to the external information in communication mode. Therefore, when the user is semi-awake and a third party is reading a book to them, an environment that does not interfere with that situation can be automatically created. As a result, the third party can read aloud smoothly, and the user can easily fall asleep.

[0066] (Termination determination process) While the first power is supplied to the backlight 21, the processor 41 performs a termination determination process (step C6). In the termination determination process, the processor 41 repeatedly determines whether the book reading has ended based on information from the outside world until it determines that it has ended. Specifically, the processor 41 determines that the book reading has ended when, for example, a third party other than the semi-awake user is no longer detected in the audio data, when a book, picture book, etc. is no longer detected, or when a voice is no longer detected in the audio data.

[0067] If the above termination determination process determines that the reading session has ended (step C6: YES), the processor 41 will execute the processes from step C1 onwards again. As a result, if it is determined that the user is asleep, the processor 41 will execute the sleep depth determination process (step C8) and the power adjustment process (step C9). Therefore, when the reading session ends, the process will proceed to the same state as when it is determined that the user is asleep, preventing the reading session from continuing indefinitely and interfering with the user's sleep.

[0068] In the above read-aloud determination process (step C4), if it is determined that read-aloud is not being performed (step C4: NO), the processor 41 performs a process in the power adjustment process to set the power supplied to the backlight 21 to the second power (step C7). The second power is greater than the power corresponding to the depth of sleep, but less than the first power. After performing the process to set the supplied power to the second power, the processor 41 performs the processes from step C1 onwards again. In addition, the processor 41 may perform a process in this power adjustment process to set the power supplied to the backlight 21 to a power different from the second power.

[0069] (Sleep depth determination process) If the sleep determination process (step C3) determines that the user is asleep (step C3: YES), the processor 41 executes a sleep depth determination process (step C8). In the sleep depth determination process, the processor 41 determines the depth of the user's sleep. After determining the depth of sleep, the processor 41 performs a power adjustment process to control the power supplied to the backlight 21 to match the depth of sleep determined in the sleep depth determination process (step C9). Specifically, the processor 41 controls the power supply circuit 7 so that the deeper the user's sleep, the less power is supplied to the backlight 21. As a result, the brightness of the head unit is adjusted to the depth of the user's sleep, preventing the light from causing the user's sleep to become lighter or waking them up. As a result, the user can deepen their sleep further. The processor 41 may also control the liquid crystal panel 22 of the projector 2 to change the light emission color of the head unit 12 in conjunction with adjusting the power supply. Alternatively, the processor 41 may change only the light emission color without adjusting the power supply. Furthermore, the processor 41 may be configured to control the projector 2 so that the face image I is not displayed on the head member 12.

[0070] (Verbal communication decision processing) While the backlight 21 is supplied with power according to the depth of sleep, the processor 41 performs a voice detection process (step C10). In the voice detection process, the processor 41 determines whether or not the user has spoken based on the voice data generated by the microphone. If the voice detection process determines that no voice was spoken (step C10: NO), the processor 41 repeats the process from step C1 onwards.

[0071] (Situation assessment and processing) If the above voice detection process determines that a voice has been called (Step C10: YES), the processor 41 executes a situation determination process (Step C11). In the situation determination process, the processor 41 determines, based on the voice data, whether or not it is an emergency voice call. Specifically, the processor 41 recognizes, for example, at least one of the user's utterance content, volume, and prosody from the voice data, and determines that it is an emergency voice call if the utterance content contains a specific word or phrase, the volume is above a predetermined level, or a predetermined part of the voice is emphasized.

[0072] In the above situation determination process, if it is determined that the voice prompt is not an emergency (step C11: No), the processor 41 proceeds to step C7. That is, it executes a process to set the power supplied to the backlight 21 to the second power. As described above, the situation determination process (step C11) is executed when it is determined in the above voice prompt determination process that a voice prompt has been made (step C10: YES), etc. Therefore, it can also be said that the process of setting the power supplied to the backlight to the second power (step C7) is executed when it is determined in the voice prompt determination process that a voice prompt has been made (step C10: YES). As a result, communication with the user when they are temporarily awake during bedtime (and will go back to sleep afterward) can be performed without fully waking the user. Furthermore, while the second power is supplied (when there is a non-emergency voice prompt, when the user is semi-awake but is not being read to, etc.: while transitioning from falling asleep to a sleep state), the processor 41 may control the liquid crystal panel 22 so that the body surface member 1 becomes warm in color.

[0073] On the other hand, if the situation assessment process determines that the call is an emergency (step C11: Yes), the processor 41 performs a process in the power adjustment process to set the power supplied to the backlight 21 to the third power (step C12). The third power is more power than the first power. Therefore, in the event of an emergency (when the user needs to move immediately), the surroundings will be brightly illuminated to a degree that allows the user to wake up and easily understand their surroundings, making it easier for the user to ensure their safety and move out of the bedroom.

[0074] (Second termination determination process) While the third power is supplied to the backlight 21, the processor 41 executes a second termination determination process (step C13). In the second termination determination process, the processor 41 determines whether the emergency has ended. In the second termination determination process according to this embodiment, the processor 41 determines that the emergency has ended when a predetermined release operation is performed by the user, or when it detects from the audio data that the ambient volume has fallen below a predetermined level. If the second termination determination process determines that the emergency has not ended (step C13: No), the processor 41 executes the processes from step C1 onwards again.

[0075] If the second termination determination process determines that the emergency has ended (step C13: Yes), the processor 41 switches the operating mode to "1", which indicates communication mode (step C14), and then executes the process from step S2 onwards (5) as shown in Figure 4 again.

[0076] [Lighting Mode] When the processor 41 transitions to lighting mode (4), as shown in Figure 4, it first supplies power to the backlight 21 according to the acquired external information in the power adjustment process (step D1). If the external information is "ambient brightness," the power supplied according to the external information in this process will be, for example, inversely proportional to the ambient brightness. That is, the processor 41 controls the backlight 21 so that it becomes dimmer when the ambient is brighter and brighter when the ambient is darker. Note that the power supplied according to the external information in this process may be the same as the power supplied according to the external information described in communication mode (1) above. Also, the power supplied according to the external information here may be less than the power supplied according to the external information described in communication mode (1) above. That is, the brightness of the body surface member 1 in lighting mode (4) may be dimmer than the brightness in communication mode (1). After executing the control to supply power according to the external information to the backlight 21, the processor 41 executes the process (5) from step S2 onwards again.

[0077] While supplying power to the backlight 21 according to the acquired external information, the processor 41 may control the projector 2 so as not to project an image onto the body surface member. As described above, lighting mode (4) is an operating mode that is transitioned to when predetermined conditions are met. Therefore, the control to prevent the projection of the face image I onto the body surface member 1 can also be said to be a control that is executed when predetermined conditions are met. As a result, the body surface member 1 will light up without displaying an image. Therefore, the user can easily understand that the robot 100 is not in a state to communicate. In addition, in lighting mode (4), the processor 41 may control the projector to display an image different from the face image I (for example, a clock). Furthermore, the body surface member 1 will light up in a color corresponding to the external information. For example, if the external information is "ambient brightness," then when the ambient light is bright (daytime), the processor 41 controls the liquid crystal panel 22 so that the body surface member 1 lights up in a cool color. On the other hand, if the surroundings are dark (nighttime), the processor 41 controls the liquid crystal panel 22 so that the body surface material 1 becomes warm in color.

[0078] Furthermore, while supplying power to the backlight 21 according to the acquired external information, the processor 41 may be configured to determine whether predetermined switching conditions (for example, a predetermined switch operation is performed, the ambient brightness is above a predetermined level, the current time is morning or daytime, etc.) are met. The processor 41 may then be configured to switch the mode number to "1" when it determines that the switching conditions are met. In addition, in lighting mode (4), the processor 41 may not adjust the power supplied to the backlight 21 according to the external information, but simply control the projector 2 so that no image is projected onto the body surface member 1. Alternatively, in lighting mode (4), the processor 41 may supply power to the backlight 21 according to the external information and project an image onto the body surface member 1.

[0079] By performing the control described above, the brightness and color of the image projected onto the body surface member 1 change in real time according to changes in ambient brightness, the presence or absence of a user, changes in the likelihood of the user communicating, changes in the user's facial expressions and speech content, time of day, etc.

[0080] [Effects and Effects] As described above, with regard to the robot 100, unnecessary power consumption can be reduced by, for example, stopping or reducing the power supply to the backlight 21 when it is not in use (not interacting with a user). Furthermore, by increasing the power supply to the backlight 21 when the user is actively communicating, it becomes possible to display images that appeal more to the user's emotions. As a result, in projection-type robots that communicate with users, it is possible to prevent wasted power consumption when not in use and to make expressions during communication easier for the user to understand.

[0081] <Embodiment 2> Other embodiments of this disclosure are described below. For the sake of clarity, components having the same function as those described in the above embodiments are denoted by the same reference numerals, and their descriptions are not repeated.

[0082] [Differences in overall structure] In the robot 100 according to Embodiment 1 described above, the thresholds for making decisions in the process of making various decisions based on information from the outside world (such as whether or not the robot is awake, whether or not it is asleep, ambient brightness, whether or not a story is being told, whether or not the story has been told, etc.) were fixed. In contrast, in the robot 100A according to this embodiment, the thresholds may fluctuate.

[0083] [Differences in electrical configuration] The primary memory 42A in this embodiment stores one or more types of trained models M constructed by machine learning. The machine learning used to construct the trained models M may be machine learning that uses pairs of external information and thresholds as training data, or it may be machine learning that does not use training data. When new external information is input, the trained models M output new thresholds. Furthermore, the primary memory 42A in this embodiment is capable of updating at least a portion of the stored thresholds.

[0084] [Differences in operation] In this embodiment, the processor 41 of the robot 100A adjusts the threshold based on machine learning using external information acquired by the sensor 3. The processor 41 then makes decisions based on new external information in light of the adjusted threshold.

[0085] [Effects and Effects] The robot 100A according to this embodiment not only provides the same effects as the robot 100 according to Embodiment 1, but also allows the accuracy of various judgments (whether or not the robot is awake, whether or not it is asleep, ambient brightness, whether or not storytelling is taking place, whether or not storytelling has finished, etc.) to be adjusted to the user's usage of the robot 100A.

[0086] [summary] A robot according to embodiment 1 of the present invention comprises a body surface member forming the body surface, a projector that projects an image onto at least a part of the body surface member, a sensor provided on at least a part of the body surface member for acquiring information from the outside world, and a control unit that controls at least the projector, wherein the control unit performs an existence determination process that determines whether or not a user is present in a predetermined area based on the information from the outside world acquired by the sensor, and a power adjustment process that adjusts the power supply to the backlight of the projector if the existence determination process determines that the user is not present in the predetermined area. In the robot according to embodiment 2 of the present invention, in embodiment 1 described above, the control unit may further perform a possibility determination process to determine the possibility of the user communicating with the robot if the presence determination process determines that the user is present in the predetermined area, and if the possibility determination process determines that the possibility of the user communicating with the robot is low, it may perform the power adjustment process. In the third aspect of the present invention, the robot may be configured such that, in the first or second aspect described above, the control unit performs control to stop or reduce the power supply to the backlight during the power adjustment process. In the fourth aspect of the present invention, the robot may be configured such that, in any one of the above aspects 1 to 3, the control unit determines that there is a high probability that the user will communicate with the robot while the power supply to the backlight is stopped or reduced, and in the power adjustment process, it executes control to restart or increase the power supply to the backlight. A robot according to aspect 5 of the present invention, in aspect 2 described above, the sensor includes an image sensor, which is mounted on the front surface of the body surface member and is configured as part of a camera capable of capturing a predetermined area and generating image data, and the control unit performs an image recognition process to recognize the user's facial expression based on the image data generated by the camera, and in the possibility determination process, if it is determined that there is a high probability that the user will communicate, the power adjustment process may adjust at least one of the power supply to the backlight and the color development of the liquid crystal panel of the projector according to the facial expression recognized in the image recognition process. A robot according to embodiment 6 of the present invention, in embodiment 2 described above, further includes a microphone that generates voice data from the user's voice, and the control unit performs voice recognition processing to recognize the content of the user's speech based on the voice data generated by the microphone, and in the possibility determination processing, if it is determined that there is a high probability that the user will communicate, the power adjustment processing adjusts at least one of the power supply to the backlight and the color development of the liquid crystal panel of the projector according to the content of the speech recognized in the voice recognition processing. In the robot according to embodiment 7 of the present invention, in embodiment 1 described above, the sensor is configured to acquire ambient brightness, and the control unit is configured to perform control in the power adjustment process to supply power to the backlight in accordance with the ambient brightness acquired by the sensor. A robot according to embodiment 8 of the present invention may be configured such that, in embodiment 1 above, it is equipped with a clock that acquires the time, and the control unit, in the power adjustment process, performs control to adjust the power supply to the backlight to a power supply corresponding to the time acquired by the clock. In the robot according to embodiment 9 of the present invention, in any one of embodiments 5 to 8 described above, the control unit may be configured to perform an awakening determination process to determine whether the user is awake or not when the presence determination process determines that the user is present; a sleep determination process to determine whether the user is asleep or semi-awake when the awakening determination process determines that the user is not awake; a sleep depth determination process to determine the depth of the user's sleep when the sleep determination process determines that the user is asleep; and in the power adjustment process, to control the power supplied to the backlight to supply power according to the depth of sleep determined in the sleep depth determination process. In the robot according to embodiment 10 of the present invention, in embodiment 9 described above, the control unit may be configured to perform a read-aloud determination process that determines whether or not a third party is reading a book to the user based on external information when the sleep determination process determines that the user is in a semi-awake state, and when the read-aloud determination process determines that a read-aloud is taking place, the power adjustment process may perform a process to set the power supplied to the backlight to a first power, which is less than the normal power supply. In the robot according to embodiment 11 of the present invention, in embodiment 10 described above, the control unit may be configured to perform a termination determination process to determine whether the reading of the book has ended based on the external information while the first power is supplied to the backlight, and if the termination determination process determines that the reading of the book has ended, it may perform the sleep depth determination process and the power adjustment process. A robot according to embodiment 12 of the present invention may be configured such that, in embodiment 10 or 11 above, the sensor further includes a microphone that generates voice data from the user's voice, and the control unit performs a voice detection process to determine whether or not the user has spoken to it based on the voice data generated by the microphone while power according to the depth of sleep is supplied to the backlight, and if the voice detection process determines that the user has spoken to it, the power adjustment process performs a process to set the power supplied to the backlight to a second power that is greater than the power according to the depth of sleep and less than the first power. In the robot according to embodiment 13 of the present invention, in embodiment 12 described above, the control unit may, when it determines in the voice judgment process that a voice has been spoken, execute a situation judgment process to determine whether or not the voice is an emergency based on the voice data, and when it determines in the situation judgment process that the voice is an emergency, execute a process in the power adjustment process to set the power supplied to the backlight to a third power, which is greater than the first power. In embodiment 14 of the present invention, the robot may be configured such that, in any one of embodiments 1 to 13 described above, the control unit performs at least one of the following operations when a predetermined condition is met: controlling the liquid crystal panel of the projector so as not to project an image onto the body surface member, and supplying power to the backlight according to the information of the external environment. In the robot according to embodiment 15 of the present invention, in any one of embodiments 1 to 14 described above, the control unit is configured to make a decision based on the external information in light of a predetermined threshold, and the threshold may be adjusted by machine learning based on the external information acquired by the sensor.

[0087] [others] This disclosure is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this disclosure.

[0088] For example, some or all of the processes performed by the processor 41 can also be implemented by logic circuits. For example, an integrated circuit in which logic circuits that perform some or all of the above processes are formed is also included in the scope of this disclosure. In addition, it is also possible to implement the functions of the above control blocks by, for example, a quantum computer. [Explanation of symbols]

[0089] 100,100A: Robot, 1: Body surface component, 11: Torso component, 11a: Torso body, 11b: Neck, 12: Head component, 2: Projector, 21: Backlight, 22: LCD panel, 3: Sensor, 31: Camera, 31a: Image sensor, 32: Microphone, 4: Control unit, 41: Processor, 42,42A: Primary memory, 43: Secondary memory, 44: Input / Output IF, 5: Base, 6: Speaker, 7: Power supply circuit

Claims

1. Body surface components that make up the body surface, A projector that projects an image onto at least a part of the body surface member, A sensor is provided on at least a part of the body surface member to acquire information from the outside world, It comprises at least a control unit for controlling the projector, The control unit, Based on the external information acquired by the sensor, an existence determination process is performed to determine whether or not a user is present within a predetermined area. If the presence determination process determines that the user is not present in the predetermined area, a power adjustment process is performed to adjust the power supply to the projector's backlight. If the existence determination process determines that the user is within the predetermined area, it executes a possibility determination process to determine the possibility of the user communicating with the robot. The control unit also executes the power adjustment process if it determines in the possibility determination process that the likelihood of the user communicating is low. A robot characterized by the following features.

2. The control unit, in the power adjustment process, performs control to stop or reduce the power supply to the backlight. The robot according to feature 1.

3. If the control unit determines that the user is likely to communicate with the robot while the power supply to the backlight is stopped or reduced, it will perform control in the power adjustment process to restart or increase the power supply to the backlight. The robot according to feature 1.

4. The aforementioned sensor includes an image sensor, The image sensor is mounted on the front surface of the body surface member and is configured as part of a camera capable of capturing images of the predetermined area and generating image data. The control unit, Based on the image data generated by the camera, an image recognition process is performed to recognize the user's facial expression. In the possibility determination process, if it is determined that the user is likely to communicate, the power adjustment process adjusts at least one of the power supply to the backlight and the color rendering of the projector's liquid crystal panel according to the facial expression recognized in the image recognition process. The robot according to feature 1.

5. The sensor further includes a microphone that generates audio data from the user's voice, The control unit, Based on the audio data generated by the microphone, a speech recognition process is performed to recognize the content of the user's speech. In the possibility determination process, if it is determined that there is a high probability that the user will communicate, the power adjustment process adjusts at least one of the power supply to the backlight and the color rendering of the projector's liquid crystal panel according to the speech content recognized in the speech recognition process. The robot according to feature 1.

6. The sensor is configured to acquire ambient brightness, The control unit, Before executing the existence determination process, the power adjustment process is also executed. In the power adjustment process prior to the execution of the presence determination process, control is performed to ensure that the power supply to the backlight is adjusted to the ambient brightness acquired by the sensor. The robot according to feature 1.

7. Equipped with a clock that obtains the time, The control unit, Before executing the existence determination process, the power adjustment process is also executed. In the power adjustment process prior to the execution of the existence determination process, control is performed to ensure that the power supply to the backlight is supplied according to the time acquired by the clock. The robot according to feature 1.

8. The control unit, If the existence determination process determines that the user exists, an awakening determination process is performed to determine whether the user is awake or not based on the image data, If the wakefulness determination process determines that the user is not awake, the following sleep determination process is performed: determining whether the user is asleep or semi-awake based on the user's body movements. The robot according to feature 4.

9. The control unit, If the sleep judgment process determines that the user is in a semi-awake state, a read-aloud judgment process determines, based on the external information, whether or not a third party is reading a book to the user. If the read-aloud determination process determines that read-aloud is being performed, the power adjustment process executes a process to set the power supplied to the backlight to a first power level, which is lower than the normal power supply level. The robot according to feature 8.

10. The control unit, While the first power is supplied to the backlight, a termination determination process is performed to determine whether the reading of the book has ended, based on the fact that no external information is detected. If the termination determination process determines that the reading aloud has ended, the power adjustment process is executed. The robot according to feature 9.

11. The control unit, when a predetermined condition is met, performs at least one of the following operations: controlling the liquid crystal panel of the projector so as not to project an image onto the body surface member, and supplying power to the backlight according to the information of the external environment. The robot according to feature 1.

12. The control unit, The system is configured to make decisions based on the aforementioned external information in light of predetermined thresholds. The threshold is adjusted by machine learning based on the external information acquired by the sensor. The robot according to feature 1.

Images