robot

Abstract

To provide a technique useful for determining a cause of collision or the like in the case of collision of a robot with an object, or the like.SOLUTION: A robot 20 comprises an operation recording unit 21. The operation recording unit 21 acquires and records operating state data indicating operating states of the robot 20 and is provided with functions similar to those of existing driver recorders. The operating state data of the robot 20 includes video data acquired by a camera mounted on the robot 20, sound data acquired by a microphone, and the like. Upon acquiring the operating state data, the operation recording unit 21 records the acquired data in a memory.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a robot. [Background technology]

[0002] In recent years, video conferencing systems using the Internet have become widespread, and robots (e.g., telepresence robots) are being used that not only allow users to talk face-to-face, but also allow users in remote locations to control the direction and position of the camera (see, for example, Patent Document 1).

[0003] Users can remotely control robots placed in facilities such as art galleries and museums, and move the robots to exhibits such as paintings, sculptures, and stuffed animals, allowing them to view the exhibits from very close range. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-062308 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when a robot moves, it may accidentally collide with a wall or an exhibit (hereinafter referred to as an object) within the facility. In such cases, it becomes necessary to identify the cause of the collision, but no specific means for identifying such a cause has yet been proposed.

[0006] The present invention has been made in consideration of the circumstances described above, and one of its objects is to provide a technique that is useful for identifying the cause of a collision between a robot and an object, for example. [Means for solving the problem]

[0007] A robot according to one embodiment of the present disclosure is a robot that operates based on operation information received from a computer via a communication network, and is equipped with an operation record unit that acquires and records operation status data that represents the operation status of the robot. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a technique that is useful for identifying the cause of a collision between a robot and an object, for example. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing the network configuration of a communication system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of a computer. [Figure 3] FIG. 10 is a diagram illustrating an example of a display screen displayed on a computer. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of the robot. [Figure 5] FIG. 10 is a diagram illustrating an example of reference information. [Figure 6] FIG. 1 is a diagram illustrating the physical configuration of a computer. [Figure 7] FIG. 1 is a diagram illustrating the physical configuration of a robot. [Figure 8] 10 is a flowchart showing the processing of the Operation Record function. DETAILED DESCRIPTION OF THE INVENTION

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described with reference to the accompanying drawings, in which the same reference numerals denote the same or similar components.

[0011] A. This embodiment Fig. 1 is a diagram showing the network configuration of a communication system 100 according to this embodiment. The communication system 100 includes free-standing robots 20, a computer 10 capable of operating each robot 20, and a management server 30. Each robot 20, the computer 10, and the management server 30 are capable of communicating with each other via a communication network N. Although Fig. 1 illustrates two robots 20, one computer 10, and one management server 30, the number of robots 20, computers 10, and management servers 30 is arbitrary.

[0012] One or more portions of the communication network N may be a wired network or a wireless network, and the communication network N may include, by way of example and without limitation, an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular network, integrated service digital networks (ISDNs), wireless LANs, long term evolution (LTE), code division multiple access (CDMA), short range wireless communications such as Bluetooth, satellite communications, or a combination of two or more thereof.

[0013] <Overview of the operation record function installed in Robot 20> Each robot 20 is equipped with an operation record function for acquiring and recording data representing the operating status of the robot 20 (e.g., video data, audio data, etc.; hereinafter collectively referred to as operating status data). Therefore, even if an event occurs in which the robot 20 collides with an exhibit or the like (i.e., an object) in a facility while a user is remotely controlling the robot 20 using the computer 10, the operating status data representing the operating status at the time of the collision is recorded in the robot 20. Therefore, the user can clearly identify the cause of the collision by analyzing the operating status data recorded in the robot 20. The configurations of the computer 10, the robot 20, etc. will be described below.

[0014] <Functional configuration> (Computer 10) The computer 10 is an information processing device that operates the robot 20 and inputs information necessary for user authentication when using the robot 20. The computer 10 is a general-purpose or dedicated information processing device such as a smartphone, tablet terminal, PDA, personal computer, or wearable terminal. Note that a VR device, haptic gloves, or the like may be used to remotely control the robot 20.

[0015] FIG. 2 is a block diagram showing the functional configuration of the computer 10. As shown in FIG. The computer includes a receiving unit 11, an input unit 12, a communication unit 13, and an output unit 14.

[0016] The reception unit 11 receives input of various operation information including remote control of the robot 20. The input unit 12 inputs video data and audio data acquired by the camera and microphone of the computer 10. The video data and audio data input to the computer 10 are, for example, data representing the user's facial image and voice.

[0017] The communication unit 13 transmits and receives various data to and from the robot 20, the management server 30, etc. Specifically, the communication unit 13 receives video data and audio data acquired by the camera and microphone of the robot 20, and transmits various pieces of operation information input to the computer 10 to the robot 20.

[0018] The output unit 14 displays video data captured by the camera of the computer 10 or the robot 20 on a display device, and outputs audio data acquired by the microphone of the computer 10 or the robot 20 from a speaker.

[0019] FIG. 3 is a diagram illustrating a display screen P1 displayed on the computer 10 when a user remotely controls the robot 20 to view exhibits in a facility. As shown in Figure 3, the display screen P1 displays a user's face image P11, as well as a facility image P12 showing the inside of the facility captured by the robot 20's first camera, and a foot image P13 showing the robot 20's feet captured by the robot's second camera.

[0020] The user can remotely control the robot 20 while checking the facility image P12 and the foot image P13, thereby viewing various exhibits displayed within the facility.

[0021] (Robot 20) The robot 20 may be, for example, a telepresence robot or an avatar robot, and may have a moving part such as wheels. The robot 20 is an unfixed robot, and multiple robots may be placed within a facility, for example. Here, "unfixed" robots include robots 20 that are mobile and have wheels or the like, and robots 20 that can be worn by a person and have a manipulator or the like. In this embodiment, a mobile robot is assumed. Mobile robots are shown, for example, in Patent Document 1. Mobile robots include robots that run on one, two, or multiple wheels, robots that run on caterpillar tracks, robots that run on rails, robots that move by hopping, robots that walk on two legs, four legs, or multiple legs, robots that navigate on or underwater using propellers, and robots that fly using propellers or the like. A wearable robot is disclosed, for example, in MHD Yamen Saraiji, Tomoya Sasaki, Reo Matsumura, Kouta Minamizawa, and Masahiko Inami, "Fusion: full body surrogacy for collaborative communication," Proceeding SIGGRAPH '18 ACM SIGGRAPH 2018 Emerging Technologies Article No. 7. Furthermore, the robot 20 may be an autonomous or semi-autonomous vehicle or heavy machinery, a drone, or various flying objects. The robot 20 may also be a robot equipped with a camera that can move on rails. Note that these robots 20 may also be equipped with a robot hand or robot arm that can grasp or adsorb an object.

[0022] The robot 20 operates based on operation information from the computer 10 that has successfully completed user authentication. Here, the user authentication may be performed using a known method, and the information for user authentication may be registered in advance.

[0023] FIG. 4 is a block diagram showing the functional configuration of the robot 20. The robot 20 includes an operation record unit 21, a communication unit 22, an analysis unit 23, a determination unit 24, a log management unit 25, and a drive unit 26.

[0024] The operation record unit 21 acquires and records operation status data that represents the operation status of the robot 20, and has the same functions as existing drive recorders. The operation status data of the robot 20 may include video data acquired by a camera mounted on the robot 20 and audio data acquired by a microphone, as well as acceleration data acquired by an acceleration sensor (not shown), time data acquired by a timing circuit (not shown), and current location data acquired by a GPS sensor (not shown). When the operation record unit 21 acquires the operation status data, it records it in memory.

[0025] The communication unit 22 transmits and receives various data to and from the computer 10 and the management server 30. Specifically, the communication unit 22 receives various operation information transmitted from the computer 10, and transmits camera image data and microphone audio data acquired by the robot 20 to the computer 10. Furthermore, the communication unit 22 transmits (uploads) operation status data acquired and recorded by the operation record unit 21 to the management server 30 at a predetermined timing. Note that, although the present embodiment illustrates an example in which the operation status data acquired and recorded by the robot 20 is stored and managed by the management server 30, it may also be stored and managed within the robot 20, for example.

[0026] The analysis unit 23 analyzes the video data acquired by the camera and outputs the analysis results. As an example, the analysis unit 23 extracts feature points and the like from the video data of the robot 20 included in the operation status data, and analyzes the video data by using machine learning and the like. The analysis of the video data includes, but is not limited to, detecting the direction of the camera mounted on the robot 20 and the surrounding environment of the robot 20 (presence or absence of an object, distance to the object, position, etc.).

[0027] The determination unit 24 determines whether or not to start acquiring and recording operation status data based on the analysis result of the video data by the analysis unit 23 and the reference information SI. The reference information SI has registered therein a plurality of conditions for starting acquiring and recording operation status data.

[0028] FIG. 5 is a diagram illustrating the reference information SI. As shown in Figure 5, the standard information SI has the following registered as the first condition: "When the distance to the target object is less than N (cm)"; the second condition: "When a suspicious object (including a suspicious person) is recognized"; and the Mth condition: "When an act violating social norms (such as theft, vandalism, or bullying) is recognized."

[0029] When the determination unit 24 determines that acquisition and recording of operation status data should be started based on the comparison between the analysis result of the video data and the reference information SI, it instructs the operation record unit 21 to start acquisition and recording of operation status data of the robot 20. Note that in this embodiment, it is assumed that the determination is made to start acquisition and recording of operation status data of the robot 20, but in addition to (or instead of) this, it may also be possible to determine the end of acquisition and recording of operation status data. In this case, for example, a condition such as "when the distance to the target is L (m) or more" may be registered in the reference information SI as a condition for ending acquisition and recording of operation status data.

[0030] The log management unit 25 acquires and manages a log of communication with the computer 10 (communication log), a log of operation of the robot 20 based on operation information received from the computer 10 (operation log), and the like.

[0031] The communication logs and operation logs managed by the log management unit 25 are used to clarify the cause of an abnormal event (for example, a collision of the robot 20) when it occurs. To explain this by taking one example, when the robot 20 collides with some object, the communication logs, operation logs, etc. are used to verify whether the cause of the collision was, for example, a problem with the user's operation (such as an operation error), a problem with the robot 20 itself (such as an abnormal operation due to component deterioration), or a problem with the communication environment (such as a communication failure).

[0032] When the log management unit 25 receives a request from the computer 10 for a log when the robot 20 collides with an object, it extracts the communication log and operation log in response to the request and transfers (outputs) them to the computer 10. In addition to (or instead of) these logs, operation status data of the robot 20 before and after the collision may be output to the computer 10.

[0033] The driving unit 26 drives each part of the robot 20 (for example, the arms, wheels, etc.) in accordance with operation information transmitted from the computer 10.

[0034] (Management Server 30) The management server 30 manages the operation status data acquired by the robot 20. The management server 30 includes a storage for holding and managing the operation status data. The management server 30 may be configured by one information processing device, or may be configured by multiple information processing devices, such as in cloud computing or edge computing.

[0035] <Physical configuration> (Computer 10) FIG. 6 is a diagram showing the physical configuration of a computer 10 according to this embodiment. The computer 10 includes a central processing unit (CPU) 10a, a random access memory (RAM) 10b, a read-only memory (ROM) 10c, a communication device 10d, an input device 10e, a display device 10f, an audio device 10g, and a camera 10h. These components are connected via a bus to enable mutual data transmission and reception. While this example describes a case where the computer 10 is configured by a single computer, the computer 10 may also be implemented by combining multiple computers. The configuration shown in FIG. 6 is merely an example, and the computer 10 may include other components or may not include some of these components.

[0036] The CPU 10a is a control unit that controls the execution of programs stored in the RAM 10b or the ROM 10c and performs data calculations and processing. The CPU 10a is a calculation unit that executes programs for viewing exhibits and the like within a facility using the robot 20. The CPU 10a receives various data from the input device 10e, the communication device 10d, and the camera 10h, and displays the results of calculations on the data on the display device 10f, outputs sound from the audio device 10g, and stores the data in the RAM 10b.

[0037] The RAM 10b is a rewritable storage device and may be implemented, for example, by a semiconductor memory device. The RAM 10b may store various programs executed by the CPU 10a and data such as user information. Note that these are merely examples, and the RAM 10b may store other data or may not store all of the data.

[0038] The ROM 10c is a storage unit from which data can be read, and may be configured, for example, with a semiconductor memory element. The ROM 10c may store, for example, programs and data that are not rewritten.

[0039] The communication device 10d is an interface that connects the computer 10 to other devices via the communication network N.

[0040] The input device 10e receives data input from a user and may include, for example, a keyboard and a touch panel, and may also include a microphone for voice input.

[0041] The display device 10f visually displays the results of calculations performed by the CPU 10a and may be configured with, for example, an LCD (Liquid Crystal Display). The display device 10f may display video data captured by the camera 20j of the robot 20, video data captured by the camera 10h mounted on the computer 10, etc.

[0042] The audio device 10g is for outputting audio and the like, and may include, for example, an audio decoder and a speaker.

[0043] The camera 10h includes an imaging element that captures still images or moving images, and transmits the captured still images or moving images to the display device 10f and the robot 20 as video data.

[0044] (Robot 20) FIG. 7 is a diagram showing the physical configuration of the robot 20 according to this embodiment. The robot 20 has a CPU 20a corresponding to a calculation unit, a RAM 20b corresponding to a memory unit, a ROM 20c corresponding to a memory unit, a communication device 20d, an input device 20e, an operation record device 20f, a display device 20g, an audio device 20h, a drive device 20i, and a camera 20j. These components are connected via a bus so that they can send and receive data to each other. Note that the configuration shown in FIG. 7 is an example, and the robot 20 may have other components or may not have some of these components.

[0045] The CPU 20a is a control unit that controls the execution of programs stored in the RAM 20b or the ROM 20c and performs data calculations and processing. The CPU 20a is a calculation unit that executes programs for using the robot 20 to view exhibits and the like within the facility. The CPU 20a receives various data from the input device 20e and the communication device 20d, and displays the results of calculations on the display device 20g and stores them in the RAM 20b. The CPU 20a also controls the operation record device 20f and the drive device 20i to control the operation of the robot 20.

[0046] The RAM 20b is a rewritable storage device and may be implemented, for example, by a semiconductor memory device. The RAM 20b may store various programs executed by the CPU 20a. Note that these are merely examples, and the RAM 20b may store other data or may not store all of the programs.

[0047] The ROM 20c is a storage unit from which data can be read, and may be configured, for example, with a semiconductor memory element. The ROM 20c may store, for example, programs and data that are not rewritten.

[0048] The communication device 20d is an interface that connects the robot 20 to other devices via the communication network N.

[0049] The input device 20e receives data input from the outside and may include, for example, a touch panel, and may also include a microphone for voice input.

[0050] The operation record device 20f, under the control of the CPU 20a, generates operation status data of the robot 20 based on video data and audio data acquired by a camera and microphone, and stores the data in a memory such as the RAM 20b.

[0051] The display device 20g visually displays the results of calculations performed by the CPU 20a and may be configured with, for example, an LCD. The display device 20g may display video data captured by the camera 10h of the computer 10, video data captured by the camera 20j of the robot 20, etc.

[0052] The driving unit 20i includes a remotely controlled actuator, a moving unit such as wheels, a manipulator, etc. When the robot 20 is a mobile robot, the driving unit 20i includes at least a moving unit such as wheels, but may also include a manipulator. When the robot 20 is a wearable robot, the driving unit 20i includes at least a manipulator.

[0053] The camera 20j includes a first camera that captures an image in front of the robot 20 (i.e., inside the facility) and a second camera that captures an image at the feet of the robot 20. The camera 20j includes an imaging element that captures still images or videos, and transmits the captured still images or videos to the display device 20g and the computer 10 as video data.

[0054] The physical configurations of the computer 10 and the robot 20 described above are merely examples, and they do not necessarily have to be independent. For example, the computer 10 and the robot 20 may be provided with an LSI in which the CPU 20a, RAM 20b, and ROM 20c are integrated.

[0055] <Processing flow> 8 is a flowchart showing the processing of the operation record function installed in the robot 20. It is assumed that the user is permitted to operate the robot 20 installed in the facility using, for example, his / her own computer 10 (i.e., the user authentication has been successful).

[0056] In order to view the exhibits in the facility, the user starts remotely controlling the robot 20 using the computer 10. When the robot 20 receives operation information from the computer 10, it starts analyzing the video data acquired by the camera (step S1). The robot 20 compares the analysis result of the video data with the reference information SI (see FIG. 5) to determine whether or not to start acquiring and recording the operation status data of the robot 20 (step S2).

[0057] When the robot 20 determines that acquisition and recording of operation status data should be started because the distance to the object is less than N (cm), for example (step S2; YES), it starts acquisition and recording of operation status data of the robot 20 (step S3). The robot 20 transmits (uploads) the acquired and recorded operation status data to the management server 30 at a predetermined timing (step S4). By performing such processing, even if an event occurs in which the robot 20 collides with an object, the robot 20 etc. has recorded operation status data indicating the operation status at the time of the collision, and the user can analyze the operation status data to clarify the cause of the collision, etc.

[0058] On the other hand, if the robot 20 determines that acquisition and recording of operation status data should not be started (step S2; NO), the process returns to step S1.

[0059] Thereafter, the robot 20 determines whether or not to end the acquisition and recording of the operation status data (step S5). If the robot 20 determines that the acquisition and recording of the operation status data should end because, for example, the distance to the object is L (m) or more (step S5; YES), the robot 20 ends the process. On the other hand, if the robot 20 determines that the acquisition and recording of the operation status data should not end (step S5; NO), the robot 20 returns to step S3 and continues the acquisition and recording of the operation status data.

[0060] As described above, according to this embodiment, when a predetermined condition is satisfied, the robot starts recording operation status data that indicates the operation status. By analyzing the operation status data recorded by the robot, the user can clearly identify the cause of a collision, etc. Furthermore, when identifying the cause of a collision, by using the communication logs and operation logs acquired and managed by the robot, it is possible to determine whether the cause of the collision was, for example, a problem with the user's operation (such as an operating error), a problem with the robot itself (such as malfunction due to component deterioration), or a problem with the communication environment (such as a communication failure).

[0061] B. Variations In the above-described embodiment, the acquisition and recording of the operation status data of the robot 20 is controlled (started or ended) based on the reference information SI, but for example, a user may use the computer 10 to include a command for controlling the acquisition and recording of the operation status data in the operation information and send it to the robot 20. In this case, the operation record unit 21 of the robot 20 starts or ends the acquisition and recording of the operation status data in accordance with the command.

[0062] C. Other The present invention is not limited to the above-described embodiment and modifications, and can be embodied in various other forms without departing from the spirit of the present invention. Therefore, the above-described embodiment and modifications are merely examples in all respects and should not be construed as limiting. For example, the order of the above-described processing steps can be arbitrarily changed or executed in parallel as long as no contradiction occurs in the processing content.

[0063] Furthermore, in this specification, the terms "unit" and "device" do not simply refer to a physical configuration, but also include cases where the processing performed by the "unit" or "device" is realized by software. Furthermore, the processing performed by one "unit" or "device" may be realized by two or more physical configurations, and the processing performed by two or more "units" or "devices" may be realized by one physical means. [Explanation of symbols]

[0064] 100...communication system, 10...computer, 11...reception unit, 12...input unit, 13...communication unit, 14...output unit, 20...robot, 21...operation record unit, 22...communication unit, 23...analysis unit, 24...judgment unit, 25...log management unit, 26...drive unit, 30...management server, N...communication network.

Claims

1. A robot that operates based on operation information received from a computer via a communication network, an operation record unit that acquires and records operation status data that indicates the operation status of the robot; a log management unit that acquires and manages an operation log of the operation information and a communication log with the computer, which are used to identify the cause of an abnormal event when the abnormal event occurs; A camera that acquires video data; an analysis unit that analyzes the video data; a determination unit that determines whether or not the operation status data should be acquired and recorded based on an analysis result of the video data, and providing reference information indicating a condition for starting acquisition and recording of the operation status data and a condition for ending acquisition and recording of the operation status data; the determination unit determines whether to start acquiring and recording the operation status data and whether to end acquiring and recording the operation status data based on the analysis result of the video data and the reference information; When the log management unit receives a request from the computer for a log when an abnormal event occurs in the robot, the log management unit outputs the operation status data before and after the abnormal event occurs to the computer along with the operation log and communication log in response to the request.

2. The robot according to claim 1 , wherein the operation status data includes the video data.

3. The robot according to claim 1 or 2, further comprising a communication unit that uploads the recorded operation status data to a management server via the communication network.

4. the operation information includes a command for controlling acquisition and recording of the operating status data; The robot according to claim 1 , wherein the operation record unit controls acquisition and recording of the operation status data in accordance with the command.

Images