Operation system, processing method, and program

Abstract

To provide an operation system which enables reduction of the amount of video image data, including the surrounding situation of a robot to be operated, transmitted to a display device from a camera when an operator is not around the robot.SOLUTION: An operation system is provided, comprising first identification means for identifying a gesture of an operator, and second identification means for identifying a camera transmitting a captured video image according to the gesture.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present disclosure relates to an operating system, a processing method, and a program.

Background Art

[0002] In recent years, the development of remotely operated robots used in warehouses and the like has been actively carried out. Patent Document 1 discloses, as a related technology, a technology related to a system for remotely operating an industrial vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when an operator operates a robot, the operator needs to grasp the state around the robot. And when the operator is not around the robot to be operated, an image of the surroundings of the robot is transmitted from a camera that captures the surroundings of the robot to a display device that the operator can view, and the operator needs to confirm the situation around the robot via the display device. Thus, when the operator is not around the robot to be operated, there is a need for a technology capable of reducing the amount of video data including the situation around the robot transmitted from the camera to the display device.

[0005] One of the purposes of each aspect of the present disclosure is to provide an operating system, a processing method, and a program that can solve the above problems.

Means for Solving the Problems

[0006] To achieve the above object, according to one aspect of the present disclosure, an operating system includes a first specifying means for specifying a gesture by an operator, and a second specifying means for specifying a camera that transmits an image captured based on the gesture.

[0007] To achieve the above object, according to another aspect of the present disclosure, a processing method includes specifying a gesture by an operator, and specifying a camera that transmits an image captured based on the gesture.

[0008] To achieve the above object, according to another aspect of the present disclosure, a program causes a computer to execute specifying a gesture by an operator, and specifying a camera that transmits an image captured based on the gesture.

Advantages of the Invention

[0009] According to each aspect of the present disclosure, when the operator is not around the robot to be operated, there is a need for a technology that can reduce the data amount of an image including the situation around the robot transmitted from the camera to the display device.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0011] <First Embodiment> Hereinafter, embodiments will be described in detail with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of an operating system 1 according to a first embodiment of the present disclosure. As shown in FIG. 1, the operating system 1 includes a first system 10 and a second system 20. The operating system 1 is a system that transmits, via the first system 10, an image captured in the second system 20 to the first system 10 according to the position of the head of an operator who operates a robot 201 (to be described later) in the second system 20 and the direction of the operator's face, and selects one from a plurality of cameras 202 (to be described later). Also, the operating system 1 is a system in which the robot 201 in the second system 20 operates according to an operation performed on the first system 10. Note that an operation of moving the body, including changing the position of the operator's head and the direction of the operator's face, in order to operate the robot 201 via the operation device 101 while viewing an image captured by the camera is an example of a gesture.

[0012] As shown in Fig. 1, the first system 10 includes an operating device 101, a head-mounted display 102 (an example of a display device), a sensor 103, and an information processing device 104. The first system 10 selects one camera that transmits the video captured in the second system 20 to the first system 10 according to the position of the operator's head and the orientation of the operator's face. Also, the first system 10 operates the robot in the second system 20 according to the operation by the operator.

[0013] The operating device 101 generates a control signal for controlling the robot 201 according to the operation by the operator. The operating device 101 transmits the generated control signal to the second system 20.

[0014] The head-mounted display 102 decodes the video signal of the video captured by the camera 202 transmitted from the second system 20 and displays the video. The operator wears this head-mounted display and performs an operation for operating the robot 201 on the operating device 101 while watching the video captured by one of the plurality of cameras 202.

[0015] The sensor 103 detects both the position of the operator's head and the orientation of the operator's face. Examples of sensors include a gyro sensor, an acceleration sensor, a motion sensor, a tracker, etc. A gyro sensor is a sensor that can detect rotation and can detect the orientation of the operator's face. When the sensor is a gyro sensor, the sensor 103 includes another sensor that can detect the position of the operator's head. An acceleration sensor is a sensor that can detect acceleration in each of the front-rear, left-right, and up-down directions and can detect both the position of the operator's head and the orientation of the operator's face. A motion sensor can detect both the position of the operator's head and the orientation of the operator's face by photographing the operator and analyzing the photographed video. A tracker irradiates light such as infrared rays or a laser on an operator wearing a tracking sensor, and when the sensor receives the light, it can detect both the position of the operator's head and the orientation of the operator's face. The sensor 103 may include any number of sensors as long as it can detect both the position of the operator's head and the orientation of the operator's face. Also, part or all of the sensor 103 may be provided on the head-mounted display 102.

[0016] The information processing device 104 identifies a combination of the position of the operator's head and the orientation of the operator's face according to the position of the operator's head and the orientation of the operator's face detected by the sensor 103. FIG. 2 is a diagram showing an example of the configuration of the information processing device 104 according to the first embodiment of the present disclosure. As shown in FIG. 2, the information processing device 104 includes a storage unit 1041 and a processing unit 1042 (an example of a first specifying means, an example of a second specifying means).

[0017] The memory unit 1041 stores thresholds for specifying the position of the operator's head and the orientation of the operator's face. FIG. 3 is a diagram showing an example of thresholds in the first embodiment of the present disclosure. The part (a) of FIG. 3 shows a first threshold, a second threshold, a third threshold, and a fourth threshold for specifying the position of the operator's head. As shown in the part (a) of FIG. 3, the first threshold and the second threshold are thresholds for specifying the position of the operator's head in the left-right direction. Also, as shown in the part (a) of FIG. 3, the third threshold and the fourth threshold are thresholds for specifying the position of the operator's head in the depth direction. The position of the operator's head is classified into three regions in the left-right direction: a region to the left of the first threshold including the first threshold, a region to the left of the second threshold including the second threshold, a region to the right of the first threshold, and a region to the right of the second threshold. Also, the position of the operator's head is classified into three regions in the depth direction: a region deeper than the third threshold including the third threshold, a region deeper than the fourth threshold including the fourth threshold, a region in front of the third threshold, and a region in front of the fourth threshold. As a result, the position of the operator's head is classified into nine regions: left back, back, right back, left center, center, right center, left front, front, and right front, as shown in the part (a) of FIG. 3, by the first threshold, the second threshold, the third threshold, and the fourth threshold. This classification is performed by the processing unit 1042, as will be described later. Note that the reference for the nine regions of left back, back, right back, left center, center, right center, left front, front, and right front is, for example, the center of the central region.

[0018] The part (b) of FIG. 3 shows a fifth threshold value, a sixth threshold value, a seventh threshold value, and an eighth threshold value for specifying the orientation of the operator's face. As shown in the part (b) of FIG. 3, the fifth threshold value and the sixth threshold value are threshold values for specifying the orientation of the operator's face in the left-right direction. Also, as shown in the part (b) of FIG. 3, the seventh threshold value and the eighth threshold value are threshold values for specifying the orientation of the operator's face in the up-down direction. The orientation of the operator's face is classified into three regions in the left-right direction: a region to the left of the fifth threshold value including the fifth threshold value, a region to the left of the sixth threshold value including the sixth threshold value, a region to the right of the fifth threshold value, and a region to the right of the sixth threshold value. Also, the orientation of the operator's face is classified into three regions in the up-down direction: a region above the seventh threshold value including the seventh threshold value, a region above the eighth threshold value including the eighth threshold value, a region below the seventh threshold value, and a region below the eighth threshold value. As a result, the orientation of the operator's face is classified into nine regions: upper left, up, upper right, left middle, middle, right middle, lower left, down, and lower right, as shown in the part (b) of FIG. 3, by the fifth threshold value, the sixth threshold value, the seventh threshold value, and the eighth threshold value. This classification is performed by the processing unit 1042 as will be described later. Note that the reference for the nine regions of upper left, up, upper right, left middle, middle, right middle, lower left, down, and lower right is, for example, the center of the middle region.

[0019] Also, the storage unit 1041 stores a table TBL1 for selecting the camera 202 that transmits the captured video to the first system 10. FIG. 4 is a diagram showing an example of the table TBL1 stored in the storage unit 1041 according to the first embodiment of the present disclosure. As shown in FIG. 4, the table TBL1 stores in association a combination of the position of the operator's head and the orientation of the operator's face, and a selection signal for selecting a camera that transmits the captured video to the first system 10 in each combination case. The selection signal is a signal for switching the switch 203 of the second system 20 described later so that the camera that transmits the captured video to the first system 10 is connected to the encoder 204 of the second system 20 described later. Since the selection signal is a signal for selecting a camera, specifying the selection signal is equivalent to specifying the camera.

[0020] Based on the detection result of the position of the operator's head by the sensor 103, the processing unit 1042 identifies which of the positions of the operator's head is in the far left, far back, far right, left center, center, right center, left front, front, or right front. Also, based on the detection result of the orientation of the operator's face by the sensor 103, the processing unit 1042 identifies which of the orientations of the operator's face is in the upper left, up, upper right, left middle, middle, right middle, lower left, down, or lower right. The processing unit 1042 identifies the combination of the identified position of the operator's head and the identified orientation of the operator's face. The processing unit 1042 identifies the combination that matches the identified combination in the table TBL1. The processing unit 1042 identifies the selection signal associated with the identified combination in the table TBL1. Then, the processing unit 1042 transmits the identified selection signal to the second system 20.

[0021] As shown in FIG. 1, the second system 20 includes a robot 201, cameras 202a1, 202a2, 202a3, 202a4, 202a5, 202a6, 202a7, 202a8, 202a9, 202a10, 202a11, 202a12, 202a13, a switch 203, and an encoder 204. The cameras 202a1, 202a2, 202a3, 202a4, 202a5, 202a6, 202a7, 202a8, 202a9, 202a10, 202a11, 202a12, 202a13 are collectively referred to as the camera 202.

[0022] The robot 201 operates in response to a control signal received from the first system 10. Each of the cameras 202 captures an image of a defined area.

[0023] The switch 203 switches its internal connection so that the camera and the encoder 204 that transmit the captured image to the first system 10 are connected in response to the selection signal received from the first system 10. The encoder 204 encodes the image captured by the camera received via the switch 203. The encoder 204 transmits the encoded video signal to the first system 10.

[0024] Next, the processing of the operating system 1 according to the first embodiment of the present disclosure will be described. Here, the processing flow of the operating system 1 according to the first embodiment of the present disclosure shown in FIG. 5 will be described.

[0025] In order to operate the robot 201 via the operating device 101 while viewing the video captured by the camera, the operator performs an operation of moving the body that changes the position of the operator's head and the orientation of the operator's face. The sensor 103 detects the position of the operator's head and the orientation of the operator's face that change as the operator moves the body (step S1).

[0026] The processing unit 1042 identifies which of the left back, back, right back, left center, center, right center, left front, front, and right front the position of the operator's head detected by the sensor 103 is (step S2).

[0027] For example, the processing unit 1042 compares the left-right direction position of the operator's head position detected by the sensor 103 with a first threshold value. When the position of the operator's head is located on the left side of the first threshold value including the first threshold value, the processing unit 1042 compares the depth direction position of the operator's head position detected by the sensor 103 with a third threshold value. When the position of the operator's head is located behind the third threshold value including the third threshold value, the processing unit 1042 identifies the position of the operator's head as left back. When the position of the operator's head is located in front of the third threshold value, the processing unit 1042 compares the depth direction position of the operator's head position detected by the sensor 103 with a fourth threshold value. When the position of the operator's head is located behind the fourth threshold value including the fourth threshold value, the processing unit 1042 identifies the position of the operator's head as left center. When the position of the operator's head is located in front of the fourth threshold value, the processing unit 1042 identifies the position of the operator's head as left front.

[0028] Also, when the position of the operator's head is located to the right of the first threshold, the processing unit 1042 compares the left - right position of the position of the operator's head detected by the sensor 103 with the second threshold. And when the position of the operator's head is located to the left of the second threshold including the second threshold, the processing unit 1042 compares the depth - direction position of the position of the operator's head detected by the sensor 103 with the third threshold. When the position of the operator's head is located behind the third threshold including the third threshold, the processing unit 1042 specifies that the position of the operator's head is in the back. Also, when the position of the operator's head is located in front of the third threshold, the processing unit 1042 compares the depth - direction position of the position of the operator's head detected by the sensor 103 with the fourth threshold. When the position of the operator's head is located behind the fourth threshold including the fourth threshold, the processing unit 1042 specifies that the position of the operator's head is in the center. Also, when the position of the operator's head is located in front of the fourth threshold, the processing unit 1042 specifies that the position of the operator's head is in the front.

[0029] Also, when the position of the operator's head is located to the right of the second threshold, the processing unit 1042 compares the depth - direction position of the position of the operator's head detected by the sensor 103 with the third threshold. When the position of the operator's head is located behind the third threshold including the third threshold, the processing unit 1042 specifies that the position of the operator's head is in the back - right. Also, when the position of the operator's head is located in front of the third threshold, the processing unit 1042 compares the depth - direction position of the position of the operator's head detected by the sensor 103 with the fourth threshold. When the position of the operator's head is located behind the fourth threshold including the fourth threshold, the processing unit 1042 specifies that the position of the operator's head is in the right - center. Also, when the position of the operator's head is located in front of the fourth threshold, the processing unit 1042 specifies that the position of the operator's head is in the front - right.

[0030] Also, the processing unit 1042 specifies whether the direction of the operator's face detected by the sensor 103 is any of upper - left, up, upper - right, left - center, center, right - center, lower - left, down, and lower - right (step S3).

[0031] For example, the processing unit 1042 compares the left - right direction of the orientation of the operator's face detected by the sensor 103 with a fifth threshold value. When the orientation of the operator's face is located on the left side of the fifth threshold value including the fifth threshold value, the processing unit 1042 compares the upward orientation of the orientation of the operator's face detected by the sensor 103 with a seventh threshold value. When the orientation of the operator's face is located above the seventh threshold value including the seventh threshold value, the processing unit 1042 identifies the orientation of the operator's face as upper - left. Also, when the orientation of the operator's face is located below the seventh threshold value, the processing unit 1042 compares the upward orientation of the orientation of the operator's face detected by the sensor 103 with a fourth threshold value. When the orientation of the operator's face is located above the fourth threshold value including the fourth threshold value, the processing unit 1042 identifies the orientation of the operator's face as left - middle. Also, when the orientation of the operator's face is located below the fourth threshold value, the processing unit 1042 identifies the orientation of the operator's face as lower - left.

[0032] Also, when the orientation of the operator's face is located on the right side of the first threshold value, the processing unit 1042 compares the left - right direction of the orientation of the operator's face detected by the sensor 103 with a sixth threshold value. When the orientation of the operator's face is located on the left side of the sixth threshold value including the sixth threshold value, the processing unit 1042 compares the upward position of the orientation of the operator's face detected by the sensor 103 with a seventh threshold value. When the orientation of the operator's face is located above the seventh threshold value including the seventh threshold value, the processing unit 1042 identifies the orientation of the operator's face as upper. Also, when the orientation of the operator's face is located below the seventh threshold value, the processing unit 1042 compares the upward position of the orientation of the operator's face detected by the sensor 103 with an eighth threshold value. When the orientation of the operator's face is located above the eighth threshold value including the eighth threshold value, the processing unit 1042 identifies the orientation of the operator's face as middle. Also, when the orientation of the operator's face is located below the eighth threshold value, the processing unit 1042 identifies the orientation of the operator's face as lower.

[0033] Also, when the orientation of the operator's face is located to the right of the sixth threshold value, the processing unit 1042 compares the upward orientation of the orientation of the operator's face detected by the sensor 103 with the seventh threshold value. When the orientation of the operator's face is located above the seventh threshold value including the seventh threshold value, the processing unit 1042 specifies that the orientation of the operator's face is upper right. Also, when the orientation of the operator's face is located below the seventh threshold value, the processing unit 1042 compares the upward orientation of the orientation of the operator's face detected by the sensor 103 with the fourth threshold value. When the orientation of the operator's face is located above the eighth threshold value including the eighth threshold value, the processing unit 1042 specifies that the orientation of the operator's face is right middle. Also, when the orientation of the operator's face is located below the eighth threshold value, the processing unit 1042 specifies that the orientation of the operator's face is lower right.

[0034] The processing unit 1042 specifies a combination of the specified position of the operator's head and the specified orientation of the operator's face (step S4). The processing unit 1042 specifies, in the table TBL1, a combination that matches the specified combination (step S5). The processing unit 1042 specifies, in the table TBL1, a selection signal associated with the specified combination (step S6). Then, the processing unit 1042 transmits the specified selection signal to the second system 20.

[0035] The switch 203 switches its internal connection so that the camera that transmits the captured video to the first system 10 in response to the selection signal received from the first system 10 is connected to the encoder 204 (step S7). The encoder 204 encodes the video captured by the camera received via the switch 203 (step S8). The encoder 204 transmits the encoded video signal to the first system 10.

[0036] The head-mounted display 102 decodes the video signal of the video captured by the camera 202 transmitted from the second system 20 and displays the video (step S9). The operator wears this head-mounted display and performs an operation for operating the robot 201 on the operation device 101 while viewing the video captured by one of the plurality of cameras 202.

[0037] The operation device 101 generates a control signal for controlling the robot 201 according to the operation by the operator (step S10). The operation device 101 transmits the generated control signal to the second system 20. The robot 201 operates according to the control signal received from the first system 10 (step S11).

[0038] As described above, the operation system 1 according to the first embodiment of the present disclosure has been described. In the operation system 1, the processing unit 1042 (an example of the first specifying means, an example of the second specifying means) specifies a gesture by the operator including the operator changing the position of the operator's head and the orientation of the operator's face. Further, the processing unit 1042 specifies a camera that transmits the video captured based on the gesture. By doing so, it is possible to reduce the amount of video data that the operator needs to view when operating the robot. As a result, when the operator is not around the robot to be operated, it is possible to reduce the amount of video data including the situation around the robot transmitted from the camera to the display device.

[0039] <First Modification Example of the First Embodiment> The operation system 1 according to the first modification example of the first embodiment is a system that identifies the position of the operator's head based on a plurality of detection results of the position of the operator's head by a plurality of sensors 103, or identifies the orientation of the operator's face based on a plurality of detection results of the orientation of the operator's face by a plurality of sensors 103. Although not clearly shown in the first embodiment, when identifying the position of the operator's head based on a plurality of detection results of the position of the operator's head by a plurality of sensors 103, or when identifying the orientation of the operator's face based on a plurality of detection results of the orientation of the operator's face by a plurality of sensors 103, there may be different detection results for each sensor 103. Therefore, in the operation system 1 according to the first modification example of the first embodiment, in steps S2 and S3, the processing unit 1042 calculates the majority vote of the detection results indicated by the plurality of sensors 103, or the logical product (AND) of the detection results indicated by the plurality of sensors 103. And the processing unit 1042 may use the calculation result as the position of the operator's head or the orientation of the operator's face. With the operation system 1 according to the first modification example of the first embodiment, compared with the case of identifying the position of the operator's head or the orientation of the operator's face by one sensor 103, the position of the operator's head and the orientation of the operator's face can be identified more accurately.

[0040] <Second Modification Example of the First Embodiment> The operation system 1 according to the second modification of the first embodiment is a system that prohibits the change of a specified camera from among a plurality of cameras 202 when an operator performs a predetermined operation. In the operation system 1 according to the first embodiment or the operation system 1 according to the second modification of the first embodiment, when the position of the operator's head or the direction of the operator's face changes while the operator is operating the robot 201, the camera may switch according to the position of the operator's head or the direction of the operator's face, and the video may switch. As a result, it may become difficult for the operator to operate the robot 201. In such a case, it is desirable to fix the specified camera so that the operator can operate the robot 201. To enable this, in the operation system 1 according to the second modification of the first embodiment, when the sensor 103 detects a predetermined operation by the operator, the processing unit 1042 (an example of a change prohibition means) prohibits the change of the specified camera. Examples of the predetermined operation by the operator include an operation of shaking the face at high speed, an operation of blinking a predetermined number of times, an operation of changing the shape of the mouth to a predetermined shape, an operation indicating the detection content where the corresponding selection signal does not exist on the table TBL1, an operation of moving the hands, feet, and body so as to make a predetermined movement, an operation of pressing a predetermined button, and the like. With the operation system 1 according to the second modification of the first embodiment, compared with the operation system 1 according to the first embodiment and the operation system 1 according to the second modification of the first embodiment, the possibility of the camera switching during the operation of the robot 201 is reduced. As a result, in the operation system 1 according to the second modification of the first embodiment, the operator can operate the robot 201 more easily compared with the operation system 1 according to the first embodiment and the operation system 1 according to the second modification of the first embodiment.

[0041] <Second Embodiment> The operation system 1 according to the second embodiment of the present disclosure specifies a camera using a gesture for specifying one from among a plurality of cameras 202 via the head-mounted display 102, instead of the gesture of the operator changing the position of the operator's head and the direction of the operator's face in the operation system 1 according to the first embodiment of the present disclosure. FIG. 6 is a diagram showing an example of the configuration of the operation system 1 according to the second embodiment of the present disclosure.

[0042] As shown in FIG. 6, the operating system 1 includes a first system 10 and a second system 20. The first system 10 includes a sensor 105 instead of the sensor 103 according to the first embodiment. The first system 10 includes an information processing device 106 instead of the information processing device 104 according to the first embodiment. The head-mounted display 102 displays the video captured by the camera 202 that the camera 205 (to be described later) of the second system 20 captures. The sensor 105 detects the operator's line of sight and blinking.

[0043] FIG. 7 is a diagram showing an example of the configuration of the information processing device 106 according to the second embodiment of the present disclosure. The information processing device 106 includes a storage unit 1061 and a processing unit 1062 (an example of the first specifying means). The storage unit 1061 stores, in association with each camera 202, a selection signal that is a signal for switching the switch 203 of the second system 20 so that each camera is connected to the encoder 204 of the second system 20. FIG. 8 is a diagram showing an example of the table TBL2 stored in the storage unit according to the second embodiment of the present disclosure. The table TBL2 is a table in which the storage unit 1061 stores, in association with each camera 202, a selection signal that is a signal for switching the switch 203 of the second system 20 so that each camera is connected to the encoder 204 of the second system 20.

[0044] Further, the processing unit 1062 looks at the video showing the camera 202 itself via the head-mounted display 102, aligns the line of sight with one of the plurality of cameras 202, and specifies one camera in response to an operation by an operator who performs a predetermined gesture including, for example, the operator blinking twice.

[0045] Next, the processing of the operating system 1 according to the second embodiment of the present disclosure will be described. Here, the processing flow of the operating system 1 according to the second embodiment of the present disclosure shown in FIG. 9 will be described.

[0046] While looking at the camera 202 itself being photographed by the camera 205 shown on the head-mounted display 102, the operator aligns the line of sight with one specific camera among the plurality of cameras 202 and performs a predetermined gesture for specifying the camera. The processing unit 1062 identifies one camera in response to such an operation by the operator (step S21). The processing unit 1062 identifies the camera that matches the identified camera in the table TBL2 (step S22). Then, the processing unit 1062 identifies the selection signal associated with the identified camera in the table TBL2 (step S23). The processing unit 1062 transmits the identified selection signal to the second system 20. Then, the processes of steps S7 to S11 are performed.

[0047] As described above, the operation system 1 according to the second embodiment of the present disclosure has been described. In the operation system 1 according to the second embodiment of the present disclosure, the processing unit 1062 identifies one camera in response to an operation by the operator. The processing unit 1062 identifies the camera that matches the identified camera in the table TBL2. Then, the processing unit 1062 identifies the selection signal associated with the identified camera in the table TBL2. The processing unit 1062 transmits the identified selection signal to the second system 20. By doing so, the operator can directly select the camera. As a result, the operation system 1 according to the second embodiment can identify the camera more reliably than the operation system 1 of the first embodiment.

[0048] FIG. 10 is a diagram showing the minimum configuration of the operation system 1 according to the embodiment of the present disclosure. As shown in FIG. 10, the operation system 1 includes a first specifying means 300 and a second specifying means 400. The first specifying means 300 specifies a gesture by the operator. The second specifying means 400 specifies the camera that transmits the video captured based on the gesture.

[0049] FIG. 11 is a diagram showing an example of a processing flow of the minimum-configuration operation system 1 according to an embodiment of the present disclosure. Next, the processing of the minimum-configuration operation system 1 according to the embodiment of the present disclosure will be described with reference to FIG. 11.

[0050] The first specifying means 300 specifies a gesture by the operator (step S101). The second specifying means 400 specifies a camera that transmits an image captured based on the gesture (step S102).

[0051] As described above, the minimum-configuration operation system 1 according to the embodiment of the present disclosure has been described. With this operation system 1, when the operator is not around the robot to be operated, the amount of video data including the situation around the robot transmitted from the camera to the display device can be reduced.

[0052] Note that in the embodiment of the present disclosure, the order of processing may be changed as long as appropriate processing is performed.

[0053] Although the embodiment of the present disclosure has been described, the above-described operation system 1, first system 10, second system 20, operation device 101, head-mounted display 102, information processing devices 104 and 106, encoder 204, and other control devices may have a computer system inside. And the processes described above are stored in a computer-readable recording medium in the form of a program, and the above processes are performed by the computer reading and executing this program. Specific examples of the computer are shown below. FIG. 12 is a schematic block diagram showing the configuration of a computer according to at least one embodiment.

[0054] As shown in FIG. 12, the computer 5 includes a CPU (Central Processing Unit) 6, a main memory 7, a storage 8, and an interface 9. For example, each of the above-described operating system 1, first system 10, second system 20, operating device 101, head-mounted display 102, information processing devices 104, 106, encoder 204, and other control devices is implemented in the computer 5. And the operations of the above-described respective processing units are stored in the storage 8 in the form of a program. The CPU 6 reads the program from the storage 8 and expands it in the main memory 7, and executes the above processing according to the program. Further, the CPU 6 secures a storage area corresponding to each of the above-described storage units in the main memory 7 according to the program.

[0055] Examples of the storage 8 include an HDD (Hard Disk Drive), an SSD (Solid State Drive), a magnetic disk, a magneto-optical disk, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), a semiconductor memory, and the like. The storage 8 may be an internal medium directly connected to the bus of the computer 5, or may be an external medium connected to the computer 5 via the interface 9 or a communication line. Also, when this program is distributed to the computer 5 via a communication line, the computer 5 that has received the distribution may expand the program in the main memory 7 and execute the above processing. In at least one embodiment, the storage 8 is a non-transitory tangible storage medium.

[0056] Also, the above program may implement a part of the above-described functions. Further, the above program may be a file, so-called differential file (differential program), that can implement the above-described functions in combination with a program already recorded in the computer system.

[0057] Although some embodiments of the present disclosure have been described, these embodiments are examples and do not limit the scope of the disclosure. Various additions, omissions, replacements, and changes may be made without departing from the gist of the disclosure.

[0058] Note that some or all of the above embodiments may also be described as follows, but are not limited thereto.

[0059] (Appendix 1) A first specifying means for specifying a gesture by an operator, A second specifying means for specifying a camera that transmits an image captured based on the gesture, An operating system comprising the same.

[0060] (Appendix 2) The gesture is An action of the operator including a change in the position of the operator's head and the direction of the operator's face, The operating system according to Appendix 1.

[0061] (Appendix 3) The gesture is An action of the operator including the operator aligning the line of sight with the camera in the image and blinking, The operating system according to Appendix 1.

[0062] (Appendix 4) A change prohibiting means for prohibiting a change in the camera specified by the second specifying means when the operator performs a predetermined operation, The operating system according to Appendix 1 or Appendix 2, comprising the same.

[0063] (Appendix 5) Specifying a gesture by an operator, Specifying a camera that transmits an image captured based on the gesture, A processing method including the same.

[0064] (Appendix 6) To a computer, Identifying a gesture by an operator, identifying a camera that transmits an image captured based on the gesture, and a program for causing the above to be executed.

Explanation of Signs

[0065] 1 ··· Operating system 5 ··· Computer 6 ··· CPU 7 ··· Main memory 8 ··· Storage 9 ··· Interface 10 ··· First system 20 ··· Second system 101 ··· Operating device 102 ··· Head-mounted display 103, 105 ··· Sensors 104, 106 ··· Information processing devices 201 ··· Robot 202, 202a1 to 202a13 ··· Cameras 203 ··· Switch 204 ··· Encoder

Claims

1. An operating system comprising a first system and a second system, the first system has a head-mounted display worn by an operator, the second system includes a plurality of first cameras that capture images of a surrounding environment of the robot that is an object of operation by the operator, and a second camera that captures images of the plurality of first cameras; a first camera identification means for identifying one first camera from the images of the plurality of first cameras projected on the head-mounted display via the second camera, based on a line of sight and a predetermined gesture of the operator; a means for transmitting an image captured by the first camera identified by the first camera identifying means to the second system; Equipped with Operation system.

2. A processing method executed by an operating system having a first system and a second system, the first system has a head-mounted display worn by an operator, the second system includes a plurality of first cameras that capture images of a surrounding environment of the robot that is an object of operation by the operator, and a second camera that captures images of the plurality of first cameras; a first camera identifying step of identifying one first camera from the images of the plurality of first cameras displayed on the head-mounted display via the second camera based on a line of sight and a predetermined gesture of the operator; a step of transmitting an image captured by the first camera identified in the first camera identifying step to the second system; A processing method comprising:

3. A first system having a head-mounted display worn by an operator; a second system having a plurality of first cameras that capture images of the surroundings of the robot that is the object of operation of the operator, and a second camera that captures images of the plurality of first cameras; A computer of an operating system comprising: a first camera identifying step of identifying one first camera from the images of the plurality of first cameras displayed on the head-mounted display via the second camera based on a line of sight and a predetermined gesture of the operator; a step of transmitting an image captured by the first camera identified in the first camera identifying step to the second system; A program that executes the following.

Images