Information processing system, information processing method, and information processing program

Abstract

To provide an information processing system, an information processing method, and an information processing program that can appropriately operate a robot.SOLUTION: An information processing system 100 comprises at least one processor. Therein the processor is configured to: acquire motion information indicating a motion of an operator U1 who operates a movable unit of a robot 50; and control the robot 50 so as to be switched from an operation mode in which the movable unit is in an operable state to a non-operation mode in which the movable unit is in an inoperable state in a case where the motion information satisfies a predetermined first condition.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing system, an information processing method, and an information processing program. [Background technology]

[0002] With the development of technologies related to virtual reality (VR), augmented reality (AR), and mixed reality (MR), a method called telexistence has been developed, in which a person operates a robot from a remote location. Telexistence allows the operator to perceive information about the robot's surroundings detected by cameras and microphones attached to the robot, and the operator can operate the robot. The operator can operate the robot while feeling as if he or she is in the same environment as the robot.

[0003] For example, Patent Document 1 discloses that an operator at a position away from a robot arm operates the robot arm by performing non-contact actions while checking images captured by a camera attached to the robot arm. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. WO2017 / 033367 Summary of the Invention [Problem to be solved by the invention]

[0005] In recent years, there has been a demand for technology that utilizes telexistence to allow robots to perform tasks on behalf of humans in areas where there are restrictions on the operator's entry or presence (hereinafter referred to as "restricted areas"). Examples of restricted areas include radiation controlled areas, high magnetic field controlled areas, biohazard controlled areas, sterile rooms, intensive care rooms, clean rooms, etc.

[0006] On the other hand, people are not completely prevented from entering and exiting the restricted area, and it is conceivable that people, including the robot operator, may enter and exit as necessary. For people entering and exiting the restricted area, if the robot makes an unexpected movement, it may interfere with the work of the people or cause a collision between the robot and the person. Therefore, technology that prevents the robot from making unexpected movements is desired for people entering and exiting the restricted area.

[0007] The present disclosure provides an information processing system, an information processing method, and an information processing program that can appropriately operate a robot. [Means for solving the problem]

[0008] A first aspect of the present disclosure is an information processing system comprising at least one processor, which acquires operation information indicating the operation of an operator manipulating a movable part of the robot, and controls the robot to switch from an operation mode in which the movable part is in an operable state to a non-operation mode in which the movable part is in an inoperable state when the operation information satisfies a predetermined first condition.

[0009] In the above aspect, the processor may acquire operation information for the movable part input by the operator, and in the operation mode, control the robot to perform an action according to the operation information, and in the non-operation mode, discard the operation information.

[0010] In the above aspect, the processor may generate operation information in the operating mode such that the movable part operates in conjunction with the operation information.

[0011] In the above aspect, the processor may control the robot to switch from the operating mode to the non-operating mode when the operation information indicates, as a first condition, that the operator has entered a predetermined area.

[0012] In the above aspect, the predetermined area may be an area within a predetermined range from the robot.

[0013] In the above aspect, the predetermined area may be at least one of a radiation controlled area, a high magnetic field controlled area, a biohazard controlled area, a sterile room, an intensive care unit, and a clean room, and the robot may operate within the predetermined area.

[0014] In the above aspect, the robot may be equipped with at least one detection unit selected from a sensor, a camera, and a microphone, and the processor may acquire detection information acquired by the detection unit of the robot and output the detection information to at least one output device.

[0015] In the above aspect, the processor may stop outputting the detection information to the output device when the operation information satisfies a second predetermined condition.

[0016] In the above aspect, the processor may acquire assistant operation information indicating the operation of an assistant other than the operator, and control the robot to switch from the operating mode to the non-operating mode when the assistant operation information satisfies a predetermined third condition.

[0017] In the above aspect, the processor may output the detection information to a first output device pre-associated with the operator and to a second output device pre-associated with an assistant other than the operator, acquire assistant operation information indicating the assistant's operation, and stop outputting the detection information to the second output device when the assistant operation information satisfies a predetermined fourth condition.

[0018] In the above aspect, the non-operating mode may be one in which functions of the robot other than the movable parts are not stopped.

[0019] In the above aspect, the robot may include at least one of a movement mechanism and an operation mechanism as a movable part.

[0020] In the above aspect, the motion information may indicate at least one of the movement of a predetermined part of the operator's body, a line of sight, a voice, an electroencephalogram, and a position.

[0021] A second aspect of the present disclosure is an information processing method in which operation information indicating the operation of an operator manipulating a movable part of a robot is acquired, and when the operation information satisfies a predetermined first condition, a computer executes a process to control the robot to switch from an operation mode in which the movable part is in an operable state to a non-operation mode in which the movable part is in an inoperable state.

[0022] A third aspect of the present disclosure is for a computer to execute a process to control the robot to acquire operation information indicating the operation of an operator manipulating a movable part of the robot, and when the operation information satisfies a predetermined first condition, switch the robot from an operation mode in which the movable part is in an operable state to a non-operation mode in which the movable part is in an inoperable state. Effect of the Invention

[0023] According to the above aspects, the information processing system, the information processing method, and the information processing program disclosed herein can appropriately operate a robot. [Brief description of the drawings]

[0024] [Figure 1] 1 is a schematic diagram showing an example of an overall configuration of an information processing system according to a first embodiment. [Diagram 2] 2 is a block diagram showing an example of a hardware configuration of an information processing device, a robot, a modality, and a terminal device. FIG. [Diagram 3] FIG. 2 is a schematic diagram illustrating an example of an external appearance of a robot. [Figure 4] FIG. 1 is a schematic diagram showing an example of how the robot and modalities are used. [Diagram 5] FIG. 2 is a block diagram showing an example of a functional configuration of an information processing device. [Figure 6] FIG. 13 is a diagram for explaining a non-operating mode of the robot. [Figure 7] FIG. 13 is a diagram for explaining disconnection from a robot. [Figure 8] 11 is a flowchart illustrating an example of information processing. [Figure 9] FIG. 11 is a schematic diagram showing an example of an overall configuration of an information processing system according to a second embodiment. [Figure 10] FIG. 13 is a diagram for explaining a non-operating mode of the robot. [Figure 11] FIG. 13 is a diagram for explaining disconnection from a robot. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0026] [First embodiment] First, an information processing system 100 according to the present embodiment will be described with reference to Fig. 1 to Fig. 4. Fig. 1 is a schematic diagram showing an example of the overall configuration of the information processing system 100. As shown in Fig. 1, the information processing system 100 includes an information processing device 10, a robot 50, a modality 80, and a terminal device 90. The information processing device 10 is configured to be connectable to each of the robot 50, the modality 80, and the terminal device 90 via a wired or wireless network.

[0027] The information processing device 10 has a console function for controlling the modality 80 in response to an operation by an operator U1. As an example, the modality 80 in this embodiment is a computed tomography (CT) device that performs radiography of a subject U2. The modality 80 performs radiography of the subject U2 under the control of the information processing device 10, and transmits the obtained radiographic image to the information processing device 10. The information processing device 10 controls the radiographic image received from the modality 80 to be displayed on the display 24A.

[0028] When performing radiography, the operator U1 performs advance preparations such as positioning the subject U2 in the radiation controlled area, then leaves the radiation controlled area and issues an instruction to start radiography using the modality operation unit 25B of the information processing device 10 (console) located outside the radiation controlled area. When radiography is completed, the operator U1 re-enters the radiation controlled area and performs post-processing such as releasing the positioning of the subject U2. In this way, by the operator U1 moving away from the subject U2 during radiation irradiation and the period before and after, unnecessary exposure of the operator U1 can be avoided, but it may be difficult to directly monitor the condition of the subject U2 in close proximity to the subject U2 during radiography.

[0029] Therefore, in the information processing system 100 according to this embodiment, a robot 50 having a detection function such as a camera is placed near the subject U2 (i.e., in a radiation controlled area), and the robot 50 is moved to detect the state of the subject U2 while changing its position, orientation, etc. This allows an operator U1 who is located away from the subject U2 to closely monitor the state of the subject U2. Fig. 1 shows the flow of information exchange when the operator U1 and the subject U2 are apart from each other.

[0030] Specifically, the robot 50 detects images, movements, sounds, etc. of the subject U2 using the detection unit 52, and transmits the detection information (i.e., information indicating the state of the subject U2) to the information processing device 10. The information processing device 10 transmits the detection information received from the robot 50 to a terminal device 90 owned by the operator U1. The terminal device 90 outputs the detection information received from the information processing device 10 to an output unit 94. This allows the operator U1 to monitor the state of the subject U2 detected by the robot 50 using the terminal device 90 owned by the operator U1.

[0031] In addition, in response to the operation of the operator U1 inputted via the robot operation unit 25A of the information processing device 10, the robot 50 moves to a position in the radiation controlled area where it is easy to monitor the condition of the subject U2, and provides assistance to the subject U2, etc.

[0032] An example of the hardware configuration of the information processing device 10, the robot 50, the modality 80, and the terminal device 90 will be described with reference to FIG.

[0033] 2, the information processing device 10 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 as a temporary storage area. The information processing device 10 also includes an output unit 24 including a display 24A and a speaker 24B, and an input unit 25 including a touch panel, a keyboard, a mouse, switches, buttons, etc. The input unit 25 also functions as a robot operation unit 25A for operating the robot 50, and a modality operation unit 25B for operating the modality 80.

[0034] The information processing device 10 also includes an I / F (Interface) unit 26 that performs wired or wireless communication with the robot 50, the terminal device 90, the modality 80, and other external devices. The CPU 21, the storage unit 22, the memory 23, the output unit 24, the input unit 25, and the I / F unit 26 are connected via a bus 28 such as a system bus and a control bus so as to be able to transmit and receive various information to and from each other.

[0035] The storage unit 22 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), and a flash memory. An information processing program 27 for the information processing device 10 is stored in the storage unit 22. The CPU 21 reads the information processing program 27 from the storage unit 22, expands it in the memory 23, and executes the expanded information processing program 27. The CPU 21 is an example of a processor of the present disclosure. For example, a personal computer, a server computer, a smartphone, a tablet terminal, a wearable terminal, or the like can be appropriately applied as the information processing device 10.

[0036] 2, the robot 50 includes a main body control unit 51 including a CPU, a ROM (Read Only Memory), and a RAM (Random Access Memory), all of which are not shown. The robot 50 also includes an output unit 54 including a display 54A and a speaker 54B, and an input unit 55 including a touch panel, a keyboard, a mouse, switches, and buttons. The robot 50 also includes an I / F unit 56 that performs wired or wireless communication with the information processing device 10 and other external devices.

[0037] The robot 50 also includes a detection unit 52 for detecting detection information such as the motion and voice of the subject U2 in the vicinity of the robot 50. For example, the detection unit 52 is at least one of a sensor 52A, a camera 52B, and a microphone 52C. As the sensor 52A, for example, an acceleration sensor, a gyro sensor, a geomagnetic sensor, a temperature sensor, and a distance measurement sensor can be applied. The distance measurement sensor is a sensor that measures the distance to an object, and is, for example, a LIDAR (Laser Imaging Detection and Ranging or Light Detection and Ranging), a TOF (Time Of Flight) camera, a stereo camera, and the like. The detection information detected by the detection unit 52 is output to the information processing device 10 via the I / F unit 56, and is further output from the information processing device 10 to the terminal device 90.

[0038] The robot 50 also includes a movable part 53 that is controlled in response to operation information input by an operator U1 via a robot operation unit 25A of the information processing device 10. For example, the movable part 53 is at least one of a moving mechanism 53A and an operation mechanism 53B.

[0039] The moving mechanism 53A is a mechanism for running and rotating the entire robot 50, and is configured to include, for example, wheels and a drive mechanism thereof. The operating mechanism 53B is a mechanism for the robot 50 to perform operations such as grasping, moving, and rotating nearby objects, and is configured to include, for example, a robot arm and a drive mechanism thereof. The operating mechanism 53B can be used, for example, to hold an IV tube and an ECG cable connected to the subject U2 so as not to interfere with imaging.

[0040] In the robot 50, the main body control unit 51, detection unit 52, movable unit 53, output unit 54, input unit 55, and I / F unit 56 are connected via a bus 58 such as a system bus and a control bus so as to be able to exchange various types of information with each other.

[0041] 2, the modality 80 includes a main body control unit 81 including a CPU, a ROM, a RAM, etc. (not shown). The modality 80 also includes an imaging unit 82 for performing radiation imaging, and a bed unit 83 on which the subject U2 lies during radiation imaging. The modality 80 also includes an I / F unit 86 for performing wired or wireless communication with the information processing device 10 and other external devices. In the modality 80, the main body control unit 81, the imaging unit 82, the bed unit 83, and the I / F unit 86 are connected via a bus 88 such as a system bus and a control bus so as to be able to transmit and receive various information to and from each other.

[0042] 2, the terminal device 90 includes a main body control unit 91 including a CPU, a ROM, a RAM, etc. (not shown). The terminal device 90 also includes an output unit 94 including a display 94A and a speaker 94B, etc., and an input unit 95 including a touch panel, a keyboard, a mouse, switches, buttons, etc. The terminal device 90 also includes an I / F unit 96 that performs wired or wireless communication with the information processing device 10 and other external devices, etc.

[0043] The terminal device 90 also includes a detection unit 92 for detecting motion information such as motion and voice of the operator U1. For example, the detection unit 92 is at least one of a sensor 92A, a camera 92B, and a microphone 92C for detecting the motion of the operator U1. For example, an acceleration sensor, a gyro sensor, a geomagnetic sensor, a temperature sensor, and a distance measurement sensor can be applied as the sensor 92A. The distance measurement sensor is a sensor that measures the distance to an object, and is, for example, a LIDAR, a TOF camera, and a stereo camera. The motion information of the operator U1 detected by the detection unit 92 is output to the information processing device 10 via an I / F unit 96.

[0044] In addition, the terminal device 90 may output the motion information of the operator U1 detected by the detection unit 92 to the output unit 94 of the terminal device 90, the output unit 24 of the information processing device, the output unit 54 of the robot 50, etc. In other words, the motion information of the operator U1 may be made available for the operator U1 himself and the subject U2 to confirm. In the terminal device 90, the main body control unit 91, the detection unit 92, the output unit 94, the input unit 95, and the I / F unit 96 are connected to each other via a bus 98 such as a system bus and a control bus so that various information can be exchanged between them. As the terminal device 90, for example, a smartphone, a wearable terminal, a tablet terminal, a personal computer, etc. can be appropriately applied. As the wearable terminal, for example, a head mounted display, smart glasses, a smart watch, etc. can be mentioned.

[0045] 1 and 2, the robot operation unit 25A and the modality operation unit 25B are included in the information processing device 10, but the present invention is not limited to this. For example, instead of or in addition to the information processing device 10, the terminal device 90 may have at least one of the functions of the robot operation unit 25A and the modality operation unit 25B. Also, for example, instead of or in addition to the information processing device 10, the robot 50 may have at least one of the functions of the robot operation unit 25A and the modality operation unit 25B. Also, for example, instead of or in addition to the information processing device 10, a dedicated operation terminal (e.g., a smartphone) may have at least one of the functions of the robot operation unit 25A and the modality operation unit 25B.

[0046] 1 and 2, the information processing device 10 controls both the robot 50 and the modality 80, but the present invention is not limited to this. The information processing device 10 may be configured with, for example, two consoles: a console for controlling the robot 50 and a console for controlling the modality 80. For example, the information processing system 100 according to the present embodiment can be applied to a case where the modality 80 and its console have already been installed, and the robot 50 and its console are newly installed as additional devices.

[0047] Fig. 3 shows an example of a schematic diagram of the appearance of the robot 50. Fig. 4 shows an example of a schematic diagram of a usage mode of the robot 50 and the modality 90 in Fig. 3. The robot 50 shown in Fig. 3 includes wheels 53A1 as an example of a moving mechanism 53A, and a robot arm 53B1 as an example of an operating mechanism 53B. The robot 50 also includes a display 54A as an example of an output unit 54, and a detection unit 52 provided on the display 54A.

[0048] 4, a CT scanner is placed in the radiation controlled area as an example of a modality 90, a subject U2 lies on a bed 83, and a robot 50 is placed beside the subject U2 to monitor his / her condition. The robot 50 detects the image, movement, sound, etc. of the subject U2 with a detection unit 52, and transmits the detected information (i.e., information indicating the condition of the subject U2) to the information processing device 10. This allows an operator U1 outside the radiation controlled area to monitor the condition of the subject U2 in detail.

[0049] The robot 50 may also be configured to communicate between the operator U1 and the subject U2 by displaying a facial image of the operator U1 on the display 54A and emitting the voice of the operator U1 from the speaker 54B. In that case, the operator U1 can move the position of the robot 50 by the moving mechanism 53A in order to closely observe the condition of the subject U2.

[0050] Meanwhile, in the process of radiography, the operator U1 may repeatedly enter and exit the radiation controlled area. For example, after the operator U1 instructs the subject U2 on the positioning and posture required for radiography, when the operator U1 goes outside the radiation controlled area and operates the modality operation unit 25B to start radiography, the subject U2 may deviate from the predetermined positioning for some reason. In this case, the operator U1 may return to the radiation controlled area again and reposition the subject U2.

[0051] In such a case, if the robot 50 moves in an unexpected manner for the operator U1 while the operator U1 is in a radiation controlled area, it may interfere with the work of the operator U1, or the robot 50 may collide with the operator U1. On the other hand, it is troublesome for the operator U1 to manually stop the robot 50 from moving every time the operator U1 enters or leaves the radiation controlled area, and the operator U1 may forget to take the action.

[0052] Therefore, the information processing device 10 according to the present embodiment controls the robot 50 so that it does not move in a way that is unexpected by the operator U1, by automatically switching the robot 50 between a movable state and a non-movable state according to the action of the operator U1. Here, the movable state of the robot 50 means that the movable part 53 of the robot 50 is in an operable state. Hereinafter, this state is referred to as an "operation mode". On the other hand, the non-movable state of the robot 50 means that the movable part 53 of the robot 50 is in an inoperable state. Hereinafter, this state is referred to as an "inoperation mode". In the non-operation mode, the movable part 53 of the robot 50 may or may not be locked. The "unlocked" state means that the robot 50 moves when the operator U1 directly touches and pushes it.

[0053] Here, the non-operation mode is preferably a mode in which functions of the robot 50 are not stopped other than the movable part 53. For example, it may be preferable to continue monitoring the subject U2, i.e., detection and output of detection information by the robot 50, regardless of whether the operator U1 is inside or outside a radiation controlled area. Therefore, the information processing device 10 may control the functions of the robot 50, such as the CPU 71, the output unit 54, and the detection unit 52, other than the movable part 53, not to be stopped even in the non-operation mode.

[0054] The information processing device 10 will be described below. An example of the functional configuration of the information processing device 10 will be described with reference to Fig. 5. As shown in Fig. 5, the information processing device 10 includes an acquisition unit 30, a determination unit 32, and a control unit 34. The CPU 21 executes the information processing program 27, causing the CPU 21 to function as each of the functional units of the acquisition unit 30, the determination unit 32, and the control unit 34.

[0055] (Robot 50 mode switching) First, a process of switching the robot 50 between an operating mode and a non-operating mode according to the action of the operator U1 will be described with reference to Fig. 1, Fig. 6 and Fig. 7. Fig. 6 is a schematic diagram showing an example of the overall configuration of the information processing system 100 in a state where the operator U1 has entered a radiation controlled area. Fig. 7 is a schematic diagram showing an example of the overall configuration of the information processing system 100 in a state where the operator U1 has left both the radiation controlled area and the medical room in which the information processing device 10 is located (left outside the medical room). Fig. 1 corresponds to the operating mode, and Figs. 6 and 7 correspond to the non-operating mode.

[0056] The acquisition unit 30 acquires motion information indicating the motion of the operator U1 operating the movable part 53 of the robot 50. The motion information indicates, for example, at least one of the motion of a predetermined part of the body of the operator U1 (e.g., a hand or a foot), a line of sight, a voice, an electroencephalogram, and a position.

[0057] The motion information may be detected by various sensors such as an acceleration sensor, a gyro sensor, a geomagnetic sensor, a gaze sensor, an electroencephalogram sensor, a temperature sensor, and a distance sensor. Alternatively, the motion information may be detected by capturing an image of the operator U1 with a camera and analyzing the resulting video. Alternatively, the motion information may be detected by collecting the speech of the operator U1 and surrounding sounds with a microphone and analyzing the resulting sounds.

[0058] Various sensors, cameras, microphones, and the like for detecting the operation information may be, for example, a detection unit 92 provided in a terminal device 90 owned by the operator U1, or may be provided in a room in which the information processing device 10 is placed and in a radiation controlled area. As an example, FIG. 1 illustrates the operation information being acquired from the detection unit 92 of the terminal device 90.

[0059] The determination unit 32 determines whether or not the operation information acquired by the acquisition unit 30 satisfies a predetermined first condition. For example, the determination unit 32 may determine whether or not a situation has arisen in which the operator U1 and the robot 50 may physically interfere with each other, based on the operation information.

[0060] Specifically, the determination unit 32 may determine whether or not the operation information indicates that the operator U1 has entered a predetermined area as a first condition. In this embodiment, the predetermined area is an area in which the robot 50 operates, which is a radiation controlled area. In other words, the determination unit 32 may determine whether or not the operation information indicates that the operator U1 has entered an area within a predetermined range from the robot 50 (see FIG. 6).

[0061] 7, when the operator U1 has left the medical room, it is considered that the operator U1 does not intend to control the robot 50. Therefore, for example, the determination unit 32 may determine whether or not the operator U1 has entered a situation in which he or she does not wish to control the robot 50, based on the operation information. Specifically, the determination unit 32 may determine whether or not the operation information indicates that the operator U1 has moved away from the robot 50 by a predetermined distance, as a first condition.

[0062] When the determination by the determination unit 32 is affirmative (i.e., when the operation information satisfies the first condition), the control unit 34 controls the robot 50 to switch from the operating mode to the non-operating mode. According to this embodiment, for example, when the operator U1 enters a predetermined area, the mode can be switched to the non-operating mode. Therefore, the robot 50 can be prevented from making an unexpected movement of the operator U1.

[0063] Furthermore, when the operation information does not satisfy the first condition while the robot 50 is in the non-operation mode, the control unit 34 may control the robot 50 to return from the non-operation mode to the operation mode. According to this embodiment, for example, after the operator U1 leaves the radiation controlled area, the robot 50 can be operated again.

[0064] (Control of robot 50) Next, the difference between the control process of the robot 50 in the operating mode and the non-operating mode will be described.

[0065] 1, the acquisition unit 30 acquires operation information for the movable unit 53 of the robot 50, which is input by an operator U1 via a robot operation unit 25A. The operation information may be, for example, input by voice.

[0066] Also, for example, the operation information may be linked to the behavior of the body of the operator U1, such as the hands and feet. In this case, the acquisition unit 30 may utilize the motion information of the operator U1 to generate operation information such that the movable part 53 of the robot 50 operates in conjunction with the motion information. For example, when the motion information indicates a predetermined gesture, the acquisition unit 30 may generate operation information such that the robot 50 performs a predetermined behavior corresponding to the gesture. Also, for example, the acquisition unit 30 may identify the moving direction and moving distance of the operator U1 based on the motion information, and generate operation information such that the robot 50 moves the same distance in the same direction.

[0067] In the operation mode, the control unit 34 controls the robot 50 to perform an operation according to the operation information acquired by the acquisition unit 30. That is, the control unit 34 controls the movable unit 53 of the robot 50 to move according to the operation information.

[0068] On the other hand, in the non-operation mode, the control unit 34 discards the operation information acquired by the acquisition unit 30, thereby preventing the robot 50 from making movements that are unexpected by the operator U1. In this embodiment, as described above, operation information based on the operation information of the operator U1 can be used. Even in the non-operation mode, the operation information may continue to be acquired for the mode switching process (see Figs. 6 and 7), so that the acquisition unit 30 may generate operation information that is not intended by the operator U1 based on the operation information. Therefore, in the non-operation mode, the control unit 34 discards the operation information, thereby preventing the robot 50 from moving.

[0069] (Monitoring Subject U2) Next, the monitoring process for the subject U2 will be described.

[0070] As shown in FIG. 1, the acquisition unit 30 acquires detection information of the subject U2 acquired by the detection unit 52 of the robot 50. The control unit 34 outputs the detection information acquired by the acquisition unit 30 to at least one output device. The output device is, for example, the output unit 94 (display 94A, speaker 94B, etc.) of the terminal device 90. In this case, the control unit 34 may output the detection information after performing a predetermined process. For example, when the detection information is a moving image of the subject U2 captured by the camera 52B, a process of enlarging the face of the subject U2 may be performed.

[0071] 6, the output of the detection information to the terminal device 90 (monitoring process of the subject U2) may be performed even when the robot 50 is in the non-operating mode. For example, even when the operator U1 is in a radiation controlled area, it may be preferable for the robot 50 to continue detecting and outputting the detection information because the robot 50 is closer to the subject U2 and easier to monitor.

[0072] 7, when the operator U1 has left the medical room, the output of the detection information to the terminal device 90 (monitoring process of the subject U2) may not be performed. This is because such a case may occur, for example, when the operator U1 is replaced by another person and the original operator U1 leaves the radiography work.

[0073] Specifically, the determination unit 32 may determine whether or not the motion information of the operator U1 satisfies a predetermined second condition. For example, the determination unit 32 may determine whether or not the motion information indicates that the operator U1 has moved away from the robot 50 by a predetermined distance, as the second condition. The control unit 34 may stop outputting the detection information to the terminal device 90 when the determination of the determination unit 32 is positive (i.e., when the motion information satisfies the second condition).

[0074] (Control of Modality 80) Next, the control process of the modality 80 will be described.

[0075] 1, the acquisition unit 30 receives an operation instruction for the modality 80 input by the operator U1 via the modality operation unit 25B. The control unit 34 controls the modality 80 to perform an operation according to the operation instruction for the modality 80 acquired by the acquisition unit 30. The operation instruction for the modality 80 is, for example, an instruction to change the imaging conditions in radiation imaging, an instruction to start radiation imaging, etc.

[0076] The operation instruction for the modality 80 may be, for example, a voice input. Also, for example, it may be linked to the body behavior of the operator U1, such as the hands and feet. In this case, the acquisition unit 30 may utilize the motion information of the operator U1 and generate an operation instruction such that the modality 80 operates in accordance with the motion information. For example, when the motion information indicates a predetermined gesture, the acquisition unit 30 may generate an operation instruction such that radiation is irradiated by the modality 80.

[0077] On the other hand, in a state in which the operator U1 has entered a radiation controlled area as shown in Fig. 6, the operator U1 does not normally operate the modality 80. However, in the information processing system 100 according to this embodiment, the robot 50 may have the function of the modality operation unit 25B as described above. In this case, the control unit 34 may also transmit to the modality 80 that it is in the non-operation mode, and may stop some or all of the operations of the modality 80. For example, the control unit 34 may permit a change in the display method on the display 24A, but prohibit the start of radiography to which the operator U1 may be exposed.

[0078] 7, even when the operator U1 has left the medical room, it is considered that the operator U1 will not operate the modality 80. It is considered that the operator U1 will not operate the robot 50. However, when the robot 50 and the modality 80 are operated according to the operation information, it is preferable to stop some or all of the operations of the modality 80 in order to prevent the robot 50 and the modality 80 from being operated unintentionally by the operator U1.

[0079] In this way, since the control of the modality 80 and the robot 50 is performed by one information processing device 10, it is also possible to perform cooperative processing between the robot 50 and the modality 80 so that the operator U1 and the subject U2 can smoothly communicate with each other. For example, even if communication is initially performed through the robot 50, it is considered that the subject U2 moves into the bore during radiography by the modality 80 (CT device), and the distance from the robot 50 increases, making communication difficult. In that case, it is also possible to switch to a camera, microphone, and display (not shown in FIG. 2, but may be provided in the imaging unit 82, for example) installed in the bore of the CT device, and communicate continuously before, during, and after radiography.

[0080] Furthermore, when the operator U1 is performing operations such as positioning beside the subject U2, the operator U1 may want to check the console information of the modality 80. In such a case, the output information of the modality 80 may be displayed on the output unit 54 of the robot 50. In addition, the input unit 55 of the robot 50 may be used to input operations to the modality 80. In that case, it is preferable that control is performed so that the start of imaging is not permitted based on the motion information of the operator U1.

[0081] In addition, the information processing device 10 may acquire detection information (e.g., an image of the posture related to positioning) of the subject U2 detected by the detection unit 52 of the robot 50, and record it in association with the radiation image captured by the modality 80 and the shooting conditions, etc.

[0082] In addition, due to the relationship between the position of the movable bed 83 on the modality 80 side and the robot 50, the bed 83 may move and collide with the robot 50. Therefore, the information processing device 10 may determine, based on the position information of the robot 50, whether the robot 50 is staying on the movement path of the bed 83 when it moves, or whether the moving position is within the movement path of the bed 83 of the modality 80 when the operator U1 moves the robot 50 during imaging. If the bed 83 and the robot 50 may come into contact with each other, a warning may be issued or the movement of the robot 50 may be restricted.

[0083] Next, the operation of the information processing device 10 according to the present embodiment will be described with reference to Fig. 8. In the information processing device 10, the CPU 21 executes the information processing program 27, thereby executing the information processing shown in Fig. 8. The information processing is executed, for example, when a user issues an instruction to start execution via the input unit 25.

[0084] In step S10, the acquisition unit 30 acquires motion information indicating the motion of the operator U1 operating the movable part 53 of the robot 50. In step S12, the acquisition unit 30 acquires operation information for the movable part 53 of the robot 50 input by the operator U1. In step S14, the determination unit 32 determines whether the motion information acquired in step S10 satisfies a predetermined first condition.

[0085] If step S14 is negative (i.e., the operation information does not satisfy the first condition), the process proceeds to step S16. In step S16, the control unit 34 sets the robot 50 to an operation mode in which the movable unit 53 of the robot 50 is in an operable state. In step S18, the control unit 34 controls the robot 50 to perform an operation according to the operation information acquired in step S12, and ends this information processing.

[0086] On the other hand, if step S14 is positive (i.e., if the operation information satisfies the first condition), the process proceeds to step S20. In step S20, the control unit 34 sets the movable unit 53 of the robot 50 to a non-operation mode in which the movable unit 53 is inoperable. In step S22, the control unit 34 discards the operation information acquired in step S12 (i.e., does not control the robot 50), and ends this information processing.

[0087] As described above, the information processing system 100 according to one embodiment of the present disclosure includes at least one processor, and the processor acquires operation information indicating the operation of the operator U1 operating the movable part 53 of the robot 50, and when the operation information satisfies a predetermined first condition, controls the robot 50 to switch from an operation mode in which the movable part 53 is in an operable state to a non-operation mode in which the movable part 53 is in an inoperable state.

[0088] That is, according to the information processing system 100 according to the present embodiment, it is possible to prevent the robot 50 from making movements that are unexpected for the operator U1 based on the actions of the operator U1. Therefore, the robot 50 can be appropriately operated.

[0089] [Second embodiment] In radiography, for example, an assistant U3 who assists with tasks such as positioning, or who assists or accompanies the subject U2 may also enter and leave the radiation controlled area. In this case, like the operator U1, it is desirable for the assistant U3 to prevent the robot 50 from making unexpected movements.

[0090] Therefore, the information processing device 10 according to this embodiment automatically switches the robot 50 between an operating mode and a non-operating mode in response to the movements of the assistant U3 as well as the movements of the operator U1, thereby controlling the robot 50 to prevent it from making movements that are unexpected by the assistant U3.

[0091] Hereinafter, the functions of the information processing device 10 according to this embodiment will be described with reference to Figs. 9 to 11, but some of the descriptions overlapping with those of the first embodiment will be omitted. Fig. 9 is a schematic diagram showing an example of the overall configuration of the information processing system 100 according to this embodiment, in which an assistant U3 is added to Fig. 1. Fig. 10 is a schematic diagram showing an example of the overall configuration of the information processing system 100 in a state in which the assistant U3 has entered a radiation controlled area. Fig. 11 is a schematic diagram showing an example of the overall configuration of the information processing system 100 in a state in which the assistant U3 has left both the radiation controlled area and the medical room in which the information processing device 10 is located (has left the medical room). Figs. 9 and 11 correspond to the operation mode, and Fig. 10 corresponds to the non-operation mode.

[0092] (Robot 50 mode switching) First, a process of switching the robot 50 between the active mode and the inactive mode in response to the action of the assistant U3 will be described.

[0093] The acquisition unit 30 acquires assistant motion information indicating the motion of assistant U3. Assistant U3 is a person other than the operator U1 who is able to enter and leave the radiation controlled area. Assistant motion information indicates, for example, at least one of the motions of a predetermined part of the assistant U3's body (e.g., hands and feet), line of sight, voice, brain waves, and position. Like the motion information of the operator U1, assistant motion information can be detected, for example, by various sensors, cameras, microphones, and the like provided in assistant terminal device 90A owned by assistant U3.

[0094] The determination unit 32 determines whether or not the assistant operation information acquired by the acquisition unit 30 satisfies a predetermined third condition. For example, the determination unit 32 may determine whether or not a situation has arisen in which the assistant U3 and the robot 50 may physically interfere with each other, based on the assistant operation information. Specifically, the determination unit 32 may determine whether or not the assistant operation information indicates that the assistant U3 has entered a predetermined area (radiation controlled area), as the third condition (see FIG. 10).

[0095] When the determination by the determination unit 32 is affirmative (i.e., when the assistant action information satisfies the third condition), the control unit 34 controls the robot 50 to switch from the operating mode to the non-operating mode. According to this embodiment, for example, when the assistant U3 enters a predetermined area, the mode can be switched to the non-operating mode. Therefore, the robot 50 can be prevented from making an unexpected movement of the assistant U3.

[0096] Furthermore, when the assistant operation information does not satisfy the third condition while the robot 50 is in the non-operation mode, the control unit 34 may control the robot 50 to return from the non-operation mode to the operation mode. According to such a configuration, for example, after the assistant U3 leaves the radiation controlled area, the robot 50 can be operated again.

[0097] (Control of robot 50) The robot 50 is controlled by the operator U1 inputting operation information, as in the first embodiment. That is, the assistant U3 does not usually operate the robot 50.

[0098] 10, when the assistant U3 enters a predetermined area (radiation controlled area), it is preferable that the control unit 34 discards operation information input to the robot 50 by the operator U1. This is because the robot 50 may perform a movement that is unintended by the assistant U3, even if the operation is intended by the operator U1.

[0099] 11, when the assistant U3 is a predetermined distance away from the robot 50, the control unit 34 may control the robot 50 to move the movable part 53 in accordance with operation information for the robot 50 input by the operator U1. This is because it is considered that the assistant U3 and the robot 50 will not interfere with each other in this case.

[0100] (Monitoring Subject U2) The subject U2 may be monitored by an assistant U3 in addition to or instead of the operator U1. Specifically, the control unit 34 may output the detection information acquired by the acquisition unit 30 to at least one of a first output device (terminal device 90) previously associated with the operator U1 and a second output device (assistant terminal device 90A) previously associated with the assistant U3. The assistant terminal device 90A is, for example, a smartphone, a wearable terminal, a tablet terminal, a personal computer, etc. Examples of the wearable terminal include a head mounted display, smart glasses, and a smart watch. The terminal device 90 and the assistant terminal device 90A may be different types of devices. The hardware configuration of the assistant terminal device 90A is the same as that of the terminal device 90 (see FIG. 2), and therefore a description thereof will be omitted.

[0101] 10, the output of the detection information to the assistant terminal device 90A (monitoring process of the subject U2) may be performed even when the robot 50 is in the non-operating mode. For example, even when the assistant U3 is in a radiation controlled area, it may be preferable for the robot 50 to continue detecting and outputting the detection information because the robot 50 is closer to the subject U2 and easier to monitor.

[0102] On the other hand, as shown in Fig. 11, when the assistant U3 has left the medical room, the output of the detection information to the assistant terminal device 90A (monitoring process of the subject U2) may not be performed. This is because such a case may occur, for example, when the assistant U3 is replaced by another person and the original assistant U3 leaves the radiography work.

[0103] The determination unit 32 may determine whether or not the assistant operation information satisfies a predetermined fourth condition. For example, the determination unit 32 may determine whether or not the assistant operation information indicates that the assistant U3 has moved away from the robot 50 by a predetermined distance, as the fourth condition. When the determination of the determination unit 32 is positive (i.e., when the assistant operation information satisfies the fourth condition), the control unit 34 may stop outputting the detection information to the assistant terminal device 90A.

[0104] (Control of Modality 80) As described above, it is preferable that the robot 50 is switched between the operating mode and the non-operating mode in consideration of the actions of both the operator U1 and the assistant U3. On the other hand, the modality 80 may be controlled according to the operation instruction of the operator U1 regardless of the action of the assistant U3. For example, when the subject U2 is a child, radiography may be performed while the assistant U3 (e.g., the child's parent) is staying together with the subject U2 in a radiation controlled area.

[0105] According to the information processing system 100 of this embodiment, it is possible to prevent the robot 50 from making unexpected movements for both the operator U1 and the assistant U3 based on the actions of both the operator U1 and the assistant U3. Therefore, the robot 50 can be appropriately operated.

[0106] In the above-described embodiments, the CT device is used as an example of the modality 80, but the present invention is not limited to this. As the modality 80, for example, a plain X-ray device, an X-ray fluoroscopy device, a mammography device, a bone mineral quantification device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, a radiation therapy device, and the like can be appropriately used.

[0107] In the above-mentioned embodiments, the determination unit 32 switches to the non-operating mode by determining whether or not at least one of the operator U1 and the assistant U3 has entered a radiation controlled area, but the present invention is not limited to this. For example, instead of the radiation controlled area, at least one of a high magnetic field controlled area (e.g., an MRI room), a biohazard controlled area, a sterile room, an intensive care room, and a clean room may be applied.

[0108] In addition, in each of the above embodiments, the operator U1 and assistant U3 monitor the state (detection information) of the subject U2, but the state of the operator U1 and assistant U3 may be confirmed by the subject U2. That is, each side may be able to confirm the state of the other in both directions. For example, the control unit 34 may control the output unit 54 (display 54A and speaker 54B) of the robot 50 to output the motion information of the operator U1 and the assistant motion information of the assistant U3 acquired by the acquisition unit 30. According to such a form, the subject U2 who is left in the radiation controlled area at the time of radiation irradiation can check the state of the operator U1 and assistant U3, thereby reducing anxiety.

[0109] In the above-described embodiments, the detection information of the subject U2 detected by the robot 50 is output to the terminal device 90 of the operator U1 and the assistant terminal device 90A of the assistant U3, but the present invention is not limited to this. For example, in addition to or instead of output to the terminal device 90 and the assistant terminal device 90A, the detection information may be controlled to be displayed on the display 24A of the information processing device 10, and the operator U1 and the assistant U3 may monitor the state of the subject U2 by checking the display 24A.

[0110] In the above embodiments, the CPU 21 of the information processing device 10 functions as the acquisition unit 30, the determination unit 32, and the control unit 34, but the present invention is not limited to this. For example, the robot 50, the modality 80, the terminal device 90, and the assistant terminal device 90A may function as at least a part of the acquisition unit 30, the determination unit 32, and the control unit 34, respectively.

[0111] In each of the above embodiments, the following various processors can be used as the hardware structure of the processing unit that executes various processes, such as the acquisition unit 30, the determination unit 32, and the control unit 34. As described above, the above various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing specific processes, such as an ASIC (Application Specific Integrated Circuit), etc.

[0112] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0113] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as typified by System on Chip (SoC), etc. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0114] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0115] In the above embodiment, the various programs are stored in the storage unit in advance, but the present invention is not limited to this. The various programs may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The various programs may be downloaded from an external device via a network. Furthermore, the technology of the present disclosure extends to a storage medium that non-temporarily stores a program in addition to the program.

[0116] The technology of the present disclosure can also be appropriately combined with the above-mentioned embodiment examples and examples. The above-mentioned description and illustrated contents are detailed descriptions of the parts related to the technology of the present disclosure, and are merely one example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the description and illustrated contents shown above, within the scope of the gist of the technology of the present disclosure.

[0117] The following supplementary notes are further disclosed regarding the above embodiment. [Appendix 1] at least one processor; The processor, acquiring motion information indicating a motion of an operator who operates a movable part of the robot; When the operation information satisfies a predetermined first condition, the robot is controlled so as to switch from an operation mode in which the movable part is in an operable state to a non-operation mode in which the movable part is in an inoperable state. Information processing system. [Appendix 2] The processor, acquiring operation information for the movable part input by the operator; In the operation mode, the robot is controlled to perform an operation according to the operation information; Discarding the operation information in the non-operational mode. 2. An information processing system according to claim 1. [Appendix 3] The processor, In the case of the operation mode, the operation information is generated so that the movable part is operated in conjunction with the operation information. 3. An information processing system according to claim 2. [Appendix 4] The processor, When the operation information indicates, as the first condition, that the operator has entered a predetermined area, the robot is controlled to switch from the operating mode to the non-operating mode. 4. An information processing system according to any one of claims 1 to 3. [Appendix 5] The predetermined area is an area within a predetermined range from the robot. 5. An information processing system according to claim 4. [Appendix 6] The predetermined area is at least one of a radiation controlled area, a high magnetic field controlled area, a biohazard controlled area, a sterile room, an intensive care room, and a clean room; The robot operates within the predetermined area. 6. The information processing system according to claim 4 or 5. [Appendix 7] The robot includes at least one detection unit selected from a sensor, a camera, and a microphone; The processor, acquiring detection information acquired by the detection unit of the robot; outputting the detected information to at least one output device; 7. An information processing system according to any one of claims 1 to 6. [Appendix 8] The processor, When the operation information satisfies a second predetermined condition, output of the detection information to the output device is stopped. 8. An information processing system according to claim 7. [Appendix 9] The processor, Acquire assistant operation information indicating an operation of an assistant other than the operator, When the assistant operation information satisfies a third predetermined condition, the robot is controlled to switch from the operating mode to the non-operating mode. 9. An information processing system according to any one of claims 1 to 8. [Appendix 10] The processor, outputting the detection information to a first output device that is pre-associated with the operator and a second output device that is pre-associated with an assistant other than the operator; Acquire assistant motion information indicating a motion of the assistant, When the assistant operation information satisfies a predetermined fourth condition, output of the detection information to the second output device is stopped. 8. An information processing system according to claim 7. [Appendix 11] In the non-operational mode, functions of the robot other than the movable parts are not stopped. 11. An information processing system according to any one of claims 1 to 10. [Appendix 12] The robot includes at least one of a moving mechanism and an operating mechanism as the movable part. 12. An information processing system according to any one of claims 1 to 11. [Appendix 13] The motion information indicates at least one of a movement of a predetermined part of the operator's body, a line of sight, a voice, an electroencephalogram, and a position. 13. An information processing system according to any one of claims 1 to 12. [Appendix 14] acquiring motion information indicating a motion of an operator who operates a movable part of the robot; When the operation information satisfies a predetermined first condition, the robot is controlled so as to switch from an operation mode in which the movable part is in an operable state to a non-operation mode in which the movable part is in an inoperable state. An information processing method in which processing is performed by a computer. [Appendix 15] acquiring motion information indicating a motion of an operator who operates a movable part of the robot; When the operation information satisfies a predetermined first condition, the robot is controlled so as to switch from an operation mode in which the movable part is in an operable state to a non-operation mode in which the movable part is in an inoperable state. An information processing program that enables a computer to execute processing. [Explanation of symbols]

[0118] 10. Information processing device 21 CPU 22 Memory section 23 Memory 24, 54 Output section 24A, 54A Display 24B, 54B speakers 25, 55 Input section 25A Robot operation unit 25B Modality Operation Unit 26, 56, 86, 96 I / F section 27 Information Processing Programs 28, 58, 88, 98 Bus 30 Acquisition Department 32 Judgment section 34 Control section 50 Robot 51, 81, 91 Main unit control section 52, 92 Detector 52A, 92A Sensor 52B, 92B Camera 52C, 92C Microphone 53 Moving parts 53A Moving mechanism 53A1 wheels 53B Operating mechanism 80 Modalities 82 Photography Department 83 Sleeping area 90 Terminal Equipment 90A Assistant terminal device 100 Information Processing Systems U1 Operator U2 Subject U3 assistant

Claims

1. At least one processor; The processor, acquiring motion information indicating a motion of an operator who operates a movable part of the robot; When the operation information satisfies a predetermined first condition, the robot is controlled so as to switch from an operation mode in which the movable part is in an operable state to a non-operation mode in which the movable part is in an inoperable state. Information processing system.

2. The processor, acquiring operation information for the movable part input by the operator; In the operation mode, the robot is controlled to perform an operation according to the operation information; Discarding the operation information in the non-operational mode. The information processing system according to claim 1 .

3. The processor, In the case of the operation mode, the operation information is generated so that the movable part is operated in conjunction with the operation information. The information processing system according to claim 2 .

4. The processor, When the operation information indicates, as the first condition, that the operator has entered a predetermined area, the robot is controlled to switch from the operating mode to the non-operating mode. The information processing system according to claim 1 .

5. The predetermined area is an area within a predetermined range from the robot.

5. The information processing system according to claim 4.

6. the predetermined area is at least one of a radiation controlled area, a high magnetic field controlled area, a biohazard controlled area, a sterile room, an intensive care room, and a clean room; The robot operates within the predetermined area.

5. The information processing system according to claim 4.

7. The robot includes at least one detection unit selected from a sensor, a camera, and a microphone; The processor, acquiring detection information acquired by the detection unit of the robot; Outputting the detected information to at least one output device. The information processing system according to claim 1 .

8. The processor, When the operation information satisfies a second predetermined condition, output of the detection information to the output device is stopped. The information processing system according to claim 7.

9. The processor, Acquire assistant operation information indicating an operation of an assistant other than the operator, When the assistant operation information satisfies a third predetermined condition, the robot is controlled to switch from the operating mode to the non-operating mode. The information processing system according to claim 1 .

10. The processor, outputting the detection information to a first output device associated in advance with the operator and a second output device associated in advance with an assistant other than the operator; Acquire assistant motion information indicating a motion of the assistant, When the assistant operation information satisfies a fourth predetermined condition, output of the detection information to the second output device is stopped. The information processing system according to claim 7.

11. In the non-operational mode, functions of the robot other than the movable parts are not stopped. The information processing system according to claim 1 .

12. The robot includes at least one of a moving mechanism and an operating mechanism as the movable part. The information processing system according to claim 1 .

13. The motion information indicates at least one of a movement of a predetermined body part of the operator, a line of sight, a voice, an electroencephalogram, and a position. The information processing system according to claim 1 .

14. acquiring motion information indicating a motion of an operator who operates a movable part of the robot; When the operation information satisfies a predetermined first condition, the robot is controlled so as to switch from an operation mode in which the movable part is in an operable state to a non-operation mode in which the movable part is in an inoperable state. An information processing method in which processing is performed by a computer.

15. acquiring motion information indicating a motion of an operator who operates a movable part of the robot; When the operation information satisfies a predetermined first condition, the robot is controlled so as to switch from an operation mode in which the movable part is in an operable state to a non-operation mode in which the movable part is in an inoperable state. An information processing program that enables a computer to execute processing.

Images