Information processing device, control method, and control program

Abstract

An information processing device (100) includes: an acquisition unit (120) that acquires biological information of an operator, robot information including a sound signal indicating sound around a robot, information indicating a sound space region, a work determination learning completion model, and a parameter determination learning completion model; the sound space area is a sound space in which an operator hears sound via the output device; a determination unit (130) that determines whether or not the operator is performing work via the robot, using the robot information and the work determination learning completion model; a specifying unit (140) that, when the operator is performing work via the robot, specifies the degree of concentration of the operator using at least one of the biological information and the robot information; a determination unit (150) that uses the parameter determination learning completion model and determines parameters for providing an operator with a sound space corresponding to the sound space region and the concentration degree; and a control unit (160) that performs signal processing on the sound signal on the basis of the parameter, and transmits the sound signal obtained by the signal processing to an output device.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a control method, and a control program. Background Art

[0002] There is known a technique for remotely operating a robot. A dummy head is provided in the robot. Sounds around the robot are input to the dummy head. An operator can hear the sounds around the robot via the dummy head. Here, a technique related to the dummy head is proposed (see Patent Document 1).

[0003] Prior Art Documents

[0004] Patent Documents

[0005] Patent Document 1: Japanese Patent Laid-Open No. 62-044384 Summary of the Invention

[0006] Problems to be Solved by the Invention

[0007] As described above, the sounds around the robot are provided to the operator. The sounds around the robot include noise. For example, when the robot is installed in a factory, the sounds around the robot include the noise in the factory. The noise is an unnecessary sound for the operator. Providing the unnecessary sound to the operator reduces the work efficiency of the operator.

[0008] An object of the present disclosure is to improve the work efficiency of the operator.

[0009] Means for Solving the Problems

[0010] An information processing apparatus providing a solution of the present disclosure. The information processing apparatus communicates with an output device that provides sound to an operator who can remotely operate a robot. The information processing apparatus includes: an acquisition unit that acquires the biological information of the operator, robot information, information indicating a sound space region, a learned model for operation determination, and a learned model for parameter determination, where the robot information includes a sound signal indicating the sound around the robot, and the sound space region is the sound space where the operator hears the sound via the output device; a determination unit that uses the robot information and the learned model for operation determination to determine whether the operator is performing an operation via the robot; a determination unit that, when the operator is performing an operation via the robot, uses at least one of the biological information and the robot information to determine the concentration of the operator; a determination unit that uses the learned model for parameter determination to determine parameters for providing a sound space corresponding to the sound space region and the concentration to the operator; and a control unit that performs signal processing on the sound signal based on the parameters and transmits the sound signal obtained by the signal processing to the output device.

[0011] Effects of the Invention

[0012] According to the present disclosure, the work efficiency of the operator can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a diagram showing a control system.

[0014] Figure 2 It is a diagram showing the hardware included in the information processing apparatus.

[0015] Figure 3 It is a block diagram showing the functions of the information processing apparatus.

[0016] Figure 4 It is a graph showing an example of the correspondence between concentration and operation time.

[0017] Figure 5 It is a diagram showing an example of the direction in which sound can be heard.

[0018] Figure 6 It is a diagram showing an example in the case of two-dimensionally representing a sound space region.

[0019] Figure 7 It is a diagram for explaining reinforcement learning.

[0020] Figure 8 It is a diagram for explaining supervised learning.

[0021] Figure 9A diagram showing an example of a rule base.

[0022] Figure 10 A flowchart showing an example of the processing executed by the information processing device. Detailed implementation mode

[0023] Hereinafter, the implementation mode will be described with reference to the drawings. The following implementation modes are merely examples, and various changes can be made within the scope of the present disclosure.

[0024] Implementation mode 1.

[0025] Figure 1 A diagram showing a control system. The control system includes an information processing device 100, a biological sensing device 200, a robot sensing device 300, and an output device 400.

[0026] The information processing device 100, the biological sensing device 200, the robot sensing device 300, and the output device 400 communicate via a network. The network is a wired network or a wireless network.

[0027] In the control system, the operator can perform remote operation of the robot.

[0028] The information processing device 100 is a device that executes a control method.

[0029] The biological sensing device 200 measures the biological information of the operator. For example, the biological information is information related to the sensory organs and motor organs. For example, the information related to the sensory organs is information such as the eyes, expression, heart rate, and brain waves. Specifically, the information of the eyes is the direction of the line of sight, the degree of eye opening, the shape of the pupil, the number of blinks per unit time, etc. The information of the eyes and the expression can be obtained from a camera. The heart rate can be obtained from a wristband meter. The brain waves can be obtained from a brain wave sensor attached to the user's head. In addition, for example, the information related to the motor organs is information such as the movement of the operator's bones and the movement of the head. The movement of the bones can be obtained from a camera. The movement of the head can be obtained from a sensor attached to the user's head.

[0030] The robot sensing device 300 obtains robot information including environmental information. The environmental information is information related to the environment around the robot. For example, the environmental information is an image or video representing the environment around the robot. In addition, for example, the environmental information is a sound signal representing the sound around the robot. An image or video can be obtained from a camera mounted on the robot. A sound signal can be obtained from a multi-channel microphone mounted on the robot. The robot information may also include robot position information and robot motion information. The robot position information is information indicating the position of the robot. For example, the robot position information can be obtained from a GPS (Global Positioning System) mounted on the robot. The robot motion information is information related to the motion of the robot. The robot motion information can also be obtained from the content input by the operator to the controller.

[0031] For example, the output device 400 is a speaker, a headset, etc. The output device 400 provides sound to the operator.

[0032] Next, the hardware of the information processing device 100 will be described.

[0033] Figure 2 It is a diagram showing the hardware of the information processing device. The information processing device 100 includes a processor 101, a volatile storage device 102, a non-volatile storage device 103, and an interface 104.

[0034] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), etc. The processor 101 can also be a multi-processor. In addition, the information processing device 100 may also have a processing circuit.

[0035] The volatile storage device 102 is the main storage device of the information processing device 100. For example, the volatile storage device 102 is a RAM (Random Access Memory). The non-volatile storage device 103 is the auxiliary storage device of the information processing device 100. For example, the non-volatile storage device 103 is an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0036] The interface 104 communicates with the biological sensing device 200, the robot sensing device 300, and the output device 400.

[0037] Next, the functions of the information processing apparatus 100 will be described.

[0038] Figure 3 FIG. is a block diagram showing the functions of the information processing apparatus. The information processing apparatus 100 includes a storage unit 110, an acquisition unit 120, a determination unit 130, a determination unit 140, a decision unit 150, and a control unit 160.

[0039] The storage unit 110 may be implemented as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103.

[0040] Part or all of the acquisition unit 120, the determination unit 130, the determination unit 140, the decision unit 150, and the control unit 160 may be implemented by a processing circuit. In addition, part or all of the acquisition unit 120, the determination unit 130, the determination unit 140, the decision unit 150, and the control unit 160 may be implemented as modules of a program executed by the processor 101. For example, the program executed by the processor 101 is also referred to as a control program. For example, the control program is recorded on a recording medium.

[0041] The storage unit 110 stores various types of information.

[0042] The acquisition unit 120 acquires the biological information of the operator from the biological sensing device 200. The acquisition unit 120 acquires the robot information from the robot sensing device 300. The acquisition unit 120 may also acquire the biological information and the robot information via other devices.

[0043] The acquisition unit 120 may store the biological information and the robot information in the storage unit 110 each time the biological information and the robot information are acquired. As a result, the biological information and the robot information are accumulated in the storage unit 110.

[0044] The acquisition unit 120 acquires information indicating the sound space area. The sound space area is the sound space in which the operator hears sounds via the output device 400. Briefly speaking, the sound space area is the sound space in which the operator currently hears sounds. The details of the sound space area will be described later. In addition, the acquisition unit 120 acquires the information indicating the sound space area from the storage unit 110 or an external device. The external device is a device that can be connected to the information processing apparatus 100. For example, the external device is a cloud server. In addition, the figure of the external device is omitted.

[0045] In addition, the acquisition unit 120 acquires the learned model for operation determination. For example, the acquisition unit 120 acquires the learned model for operation determination from the storage unit 110. In addition, for example, the acquisition unit 120 acquires the learned model for operation determination from an external device.

[0046] The determination unit 130 uses the robot information and the learned job determination model to determine whether the operator is performing a job via the robot. In other words, the determination unit 130 uses the robot information and the learned job determination model to determine whether the operator is remotely operating the robot to perform an action. In addition, the determination unit 130 may also be configured to use the robot information and the learned job determination model to determine whether the operator is performing a specific job via the robot. For example, the determination unit 130 inputs the robot information to the learned job determination model, and the learned job determination model outputs information indicating whether the operator is performing a job via the robot. The determination unit 130 determines whether the operator is performing a job via the robot based on this information. Additionally, for example, the learned job determination model estimates whether the operator is performing a job via the robot based on the sound signal included in the robot information. Furthermore, for example, the learned job determination model estimates whether the operator is performing a job via the robot based on the robot motion information included in the robot information.

[0047] When the operator is performing a job via the robot, the determination unit 140 uses at least one of the biological information and the robot information to determine the operator's concentration.

[0048] The method for determining the concentration will be described. The determination unit 140 uses the acquired biological information to determine the concentration. For example, the determination unit 140 uses a table showing the correspondence between the biological information and the concentration to determine the concentration corresponding to the acquired biological information. Additionally, for example, the determination unit 140 may input the acquired biological information to the learned model, and the learned model outputs the concentration.

[0049] The determination unit 140 may also use the acquired biological information (i.e., the current biological information) and the previously acquired biological information to determine the operator's concentration.

[0050] In addition, the determination unit 140 may also use the acquired biological information, the acquired robot information (i.e., the current robot information), the previously acquired robot information, and the learned model to determine the operator's concentration. Here, the reason for using the robot information will be explained. The motion of the robot indicated by the robot motion information included in the robot information is related to the concentration. When the robot performs a motion without redundant actions, it can be said that the operator's concentration is high. On the other hand, when the robot performs a non-smooth motion (i.e., redundant actions), it can be said that the operator's concentration is low. Thus, the motion of the robot is related to the concentration. Therefore, using the robot information as one factor when determining the concentration can obtain a high-precision concentration.

[0051] In addition, the determination unit 140 can also determine the operator's concentration based on the operation time of the operator obtained from the acquired robot information and the robot information acquired in the past. A learned model can also be used in the determination of this concentration. For example, the determination unit 140 inputs the operation time into the learned model, and thus, this learned model outputs the concentration. For example, this learned model is obtained by learning information indicating the correspondence relationship between the concentration and the operation time. Here, an example of the correspondence relationship between the concentration and the operation time is shown.

[0052] Figure 4 It is a curve graph showing an example of the correspondence relationship between the concentration and the operation time. Figure 4 The vertical axis of represents the concentration. Figure 4 The horizontal axis of represents the operation time. This learned model is obtained by learning the learning data shown in the curve graph.

[0053] In addition, in the determination of this concentration, a table capable of determining the concentration can also be used.

[0054] In addition, the determination unit 140 can also determine the operator's concentration based on the operation time and operation content of the operator obtained from the acquired robot information and the robot information acquired in the past. In the determination of this concentration, a table or a learned model capable of determining the concentration can also be used.

[0055] In addition, when the determination unit 140 uses the learned model, this learned model is acquired by the acquisition unit 120. For example, the acquisition unit 120 acquires this learned model from the storage unit 110 or an external device. In addition, this learned model is also referred to as a learned model for concentration determination.

[0056] The decision unit 150 uses a learned model for parameter decision to decide the parameters for providing a sound space corresponding to the sound space region and the concentration to the operator. In addition, this statement can also be expressed as follows. The decision unit 150 uses a learned model for sound space region, concentration, and parameter decision to decide the parameters for providing a sound space for improving the operation efficiency of the operator.

[0057] Specifically, the decision unit 150 inputs the sound space region and the concentration into the learned model for parameter decision, and the learned model for parameter decision outputs the parameters. In addition, the learned model for parameter decision and the sound space region will be described later.

[0058] Here, the direction in which the sound can be heard can be represented as follows.

[0059] Figure 5 It is a diagram showing an example of the direction in which the sound can be heard.Figure 5 This shows a situation where the direction from which a sound can be heard is represented on a spherical surface. Also, in Figure 5 , the direction from which a sound can be heard is indicated by arrow 10. Additionally, Figure 5 it is represented using the sound space region 11.

[0060] Furthermore, a situation where the sound space region is represented two-dimensionally is shown.

[0061] Figure 6 FIG. is an example showing the case where the sound space region is represented two-dimensionally. In Figure 6 , the sound space region is represented by an angle. For example, in Figure 6 , the sound space region is represented by 90 degrees or 270 degrees. Also, the reference for the angle can be the front direction of the operator. To simplify the explanation, hereinafter, the sound space region is represented two-dimensionally.

[0062] Next, an explanation will be given of the parameter determination learned model. The parameter determination learned model is obtained by the acquisition unit 120. For example, the acquisition unit 120 obtains the parameter determination learned model from the storage unit 110 or an external device. The parameter determination learned model can be obtained through machine learning. For example, the parameter determination learned model can be obtained through reinforcement learning. An explanation of reinforcement learning will be given using a figure.

[0063] Figure 7 FIG. is a figure for explaining reinforcement learning. The environment part in reinforcement learning corresponds to the operator's concentration. The agent in reinforcement learning corresponds to the sound space region controller.

[0064] For example, in the case of maintaining a high concentration state, the reward function is designed such that a higher reward is obtained for an action with a high concentration. Then, the optimal policy is learned.

[0065] Furthermore, the parameter determination learned model can also be obtained by a learning method other than reinforcement learning. For example, the parameter determination learned model can be obtained through supervised learning. An explanation of supervised learning will be given using a figure.

[0066] Figure 8 FIG. is a figure for explaining supervised learning. Twelve states representing the relationship between the operator's concentration and trend over a certain period of time are created. The twelve states can also be represented as the operator's state. In machine learning, the operator's concentration over a certain period of time is used as input data. Additionally, in machine learning, the operator's state is used as the correct answer label. The parameters are determined by a rule base based on the operator's state. An example of the rule base is shown.

[0067] Figure 9It is a diagram showing an example of a rule base. For example, when the operator's state is "S2" and the sound space region is less than 90 degrees, parameters for widening the sound space region are output.

[0068] In addition, the following supervised learning can also be performed. Time series data of parameters is used as input data. Moreover, in machine learning, learning is performed in such a way that the output is the parameter expected to have the highest concentration improvement.

[0069] In this way, the parameter obtained through learning determines the learned model.

[0070] Here, a specific example is used to illustrate the method of determining parameters.

[0071] For example, when the current sound space region is wide (e.g., 270 degrees) and the concentration is low, the determination unit 150 determines parameters for narrowing the sound space region to improve the operator's concentration. For example, the determination unit 150 determines parameters for changing the sound space region from 270 degrees to 90 degrees.

[0072] In addition, for example, when the current sound space region is narrow (e.g., 90 degrees) and the concentration is low, it can be said that the operator is overly concentrated and feels fatigued. Then, the determination unit 150 determines parameters for widening the sound space region to relax the operator. For example, the determination unit 150 determines parameters for changing the sound space region from 90 degrees to 270 degrees.

[0073] The control unit 160 performs signal processing on the sound signal included in the robot information based on the determined parameters. For example, the signal processing is beamforming processing, sound masking processing, etc. Thus, the sound signal is converted into a sound signal corresponding to the parameter.

[0074] The control unit 160 sends the sound signal obtained through signal processing to the output device 400. Thus, for example, an operator who is provided with a wide sound space region and has low concentration can hear the sound with a narrowed sound space region. As a result, the operator's concentration increases, and thus the operator's work efficiency improves. In addition, for example, an operator who is provided with a narrow sound space region and has low concentration can hear the sound with a widened sound space region. As a result, the operator feels relaxed, and thus the operator's work efficiency improves.

[0075] Next, the processing performed by the information processing device 100 is described using a flowchart.

[0076] Figure 10 It is a flowchart showing an example of the processing performed by the information processing device.

[0077] (Step S11) Acquisition unit 120 acquires biological information, robot information, and information indicating the current sound space area.

[0078] (Step S12) Determination unit 130 uses the robot information and the operation determination learned model to determine whether the operator is performing an operation via the robot. If an operation is being performed, the process proceeds to Step S13. If no operation is being performed, the process ends.

[0079] (Step S13) Determination unit 140 uses the biological information and the robot information to determine the operator's concentration.

[0080] (Step S14) Decision unit 150 uses the parameter decision learned model to decide on the parameters for providing a sound space corresponding to the sound space area and the concentration to the operator.

[0081] (Step S15) Control unit 160 performs signal processing on the sound signal included in the robot information based on the decided parameters.

[0082] (Step S16) Control unit 160 sends the sound signal obtained through the signal processing to the output device 400.

[0083] According to the embodiment, the information processing device 100 decides on the parameters for providing a sound space that is used to improve the operation efficiency of the operator. The information processing device 100 performs signal processing on the sound signal based on the decided parameters. The information processing device 100 provides the sound signal obtained through the signal processing to the operator via the output device 400. Therefore, the information processing device 100 can improve the operation efficiency of the operator.

[0084] Description of Reference Numerals

[0085] 10 Arrow, 11 Sound space area, 100 Information processing device, 101 Processor, 102 Volatile storage device, 103 Non-volatile storage device, 104 Interface, 110 Storage unit, 120 Acquisition unit, 130 Determination unit, 140 Determination unit, 150 Decision unit, 160 Control unit, 200 Biological sensing device, 300 Robot sensing device, 400 Output device.

Claims

1. An information processing apparatus that communicates with an output device that provides sound to an operator who can remotely operate a robot, wherein, the information processing apparatus has: an acquisition unit that acquires the biological information of the operator, robot information, information indicating a sound space area, a learned model for job determination, and a learned model for parameter determination, where the robot information includes a sound signal indicating the sound around the robot, and the sound space area is the sound space where the operator hears the sound via the output device; a determination unit that uses the robot information and the learned model for job determination to determine whether the operator is performing a job via the robot; a determination unit that, when the operator is performing a job via the robot, uses at least one of the biological information and the robot information to determine the concentration of the operator; a decision unit that uses the learned model for parameter determination to determine parameters for providing a sound space corresponding to the sound space area and the concentration to the operator; and a control unit that performs signal processing on the sound signal based on the parameters and sends the sound signal obtained through the signal processing to the output device.

2. The information processing apparatus according to claim 1, wherein, when the operator is performing a job via the robot, the determination unit uses the acquired biological information and the previously acquired biological information to determine the concentration of the operator.

3. The information processing apparatus according to claim 1, wherein, the robot information includes information related to the movement of the robot, i.e., robot movement information, the acquisition unit acquires a learned model for concentration determination, when the operator is performing a job via the robot, the determination unit uses the biological information, the acquired robot information, the previously acquired robot information, and the learned model for concentration determination to determine the concentration of the operator.

4. The information processing apparatus according to claim 1, wherein, when the operator is performing a job via the robot, the determination unit determines the concentration of the operator based on the operation time of the operator obtained from the acquired robot information and the previously acquired robot information.

5. The information processing apparatus according to claim 4, wherein, when the operator is performing a job via the robot, the determination unit determines the concentration of the operator based on the operation time and the operation content of the operator obtained from the acquired robot information and the previously acquired robot information.

6. A control method, wherein, an information processing apparatus that communicates with an output device that provides sound to an operator who can remotely operate a robot performs the following processing: Obtain the biological information of the operator, the robot information, the information indicating the sound space area, the learned model for job determination, and the learned model for parameter determination, where the robot information includes a sound signal representing the sound around the robot, and the sound space area is the sound space where the operator hears the sound via the output device. Use the robot information and the learned model for job determination to determine whether the operator is performing an operation via the robot. In the case where the operator is performing an operation via the robot, use at least one of the biological information and the robot information to determine the concentration of the operator. Use the learned model for parameter determination to determine the parameters for providing a sound space corresponding to the sound space area and the concentration to the operator; Perform signal processing on the sound signal based on the parameters; Send the sound signal obtained through the signal processing to the output device.

7. A control program, wherein, The control program causes an information processing device that communicates with an output device that provides sound to an operator who can remotely operate a robot to perform the following processing: Obtain the biological information of the operator, the robot information, the information indicating the sound space area, the learned model for job determination, and the learned model for parameter determination, where the robot information includes a sound signal representing the sound around the robot, and the sound space area is the sound space where the operator hears the sound via the output device. Use the robot information and the learned model for job determination to determine whether the operator is performing an operation via the robot. In the case where the operator is performing an operation via the robot, use at least one of the biological information and the robot information to determine the concentration of the operator. Use the learned model for parameter determination to determine the parameters for providing a sound space corresponding to the sound space area and the concentration to the operator; Perform signal processing on the sound signal based on the parameters; Send the sound signal obtained through the signal processing to the output device.

Images