INFORMATION PROCESSING DEVICE, CONTROL METHOD AND CONTROL PROGRAM

Abstract

An information processing device (100) comprises an acquisition unit (120) that acquires biological information about an operator, robot information including a sound signal indicating sound in an environment of the robot, information indicating a sound space range as a sound space in which the operator hears sound via an output device, a learned work evaluation model, and a learned parameter determination model; a judgment unit (130) that judges whether or not the operator is performing work via the robot using the robot information and the learned work evaluation model; an identification unit (140) that identifies a concentration level of the operator using at least one piece of information from the biological information and the robot information when the operator is performing work via the robot; a determination unit (150),which determines a parameter to provide the operator with a tone space corresponding to the tone space range and the concentration level by using the learned parameter determination model, and a control unit (160) which performs signal processing of the tone signal based on the parameter and transmits a tone signal obtained by the signal processing to the output device.

Description

FIELD OF TECHNOLOGY

[0001] The present disclosure relates to an information processing device, a control method and a control program. TECHNICAL BACKGROUND

[0002] Technologies are already known that allow an operator to remotely control a robot. The robot has an artificial head. The sound surrounding the robot is fed into the artificial head. The operator can hear the sound surrounding the robot through the artificial head. A technology related to the artificial head has been proposed here (see patent reference 1). PRIOR ART PATENT REFERENCES

[0003] Patent Reference 1: Publication of Japanese Patent Application No. 62-044384 SUMMARY OF THE INVENTION TASK TO BE SOLVED BY THE INVENTION

[0004] As described above, the operator is exposed to the sound around the robot. The sound around the robot includes noise. For example, if the robot is installed in a factory, the sound around the robot also includes the noise in the factory. This noise is unnecessary for the operator. Unnecessary sound supplied to the operator reduces the operator's work efficiency.

[0005] An object of the present disclosure is to increase the work efficiency of the operator. MEANS TO SOLVE THE PROBLEM

[0006] An information processing device according to one aspect of the present disclosure is provided. The information processing device communicates with an output device that provides sound to an operator capable of operating a robot by remote control. The information processing device includes an acquisition unit that acquires biological information about the operator, robot information including a sound signal indicating sound in an environment of the robot, information indicating a sound space range as a sound space in which the operator hears sound via the output device, a learned work judgment model, and a learned parameter determination model; a judgment unit that judges whether or not the operator is performing work via the robot using the robot information and the learned work judgment model; an identification unit;which identifies a concentration level of the operator using at least one piece of information from the biological information and the robot information when the operator performs the work via the robot, a determination unit that determines a parameter to provide the operator with a sound space corresponding to the sound space range and the concentration level by using the learned parameter determination model, and a control unit that performs signal processing of the sound signal based on the parameter and transmits a sound signal obtained by the signal processing to the output device. EFFECT OF THE INVENTION

[0007] According to the present disclosure, the work efficiency of the operator can be increased. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing a control system. Fig. 2 is a diagram showing hardware in an information processing device. Fig. 3 is a block diagram showing functions of the information processing device. Fig. Figure 4 is a graph showing an example of a correspondence relationship between concentration level and working time. Fig. Figure 5 is a diagram showing an example of a direction in which sound is audible. Fig. 6 is a diagram showing an example of a case where a pitch range is represented two-dimensionally. Fig. Figure 7 is a diagram explaining reinforcement learning. Fig. Figure 8 is a diagram explaining supervised learning. Fig. Figure 9 is a diagram showing an example of a rule database. Fig. 10 is a flowchart showing an example of a process executed by the information processing device. MODE FOR CARRYING OUT THE INVENTION

[0008] An embodiment will be described below with reference to the drawings. The following embodiment is only an example, and various modifications are possible within the scope of the present disclosure. First embodiment

[0009] Fig. Figure 1 is a diagram showing a control system. The control system includes an information processing device 100, a biological scanning device 200, a robot scanning device 300, and an output device 400.

[0010] The information processing device 100, the biological scanning device 200, the robot scanning device 300, and the output device 400 communicate via a network. The network is a wired network or a wireless network.

[0011] The control system allows an operator to operate a robot remotely.

[0012] The information processing device 100 is a device that executes a control process.

[0013] The biological sensing device 200 measures biological information about the operator. The biological information is, for example, information about a sensory organ or a motor organ. The information about a sensory organ is, for example, information about the eyes, facial expression, heart rate, brain waves, or the like. Specifically, the information about the eyes is the direction of the line of sight, the degree (width) of eye opening, the shape of each pupil, the number of blinks per unit time, or the like. The information about the eyes and facial expression can be acquired from a camera. The heart rate can be acquired using a wristband-type measuring device. The brain waves can be acquired from a brainwave sensor attached to the user's head.Furthermore, information about a locomotor system includes, for example, information about the movement of the operator's skeletal structure, head movement, or similar. The movement of the skeletal structure can be acquired by a camera. The movement of the head can be acquired by a sensor attached to the user's head.

[0014] The robot sensing device 300 acquires robot information, including environmental information. The environmental information is information about the robot's environment. The environmental information is, for example, an image or video indicating the robot's environment. The environmental information is, for example, an audio signal indicating audio in the robot's environment. The image or video may be acquired from a camera mounted on the robot. The audio signal may be acquired from a multi-channel microphone mounted on the robot. The robot information may include robot position information and robot motion information. The robot position information is information indicating the position of the robot. The robot position information may be acquired, for example, from a Global Positioning System (GPS) mounted on the robot.Robot motion information is information that specifies a robot's movement. Robot motion information can be obtained from information input by the operator into a controller.

[0015] Output device 400 is, for example, a loudspeaker, headphones, or the like. Output device 400 provides sound to the operator.

[0016] Next, hardware included in the information processing device 100 will be described below.

[0017] Fig. Figure 2 is a diagram showing the hardware included in the information processing device. The information processing device 100 includes a processor 101, a volatile memory device 102, a non-volatile memory device 103, and an interface 104.

[0018] The processor 101 controls the information processing device 100 as a whole. The processor 101 is, for example, a central processing unit (CPU), a field-programmable gate array (FPGA), or the like. The processor 101 may also be a multiprocessor. The information processing device 100 may further include processing circuitry.

[0019] The volatile storage device 102 is a main memory of the information processing device 100. The volatile storage device 102 is, for example, a random access memory (RAM). The non-volatile storage device 103 is an auxiliary memory of the information processing device 100. The non-volatile storage device 103 is, for example, a hard disk drive (HDD) or a solid state drive (SSD).

[0020] The interface 104 carries out communication with the biological scanner 200, the robot scanner 300 and the output device 400.

[0021] The functions of the information processing device 100 are described below.

[0022] Fig. Figure 3 is a block diagram showing the functions of the information processing device. The information processing device 100 includes a storage unit 110, an acquisition unit 120, a judgment unit 130, an identification unit 140, a determination unit 150, and a control unit 160.

[0023] The storage unit 110 may be implemented as a memory area reserved in the volatile memory device 102 or in the non-volatile memory device 103.

[0024] The acquisition unit 120, the assessment unit 130, the identification unit 140, the determination unit 150, and the control unit 160 may be partially or entirely implemented by a processing circuit. Part or all of the acquisition unit 120, the assessment unit 130, the identification unit 140, the determination unit 150, and the control unit 160 may also be implemented as modules of a program executed by the processor 101. The program executed by the processor 101 is also referred to, for example, as a control program. The control program is recorded, for example, on a recording medium.

[0025] The storage unit 110 stores a variety of information.

[0026] The acquisition unit 120 acquires the biological information about the operator from the biological scanning device 200. The acquisition unit 120 acquires the robot information from the robot scanning device 300. The acquisition unit 120 may acquire the biological information and the robot information through another device.

[0027] The acquisition unit 120 may store the biological information and the robot information in the storage unit 110 each time the biological information and the robot information are acquired. In this way, the biological information and the robot information are accumulated in the storage unit 110.

[0028] The acquisition unit 120 acquires information indicating a sound space range. The sound space range is a sound space in which the operator hears the sound via the output device 400. Simply put, the sound space range is a sound space in which the operator is currently hearing sound. The details of the sound space range will be described later. Incidentally, the acquisition unit 120 acquires the information indicating the sound space range from the storage device 110 or an external device. The external device is a device that can be connected to the information processing device 100. The external device is, for example, a cloud server. Incidentally, the illustration of the external device is omitted.

[0029] The procurement unit 120 further acquires a learned job evaluation model. For example, the procurement unit 120 acquires the learned job evaluation model from the storage unit 110. Alternatively, the procurement unit 120 acquires the learned job evaluation model from the external facility, for example.

[0030] The judgment unit 130 judges whether the operator is performing work via the robot using the robot information and the learned work judgment model. In other words, the judgment unit 130 judges whether the operator is operating the robot remotely using the robot information and the learned work judgment model. It can also be expressed that the judgment unit 130 judges whether the operator is performing work via the robot using the robot information and the learned work judgment model. For example, when the judgment unit 130 inputs the robot information into the learned work judgment model, the learned work judgment model outputs information indicating whether the operator is performing work via the robot.The judgment unit 130 judges whether the operator is performing work via the robot based on the information. The learned work judgment model determines whether the operator is performing work via the robot based on, for example, the sound signal included in the robot information. The learned work judgment model further determines whether the operator is performing work via the robot based on, for example, the robot motion information included in the robot information.

[0031] When the operator performs the work via the robot, the identification unit 140 identifies a concentration level of the operator using at least one piece of information from the biological information and the robot information.

[0032] The following describes a method for identifying the concentration level. The identification unit 140 identifies the concentration level based on the acquired biological information. For example, the identification unit 140 identifies the concentration level corresponding to the acquired biological information using a table indicating a correspondence relationship between the biological information and the concentration level. Further, a learned model can output the concentration level, for example, when the identification unit 140 inputs the acquired biological information into the learned model.

[0033] The identification unit 140 can identify the concentration level of the operator using the acquired biological information (ie, the current biological information) and the biological information acquired in the past.

[0034] Furthermore, the identification unit 140 can identify the operator's concentration level using the acquired biological information, the acquired robot information (i.e., the current robot information), the robot information acquired in the past, and a learned model. The reason for using the robot information is described below. There is a relationship between the robot's movement indicated by the robot movement information included in the robot information and the concentration level. When the robot moves efficiently, the operator's concentration level can be considered high. Conversely, when the robot moves clumsily (i.e., inefficient movement), the operator's concentration level can be considered low. As described above, there is a relationship between the robot's movement and the concentration level.By using the robot information as an element to identify the concentration level, the concentration level can be obtained with high accuracy.

[0035] Furthermore, the identification unit 140 can identify the operator's concentration level based on the operator's working hours obtained from the acquired robot information and robot information acquired in the past. A learned model can be used to identify the concentration level. For example, when the identification unit 140 inputs the working hours into the learned model, the learned model outputs the concentration level. The learned model is obtained, for example, by learning information indicating a correspondence between the concentration level and the working hours. The following shows an example of the correspondence between the concentration level and the working hours.

[0036] Fig. Figure 4 is a graph showing an example of the correspondence relationship between the concentration level and working time. The vertical axis in Fig. 4 represents the degree of concentration. The horizontal axis in Fig. 4 shows the working hours. The learned model is obtained by learning learning data as shown in the graph.

[0037] When identifying the concentration level, a table can also be used to identify the concentration level.

[0038] Furthermore, the identification unit 140 can identify the operator's concentration level based on the type of work the operator is performing and the operator's working hours, which are obtained from the acquired robot information and robot information acquired in the past. When identifying the concentration level, a learned model or a table can be used to identify the concentration level.

[0039] Incidentally, when the identification unit 140 uses a learned model, the learned model is acquired by the acquisition unit 120. The acquisition unit 120 acquires the learned model from, for example, the storage unit 110 or the external device. This learned model is also referred to as a learned concentration level identification model.

[0040] The determination unit 150 determines a parameter for providing the operator with a pitch range corresponding to the pitch range and the concentration level by using a learned parameter determination model. This sentence can also be expressed as follows: The determination unit 150 determines a parameter for providing a pitch range to the operator, using the pitch range, the concentration level, and the learned parameter determination model, to increase the operator's work efficiency.

[0041] When the determination unit 150 inputs the pitch range and the concentration level to the learned parameter determination model, the learned parameter determination model outputs the parameter. Incidentally, the learned parameter determination model and pitch range will be described later.

[0042] Here, a direction in which sound is audible can be shown as in the following illustration.

[0043] Fig. Figure 5 is a diagram showing an example of the direction in which sound is audible. Fig. Figure 5 shows a case in which the direction in which sound is audible is represented on a spherical surface. In Fig. 5, the direction in which sound is audible is indicated by an arrow 10. In addition, Fig. 5 a pitch range 11.

[0044] The following shows a case in which the tonal space area is represented two-dimensionally.

[0045] Fig. Figure 6 is a diagram showing an example of the case where the pitch range is represented two-dimensionally. Fig. 6, the pitch range is represented by an angle. In Fig. In Figure 6, the pitch range is represented by 90 degrees or 270 degrees, for example. Incidentally, the angle reference can be set to the direction in front of the operator. To simplify the explanation, the pitch range is assumed to be represented two-dimensionally in the following description.

[0046] The learned parameter determination model is described below. The learned parameter determination model is acquired by the acquisition unit 120. The acquisition unit 120 acquires the learned parameter determination model, for example, from the storage unit 110 or the external device. The learned parameter determination model can be obtained through machine learning. The learned parameter determination model can be obtained, for example, through reinforcement learning. Reinforcement learning is explained below using an illustration.

[0047] Fig. Figure 7 is a diagram explaining reinforcement learning. An environment part in reinforcement learning corresponds to the operator's concentration level. An agent in reinforcement learning corresponds to a controller for the tonal range.

[0048] For example, if the operator is encouraged to maintain a state of high concentration, a reward function is designed so that a high reward can be obtained for an action with high concentration. An optimal strategy is then learned.

[0049] The learned parameter determination model can also be obtained using a learning method other than reinforcement learning. For example, the learned parameter determination model can be obtained using supervised learning. Supervised learning is explained below using a diagram.

[0050] Fig. Figure 8 is a diagram explaining supervised learning. Twelve types of states are generated, each of which indicates a relationship between the operator's concentration level and a trend in a constant time. The twelve types of states can also be represented as operator states. In machine learning, the operator's concentration level in a constant time is used as input data. In machine learning, the operator's states are also used as correct answer labels. The parameter is determined based on the operator's state using a rule database. An example of the rule database is shown below.

[0051] Fig. Figure 9 is a diagram showing an example of the rule database. For example, if the operator's state is "S2" and the pitch range is less than or equal to 90 degrees, a pitch range extension parameter is output.

[0052] Additionally, supervised learning can be performed, as described below. Time series data of the parameter is used as input data. In machine learning, the learning is performed to output a parameter that is expected to increase the concentration level the most.

[0053] As described above, the learned parameter determination model is obtained by learning.

[0054] A procedure for determining the parameter is described below using concrete examples.

[0055] For example, if the current pitch range is wide (e.g., 270 degrees) and the concentration level is low, the determination unit 150 determines a parameter for narrowing the pitch range to increase the operator's concentration level. For example, the determination unit 150 determines a parameter for narrowing the pitch range from 270 degrees to 90 degrees.

[0056] If the pitch range is narrow (e.g., 90 degrees) and the concentration level is low, it can be considered that the operator is fatigued due to hyperfocus. Therefore, the determination unit 150 determines a parameter for expanding the pitch range to relax the operator. For example, the determination unit 150 determines a parameter for expanding the pitch range from 90 degrees to 270 degrees.

[0057] The control unit 160 performs signal processing on the sound signal contained in the robot information based on the determined parameter. The signal processing may include, for example, beamforming processing, sound masking processing, or the like. This signal processing converts the sound signal into a sound signal corresponding to the parameter.

[0058] The control unit 160 transmits the sound signal obtained by signal processing to the output device 400. In this way, for example, an operator who is provided with a sound with a wide pitch range and is at a low level of concentration can hear a sound whose pitch range has been narrowed. Accordingly, the operator's level of concentration increases, and thus the operator's work efficiency also increases. In this way, for example, an operator who is provided with a sound with a narrow pitch range and is at a low level of concentration can hear a sound whose pitch range has been expanded. Accordingly, the operator is relaxed, and thus the operator's work efficiency also increases.

[0059] A process executed by the information processing device 100 will be described below using a flowchart.

[0060] Fig. 10 is a flowchart showing an example of the process executed by the information processing device. (Step S11) The acquisition unit 120 acquires the biological information, the robot information, and the information indicating the current sound space range. (Step S12) The judgment unit 130 judges whether the operator is performing work via the robot using the robot information and the learned work judgment model. If the operator is performing work, the process proceeds to step S13. If the operator is not performing work, the process ends. (Step S13) The identification unit 140 identifies the concentration level of the operator using the biological information and the robot information. (Step S14) The determination unit 150 determines the parameter to provide the operator with the pitch range corresponding to the pitch range and the concentration level by using the learned parameter determination model. (Step S15) The control unit 160 performs the signal processing of the sound signal included in the robot information based on the determined parameter. (Step S16) The control unit 160 transmits the sound signal obtained by the signal processing to the output device 400.

[0061] According to the embodiment, the information processing device 100 determines the parameter for providing a sound space to increase the operator's work efficiency. The information processing device 100 performs signal processing of the sound signal based on the determined parameter. The information processing device 100 provides the sound signal obtained by the signal processing to the operator via the output device 400. Accordingly, the information processing device 100 is capable of increasing the operator's work efficiency. LIST OF REFERENCE SYMBOLS

[0062] 10: Arrow, 11: Tone space area, 100: Information processing device, 101: Processor, 102: Volatile storage device, 103: Non-volatile storage device, 104: Interface, 110: Storage unit, 120: Acquisition unit, 130: Assessment unit, 140: Identification unit, 150: Determination unit, 160: Control unit, 200: Biological scanning device, 300: Robot scanning device, 400: Output device

Claims

[1] An information processing device that carries out communication with an output device that provides sound to an operator capable of operating a robot by remote control, the information processing device comprising: an acquisition unit that acquires biological information about the operator, robot information including a sound signal indicating sound in an environment of the robot, information indicating a sound space range as a sound space in which the operator hears sound through the output device, a learned work judgment model, and a learned parameter determination model; a judgment unit that judges whether or not the operator performs a work via the robot using the robot information and the learned work judgment model; an identification unit that identifies a concentration level of the operator using at least one piece of information from the biological information and the robot information when the operator performs the work via the robot; a determination unit that determines a parameter to provide the operator with a pitch range corresponding to the pitch range and the concentration level by using the learned parameter determination model; and a control unit that performs signal processing of the sound signal based on the parameter and transmits a sound signal obtained by the signal processing to the output device. [2] The information processing device according to claim 1, wherein the identification unit acquires the concentration level of the operator using the acquired biological information and previously acquired biological information when the operator performs the work via the robot. [3] Information processing device according to claim 1, wherein the robot information includes robot movement information as information about the movement of the robot, the procurement unit procures a learned concentration level identification model, and the identification unit identifies the concentration level of the operator using the biological information, the acquired robot information, previously acquired robot information, and the learned concentration level identification model when the operator performs the work via the robot. [4] The information processing device according to claim 1, wherein the identification unit identifies the concentration level of the operator based on the working hours of the operator obtained from the acquired robot information and previously acquired robot information when the operator performs the work via the robot. [5] The information processing device according to claim 4, wherein the identification unit identifies the concentration level of the operator based on the type of work and the working hours of the operator obtained from the acquired robot information and previously acquired robot information when the operator performs the work via the robot. [6] A control method performed by an information processing device that communicates with an output device that provides sound to an operator capable of operating a robot by remote control, the control method comprising: Obtaining biological information about the operator, robot information including a sound signal indicating sound in the environment of the robot, information indicating a sound space range as a sound space in which the operator hears sound via the output device, a learned work assessment model, and a learned parameter determination model that judges whether or not the operator is performing work via the robot using the robot information and the learned work assessment model, identifying a concentration level of the operator using at least one piece of information from the biological information and the robot information when the operator performs work via the robot, determining a parameter to provide the operator with a sound space corresponding to the sound space range and the concentration level,by using the learned parameter determination model;, Performing signal processing of the audio signal based on the parameter; and Transmitting an audio signal obtained by signal processing to the output device. [7] A control program that causes an information processing device that communicates with an output device that provides sound to an operator capable of operating a robot by remote control to perform the following process: Obtaining biological information about the operator, robot information including a sound signal indicating sound in the environment of the robot, information indicating a sound space range as a sound space in which the operator hears sound via the output device, a learned work assessment model, and a learned parameter determination model; judging whether or not the operator is performing work via the robot using the robot information and the learned work assessment model; identifying a concentration level of the operator using at least one piece of information from the biological information and the robot information when the operator performs work via the robot; determining a parameter to provide the operator with a sound space corresponding to the sound space range and the concentration level;by using the learned parameter determination model;, Performing signal processing of the audio signal based on the parameter; and Transmitting an audio signal obtained by signal processing to the output device.

Images