Information processing system, information processing method, and information processing program

Abstract

An information processing system according to the present disclosure includes a robot device, wherein the robot device includes: a sensor that senses a surrounding environment; an acquisition unit that acquires robot information relating to the robot device and environment information from the sensor; a detection unit that detects whether or not the robot device is being powered; and a training unit that, when it is detected that the robot device is being powered, trains a machine learning model using the robot information and the environment information acquired by the sensor.

Description

Information Processing System, Information Processing Method, and Information Processing Program

[0009] ,

[0001] The present disclosure relates to an information processing system, an information processing method, and an information processing program.

[0002] Since learning of a machine learning model involves repeated computations, it is known to consume a large amount of power. Therefore, in the prior art, it is known to perform learning of a machine learning model according to the available amount of power.

[0003] For example, in Patent Document 1, it is proposed to perform learning of a machine learning model so that a battery used for learning the machine learning model mounted on a vehicle does not run out of power.

[0004] Japanese Unexamined Patent Application Publication No. 2022-53033

[0005] In the prior art, in order to perform learning of a machine learning model using a large-capacity battery provided in a device that can handle a large amount of power consumption, there are cases where a device equipped with the machine learning model cannot be designed flexibly. For example, a large-capacity battery is large and heavy, which increases the size of the device.

[0006] In addition, data used for learning a machine learning model acquired by a device may not be transmitted externally due to concerns about information leakage or the communication environment.

[0007] Therefore, the present disclosure proposes an information processing system, an information processing method, and an information processing program that can flexibly design a device equipped with a machine learning model.

[0008] In order to solve the above problems, an information processing system according to one aspect of the present disclosure is an information processing system including a robot device, wherein the robot device includes a sensor that senses the surrounding environment, robot information regarding the robot device, an acquisition unit that acquires the environmental information from the sensor, a detection unit that detects whether the robot device is in a power supply state, and a learning unit that performs learning of a machine learning model using the robot information and the environmental information acquired by the sensor when it is detected that the robot device is in a power supply state.

[0009] Figure 1 is a diagram showing an overview of an information processing system according to an embodiment. Figure 2 is a diagram showing an example configuration of a power supply device and a robot device according to an embodiment. Figure 3 is a sequence diagram showing an example of information processing in an embodiment. Figure 4 is a diagram illustrating a modified example in this disclosure. Figure 5 is a hardware configuration diagram showing an example of a computer that realizes the functions of a robot device.

[0010] Embodiments of this disclosure will be described in detail below with reference to the drawings. In each of the following embodiments, the same parts will be denoted by the same reference numerals to avoid redundant descriptions.

[0011] This disclosure will be described in the following order of items: 1. Prior Art 2. Embodiments 2-1. Overview of the Information Processing System According to the Embodiment 2-2. Configuration of the Information Processing System According to the Embodiment 2-2-1. Configuration of the Robot Device According to the Embodiment 2-2-2. Configuration of the Power Supply Device According to the Embodiment 2-3. Information Processing Procedure of the Information Processing System According to the Embodiment 2-4. Modifications According to the Embodiment 3. Other Embodiments 4. Effects of the Information Processing System According to the Disclosure 5. Hardware Configuration

[0012] (1. Conventional Technology) Conventional technology includes autonomous robots, mobile devices, and self-driving cars (hereinafter, devices are also referred to as robots), which are equipped with sensors to recognize the environment in which the robot is placed and a computing unit to perform inference using machine learning. Robots equipped with sensors and computing units can continuously improve their performance by learning a machine learning model using data collected by the sensors.

[0013] Training a machine learning model for a robot equipped with sensors and computing units involves sending data collected by the sensors to an external cloud or server, where the training is performed. Alternatively, the training of the machine learning model for a robot equipped with sensors and computing units can be performed using the computing unit installed in the robot.

[0014] For example, a robot equipped with sensors and computing units will transmit large amounts of data, such as data collected by sensors, when training a machine learning model on an external cloud. Specifically, training a machine learning model on an external cloud incurs communication costs and time proportional to the amount of data. Furthermore, training a machine learning model on an external cloud cannot be performed in environments where communication is impossible or bandwidth is limited.

[0015] The data may contain important information or content related to privacy. Robots equipped with sensors and computing units cannot transmit data if the data in the robot's surroundings contains highly private or confidential data, such as facial images or important information. Important information includes, for example, images of the inside of a factory.

[0016] On the other hand, robots equipped with sensors and computing units consume a large amount of power when using their built-in computing units to train machine learning models due to the large-scale calculations involved. Furthermore, robots equipped with sensors and computing units generate heat during the training process when using their built-in computing units to train machine learning models.

[0017] Robots equipped with sensors and computing units, when using the computing unit to train machine learning models, utilize the robot's battery for this purpose. Therefore, they are equipped with high-capacity batteries and cooling devices to dissipate the heat generated during training. Robots with high-capacity batteries and cooling devices have increased weight and volume, and the large power consumption during machine learning model training reduces the operating time of battery-powered robots. Furthermore, robots equipped with high-capacity batteries and cooling devices may experience increased manufacturing costs and a decrease in payload capacity due to the inclusion of these devices.

[0018] Therefore, this disclosure proposes an information processing system comprising a robotic device that learns a machine learning model from data acquired through autonomous movement and the actions of the robotic device, and a power supply device that provides power and cooling for learning the machine learning model.

[0019] (2. Embodiments) (2-1. Overview of the Information Processing System According to the Embodiment) Next, an overview of the information processing system 1 according to the embodiment will be described using Figure 1. Figure 1 is a diagram showing an overview of the information processing system 1 according to the embodiment.

[0020] In Figure 1, the information processing system 1 includes a power supply device 50 and a robot device 100. The power supply device 50 and the robot device 100 are connected, for example, via a network.

[0021] The power supply device 50 includes a power supply device 51 and a cooling device 52. For example, the power supply device 50 is a device that is communicatively connected to the robot device 100 and provides power and cooling to the robot device 100. For example, the power supply device 51 provides power to the robot device 100, and the cooling device 52 provides cooling to the robot device 100. Note that the power supply device 51 and the cooling device 52 do not have to be a single device, but may be separate devices.

[0022] The robot device 100 is, for example, an autonomous robot. Alternatively, the robot device 100 may be a device that uses sensors to acquire information about the surrounding environment 10, such as an autonomously moving vehicle, a smartphone, a camera, or a portable game device, and can act on the environment 10 to store the collected surrounding environment 10 and the actions taken. The actions taken on the environment 10 refer to actions of the robot device 100, such as movement or taking moving images.

[0023] The robot device 100 is equipped with sensors and a machine learning model. The machine learning model is, for example, a model that takes environmental information as input and outputs the optimal action. The machine learning model is trained to output the optimal action using a learning database that stores environmental information sensed by the sensors and robot information about the robot device 100. The robot information refers to information about the robot device 100 itself and information from user operations.

[0024] Next, the information processing flow of the information processing system 1 according to the embodiment will be explained using Figure 1. First, the robot device 100 recognizes the surrounding environment 10 using the sensors provided by the robot device 100 and acts autonomously (step S1). Specifically, the robot device 100 inputs environmental information about the surrounding environment 10 and robot information about the robot device 100 into a machine learning model and acts autonomously based on the optimal action output.

[0025] The robot device 100 acquires environmental information and information about the robot device 100 (step S2). The robot device 100 stores the acquired environmental information and robot information about the robot device 100 in the storage unit as a learning database.

[0026] After acquiring environmental information and robot information, the robot device 100 connects to the power supply device 51 and accepts power (step S3). By accepting power, the robot device 100 can receive power to be used for training the machine learning model. The robot device 100 may also accept power from the power supply device 50 for a battery used for movement to acquire environmental information, for example.

[0027] When the robot device 100 detects that it is receiving power, it uses the supplied power, environmental information acquired from sensors, and robot information to train a machine learning model (step S4). For example, the robot device 100 controls the cooling of the cooling device 52 to cool down the heat generated by the training of the machine learning model (step S5). For example, when the temperature of the robot device 100 exceeds a preset threshold, it controls the cooling device 52 to activate and cool the robot device 100.

[0028] The robot device 100 uses environmental information acquired during its autonomous actions, along with robot information, to train a machine learning model and optimize its actions. The robot device 100 does not transmit environmental information acquired by its sensors to an external computing cloud 20 or the like.

[0029] In this way, the robot device 100 can train a machine learning model without transmitting the environmental information it acquires to an external computing cloud 20, thus suppressing the leakage of private data and confidential information. Furthermore, since the robot device 100 receives power from the power supply device 51 and cooling from the cooling device 52, it does not require the installation of a large battery or cooling device, allowing for flexible design of robots equipped with machine learning models.

[0030] (2-2. Configuration of the Information Processing System According to the Embodiment) Next, the configuration of the power supply device 50 and the robot device 100 included in the information processing system 1 according to the embodiment will be described using Figure 2. Figure 2 is a diagram showing an example of the configuration of the power supply device 50 and the robot device 100 according to the embodiment. The information processing system 1 includes the power supply device 50 and the robot device 100.

[0031] (2-2-1. Configuration of the robot device according to the embodiment) As shown in Figure 2, the robot device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

[0032] The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 may be connected to a network (Internet, NFC (Near Field Communication), Bluetooth®, etc.) by wire or wireless connection, and may transmit and receive information with, for example, a power supply device 50, via the network.

[0033] The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks and optical discs. As shown in Figure 2, the storage unit 120 has an environmental information storage unit 121 and a machine learning model storage unit 122.

[0034] The environmental information storage unit 121 stores robot information and environmental information. For example, the environmental information storage unit 121 stores environmental information acquired from sensors equipped on the robot device 100 as a result of the robot device 100's actions, as well as the results of those actions. The environmental information storage unit 121 also stores environmental information, robot information, and the results of movement acquired from sensors equipped on the robot device 100 as a result of the robot device 100's movement. Specifically, the environmental information storage unit 121 stores the environmental information, robot information, and the results of movement acquired from sensors equipped on the robot device 100 as a learning database. Note that robot information may be acquired, for example, while the robot device 100 is moving.

[0035] The machine learning model storage unit 122 stores a trained model that is learned using robot information and acquired environmental information. For example, the machine learning model is a model that takes robot information and environmental information as input and outputs the optimal action. The optimal action refers to the best action according to the purpose of use of the robot device 100. The purpose of use is not limited to an objective that can be achieved by the robot device 100 moving, but may also be an objective that can be achieved without the robot device 100 moving. For example, the purpose of use may be obstacle avoidance. Alternatively, the purpose of use may be to discover and remember unknown objects, unknown places, and unknown people.

[0036] The control unit 130 is implemented, for example, by a CPU (Central Processing Unit) or MPU (Micro Processing Unit) executing a program (for example, the information processing program according to this disclosure) stored inside the robot device 100 using RAM (Random Access Memory) or the like as the work area. The control unit 130 is also a controller and may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0037] The control unit 130 includes a sensor 131, an acquisition unit 132, a detection unit 133, a learning unit 134, an operation control unit 135, and a cooling control unit 136, and realizes or executes the information processing functions and operations described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in Figure 2, and other configurations are also acceptable as long as they perform the information processing described later.

[0038] Sensor 131 senses the surrounding environment. For example, sensor 131 is a sensor used to understand the surrounding environment. Specifically, sensor 131 is a pressure sensor, electrostatic sensor, human presence sensor, acceleration sensor, gyroscope, etc. Sensor 131 may also be a sensor that senses hearing or the condition of the ground. The surrounding environment refers to information about the area around the robot device 100, such as the presence or absence of obstacles.

[0039] The acquisition unit 132 acquires robot information related to the robot device 100 and environmental information from the sensor 131. The acquisition unit 132 acquires the surrounding environment sensed by the sensor 131 as environmental information. For example, the acquisition unit 132 detects surrounding objects and other elements of the environment from the sensor 131 and acquires the detected objects as environmental information.

[0040] Robot information is at least one of the following: user operation information for the robot device 100, or information obtained from various devices equipped with the robot device 100 through the actions of the robot device 100 itself. For example, user operation information for the robot device 100 includes at least one of user operations on buttons attached to the robot device 100 and user instructions given to the robot device 100 by voice. Existing technologies may be used for voice recognition of user speech.

[0041] For example, the information acquired by the robot device 100 through its own actions from various devices equipped with the robot device 100 includes at least one of the following: the angle of the robot arm joints, a moving image of the surroundings acquired from a camera, the distance traveled, speed, acceleration, angular velocity, and the power consumption of the battery. For example, the distance traveled and speed are acquired from an encoder. For example, acceleration and angular velocity are acquired from an IMU (inertial measurement unit). The encoder is a device equipped in the robot device 100 that detects the amount of movement, direction, angle, etc. The encoder may be, for example, a gyro sensor or an acceleration sensor.

[0042] The acquisition unit 132 may acquire any information obtained from the activities of the robot device 100 other than the environmental information acquired by the sensor 131 as robot information. For example, the acquisition unit 132 may acquire user operations on the robot device 100 as robot information relating to the robot device 100. Specifically, the acquisition unit 132 may include user operations on buttons attached to the robot device 100 as information relating to the robot device 100. For example, if the robot device 100 is equipped with a robot arm, the acquisition unit 132 may acquire joint angles as robot information relating to the robot device 100. The acquisition unit 132 may also acquire the angles of the robot arm joints, moving images of the surroundings acquired from the camera, travel distance acquired from the encoder, and speed acquired from the encoder as robot information relating to the robot device 100. The acquisition unit 132 may also acquire acceleration acquired from the IMU, angular velocity acquired from the IMU, and battery power consumption.

[0043] For example, the acquisition unit 132 further acquires the result of the action of the robot device 100 based on the robot information and the environmental information acquired by the sensor 131. Specifically, the acquisition unit 132 inputs the robot information and the environmental information into a machine learning model that outputs an optimal action, and acquires the result of the action executed according to the optimal action output by the robot device 100. Note that the action of the robot device 100 refers to any operation of the robot device 100. For example, the action of the robot device 100 refers to at least one of the robot device 100 moving, moving a robot arm attached to the robot device 100 to handle an object, moving the face of the robot device 100 toward the surroundings, and communicating by voice.

[0044] For example, as a result of the action, the acquisition unit 132 acquires the result of the movement. Specifically, the acquisition unit 132 inputs the robot information and the environmental information such as surrounding obstacles into a machine learning model that outputs an optimal action, and acquires the result of the movement executed according to the optimal action output by the robot device 100. Note that the result of the movement is data indicating the result accompanying the action of the robot device 100. In other words, the result of the movement is data of the result of the robot device 100 acting on the environment. For example, the result of the movement refers to the result of evaluating whether the robot device 100 should have moved.

[0045] The detection unit 133 detects whether the robot device 100 is in a power supply state. In addition, the detection unit 133 detects whether the robot device 100 is connected to the power supply device 50. The power supply state refers to a state in which power is supplied to the robot device 100. Note that the power supply method is not limited to power supply by directly connecting to the power supply device 50, and may be any power supply method such as wireless power supply.

[0046] When the learning unit 134 detects that the robot device 100 is in a power supply state, it learns a machine learning model using the robot information and the environmental information acquired by the sensor 131. The learning unit 134 uses the power supplied from the power supply device 50 to learn a machine learning model with the robot information and the environmental information including information such as surrounding obstacles as learning data. For example, the machine learning model is a model that improves the performance of the robot device 100 in processing tasks such as autonomous movement. For example, the machine learning model may be a machine learning model that learns by reinforcement learning. Also, the learning method of the machine learning model may utilize existing technologies.

[0047] For example, when the learning unit 134 detects that the robot device 100 is in a power supply state, it learns a machine learning model using the robot information, the environmental information acquired by the sensor 131, and the result of the action of the robot device 100. In other words, the learning unit 134 learns the machine learning model so that the actions such as the behavior of the robot device 100 are improved. Specifically, the learning unit 134 uses a learning dataset including the robot information, the environmental information, and the result of the action to learn a machine learning model that outputs an optimal action with the robot information, the environmental information, and the result of the action as inputs.

[0048] For example, when the learning unit 134 detects that the robot device 100 is in a power supply state, it learns a machine learning model using the robot information, the environmental information acquired by the sensor 131, and the result of the movement of the robot device 100. Specifically, the learning unit 134 uses a learning dataset including the robot information, the environmental information, and the result of the movement to learn a machine learning model that outputs an optimal action with the robot information, the environmental information, and the result of the movement as inputs.

[0049] The motion control unit 135 controls the movement of the robot device 100. For example, the motion control unit 135 controls the movement of the robot device 100 based on a learned machine learning model. Specifically, the motion control unit 135 takes robot information and environmental information such as surrounding obstacles as inputs, and outputs the optimal action to a machine learning model. By inputting the environmental information, the optimal action is output, and the motion control unit 135 controls the movement of the robot device 100 to follow that optimal action.

[0050] For example, the motion control unit 135 controls the movement of the robot device 100 based on the optimal action output by a machine learning model that has learned from a learning database containing robot information, environmental information, and the results of the robot device 100's movement, all stored in the environmental information storage unit 121. Specifically, the motion control unit 135 controls the robot device 100's actions based on the optimal action output by a machine learning model that has been learned without taking in or transmitting information to the outside.

[0051] The cooling control unit 136 controls the cooling of the robot device 100 in response to temperature changes of the robot device 100 caused by the learning of a machine learning model. For example, if the cooling device 52 included in the power supply device 50 is a fan or cooling fan, the cooling control unit 136 controls the airflow of the cooling device 52 to adjust in accordance with the temperature changes of the robot device 100. For example, if the cooling control unit 136 detects a temperature above a preset temperature, it controls the robot device 100 to be cooled.

[0052] For example, the cooling control unit 136 controls the cooling of the robot device 100 when the temperature of the robot device 100 is above a threshold. Specifically, the cooling control unit 136 controls the start of cooling by the cooling function of the cooling device 52 included in the power supply device 50 when the temperature of the computing unit, which generates heat due to the learning of the machine learning model, is above a preset temperature. Furthermore, the cooling control unit 136 controls the end of cooling by the cooling function of the cooling device 52 included in the power supply device 50 when the learning of the machine learning model is completed and the temperature of the computing unit is below a preset temperature. The computing unit refers to, for example, a CPU, MPU, and GPU (Graphical Processing Unit).

[0053] For example, the cooling control unit 136 controls the coolant supplied from the cooling device 52 of the power supply device 50 (described later) in response to temperature changes in the robot device 100 due to the learning of a machine learning model. In other words, if the robot device 100 is cooled using a so-called water-cooling method, the cooling control unit 136 controls the supply of coolant.

[0054] (2-2-2. Configuration of the power supply device according to the embodiment) As shown in Figure 2, the power supply device 50 includes a power supply device 51 and a cooling device 52. Note that the power supply device 51 and the cooling device 52 may be separate devices.

[0055] The power supply device 50, the power supply device 51, and the cooling device 52 may each have a communication unit, a storage unit, and a control unit. The communication unit may be implemented by, for example, a NIC. The communication unit may be connected to a network by wire or wireless and may transmit and receive information to and from, for example, the robot device 100, via the network. The storage unit may be implemented by, for example, a semiconductor memory element such as RAM or flash memory, or a storage device such as a hard disk or optical disc. The control unit may be implemented by, for example, a CPU or MPU, which executes a program stored inside the robot device 100 using RAM or the like as a work area. The control unit 130 is a controller and may be implemented by, for example, an integrated circuit such as an ASIC or FPGA. Note that the power supply device 50, the power supply device 51, and the cooling device 52 may not all or part of have a communication unit, a storage unit, and a control unit.

[0056] For example, the control unit of the cooling device 52 may have a cooling control unit 136 of the robot device 100. Specifically, the control unit of the cooling device 52 measures the temperature of the robot device 100 and controls it to cool if the temperature is above a preset temperature.

[0057] The power supply device 51 supplies power to the robot device 100. Specifically, the power supply device 51 connects to the robot device 100 and supplies power. The power supply device 51 may supply power not only for training the machine learning model but also for the battery of the robot device 100.

[0058] The cooling device 52 cools the robot device 100. The cooling device 52 can have any configuration as long as it can cool the robot device 100. In addition, existing technologies may be used for the cooling method of the robot device 100.

[0059] For example, the cooling device 52 may be a fan, a device for supplying coolant, a heat sink, a cooling fan, or a water tank for holding coolant. Specifically, if the cooling device 52 is a fan, the fan cools the robot device 100 by blowing air onto it.

[0060] For example, the cooling device 52 cools the robot device 100 by supplying coolant to it. Specifically, if the cooling device 52 is a device that supplies coolant, the device supplies coolant to the robot device 100. The device that supplies coolant may also cool the battery and the charging cable used for power supply, not just the computing unit. For example, the device that supplies coolant is suitable when the robot device 100 is an electric vehicle, and by cooling the battery and charging cable, it enables rapid charging.

[0061] For example, the robot device 100 is further equipped with a Peltier element, and the cooling device 52 is a heat sink or a cooling fan. Specifically, the heat sink or cooling fan dissipates the heat transferred by the Peltier element when the robot device 100 is equipped with a Peltier element.

[0062] For example, the cooling device 52 is a water tank that holds a cooling liquid, and the robot device 100 is immersed in the water tank. Specifically, the cooling device 52 cools the robot device 100 with the cooling liquid in the water tank when the robot device 100 is placed in the tank. For example, the robot device 100 that is placed in the water tank that holds the cooling liquid is waterproof. The water tank could be, for example, a swimming pool.

[0063] (2-3. Information Processing Procedure of the Information Processing System According to the Embodiment) Next, the information processing procedure of the information processing system 1 according to the embodiment will be described using Figure 3. Figure 3 is a sequence diagram showing an example of information processing in the embodiment.

[0064] As shown in Figure 3, the sensor 131 of the robot device 100 senses the surrounding environment (step S101). The acquisition unit 132 of the robot device 100 acquires robot information and environmental information (step S102).

[0065] The detection unit 133 of the robot device 100 acquires robot information and environmental information, and then detects whether it is connected to the power supply device 50 (step S103). If the robot device 100 is not connected to the power supply device 50, the detection unit 133 of the robot device 100 terminates the process (step S103: No).

[0066] The power supply device 51 of the power supply device 50 supplies power to the robot device 100 when the robot device 100 is connected to the power supply device 50 (step S103: Yes) (step S104). The detection unit 133 of the robot device 100 detects whether or not it is in a power supply state (step S105). If the detection unit 133 of the robot device 100 detects that it is not in a power supply state, it terminates the process (step S105: No).

[0067] If the detection unit 133 of the robot device 100 detects that it is in a power supply state (step S105: Yes), it trains a machine learning model (step S106). The cooling control unit 136 of the robot device 100 controls the cooling of the robot device 100 that occurs as a result of the machine learning model training (step S107). The cooling device 52 of the power supply device 50 cools the robot device 100 in accordance with the control of the cooling control unit 136 of the robot device 100 (step S108).

[0068] (2-4. Modifications of the Embodiment) The information processing according to the embodiment described above may be modified in various ways. Modifications of the embodiment will be described below.

[0069] Figure 4 illustrates a modified example in this disclosure. In Figure 4, explanations of content common to Figure 1 are omitted as appropriate. As shown in Figure 4, the robot device 100 may acquire environmental information from devices connected to the home network 200, unless it transmits data to a computing cloud 20, such as an external cloud server. For example, the robot device 100 may acquire environmental information acquired by a terminal device 300, such as a smartphone, tablet, or portable game device, connected to the home network 200 (step S11). The robot device 100 may also acquire environmental information acquired by an imaging device 400, such as a camera, connected to the home network 200 (step S12). The robot device 100 may also have an information processing device such as a PC (personal computer) mounted on the power supply device 50 and acquire environmental information acquired by the PC.

[0070] Furthermore, the home network 200 is not limited to a general network such as Ethernet (registered trademark). For example, the home network 200 refers to a network to which only devices in a specific area that are not connected to an external server are connected. Alternatively, the home network 200 may be constructed by mounting an information processing device such as a PC (personal computer) on the power supply device 50 and communicating between the robot and the PC using an interface such as USB or PCI-Express.

[0071] The robot device 100 trains a machine learning model using environmental information acquired from the robot device 100, the terminal device 300, and at least one of the imaging device 400. Note that the training of the machine learning model may be performed on terminal devices 300 or imaging device 400 other than the robot device 100 that are connected to the robot device 100 within the home network 200.

[0072] The terminal device 300, the imaging device 400, and the information processing device may each have a sensor for sensing the surrounding environment, a communication unit, a storage unit, and a control unit. The communication unit may be implemented by, for example, a NIC. The communication unit may be connected to a network by wire or wireless and may transmit and receive information to and from, for example, a robot device 100, via the network. The storage unit may be implemented by, for example, a semiconductor memory element such as RAM or flash memory, or a storage device such as a hard disk or optical disc. The control unit may be implemented by, for example, a CPU or MPU, which executes a program stored inside the robot device 100 using RAM or the like as a working area. The control unit 130 is a controller and may be implemented by, for example, an integrated circuit such as an ASIC or FPGA.

[0073] (3. Other Embodiments) The processes according to each of the embodiments described above may be carried out in various other forms besides those described above.

[0074] Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be changed at will unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0075] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0076] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0077] Furthermore, the effects described herein are merely illustrative and not limiting; other effects may also occur.

[0078] (4. Effects of the Information Processing System in the Disclosure) As described above, the information processing system in the disclosure (information processing system 1 in the embodiment) includes a robot device (robot device 100 in the embodiment). The robot device includes a sensor (sensor 131 in the embodiment), an acquisition unit (acquisition unit 132 in the embodiment), a detection unit (detection unit 133 in the embodiment), and a learning unit (learning unit 134 in the embodiment). The sensor senses the surrounding environment. The acquisition unit acquires robot information related to the robot device and environmental information from the sensor. The detection unit detects whether the robot device is in a powered state or not. If the learning unit detects that the robot device is in a powered state, it uses the robot information and the environmental information acquired by the sensor to learn a machine learning model.

[0079] Thus, the information processing system related to this disclosure can suppress the leakage of private data and confidential information because it does not transmit data outside the robot by training a machine learning model within the robot device.

[0080] Furthermore, by performing machine learning model training only when power is supplied, the information processing system eliminates the need for large batteries in the robotic device, thereby reducing design constraints on the robot. Specifically, by eliminating the need for large batteries, the information processing system can reduce design constraints such as the size of the robot itself and the robot's operating time. Therefore, equipment equipped with machine learning models can be designed more flexibly.

[0081] Furthermore, the acquisition unit acquires the results of the robot device's actions based on the robot information and the environmental information acquired by the sensors. When the learning unit detects that the robot device is powered, it uses the robot information, the environmental information acquired by the sensors, and the results of the robot device's actions to train a machine learning model.

[0082] In this way, information processing systems can improve any action performed by a robotic device.

[0083] Furthermore, the acquisition unit acquires the result of the action, specifically the movement. When the learning unit detects that the robot device is powered, it uses the robot information, the environmental information acquired by the sensors, and the result of the robot device's movement to train a machine learning model.

[0084] In this way, information processing systems can learn more accurate machine learning models by acquiring information about the robot's movements, in addition to robot information and environmental information.

[0085] The robot device also includes a motion control unit (motion control unit 135 in this embodiment) and an environmental information storage unit (environmental information storage unit 121 in this embodiment). The motion control unit controls the movement of the robot device. The environmental information storage unit stores environmental information, robot information, and the results of the movement acquired from sensors on the robot device as the robot device moves.

[0086] In this way, the information processing system can store information acquired by the robotic device as it moves, allowing it to train machine learning models while powered on. Furthermore, since the information processing system does not transmit information externally, there are no communication costs, and it can handle highly confidential data.

[0087] Furthermore, the motion control unit controls the movement of the robotic device based on a learned machine learning model.

[0088] In this way, the information processing system can provide a robot whose performance continuously improves by moving based on a learned machine learning model.

[0089] The information processing system further includes a power supply device (power supply device 50 in this embodiment). The power supply device includes a power supply unit (power supply device 51 in this embodiment) and a cooling device (cooling device 52 in this embodiment). The power supply unit supplies power to the robot device. The cooling device cools the robot device.

[0090] In this way, the information processing system provides power and cooling to the robotic device, enabling the robot to train a machine learning model without transmitting environmental information or other data acquired externally, and without requiring a large battery.

[0091] The robot device also includes a cooling control unit (a cooling control unit 136 in this embodiment). The cooling control unit controls the cooling of the robot device in response to temperature changes in the robot device caused by the learning of a machine learning model.

[0092] In this way, the information processing system can suppress the temperature rise of the robotic device caused by the learning of the machine learning model.

[0093] Furthermore, the cooling control unit controls the cooling of the robot device if its temperature exceeds a threshold.

[0094] In this way, the information processing system can suppress the rise in temperature of the robot device by cooling the overheated robot device, and can also save energy by not cooling it when the temperature is below a threshold.

[0095] Furthermore, the cooling control unit controls the coolant supplied from the cooling system in response to temperature changes in the robot device caused by the learning of the machine learning model. The cooling system cools the robot device by supplying coolant to it.

[0096] In this way, the information processing system can suppress the temperature rise of the robot device caused by the learning of the machine learning model. Furthermore, the information processing system can provide rapid charging to the robot device by simultaneously cooling the battery mounted on the robot device and the charging cable used for power supply.

[0097] Furthermore, the robotic device is further equipped with a Peltier element, and the cooling device is either a heat sink or a cooling fan.

[0098] Thus, even if a robotic device is equipped with a Peltier element, which is small, has no moving parts, and is easy to control the temperature of, the information processing system can still dissipate heat through a cooling device, even if the Peltier element has poor power efficiency as a cooling mechanism. Therefore, the robotic device can be cooled, and machine learning models that generate heat through calculations can be trained.

[0099] Furthermore, the cooling system is a water tank that holds a cooling liquid, and the robotic device is immersed in the water tank.

[0100] Thus, by using a water tank as a cooling device, the heat dissipation design for robotic devices can be simplified. Furthermore, by using a water tank to hold the coolant, the design of the information processing system can be simplified and costs can be reduced.

[0101] Furthermore, robot information is at least one of the following: user operation information to the robot device, or information obtained from various devices equipped on the robot device through the actions of the robot device itself. User operation information to the robot device includes at least one of user operations on buttons attached to the robot device and user verbal instructions to the robot device. Information obtained from various devices equipped on the robot device through the actions of the robot device itself includes at least one of the following: the angle of the robot arm joints, a video image of the surroundings taken from a camera, the distance traveled obtained from an encoder, the speed obtained from an encoder, the acceleration obtained from an IMU (inertial measurement unit), the angular velocity obtained from an IMU, and the power consumption of the battery.

[0102] In this way, information processing systems can learn machine learning models based on various types of information that are useful for improving the performance of robotic devices.

[0103] (5. Hardware Configuration) The robot device 100 and the like according to the embodiments of this disclosure described above are realized by a computer 1000 having a configuration such as that shown in Figure 5. The robot device 100 will be used as an example for explanation. Figure 5 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the robot device 100. The computer 1000 has a processing circuitry 1100, RAM 1200, ROM (Read Only Memory) 1300, secondary storage device 1400, communication interface 1500, input / output interface 1600, display unit 1700, camera unit 1800, microphone 1900, and speaker 2000. The parts of the computer 1000 are connected by a bus 1050.

[0104] The processing circuit 1100 operates based on a program stored in the ROM 1300 or secondary storage device 1400, and controls each part. For example, the processing circuit 1100 loads the program stored in the ROM 1300 or secondary storage device 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0105] ROM 1300 stores boot programs such as the BIOS (Basic Input Output System) that are executed by the processing circuit 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0106] The secondary storage device 1400 is a computer-readable recording medium that non-temporarily records programs executed by the processing circuit 1100 and data used by such programs. Specifically, the secondary storage device 1400 is a recording medium that records programs for each process of the robot device 100 according to the embodiment of this disclosure, which is an example of program data 1450.

[0107] The communication interface 1500 is an interface for the computer 1000 to connect to the external network 1550. The communication interface 1500 corresponds to the communication unit 110 of the robot device 100. For example, the processing circuit 1100 receives data from other devices or transmits data generated by the processing circuit 1100 to other devices via the communication interface 1500.

[0108] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the processing circuit 1100 receives data from input devices such as a microphone 1900 or a touch panel via the input / output interface 1600. The processing circuit 1100 also transmits data to output devices such as a display unit 1700 or a speaker 2000 via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium (media). Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

[0109] The display unit 1700 is an interface for displaying information processed by the computer 1000. The display unit 1700 is, for example, a liquid crystal display or an organic electroluminescent display (Organic Electro Luminescence Display). Alternatively, the display unit 1700 may be a touch panel display device or an image projection device.

[0110] The camera unit 1800 is an interface for the computer 1000 to capture images. The microphone 1900 is an interface for the computer 1000 to capture sound. The speaker 2000 is an interface for the computer 1000 to output processed sound. The various parts of the computer 1000 are connected by the bus 1050. Each interface does not necessarily have to be located inside the computer 1000, but may be located outside the computer 1000 via a network or the like. Furthermore, each part of the computer 1000 may be controlled by a circuit different from the processing circuit 1100. For example, the display unit 1700 may be controlled not by the processing circuit 1100, but by a circuit dedicated to display processing provided within the display unit 1700.

[0111] For example, when the computer 1000 functions as a robot device 100 according to an embodiment of this disclosure, the processing circuit 1100 of the computer 1000 functions as a control unit 130 by executing a program loaded onto the RAM 1200. The secondary storage device 1400 stores the information processing program according to this disclosure and various data stored by the storage unit 120. The processing circuit 1100 reads and executes the program data 1450 from the secondary storage device 1400, but as another example, these programs may be obtained from other devices via an external network 1550. In other words, the secondary storage device 1400 is not limited to being inside the computer 1000, but may be located outside the computer 1000. The processing circuit 1100 is an example of an integrated circuit, and CPU, MPU, GPU, APU (Accelerated Processing Unit), ASIC, and FPGA can all be considered integrated circuits.

[0112] Furthermore, this technology can also take the following configurations: (1) An information processing system including a robot device, wherein the robot device comprises: a sensor for sensing the surrounding environment; an acquisition unit for acquiring robot information relating to the robot device and environmental information from the sensor; a detection unit for detecting whether or not the robot device is in a powered state; and, if the robot device is detected to be in a powered state, a learning unit for learning a machine learning model using the robot information and the environmental information acquired by the sensor. (2) The information processing system according to (1), wherein the acquisition unit further acquires the results of actions taken by the robot device based on the robot information and the environmental information acquired by the sensor, and the learning unit, if the robot device is detected to be in a powered state, learns a machine learning model using the robot information, the environmental information acquired by the sensor, and the results of actions taken by the robot device. (3) The information processing system according to (2), wherein the acquisition unit acquires the result of movement as a result of the action, and the learning unit, when it is detected that the robot device is in a power supply state, learns a machine learning model using the robot information, the environmental information acquired by the sensor, and the result of movement of the robot device. (4) The information processing system according to (3), wherein the robot device further comprises an action control unit that controls the movement of the robot device, and an environmental information storage unit that stores environmental information acquired from the sensor provided by the robot device as a result of the movement of the robot device, the robot information, and the result of movement. (5) The information processing system according to (4), wherein the action control unit controls the movement of the robot device based on the learned machine learning model. (6) The information processing system according to any one of (1) to (5), wherein the information processing system further includes a power supply device, and the power supply device includes a power supply device that supplies power to the robot device, and a cooling device that cools the robot device.(7) The information processing system according to (6), wherein the robot device further comprises a cooling control unit that controls the cooling of the robot device in accordance with the temperature change of the robot device due to the learning of the machine learning model. (8) The information processing system according to (7), wherein the cooling control unit controls the cooling of the robot device when the temperature of the robot device is above a threshold. (9) The information processing system according to (7), wherein the cooling control unit controls the coolant supplied from the cooling device in accordance with the temperature change of the robot device due to the learning of the machine learning model, and the cooling device supplies coolant to the robot device to cool the robot device. (10) The information processing system according to (7) or (8), wherein the robot device further comprises a Peltier element, and the cooling device is a heat sink or a cooling fan. (11) The information processing system according to any one of (6) to (10), wherein the cooling device is a water tank that holds coolant, and the robot device is immersed in the water tank. (12) The robot information is at least one of user operation information to the robot device or information obtained from various devices provided by the robot device through the actions of the robot device itself, the user operation information to the robot device includes at least one of user operation to buttons attached to the robot device and user verbal instructions to the robot device, and the information obtained from various devices provided by the robot device through the actions of the robot device itself includes at least one of the angles of the joints of the robot arm, moving images of the surroundings taken from a camera, travel distance taken from an encoder, speed taken from an encoder, acceleration taken from an IMU (inertial measurement unit), angular velocity taken from an IMU, and power consumption of the battery, the information processing system according to any one of (1) to (11) above.(13) An information processing method to be performed by an information processing system including a robot device, the robot device acquiring robot information relating to the robot device and environmental information from a sensor that senses the surrounding environment, detecting whether or not the robot device is in a powered state, and if the robot device is detected to be in a powered state, training a machine learning model using the robot information and the environmental information acquired by the sensor. (14) An information processing program to be performed by an information processing system including a robot device, the program to cause the robot device to function as an information processing system comprising: an acquisition unit that acquires robot information relating to the robot device and environmental information from a sensor that senses the surrounding environment; a detection unit that detects whether or not the robot device is in a powered state; and a learning unit that, if the robot device is detected to be in a powered state, trains a machine learning model using the robot information and the environmental information acquired by the sensor.

[0113] 1 Information processing system 50 Power supply device 51 Power supply device 52 Cooling device 100 Robot device 110 Communication unit 120 Memory unit 121 Environmental information memory unit 122 Machine learning model memory unit 130 Control unit 131 Sensor 132 Acquisition unit 133 Detection unit 134 Learning unit 135 Motion control unit 136 Cooling control unit

Claims

1. An information processing system including a robot device, wherein the robot device comprises: a sensor for sensing the surrounding environment; an acquisition unit for acquiring robot information relating to the robot device and environmental information from the sensor; a detection unit for detecting whether or not the robot device is in a powered state; and, if the robot device is detected to be in a powered state, a learning unit for learning a machine learning model using the robot information and the environmental information acquired by the sensor.

2. The information processing system according to claim 1, wherein the acquisition unit further acquires the results of the actions of the robot device based on the robot information and the environmental information acquired by the sensor, and the learning unit, when it is detected that the robot device is in a power supply state, learns a machine learning model using the robot information, the environmental information acquired by the sensor, and the results of the actions of the robot device.

3. The information processing system according to claim 2, wherein the acquisition unit acquires the result of movement as a result of the action, and the learning unit, when it is detected that the robot device is in a power supply state, learns a machine learning model using the robot information, the environmental information acquired by the sensor, and the result of movement of the robot device.

4. The information processing system according to claim 3, further comprising: an action control unit that controls the movement of the robot device; and an environmental information storage unit that stores environmental information acquired from sensors provided by the robot device as a result of the movement of the robot device, the robot information, and the result of the movement.

5. The information processing system according to claim 4, wherein the motion control unit controls the movement of the robot device based on the learned machine learning model.

6. The information processing system according to claim 1, further comprising a power supply device, wherein the power supply device comprises a power supply device for supplying power to the robot device and a cooling device for cooling the robot device.

7. The information processing system according to claim 6, further comprising a cooling control unit that controls the cooling of the robot device in accordance with the temperature change of the robot device due to the learning of the machine learning model.

8. The information processing system according to claim 7, wherein the cooling control unit controls the cooling of the robot device when the temperature of the robot device is above a threshold.

9. The information processing system according to claim 7, wherein the cooling control unit controls the coolant supplied from the cooling device in accordance with the temperature change of the robot device due to the learning of the machine learning model, and the cooling device supplies the coolant to the robot device to cool the robot device.

10. The robotic device further comprises a Peltier element, and the cooling device is a heat sink or a cooling fan, the information processing system according to claim 6.

11. The information processing system according to claim 6, wherein the cooling device is a water tank that holds a cooling liquid, and the robot device is immersed in the water tank.

12. The robot information is at least one of user operation information to the robot device or information obtained from various devices provided by the robot device through the actions of the robot device itself, the user operation information to the robot device includes at least one of user operation to buttons attached to the robot device and user verbal instructions to the robot device, and the information obtained from various devices provided by the robot device through the actions of the robot device itself includes at least one of the angles of the joints of the robot arm, moving images of the surroundings obtained from a camera, travel distance obtained from an encoder, speed obtained from an encoder, acceleration obtained from an IMU (inertial measurement unit), angular velocity obtained from an IMU, and power consumption of the battery, the information processing system according to claim 2.

13. An information processing method to be performed by an information processing system including a robot device, the method comprising: the robot device acquiring robot information relating to the robot device and environmental information from sensors that sense the surrounding environment; detecting whether or not the robot device is in a powered state; and, if it is detected that the robot device is in a powered state, training a machine learning model using the robot information and the environmental information acquired by the sensors.

14. An information processing program to be executed in an information processing system including a robot device, the program to cause the robot device to function as an information processing system comprising: an acquisition unit that acquires robot information relating to the robot device and environmental information from sensors that sense the surrounding environment; a detection unit that detects whether or not the robot device is in a powered state; and, if the robot device is detected to be in a powered state, a learning unit that learns a machine learning model using the robot information and the environmental information acquired by the sensors.

Images