System

The system addresses the challenge of maintaining automated scientific experiment systems by using a management server and robot with sensors and manipulators to autonomously adjust and maintain the robot's state, enhancing operational efficiency and availability through reinforcement learning and predictive maintenance.

WO2024237329A1PCT designated stage expired Publication Date: 2024-11-21RIKEN CO LTD

Patent Information

Application Number
PCT/JP2024/018261
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-05-02
Filing Date
2024-05-17
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Automated systems used for scientific experiments are difficult to maintain and manage, lacking self-maintenance capabilities.

Method used

A system comprising a management server and a robot equipped with sensors and a manipulator, where the management server generates instructions based on environmental data to guide the robot in identifying and adjusting its state to match a desired configuration, using reinforcement learning to optimize control actions and maintain an ideal operational state.

Benefits of technology

Enables autonomous maintenance of the robot's state and environment, improving system availability by autonomously correcting deviations and optimizing operations, while also predicting and preparing for future missions based on past data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024018261_21112024_PF_FP_ABST
    Figure JP2024018261_21112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided is a system comprising a management server, a robot, and a sensor that measures an environment including a space of an action range of the robot. The management server is provided with: a reception unit that receives a mission; and a command unit that generates a command on the basis of the mission and transmits the command to the robot. The command includes a first command representing a state after execution of the mission. The robot is provided with: a state specification unit that specifies an environmental state of the robot including a state of an article handled by the robot on the basis of measurement data of the sensor; and an instruction unit that receives the command and drives the robot so that the state of the robot approaches a state represented by the command on the basis of the difference between the specified state and the state represented by the command.
Need to check novelty before this filing date? Find Prior Art

Description

system

[0001] This application claims priority to Japanese Patent Application No. 2023-081712 filed in Japan on May 17, 2023, and Japanese Patent Application No. 2024-074900 filed in Japan on May 2, 2024, the contents of which are incorporated herein by reference.

[0002] Research and development is underway to automate tasks such as scientific experiments.

[0003] US Patent Application Publication No. 2015 / 242395

[0004] An object of the present invention is to solve the above-mentioned problems and to provide a system having self-maintenance capabilities.

[0005] One aspect of the present invention is a system comprising a management server, a robot, and a sensor that measures an environment including a space within the robot's range of movement, wherein the management server comprises a reception unit that receives missions, and a command unit that generates commands based on the mission and transmits the commands to the robot, the commands including a first command that represents the state after the mission is executed, and the robot comprises a state identification unit that identifies the environmental state of the robot including the state of an item handled by the robot based on measurement data from the sensor, and an instruction unit that receives the command and drives the robot so that the state of the robot approaches the state represented by the command based on the difference between the identified state and the state represented by the command.

[0006] The present invention provides a system that has self-maintenance capabilities.

[0007] 1 is a diagram showing the configuration of a system 1 according to a first embodiment. FIG. 2 is a diagram showing the configuration of a robot 21. FIG. 3 is a diagram showing an example of the configuration of a control unit 25. FIG. 4 is a diagram showing the configuration of a management server 11. FIG. 5 is a flowchart showing the operation of the management server 11. FIG. 6 is a flowchart showing the operation of the management server 11 in step S12 and the operation of the robot 21 at that time. FIG. 7 is a flowchart showing the operation of the management server 11 in step S13 and the operation of the robot 21 at that time. FIG. 8 is a diagram showing an example of an implementation of the system 1. FIG. 9 is a diagram showing the relationship between a camera, a computer, an actuator, and installation space.

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First Embodiment Fig. 1 is a diagram showing the configuration of a system 1 according to a first embodiment. The system 1 includes a management server 11 and a robot 21. The management server 11 accepts a mission from a user U. The mission includes, for example, the content of an experiment to be performed by the robot 21. The management server 11 acquires data relating to a predetermined range of movement 2 of the robot 21 from the robot 21. The predetermined range of movement 2 is, for example, a laboratory where the robot 21 performs experimental work. The management server 11 generates commands and issues instructions to the robot 21. The robot 21 drives in the predetermined range of movement 2 in accordance with the commands from the management server 11.

[0009] 2 is a diagram showing the configuration of the robot 21. The robot 21 includes a running device 22, a manipulator 23, a sensor 24, a control unit 25, and a communication unit 26. The running device 22 is provided at the bottom of the robot 21 and supports the entire robot 21. The running device 22 is, for example, an endless track that moves the robot 21 within the movement range 2 by rotating. The manipulator 23 grasps fixtures within the movement range 2. For example, servo motors are provided at each joint of the manipulator 23, and the joint angles are controlled. The manipulator 23 feeds back information regarding the joint angles and whether or not it is grasping to the control unit 25. The manipulator 23 may be provided with a sensor in the grasping portion to determine whether or not it is grasping an object and provide feedback.

[0010] The sensor 24 senses the surroundings of the robot 21 and acquires sensing data. The sensor 24 is, for example, a camera, and captures an image of at least the area in front of the robot 21. The sensing data includes the image captured by the camera. The sensor 24 is, for example, a LiDAR, and measures three-dimensional space. The sensing data includes data of the three-dimensional space measured by the LiDAR.

[0011] The control unit 25 controls the traveling device 22 and the manipulator 23 based on sensing data from the sensors 24 and commands received from the management server 11. Fig. 3 is a diagram showing an example of the configuration of the control unit 25. The control unit 25 includes a state identification unit 251, a control amount calculation unit 252, and an instruction unit 253. The state identification unit 251 identifies the self-state and environmental state of the robot 21 based on measurement data from the sensors 24. The self-state of the robot 21 indicates the state of the robot 21. The environmental state of the robot 21 indicates the state of the surrounding environment of the robot 21 and the state of equipment used by the robot 21.

[0012] The state identification unit 251 recognizes equipment such as test tubes by analyzing images. Image analysis is realized, for example, by a machine learning object recognition model. The state identification unit 251 generates a map using SLAM based on measurement data obtained by LiDAR. The state identification unit 251 identifies the three-dimensional position and three-dimensional orientation of the equipment based on the map generated by SLAM. The state identification unit 251 obtains information on whether an object is being grasped by feedback from the manipulator. In recognizing equipment through image analysis, the number of equipment items may be identified by an object recognition model. Furthermore, the volume of equipment such as liquid is detected using a dedicated trained model. The model for detecting the volume of equipment is trained to calculate volume from an image using, for example, a training dataset consisting of a combination of images of a container containing liquid and the volume of the liquid.

[0013] The control amount calculation unit 252 calculates the control amounts of the running device 22 and the manipulator 23 based on the difference between the self-state and environmental state of the robot 21 identified by the state identification unit 251 and the state indicated by the command from the management server 11. The control amount of the running device 22 is, for example, the running distance, the number of rotations, and the running direction. The control amount of the manipulator 23 is, for example, the joint angle. The calculation by the control amount calculation unit 252 is realized by reinforcement learning. More specifically, the difference between the self-state and environmental state of the robot 21 identified by the state identification unit 251 and the state indicated by the command from the management server 11 is used as a reward to calculate the behavior of the robot 21 in the self-state and environmental state identified by the state identification unit 251. Here, the behavior is the control amount of the running device 22 and the manipulator 23. As described above, the control amount calculation unit controls the running device 22 and the manipulator 23 so as to reduce the difference between the self-state and environmental state of the robot 21 at that time and the state indicated by the command, and therefore can make adjustments even when a disturbance occurs.

[0014] The instruction unit 253 instructs the travel device 22 and the manipulator 23 on the amount of control. The travel device 22 and the manipulator 23 receive the instruction and perform movement, rotation, etc.

[0015] The control amount calculation unit 252 may calculate the control amounts of the traveling device 22 and the manipulator 23 even when there is no command from the management server 11. In this case, the control amount calculation unit 252 calculates the control amounts of the traveling device 22 and the manipulator 23 based on an ideal state. The ideal state is a state in which fixtures are in standard positions and have standard remaining amounts. This allows the robot 21 to autonomously maintain its own state and the environmental state of the robot 21 in the ideal state.

[0016] The communication unit 26 receives commands from the management server 11. The communication unit 26 notifies the management server 11 of the robot 21's own state and the environmental state.

[0017] FIG. 4 is a diagram showing the configuration of the management server 11. The management server 11 includes a reception unit 111, a communication unit 112, and a calculation unit 113. The reception unit 111 receives missions from a user U. The missions are experimental instructions to be performed by the robot 21. The experimental instructions are expressed as changes in the state of the system 1. The experimental instructions include information on the substance contained in the test tube (e.g., test tube X contains 1 mg of reagent A), the position of the test tube (e.g., test tube X is placed on rack Y), and the process to be applied to the test tube (e.g., test tube X is heated for 100 seconds, or the contents of test tube X are analyzed using the Z method). The reception unit 111 receives the missions via an input device such as a keyboard or via wireless communication using the communication unit 112, which will be described later. The missions also include a product delivery instruction to have the robot 21 send the experimental product.

[0018] The communication unit 112 transmits and receives data to and from the robots 21. The communication unit 112 receives the self-state and environmental state of each robot 21 from each robot 21. The self-state of the robot 21 includes, for example, the position and working state of the robot 21. The environmental state of the robot 21 includes, for example, the position and remaining amount of equipment around the robot 21. The communication unit 112 may also receive data including mission information from another communication device.

[0019] The calculation unit 113 includes a state recognition unit 114, an ordering unit 115, an ideal state estimation unit 116, and a command unit 117. The state recognition unit 114 recognizes the state of the system 1 based on the self-state and environmental state of the robot 21 acquired from each robot 21. Note that the state recognition unit 114 recognizes the state of the system 1 by receiving the results of image analysis and the like as the self-state and environmental state of the robot 21, but the environmental state of the robot 21 may also be acquired by acquiring raw data such as images from the robot 21 and analyzing the images. In this case, the state identification unit 251 of the robot 21 does not need to perform processing such as image analysis.

[0020] The state recognition unit 114 may determine whether or not the command has been completed based on the state of the system 1 after sending the command to the robot 21.

[0021] The ordering unit 115 orders supplies based on the state of the system 1 recognized by the state recognition unit 114. For example, the ordering unit 115 orders supplies when the remaining amount of supplies is low in the recognized state of the system 1. The ordering unit 115 also orders repairs to the robot 21. For example, when the rate at which the robot 21 completes commands is equal to or less than a predetermined value, the ordering unit 115 determines that the robot 21 needs repairs and orders the repairs. For example, when the difference between the robot 21's own state and environmental state and the state indicated by the command does not decrease and is equal to or greater than a predetermined value, the ordering unit 115 determines that the robot 21 needs repairs and orders the repairs.

[0022] The ideal state estimation unit 116 estimates the ideal state of the robot 21 in the movement range 2 based on the history of past missions. The ideal state estimation unit 116 predicts a mission that will be performed in the future based on the history of past missions, for example, and performs a simulation of the predicted future mission being performed in a plurality of different states, thereby estimating an optimal state from the plurality of different states. The ideal state estimation unit 116 predicts a mission that will be performed in the future by, for example, recognizing patterns of past missions.

[0023] The command unit 117 generates commands for controlling the robots 21. The commands include a first command representing the state after the mission is executed and a second command representing an ideal state of the system 1. The command unit 117 transmits the generated commands to each robot 21. The command unit 117 generates the first command based on the mission accepted by the acceptance unit 111 and the state of the system 1 recognized by the state recognition unit 114. The command unit 117 identifies a robot 21 that can implement each experimental instruction included in the mission and assigns each experimental instruction to the identified robot 21. When a mission is performed by multiple robots 21, the mission may include an instruction for the robot to hand over equipment. The command unit 117 generates a second command based on the ideal state estimated by the ideal state estimation unit 116 and the state of the system 1 recognized by the state recognition unit 114. The second command includes, for example, an instruction for the robot 21 to move the equipment from an inventory area where the equipment is placed to a predetermined location.

[0024] 5 is a flowchart showing the operation of the management server 11. If the reception unit 111 has received a mission (step S11: YES), the management server 11 performs an operation to complete the mission (step S12). If the reception unit 111 has not received a mission (step S11: NO), the management server 11 performs an operation to maintain the ideal state (step S13).

[0025] FIG. 6 is a flowchart showing the operation of the management server 11 in step S12 and the operation of the robot 21 at that time. The command unit 117 generates a first command representing the state after the mission is executed (step S121). The command unit 117 then outputs the command to the robot 21 (step S122). Upon receiving the command, the control amount calculation unit 252 calculates the control amounts for the running device 22 and the manipulator 23, and the instruction unit 253 instructs the running device 22 and the manipulator 23 on the control amounts, thereby controlling the robot 21 in accordance with the command (step S221). After the command is completed, the communication unit 26 of the robot 21 notifies the management server 11 of the robot's own state and the environmental state after the command is completed (step S222). In the management server 11, the state recognition unit 114 recognizes the state of the robot 21 after the command is completed and outputs the results of the experiment (step S123).

[0026] 5 and 6, whether or not there is a mission is determined in step S11 for each mission, but if there is a mission, it may be possible to complete all of the missions together in step S12. In this case, the final experiment results may be output for all missions together.

[0027] 7 is a flowchart showing the operation of the management server 11 in step S13 and the operation of the robot 21 at that time. The ideal state estimation unit 116 estimates the ideal state of the robot 21 (step S131). The command unit 117 generates a second command and outputs it to the robot 21 (step S132). The robot 21 receives the command, the control amount calculation unit 252 calculates the control amounts of the traveling device 22 and the manipulator 23 based on the ideal state, and the instruction unit 253 instructs the traveling device 22 and the manipulator 23 on the control amounts (step S231). The ordering unit 115 of the management server 11 orders equipment, if necessary, based on the state of the system 1 (step S133).

[0028] As described above, according to this embodiment, the management server 11 accepts a mission, generates a command based on the mission, and transmits it to the robot 21. The robot 21 identifies the environmental state of the robot 21, receives the command, and drives the robot so that its state approaches the state represented by the command. This allows the robot 21 to complete the mission even when there is a change in the environment or a disturbance.

[0029] Furthermore, when the robot 21 has not received the first command indicating the state after the mission has been executed, the robot 21 is driven so that the difference between the state of the robot 21 and the ideal state of the robot 21 satisfies a predetermined requirement, thereby keeping the robot 21 in the ideal state.

[0030] Furthermore, by transmitting a second command representing the ideal state from the management server 11 to the robot 21, the ideal state can be changed appropriately from the management server 11 side. This makes it possible to increase the availability of the system 1.

[0031] Furthermore, the robot 21 can perform optimal driving by determining the amount of driving using a reinforcement learning algorithm that uses a reward based on the difference between the state of the robot 21 and the state represented by the command.

[0032] Furthermore, the management server 11 can prevent shortages of items and increase availability by ordering the replenishment of items that are lacking in the robot's state according to the command. Furthermore, the robot 21 can automatically replenish items and increase availability by comparing the data of the items that have arrived in response to the order with the order contents and managing the items.

[0033] Furthermore, the management server 11 can automatically bring the ideal state closer to the optimal state by estimating the state in which a future mission can be executed based on missions that have been executed in the past as the ideal state.

[0034] Fig. 8 is a diagram showing an example of an implementation of the system 1. In the example shown in Fig. 8, the system 1 includes a camera, an actuator, and a computer. The camera corresponds to the sensor 24 in the embodiment. The actuator corresponds to the traveling device 22 and the manipulator 23 in the embodiment. The computer corresponds to the control unit 25 and the management server 11 in the embodiment.

[0035] 9 is a diagram showing the relationship between the camera, computer, actuator, and installation space. The camera captures the installation space of the appliance. The installation space of the appliance corresponds to movement range 2 in the embodiment. Disturbances in the installation space of the appliance include the replenishment of appliances and the disposal of appliances. If the installation space is a shared space shared with other systems, disturbances from the other systems may also be present.

[0036] The image captured by the camera is sent to a computer. The computer uses an object detection model to detect the appliances placed in the appliance installation space. The object detection model used is the YOLOv5 model. This detection means corresponds to the operation of the state identification unit 251.

[0037] When a question about the location of an appliance is input to a computer, the computer returns the location of the appliance. In the example shown in Figure 8, the computer used a REST API and the YOLOv5 model returned the location of the appliance.

[0038] When an instruction for an actuator is input to the computer, the computer controls the actuator according to the instruction. In the example shown in Figure 8, the computer uses the API of ChatGPT to control the actuator based on the input prompt.

[0039] The actuator, when controlled by a computer, moves the instruments placed in the installation space, thereby allowing the instruments placed in the installation space to be moved.

[0040] Other Embodiments One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like can be made within the scope that does not deviate from the gist of the present invention.

[0041] The present invention provides a system that has self-maintenance capabilities.

[0042] 1 System, 11 Management Server, 111 Reception Unit, 112 Communication Unit, 113 Calculation Unit, 114 State Recognition Unit, 115 Ordering Unit, 116 Ideal State Estimation Unit, 117 Command Unit, 21 Robot, 22 Traveling Device, 23 Manipulator, 24 Sensor, 25 Control Unit, 251 State Identification Unit, 252 Control Amount Calculation Unit, 253 Instruction Unit, 26 Communication Unit

Claims

1. A system comprising a management server, a robot, and a sensor that measures an environment including a space within the robot's range of movement, wherein the management server comprises: a reception unit that receives a mission; and a command unit that generates commands based on the mission and transmits the commands to the robot, wherein the commands include a first command that represents a state after the mission is executed, and the robot comprises: a state identification unit that identifies an environmental state of the robot including a state of an item handled by the robot based on measurement data of the sensor; and an instruction unit that receives the command and drives the robot so that the state of the robot approaches the state represented by the command based on the difference between the identified state and the state represented by the command.

2. The system described in claim 1, wherein the instruction unit drives the robot so that the difference between the robot's state and the state represented by the first command satisfies a specified requirement, and when the first command is not received, drives the robot so that the difference between the robot's state and an ideal state of the robot satisfies a specified requirement.

3. The system according to claim 2, wherein the robot receives a second command representing an ideal state of the robot from the management server.

4. The system according to claim 1, further comprising: a control amount calculation unit for determining a drive amount by a reinforcement learning algorithm using a reward based on the difference between the state of the robot and the state represented by the command.

5. The system according to claim 1, wherein the management server includes an ordering unit that places an order for replenishing items that are in short supply in the state of the robot according to the state indicated by the command.

6. The system according to claim 5, wherein the status determination unit compares data on the items delivered in response to the order measured by the sensor with the contents of the order placed by the ordering unit, and manages the items.

7. The system according to claim 2, wherein the management server comprises an ideal state estimation unit that estimates, as the ideal state, a state in which a mission to be generated in the future can be executed based on the missions executed in the past.

Citation Information

Patent Citations

  • Robot system, controlling method and record medium

    JP1999231910A

  • Learning device, robot control method, and robot control system

    JP2020082314A

  • Systems, devices, and methods for robotic learning and execution of skills

    JP2021534988A

  • Autonomous locating system, operation terminal, mobile locating device, and locating control method

    WO2015060182A1

  • Autonomous mobile object, information processing method, program, and information processing device

    WO2021085175A1

Cited By

  • Information processing device, information processing method, and program

    WO2026210716A1