Information processing system and information processing method
The information processing system enables real-time correction of automated machinery operations by using a scenario program with multiple code programs, addressing the challenge of maintaining consistent product quality during disturbances.
Patent Information
- Application Number
- PCT/JP2025/017026
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2026-11-12
AI Technical Summary
Existing systems for automated machinery, such as industrial robots and machines in regenerative medicine, struggle to maintain consistent product quality when disturbances occur, requiring manual correction that can be time-consuming and risk producing products with varying performance or characteristics.
An information processing system that includes a control device controlling an operating device based on a scenario program composed of multiple code programs for each processing step, allowing real-time correction of automated machinery operations through user input and sensor feedback.
Facilitates real-time correction of automated machinery operations, ensuring consistent product quality by enabling immediate adjustments to speed, path, and position, thereby reducing the time required for manual intervention.
Smart Images

Figure JP2025017026_12112026_PF_FP_ABST
Abstract
Description
Information Processing System and Information Processing Method
[0001] The present invention relates to an information processing system and an information processing method.
[0002] Conventionally, predetermined repetitive tasks have been automatically performed by machines such as industrial robots at manufacturing sites and the like. Automation by industrial robots and the like is suitable, for example, for manufacturing the same product by performing repetitive operations exactly as designed in advance. However, if a state different from the assumed state occurs during automatic operation, it may not be possible to manufacture the same product through the repetitive operations.
[0003] Also in the life science industry, automation by machines in regenerative medicine, cell manufacturing and the like is being promoted. In processes such as cell manufacturing in life science, it may not be possible to manufacture the same product through repetitive operations due to the occurrence of disturbances or disruptions in cell manufacturing. Disturbances in cell manufacturing include, for example, disturbances from the culture medium environment (deterioration caused by the environment surrounding the cells), disturbances caused by operations (deterioration of cell characteristics caused by actions and operations), and disturbances originating from the cells themselves (deterioration of cell characteristics caused by cell lifespan).
[0004] Accordingly, there are existing systems that allow the operation of a machine under automatic operation to be supplemented or corrected by manual operation. Patent Document 1 describes controlling the operation of a robot main body in an automatic operation mode in which the robot main body is operated according to a program for predetermined operations, and a correction operation mode in which the operation of the robot main body in the automatic operation mode is corrected by a command input from a manual operation input device.
[0005] Japanese Unexamined Patent Application Publication No. 2022-119943
[0006] In the invention described in Patent Document 1, while it is possible to correct the automatic operation by manual operation by an operator, manual correction must be continued until the automatic operation returns to the predetermined trajectory. For this reason, it is thought that correcting the automatic operation by manual operation takes a considerable amount of time, and depending on the product being manufactured, there is a risk that it may not be possible to manufacture products with the same performance or characteristics. For example, in the production of cells in life sciences, time is an important requirement, and if it takes time to correct the operation of machinery such as robotic devices, there is a risk that it may not be possible to manufacture cells with appropriate characteristics.
[0007] This invention has been made in view of these circumstances, and aims to facilitate real-time correction of the operation of automated machinery by an operator's manual control.
[0008] To solve the above problems, the following means are employed. The present invention is an information processing system comprising, for example, an operating device that holds a processing device for processing an object and operates the processing device, and a control device that controls the operation of the operating device based on a scenario program that defines the continuous operation of the operating device for each processing step, wherein the scenario program corresponding to each processing step is composed of a plurality of code programs for predetermined time intervals, and the control device controls the continuous operation of the operating device for each processing step by causing the plurality of code programs to execute continuously as the scenario program.
[0009] The mode of disclosure may be realized by the execution of a program by an information processing device. That is, the configuration of the disclosure can be identified as a program that causes an information processing device to execute the processing performed by each of the means in the above mode, or as a recording medium readable by the information processing device that stores the program. Alternatively, the configuration of the disclosure may be identified as a method by which the information processing device executes the processing performed by each of the means described above. Alternatively, the configuration of the disclosure may be identified as a system including an information processing device that performs the processing performed by each of the means described above. The information processing device is, for example, a computer. A computer is sometimes called a personal computer or a server.
[0010] According to the present invention, it is possible to provide a technology that facilitates real-time correction of the operation of automated machinery by an operator's manual control.
[0011] This is a diagram showing an example of the configuration of an information processing system according to an embodiment. This is a diagram showing an example of the configuration of an information processing system according to an embodiment. This is a diagram showing an example of the hardware configuration of an information processing device. This is a diagram illustrating an example of the configuration of a task program which is a scenario. This is a diagram illustrating an example of the configuration of a task program which is a scenario. This is a sequence diagram showing an example of the execution procedure of a task program. This is a sequence diagram showing an example of the implementation procedure of real-time correction.
[0012] Embodiments of the present invention will be described below with reference to the drawings. The configurations of the embodiments are illustrative, and the configuration of the invention is not limited to the specific configurations of the disclosed embodiments. In carrying out the invention, specific configurations depending on the embodiment may be adopted as appropriate.
[0013] The information processing system according to the present invention comprises an operating device (also called an operating unit) which is the object to be controlled, and a control device which controls the operation of the operating device. The system processes information for controlling the operation of the operating device based on user input and detection results from various sensors. More specifically, the operating device, which is the object to be controlled, is a device that holds a processing instrument for processing an object (object to be processed) as the object to be operated, and operates the held processing instrument (object to be operated). This operating device can also be called an automation device for automating processing of an object. The information processing system comprising the operating device and the control device can also be called an automation system.
[0014] [System Configuration Example] Figure 1 is a block diagram showing an example configuration of the information processing system of the embodiment, and Figure 2 is a schematic diagram showing an example configuration of the information system of the embodiment.
[0015] As shown in Figures 1 and 2, the information processing system 10 includes a control device 100, a terminal device 200, a robot device 300 to be controlled, and a detection device 400. The control device 100 includes a control unit 102, a storage unit 104, and a communication unit 106. The terminal device 200 includes an input unit 202 and a display unit 204. The control device 100 is connected to the terminal device 200, the robot device 300, and the detection device 400 in a communication manner.
[0016] The information processing system 10, in the control device 100, creates a scenario to be executed by the robot device 300, which acts on the target device, based on user input to terminal devices 200, etc. In other words, the information processing system 10 creates a scenario to be executed by the robot device 300, which operates the target device. The scenario is a task program that shows the procedure for operation by the robot device 300. The control device 100 operates the robot device 300 based on the created scenario.
[0017] The control device 100 accepts user input for correction of operation from the terminal device 200 while the robot device 300 is operating based on a scenario, and modifies the scenario. Examples of processes performed by the robot device 300 based on the scenario created by the control device 100 include cell production (cell culture) in life sciences. In this embodiment, as an example of cell culture processing, the robot device 300 uses an instrument to be operated to aspirate a liquid sample, move it, and discharge the liquid to a predetermined position. Another example of the operation of the robot device 300 is scraping in cell culture. The sample includes, for example, chemical substances, bacteria, cells, and other living organisms.
[0018] The control device 100 includes a control unit 102, a storage unit 104, and a communication unit 106. The control unit 102 controls the operation of terminal devices 200, robot devices 300, detection devices 400, etc. The storage unit 104 stores various data, programs, etc., used by the control device 100. The communication unit 106 is a communication interface for sending and receiving data with terminal devices 200, robot devices 300, detection devices 400, etc.
[0019] The terminal device 200 includes an input unit 202 and a display unit 204 as input / output means for receiving information input from the user and outputting information to the user. The terminal device 200 can be implemented as, for example, a tablet terminal, a personal computer, or a smartphone.
[0020] The input unit 202 is an input means that accepts information input from the user. The input unit 202 can be composed of, for example, a keyboard, a pointing device, a wireless remote control, a touch panel, a camera, a microphone, a joystick (also called a jog), a foot pedal, etc. The terminal device 200 may have multiple input units 202. In addition to input units composed of, for example, a keyboard or a touch panel, the terminal device 200 may also have an input unit composed of a joystick. This allows multiple operators to input multiple different pieces of information at the same time.
[0021] The display unit 204 is an output means that outputs information to the user, etc., and is a display means having a display screen. The display unit 204 is a display that displays text information, images, etc. In this example, the display screen of the display unit 204 is a touch panel and also serves as the input unit 202. The display unit 204 may also include a speaker that outputs sound, etc.
[0022] The robotic device 300 is a device that performs predetermined operations based on instructions from the control device 100. The robotic device 300 performs operations such as grasping, moving, and releasing the instrument to be operated. In this embodiment, the instrument to be operated is, for example, a well plate, petri dish, test tube, beaker, flask, container, pipette, etc. Of course, the instrument to be operated is not limited to these. The robotic device 300 operates the instrument to perform predetermined processing on the sample to be processed. For example, the robotic device 300 can operate a pipette, which is an example of an instrument to be operated, to aspirate a liquid such as a sample from one well of a well plate and discharge it into another well of the well plate. It should be noted that the well plate, etc., which contains the liquid such as a sample, can also be said to be the object to be processed using an instrument such as a pipette.
[0023] Furthermore, the robotic device 300 performs a scraping operation as part of the cell culture process, specifically the detachment process, by using a scraper to remove the cultured adherent cells from the petri dish. More specifically, the robotic device 300 operates the scraper, which is the processing instrument to be manipulated, and moves it along the surface of the petri dish while keeping the scraper in contact with the surface (bottom) of the petri dish, thereby performing a scraping operation to remove the adherent cells from the petri dish. In this scraping operation, the petri dish that the scraper contacts is considered to be included in the above-mentioned object along with the adherent cells.
[0024] Furthermore, when performing a scraping operation, the robot device 300 may be configured to hold the petri dish independently of the scraper and to adjust the position and orientation of the petri dish as appropriate. The robot device 300 may be configured to hold the scraper, which is a processing instrument, and the petri dish, which is the object, individually and operate them independently. The information processing system 10 may include multiple robot devices 300 that individually hold the scraper and the petri dish. In any case, the information processing system 10 is configured to perform a scraping operation by moving the scraper, which is the object to be operated, relative to the petri dish using the robot device 300. During the scraping operation, the robot device 300 may move or change the orientation of the petri dish. In this case, the petri dish used for cell culture is included in the object as described above, but it also corresponds to the object to be operated by the robot device 300. The petri dish does not necessarily have to be held by the robot device 300; for example, it may be fixed in a predetermined position such as a table 620.
[0025] The detection device 400 monitors the operation of the robot device 300. The detection device 400 includes sensors that detect the operation of the robot device 300 by visual means or the like. The detection device 400 includes, for example, a camera that photographs an area including the robot device 300 and the instrument to be operated by the robot device 300. The detection results from the detection device 400, such as the image captured by the camera, are output to the user from the terminal device 200. The detection device 400 may also include sensors that detect the state of the sample being processed.
[0026] Furthermore, as shown in Figure 2 as an example, the robot device 300 has a base placed on the floor. The robot device 300 also includes a rotatable joint connected to the base and an arm connected to the joint. Further joints and arms are connected to the end of the arm. A gripping part for gripping the instrument 610 is also connected to the tip of the arm. The robot device 300 moves and operates the instrument 610 using the joints, arm, gripping part, etc. The configuration of the robot device 300 is not particularly limited and is sufficient as long as it can hold various instruments 610 and operate the instruments 610 appropriately.
[0027] Multiple instruments 610, which are the targets of the robotic device 300, are placed on a table 620. The multiple instruments 610 include test tubes 611, pipettes 612, containers 613, well plates 614, flasks 615, etc., and when scraping is performed, scrapers, petri dishes, etc. are also included. The test tubes 611, pipettes 612, containers 613, etc., included in the instruments 610 are each operated by the robotic device 300. The robotic device 300, the multiple instruments 610, etc. are photographed by a detection device 400, which is a camera. The images captured by the camera of the detection device 400 are displayed on the display unit 204 of the terminal device 200, for example, via the control device 100.
[0028] These robotic devices 300, tables 620, instruments 610, etc., are placed in a so-called aseptic area. By placing the instruments 610, etc., in the aseptic area, contamination of samples with foreign matter can be suppressed. Other devices, such as devices for transporting the instruments 610, etc., may also be placed in the aseptic area. The terminal device 200 is located outside the aseptic area and can be operated by the user. The information processing system 10 can realize the aseptic area as a cleanliness control area that does not require human entry.
[0029] Incidentally, the control device 100 can be implemented using a dedicated or general-purpose computer such as a PC (Personal Computer) or workstation (WS), or an electronic device equipped with a computer. Alternatively, the terminal device 200 can be implemented using a dedicated or general-purpose computer such as a PC, smartphone, mobile phone, tablet terminal, or PDA (Personal Digital Assistant), or an electronic device equipped with a computer.
[0030] Figure 3 shows an example of the hardware configuration of an information processing device. The information processing device shown in Figure 3 has the configuration of a typical computer. The control device 100 and terminal device 200 are implemented by the information processing device 90 shown in Figure 3. The information processing device 90 shown in Figure 3 has a processor 91, memory 92, storage unit 93, input unit 94, output unit 95, and communication control unit 96. These are connected to each other by a bus. The memory 92 and storage unit 93 are computer-readable recording media. The hardware configuration of the computer that implements the control device 100 and terminal device 200 is not limited to the example shown in Figure 3, and components may be omitted, replaced, or added as appropriate.
[0031] The information processing device 90 can realize functions that match a predetermined purpose by having the processor 91 load a program stored on a recording medium into the working area of the memory 92 and execute it, thereby controlling each component through the execution of the program. The processor 91 is, for example, a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). The memory 92 includes, for example, RAM (Random Access Memory) or ROM (Read Only Memory). The memory 92 is also called the main memory.
[0032] The storage unit 93 may be, for example, an EPROM (Erasable Programmable ROM), a hard disk drive (HDD), or a solid state drive (SSD). The storage unit 93 may also include removable media, i.e., portable recording media. Removable media may be, for example, a USB (Universal Serial Bus) memory, or a disk recording medium such as a CD (Compact Disc) or DVD (Digital Versatile Disc). The storage unit 93 is also called a secondary storage device.
[0033] The storage unit 93 stores the operating system (OS), various programs, various tables, various data, etc. The information stored in the storage unit 93 may also be stored in the memory 92. Conversely, the information stored in the memory 92 may also be stored in the storage unit 93.
[0034] An operating system is software that acts as an intermediary between software and hardware, manages memory space, manages files, and manages processes and tasks. The operating system includes a communication interface. The communication interface is a program that exchanges data with other external devices connected via the communication control unit 96. These external devices include, for example, other computers and external storage devices.
[0035] The input unit 94 includes a keyboard, pointing device, wireless remote control, touch panel, etc. The input unit 94 may also include video or image input devices such as a camera, and audio input devices such as a microphone.
[0036] The output unit 95 includes display devices such as LCDs (Liquid Crystal Displays), EL (Electroluminescence) panels, CRT (Cathode Ray Tube) displays, and PDPs (Plasma Display Panels), as well as output devices such as printers. The output unit 95 may also include audio output devices such as speakers.
[0037] The communication control unit 96 connects to other devices and controls communication between the information processing device 90 and other devices. The communication control unit 96 is, for example, a LAN (Local Area Network) interface board, a wireless communication circuit for wireless communication, or a communication circuit for wired communication. The LAN interface board and wireless communication circuit are connected to a network such as the Internet.
[0038] The computer that implements the control device 100 performs the functions of the control unit 102 and the communication unit 106 by having the processor load a program stored in the auxiliary storage device into the main memory and execute it. On the other hand, the storage unit 104 is provided in the storage area of the main memory or the auxiliary storage device.
[0039] [Example of Operation] Next, an overview of the operation of the information processing system 10 will be described. The control device 100 of the information processing system 10 creates a scenario to be executed by the robot device 300 that applies an action to the target of operation, based on user input to terminal devices 200, etc. The created scenario is stored in the storage unit 104, and the control device 100 can operate the robot device 300 according to the scenario. In other words, a scenario is a task program that shows the procedure for operation by the robot device 300, etc., which is the target of control of the control device 100. The task program (scenario) created by the control device 100 defines the operation of the target device, including parameters such as the position, trajectory, and speed of the device being controlled. For example, this task program (scenario) defines the operation of the robot device 300 so that the position, trajectory, and speed of the instrument that is the target of operation of the robot device 300 are appropriate. The operation of the robot device 300 defined in the task program (scenario) is, for example, the operation of the robot device 300 in processing in cell manufacturing.
[0040] The control device 100 of the information processing system 10 basically operates the robot device 300 based on the created task program (scenario). However, if the control device 100 receives input from the user to correct the operation of the robot device 300 via the terminal device 200 while the robot device 300 is operating based on the task program (scenario), the control device 100 interrupts the operation of the robot device 300 based on the task program and performs real-time correction of the operation of the robot device 300. The real-time correction of the operation of the robot device 300 will be described in detail later.
[0041] [Task Program Structure] The structure of the task program (scenario) will be explained with reference to Figure 4. For the sake of simplicity, a scenario in which the robot device 300 is moved will be used as an example. The task program TP is generated by combining multiple scenario programs SP corresponding to each processing step, as shown in an example in Figure 4. The processing steps referred to here are steps executed by the continuous operation of the robot device 300 in processes such as moving an operating instrument gripped by the robot device 300, opening and closing a lid, and gripping a centrifuge tube, and can be set arbitrarily. For example, if the movement of an instrument by the robot device 300 is a combination of linear and curved movement, then the linear movement and the curved movement can each be treated as processing steps. In this case, each program for executing the linear motion and the curved movement corresponds to a scenario program SP.
[0042] In this example, the task program TP includes a first scenario program SP1, a second scenario program SP2, and a third scenario program SP3 as scenario programs SP, and is configured to be executed sequentially. Note that the task program TP does not necessarily have to include multiple scenario programs SP; for example, it may consist of only one scenario program SP.
[0043] Furthermore, as shown in Figure 5, each scenario program SP (SP1, SP2, SP3) is composed of multiple code programs CP (CP1, CP2, CP3) at predetermined time intervals. Each scenario program SP is code-divided and generated by a combination of multiple code programs CP at predetermined time intervals (e.g., 60 ms). In other words, each scenario program SP is composed of multiple code programs CP divided at predetermined time intervals. Note that code programs CP can also be called technical codes.
[0044] As an example, the first scenario program SP1 shown in FIG. 5 defines the movement of the robot apparatus 300 from the start point to end point 1, and is composed of a plurality of code programs CP1(1) to CP1(n). In each of the code programs CP1(1) to CP1(n), a movement distance and an end point position as target values are respectively set according to the movement path from the start point to end point 1. The second scenario program SP2 defines the movement of the robot apparatus 300 from the aforementioned end point 1 to end point 2, and is composed of a plurality of code programs CP2(1) to CP2(n). In each of the code programs CP2(1) to CP2(n), a movement distance and an end point position as target values are respectively set according to the movement path from end point 1 to end point 2. The third scenario program SP3 defines the movement of the robot apparatus 300 from the aforementioned end point 2 to end point 3, and is composed of a plurality of code programs CP3(1) to CP3(n). In each of the code programs CP3(1) to CP3(n), a movement distance and an end point position as target values are respectively set according to the movement path from end point 2 to end point 3.
[0045] The number of code programs CP constituting each scenario program SP may be appropriately determined according to the processing content, and is not particularly limited. The aforementioned n is a different value in each scenario program SP in this example, but may be the same value.
[0046] Note that a plurality of each code program CP constituting the scenario program SP is generated in advance for each content of the processing step, that is, for each operation content of the robot apparatus 300. Each scenario program SP is generated by appropriately selecting and combining predetermined code programs CP according to the operation content of the robot apparatus 300. Further, the task program TP is generated, for example, by selecting and combining the plurality of scenario programs SP generated in this manner.
[0047] [Execution of Task Program] In the information processing system 10 of the present embodiment, the control device 100 controls the operation of the robot device 300 by executing a task program TP composed of a plurality of scenario programs SP in response to a command or notification from the terminal device 200. In this example, the control device 100 controls the operation of the robot device 300 by sequentially executing a first scenario program SP1, a second scenario program SP2, and a third scenario program SP3 as the task program TP.
[0048] At that time, the control device 100 controls the continuous operation of the robot device 300 for each processing step by continuously executing a plurality of code programs CP as each scenario program SP. In this example, the control device 100 continuously executes a plurality of code programs CP1(1) to CP1(n) as the first scenario program SP1. The control device 100 continuously executes a plurality of code programs CP2(1) to CP2(n) as the second scenario program SP2. The control device 100 continuously executes a plurality of code programs CP3(1) to CP3(n) as the third scenario program SP3. In short, during the execution of the scenario program SP, each of the plurality of code programs CP is executed sequentially. In other words, during the execution of the scenario program SP, the process in which a new code program CP is executed when one code program CP is completed is repeated.
[0049] The execution procedure of the task program TP will be described in more detail using the sequence diagram of Fig. 6. Note that the example shown in Fig. 6 is an example of moving the robot device 300 holding an operation tool from a predetermined start point through end point 1 and end point 2 to end point 3.
[0050] In the example shown in Figure 6, at time t1, in response to the operator's operation, an execution command (also called a movement notification) is sent from the terminal device 200 to the control device 100 instructing the robot device 300 to move from the starting point to the ending point 1. In this example, along with the movement notification, information specifying the type of movement is sent from the terminal device 200 to the control device 100. The type of movement here includes "continuous movement" and "single movement". If multiple scenario programs SP are executed after the transmission of the movement type specification information, "continuous movement" is specified as the type of movement; if only one scenario program SP is executed, "single movement" is specified as the type of movement.
[0051] When the control device 100 receives a movement notification, for example, at time t2, the main thread sets the endpoint position (endpoint 1) of the robot device 300's movement according to the next scenario program to be executed (in this example, the first scenario program SP1) (endpoint setting). Based on the set endpoint position and movement path information, target values such as movement distance and endpoint position are set for each code program CP1(1) to CP(n). Subsequently, the first scenario program SP1, which constitutes the task program TP, is executed in the movement thread of the control device 100. More specifically, each code program CP1(1) to CP1(n) that constitutes the first scenario program SP1 is executed sequentially. Note that a thread (main thread and movement thread) is a unit of processing that is executed independently within a computer program. By utilizing multiple threads simultaneously, it becomes possible to execute multiple processes in parallel.
[0052] As the completion of the first scenario program SP1 approaches, at a predetermined timing (for example, time t3), the control device 100's movement thread sends information (completion response) to the terminal device 200 indicating that the robot device 300 has completed its movement to endpoint 1. The timing of sending the completion response may be when the movement of the robot device 300 to the endpoint position (endpoint 1) based on the currently executing scenario program SP is actually completed, but it is preferable that it be sent before the actual completion of the robot device 300's movement to the endpoint position. In other words, it is preferable that the control device 100 sends a notification response to the terminal device 200 to announce completion before the robot device 300's movement to the endpoint position is completed.
[0053] In particular, if there is a scenario program SP to be executed next, it is preferable that the completion response is sent before the robot device 300 completes its movement to the endpoint. In this example, since the second scenario program SP2 is executed following the first scenario program SP1, it is preferable that the completion response is sent before the robot device 300 completes its movement to endpoint 1. In other words, if "continuous movement" is selected as the type of movement, it is preferable that the completion response is sent before the robot device 300 completes its movement to endpoint 1.
[0054] Once the operator confirms the completion of the response received by the terminal device 200, the next movement notification is then sent to the control device 100 in accordance with the operator's operation. In this example, at time t4, a movement notification instructing the robot device 300 to move from endpoint 1 to endpoint 2, along with information on the type of movement, is sent from the terminal device 200 to the control device 100 in accordance with the operator's operation. The transmission of this next movement notification should be performed as soon as possible after the terminal device 200 receives the completion response, but it is preferable that it be slightly before the robot device 300 completes its movement to endpoint 1. Therefore, it is preferable that the above-mentioned completion response is transmitted at a timing that is slightly before the transmission of the next movement notification completes the robot device 300's movement to endpoint 1.
[0055] When the control device 100 receives a movement notification or the like instructing the robot device 300 to move to endpoint 2, for example, at time t5, the main thread sets the endpoint position (endpoint 2) of the robot device 300's movement according to the second scenario program SP2. Based on the set endpoint position and movement path information, target values such as movement distance and endpoint position are set in each code program CP2(1) to CP2(n). However, at this point, the movement of the robot device 300 to endpoint 2 according to the second scenario program SP2 has not yet started; only the setting of endpoint 2 and the setting of the above target values in code programs CP2(1) to CP(n) are performed. At time t6, when the movement of the robot device 300 to endpoint 1 according to the first scenario program SP1 is completed, the movement of the robot device 300 to endpoint 2 begins. That is, in the movement thread of the control device 100, the execution of the first scenario program SP1, which constitutes the task program TP, is completed, and the execution of the second scenario program SP2 begins. In this example, the endpoint position (endpoint 2) of the robot device 300 is set in advance before time t6. Therefore, at time t6, there can be a smooth transition from the operation of the robot device 300 by the first scenario program SP1 to the operation of the robot device 300 by the second scenario program SP2.
[0056] Then, at time t7, the movement thread of the control device 100 sends information (completion response) that the robot device 300 has completed its movement to endpoint 2 to the terminal device 200. As described above, it is preferable that the timing for sending the completion response is before the robot device 300 completes its movement to endpoint 2. Next, at time t8, a movement notification and movement type information instructing the robot device 300 to move from endpoint 2 to endpoint 3 in response to the operator's operation are sent from the terminal device 200 to the control device 100. In this example, since the execution of the task program TP is completed by moving the robot device 300 to endpoint 3 by the third scenario program SP3, a single movement is specified as the movement type. It is preferable that the next movement notification is sent after the completion response is received by the terminal device 200 as described above, and slightly before the robot device 300 completes its movement to endpoint 2.
[0057] When the control device 100 receives a movement notification or the like instructing the robot device 300 to move to endpoint 3, for example, at time t9, the endpoint position (endpoint 3) of the robot device 300 is set in the main thread. However, at this point, the movement of the robot device 300 to endpoint 3 has not started, and only the setting of endpoint 3 and the setting of the target values in code programs CP3(1) to CP3(n) are performed. At time t10, when the movement of the robot device 300 to endpoint 2 by the second scenario program SP2 is actually completed, the movement of the robot device 300 to endpoint 3 by the third scenario program SP3 is started. That is, when the execution of the second scenario program SP2, which constitutes the task program TP, is completed in the movement thread of the control device 100, the execution of the third scenario program SP3 is started. That is, each code program CP3(1) to CP3(n) that constitutes the third scenario program SP3 is executed sequentially.
[0058] When the execution of the third scenario program SP3 is completed at time t11, then at time t12, the movement thread of the control device 100 sends information (completion response) to the terminal device 200 indicating that the robot device 300 has completed its movement to the endpoint 3. This completes the series of processes based on the task program (scenario) TP.
[0059] In this example, the third scenario program SP3 is the last scenario program that makes up the task program TP. Therefore, the completion response of the third scenario program SP3 is sent at a later time (time t12) than the time (time t11) when the robot device 300 completes its movement to endpoint 3. In other words, if "single movement" is specified as the type of movement, it is sent at a later time than when the robot device 300 actually completes its movement to endpoint 1.
[0060] [Real-time Correction] As described above, while the robot device 300 is operating according to the task program (scenario) TP, the control device 100 can perform real-time correction of the robot device 300's operation in response to correction requests input by the operator to the terminal device 200. Real-time correction here refers to interrupting the execution of the task program TP and correcting parameters related to the operation of the robot device 300, such as speed, path, and position, based on a correction program, and is different from correcting the task program TP itself.
[0061] The real-time correction procedure during the execution of the task program TP will be explained using the sequence diagram in Figure 7. The example shown in Figure 7 is an example of performing real-time correction during the operation of a robot device 300 based on a task program TP composed of a single scenario program SP4. The scenario program SP4 shown in Figure 7 defines the movement of the robot device 300 from a starting point to an ending point 4 and is composed of multiple code programs CP4(1) to CP4(n). Of course, real-time correction can also be performed during the operation of a robot device 300 based on a task program composed of multiple scenario programs.
[0062] In the example shown in Figure 7, at time t21, an execution command (movement notification) is first sent from the terminal device 200 to the control device 100 in response to the operator's operation, instructing the robot device 300 to move from the starting point to the ending point 4. In this example, at time t21, along with the movement notification, information specifying the type of movement is sent from the terminal device 200 to the control device 100. In this example, "single movement" is specified as the type of movement.
[0063] When the control device 100 receives a movement notification, for example, at time t22, the endpoint position (endpoint 4) of the robot device 300 is set in the main thread (endpoint setting). Based on the set endpoint position and movement path information, target values such as movement distance and endpoint position are set in each code program CP4(1) to CP4(n). Subsequently, the scenario program SP4 that constitutes the task program TP is executed in the movement thread of the control device 100. More specifically, each code program CP4(1) to CP4(n) that constitutes the scenario program SP4 is executed sequentially.
[0064] While multiple code programs CP4 (CP4(1) to CP4(n)) are being executed in succession, if the terminal device 200 issues a correction start instruction to the control device 100, the correction program AP (for example, AP(1) to AP(n)) is executed in place of the code program CP4, thereby performing real-time correction of the robot device 300's operation.
[0065] In this example, at time t23, an instruction to start correction is sent from the terminal device 200 to the control device 100. That is, a correction request requesting the execution of real-time correction of the operation of the robot device 300 is input to the terminal device 200 via the input unit 202, and a correction instruction is sent from the terminal device 200 to the control device 100 instructing the execution of real-time correction. Specifically, for example, a correction instruction is sent from the terminal device 200 to the control device 100 instructing the start of correction according to the operator's operation content (correction request) to the input unit 202, which is a joystick. The correction instruction includes, for example, operation information such as the amount of operation of the input unit 202, the joystick, operation direction, operation speed, or acceleration. When the control device 100 receives the correction instruction including the operation information, for example, the correction start setting is performed in the main thread at time t24. In the correction start setting, for example, the content of the correction of the operation of the robot device 300 is determined based on the operation information. In other words, in the correction start setting, a correction program AP to be executed in place of the code program CP4 is selected based on the operation information. In other words, the operator's operation of the input unit 202 can be described as the operation of selecting the correction program AP.
[0066] Here, the scenario program SP is generated in advance as described above and stored in the memory unit 104, etc. Furthermore, for each stored scenario program SP, a selection of correction program candidates applicable to the multiple code programs CP that constitute each scenario program SP is pre-selected and set based on various information such as the purpose of operation and target values of each scenario program SP. In this example, the movement of the robot device 300 is controlled based on the scenario program SP4. For this reason, for the scenario program SP4, a selection of correction program candidates for correcting target values such as the distance traveled and the end point position in each code program CP4(1) to CP4(n) is pre-set. The number of correction program candidates set for each scenario program SP is not particularly limited, but it is preferable to keep it as small as possible, and in this example, it is set to about a few. Furthermore, the correction program candidates corresponding to each scenario program SP may be updated to a more appropriate correction program depending on the usage situation, etc.
[0067] When the control device 100 receives an instruction to start correction, it appropriately selects a correction program AP to be executed in place of the code program CP from among such correction program candidates. In the example shown in Figure 7, the correction start setting is performed while the code program CP4(x) is being executed. Therefore, the control device 100 appropriately selects one of several correction programs AP (in this example, correction programs AP(1) to AP(n)) to be executed in place of multiple code programs CP4 from CP4(x+1) onward. The selection conditions for correction programs AP can be determined arbitrarily, but examples include the timing of the execution of the code program. For example, correction programs AP may be appropriately selected depending on whether it is the first or second half of the scenario program.
[0068] In this case, one of the correction start settings may be a change in the speed of the robot device 300. For example, in the correction start setting, the unit of movement of the robot device 300 may be changed as a speed change. In this example, the unit of movement of the robot device 300 is changed from 60 ms to 120 ms. This makes it possible to instantly change the movement speed of the robot device 300 by half. In other words, each correction program AP executed in place of the code program CP will be set every 120 ms. As a result, the movement speed of the robot device 300 will be slowed down, making it easier to correct the operation of the robot device 300 more appropriately by real-time correction. However, in real-time correction, the unit of movement of the robot device 300 is not particularly limited and does not need to be changed.
[0069] Furthermore, the conditions under which the instruction to start correction is transmitted to the control device 100 are preferably a combination of multiple operations on the input unit 202. For example, in this example, it is preferable to include not only the operation of tilting the input unit 202, which is a joystick, but also the operation of pressing a predetermined button provided on the joystick. That is, it is preferable that the instruction to start correction is transmitted to the control device 100 when the joystick is tilted while the predetermined button is pressed. This allows for a more appropriate determination of the operator's intention to start correction.
[0070] When the correction start setting described above is performed, the robot movement to the corrected position is executed in the movement thread according to the content of the set correction start setting. In other words, the operation of the robot device 300 is corrected based on the correction program AP. For example, if the timing of the correction start setting is during the execution of the code program CP(x), the control device 100 waits for the completion of the code program CP(x) to execute the correction program AP and starts correcting the operation of the robot device 300 based on the correction program AP.
[0071] Although not shown in the diagram, if another operation is performed on the input unit 202 while the operation of the robot device 300 is being corrected based on the correction programs AP(1) to AP(n) set by the correction start setting at time t24, a new correction start instruction is sent from the terminal device 200 to the control device 100. For example, if the joystick that was tilted to the right is tilted to the left, a new correction start instruction is sent from the terminal device 200 to the control device 100. In response to the new correction start instruction, the control device 100 selects the correction program AP again.
[0072] The correction of the robot device 300's operation based on the correction program AP continues until a correction completion instruction is sent from the terminal device 200 to the control device 100 at time t25. For example, a correction completion instruction is sent from the terminal device 200 to the control device 100 in response to the operation of the input unit 202, which is a joystick. If, as described above, a correction start instruction is sent from the terminal device 200 to the control device 100 when the joystick is tilted while a predetermined button is pressed, then, for example, releasing the pressure on the predetermined button will cause a correction completion instruction to be sent from the terminal device 200 to the control device 100, regardless of whether the joystick is tilted or not.
[0073] When a correction completion instruction is sent from the terminal device 200 to the control device 100 at time t25, the correction completion setting is performed in the main thread at time t26, and the operation control of the robot device 300 based on the code program CP4 is resumed in the movement thread according to the content of the correction completion setting.
[0074] Here, as one of the correction completion settings, for example, the return process (return method) from real-time correction based on the correction program AP to the operation control of the robot device 300 by the scenario program SP may be appropriately determined. For example, multiple return process candidates may be set in advance as the above return process, and a return process may be appropriately selected from the multiple return process candidates depending on the state of the robot device 300 during real-time correction. The selection criteria in this case can be appropriately set in advance.
[0075] Specific examples of recovery process candidates include, for example, the following recovery process candidates 1 to 3. Recovery process candidate 1 is "to move towards the endpoint position (endpoint 4) of scenario program SP4". In this example, if recovery process candidate 1 is selected, the above target values are reset for code programs CP4(y) to CP4(n) so that the robot device 300 moves from its position at the time real-time correction based on correction program AP(n) is completed towards the endpoint position (endpoint 4). Then, the operation of the robot device 300 is controlled based on code programs CP4(y) to CP4(n) with reset target values.
[0076] Recovery process candidate 2 is to "start a movement defined to aim for the starting position of the next step in the procedure that has been corrected." In other words, recovery process candidate 2 means to start a movement defined to aim for the starting position of the next code program CP after the code program CP that has been replaced by the correction program AP. In this example, if recovery process candidate 2 is selected, once real-time correction is completed, the robot device 300 will perform an action aiming for the starting position of code program CP4(y). Recovery process candidate 1 prioritizes moving towards the endpoint 4, that is, achieving the objective of scenario program SP. In contrast, recovery process candidate 2 prioritizes returning to the movement path of the robot device 300 based on scenario program SP, that is, returning to scenario program SP.
[0077] Recovery process candidate 3 is "end the procedure at this point and proceed to the next step." For example, recovery process candidate 3 is selected if the position of the robot device 300 deviates significantly from the scenario program SP4 due to real-time correction. When recovery process candidate 3 is selected, the operation of the robot device 300 is temporarily stopped at the position at which the real-time correction was completed, and then the operation of the robot device 300 is resumed after waiting for a new movement notification from the operator.
[0078] The example shown in Figure 7 is an example in which the above-mentioned recovery process candidate 2 is selected as the recovery process candidate. Therefore, if the correction end setting is performed during real-time correction of the operation of the robot device 300 based on the correction program AP(n), the control device 100 waits for the completion of real-time correction based on the correction program AP(n) and then moves the robot device 300 toward the starting position of the code program CP4(y). After that, it executes control of the operation of the robot device 300 based on the code programs CP4(y) to CP4(n).
[0079] Furthermore, once the execution of the scenario program SP4 is completed at time t27, at time t28, the movement thread of the control device 100 sends information (completion response) to the terminal device 200 indicating that the robot device 300 has completed its movement to the endpoint 4. This completes the series of processes based on the task program (scenario) TP.
[0080] [Operation and Effects of the Embodiment] As described above, in the information processing system 10 of this embodiment, the scenario program SP corresponding to each processing step is composed of a plurality of code programs CP at predetermined time intervals. Furthermore, a task program TP is generated by the plurality of scenario programs SP. The control device 100 controls the continuous operation of the robot device 300 for each processing step by causing the plurality of code programs CP constituting the scenario program SP to execute continuously as the task program TP.
[0081] This provides a technology that facilitates real-time manual correction of automated machine movements by an operator. For example, if a scenario program SP corresponding to each processing step is composed of multiple code programs CP at predetermined time intervals, that is, if each scenario program SP is code-divided, then corrective intervention on the scenario programs CP becomes easier. Specifically, by executing a correction program AP in place of the code program CP at a predetermined timing, the operation of the robot device 300 can be corrected in real time without terminating the task program TP.
[0082] Furthermore, because each scenario program SP is divided into code, it has the effect of making it easier to return to the motion control of the robot device 300 based on the scenario program SP once real-time correction is completed. For example, by resetting the target value for the corresponding code program CP between the point where real-time correction is completed and the endpoint, as described above, the speed of returning to the motion control of the robot device 300 based on the code program CP can be increased. In other words, because each scenario program SP is divided into code, it has the effect of shortening the time required to return to the motion control of the robot device 300 based on the scenario program SP.
[0083] For example, in a cell culture process, it may be necessary for the user, an operator, to correct the operation of the robotic device 300. As an example, the processing operation in a peeling process using a scraper is difficult to automate, and real-time correction of the operation of the robotic device 300 by an operator is often necessary. The information processing system 10 according to this embodiment is particularly effective in such cell culture processes.
[0084] Furthermore, in the information processing system 10 of this embodiment, the correction content of the robot device 300 is determined by selecting a correction program from a pre-selected list of correction program candidates. Since the number of candidates for selection as a correction program is limited, the time required for correction can be significantly reduced compared to when the correction content is determined by calculation based on various information. In other words, the correction of the robot device 300's operation can be started and ended quickly. To put it another way, the simultaneity and synchronization of the correction for the robot device 300's operation can be satisfied. In short, the operation of the robot device 300 can be corrected in real time while suppressing delays in the processing steps performed by the robot device 300.
[0085] [Computer-readable recording medium] A program that enables a computer or other machine or device (hereinafter referred to as "computer, etc.") to perform any of the above functions can be recorded on a recording medium that is readable by a computer, etc. Then, by having the computer, etc. read and execute the program on this recording medium, it can provide that function.
[0086] Here, a recording medium readable by a computer refers to a recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical means, and can be read by a computer. Such a recording medium may contain computer components such as a CPU and memory, and the CPU may be used to execute programs.
[0087] Examples of such recording media that can be removed from computers include flexible disks, magneto-optical disks, CD-ROMs, CD-R / Ws, DVDs, DATs, 8mm tapes, and memory cards. Examples of recording media fixed to computers include hard disks and ROMs.
[0088] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above and includes various modifications. The present invention is not necessarily limited to having all the configurations described above. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations.
[0089] For example, in the above embodiment, an example was described in which the task program TP is composed of multiple scenario programs SP, and each scenario program SP is composed of multiple code programs CP. However, the configuration of the task program TP is not limited to this. For example, each scenario program SP may be further divided into multiple action programs, and each action program may be composed of multiple code programs. In this case, correction program candidates are selected and pre-set for each action program. This makes it possible to select a correction program corresponding to each code program more appropriately and quickly, and to further reduce the time required for correction.
[0090] Furthermore, in the above-described embodiment, an example was explained in which a suitable correction program to replace the code program is pre-registered as a candidate correction program from among multiple correction programs, and a correction program to be executed is selected from the registered candidate correction program. However, the method of selecting a correction program is not limited to this. For example, an unsuitable correction program to replace the code program may be pre-registered from among multiple correction programs, and a correction program to be executed may be selected from the unregistered correction programs.
[0091] Furthermore, as described in the embodiments above, the information processing system according to the present invention is suitably used in fields such as cell manufacturing (cell culture) in life sciences. However, the application fields of the information processing system according to the present invention are not particularly limited. The information processing system according to the present invention can also be applied to fields other than life sciences.
[0092] 10: Information processing system 90: Information processing device 91: Processor 92: Memory 93: Storage unit 94: Input unit 95: Output unit 96: Communication control unit 100: Control device 102: Control unit 104: Storage unit 106: Communication unit 200: Terminal device 202: Input unit 204: Display unit 300: Robot device 400: Detection device 610: Instrument 611: Test tube 612: Pipette 613: Container 614: Well plate 615: Flask 620: Table
Claims
1. An information processing system comprising: an operating device for holding a processing device for processing an object and for operating the processing device; and a control device for controlling the operation of the operating device based on a scenario program that defines the continuous operation of the operating device for each processing step, wherein the scenario program corresponding to each processing step is composed of a plurality of code programs for predetermined time intervals, and the control device controls the continuous operation of the operating device for each processing step by causing the plurality of code programs to execute continuously as the scenario program.
2. An information processing system according to claim 1, wherein the control device controls a series of operations of the operating device based on a task program which includes a plurality of scenario programs.
3. An information processing system according to claim 2, comprising a terminal device having an input unit for receiving input to the control device, wherein the control device executes operation control of the operating device based on each scenario program in response to execution commands transmitted from the terminal device at predetermined timings, and the terminal device transmits an execution command for operation control of the operating device according to the next scenario program before the completion of the scenario program being executed by the control device.
4. An information processing system according to claim 3, wherein, upon receiving the execution command, the control device sets parameters that define the operation of the operating device according to the next scenario program, before the completion of the currently executing scenario program.
5. An information processing system according to claim 3, wherein the control device transmits a completion response to the terminal device prior to the completion of the currently executing scenario program, and the terminal device, upon receiving the completion response, transmits an execution command for controlling the operating device according to the next scenario program.
6. An information processing system according to claim 1, comprising a terminal device having an input unit for receiving input to the control device, wherein, while the operating device is being operated based on the code program, a correction request requesting the execution of real-time correction of the operation of the operating device is input to the terminal device via the input unit, and when a correction instruction instructing the control device to execute the real-time correction is transmitted from the terminal device, the control device performs real-time correction of the operation of the operating device by executing a correction program corresponding to the code program instead of the code program.
7. An information processing system according to claim 6, wherein the control device, upon receiving the correction instruction, waits for the completion of the code program currently being executed and then begins real-time correction of the operation of the operating device.
8. An information processing system according to claim 6, wherein the control device continues to correct the operation of the operating device in real time while the correction request is input to the terminal device.
9. An information processing system according to claim 8, wherein the control device executes a plurality of correction programs corresponding to each code program instead of a plurality of code programs while the correction request is input to the terminal device.
10. An information processing system according to claim 9, wherein each of the scenario programs has a plurality of correction program candidates that are applicable to the code programs that constitute each scenario program, and the control device, upon receiving the correction instruction, selects one of the correction program candidates as the correction program.
11. An information processing system according to claim 6, wherein the control device terminates real-time correction of the operation of the operating device in response to a termination request transmitted from the terminal device, and resumes control of the operation of the operating device based on the scenario program, prioritizing the achievement of the objective of the scenario program.
12. An information processing system according to claim 6, wherein the control device terminates real-time correction of the operation of the operating device in response to a termination request transmitted from the terminal device, prioritizes returning to the scenario program, and resumes control of the operation of the operating device based on the scenario program.
13. An information processing method for controlling the operation of an operating device that holds a processing device for processing an object and operates the processing device, based on a scenario program that defines the continuous operation of the operating device for each processing step, wherein the scenario program corresponding to each processing step is composed of a plurality of code programs for predetermined time intervals, and the continuous operation of the operating device for each processing step is controlled by continuously executing the plurality of code programs as the scenario program.
