Information processing system and information processing method

Abstract

The present invention provides a technology which enables correction of a movement of a machine by a user's operation so as to achieve a movement of the machine desired by the user. This information processing system includes: an operation device that holds a processing instrument for processing an object and operates the processing instrument; and a control device that controls movement of the operation device. This information processing system comprises a force sensor that is provided in the operation device and detects a pressure which is generated when the processing instrument comes into contact with the object along with the movement of the operation device. The control device acquires, from a detection result by the force sensor, a normal pressure which is generated in the normal direction, with respect to a surface of the object, and controls the movement of the operation device on the basis of the normal pressure.

Description

Information Processing System and Information Processing Method 【0001】 The present invention relates to an information processing system and an information processing method. 【0002】 Conventionally, in a manufacturing site or the like, a predetermined repetitive operation has been automatically performed by a machine such as an industrial robot. In the life science industry as well, automation by machines in regenerative medicine, cell manufacturing, etc. has been promoted. 【0003】 Japanese Patent Application Laid-Open No. 2022-119943, Japanese Patent Application Laid-Open No. 2021-24025, International Publication No. 2012 / 029227 【0004】 Automation by an industrial robot or the like is suitable for, for example, repeatedly operating as designed in advance to manufacture the same product. However, in cell manufacturing in life science, etc., due to the occurrence of disturbances in cell manufacturing, etc., it may not be possible to manufacture the same thing by repeated operation. Disturbances in cell manufacturing include, for example, disturbances from the culture medium environment (alteration due to the environment around the cells), disturbances due to operation (alteration of cell characteristics due to actions and operations), disturbances from the cells themselves (alteration of cell characteristics due to cell lifespan), etc. Disturbances in cell manufacturing make automation by machines in cell manufacturing difficult. 【0005】 The technology of the present disclosure aims to reduce the influence on the operation of an automated machine and facilitate the correction of the operation of the machine by the user. 【0006】 To solve the above problems, the following means are adopted. If an example is given, the present application is an information processing system having an operating device that holds a processing tool for processing an object, an operating device that operates the processing tool, and a control device that controls the operation of the operating device, the operating device is provided with a force sensor that detects the pressure generated when the processing tool contacts the object as the operating device operates, the control device acquires the normal pressure generated in the normal direction with respect to the surface of the object from the detection result by the force sensor, and controls the operation of the operating device based on the normal pressure. 【0007】The mode of disclosure may be realized by the execution of a program by an information processing device. That is, the configuration of the disclosure can be identified as a program that causes an information processing device to execute the processing performed by each of the means in the above mode, or as a recording medium readable by the information processing device that stores the program. Alternatively, the configuration of the disclosure may be identified as a method by which the information processing device executes the processing performed by each of the means described above. Alternatively, the configuration of the disclosure may be identified as a system including an information processing device that performs the processing performed by each of the means described above. The information processing device is, for example, a computer. A computer is sometimes called a personal computer or a server. 【0008】 According to the disclosed technology, it is possible to reduce the impact on the operation of automated machinery while allowing the machine's operation to be corrected by the user. 【0009】This is a diagram showing an example configuration of the information processing system of the embodiment. This is a diagram showing an example configuration of the information processing system of the embodiment. This is a diagram showing an example hardware configuration of the information processing device. This is a diagram showing an example of the operation flow of scenario creation by the control device. This is a diagram showing an example of the overall operation flow of scenario execution and correction by the control device. This is a diagram showing an example of the operation flow of manual operation mode switching determination in S203. This is a diagram showing an example of the operation flow of manual operation mode in S205. This is a diagram showing an example of the screen displayed on the display unit 204. This is a diagram showing an example of the operation flow of separation determination in S403. This is a diagram showing an example of the operation flow of speed correction processing in S406. This is a diagram showing an example of the operation flow of trajectory correction processing in S408. This is a diagram showing an example of the screen displayed in trajectory correction processing. This is a diagram showing an example of the operation flow of position correction processing in S409. This is a diagram showing an example of the screen displayed on the display unit 204. This is a diagram showing an example of the operation flow of position correction in S803. This is a diagram showing an example of the operation flow of timing correction processing in S410. This is a diagram showing an example of the screen displayed on the display unit 204. This is a diagram showing a robot device according to Embodiment 2. This is a diagram illustrating the normal pressure generated in the direction normal to the ground surface. This is a flowchart illustrating an example of robot device control during scraping. This is a diagram illustrating the relationship between the world coordinate system and the local coordinate system. This is a diagram illustrating the relationship between the three-dimensional Cartesian coordinate system and the spherical coordinate system. This is a diagram illustrating an example of the normal direction to the surface of a petri dish. This is a diagram showing a modified example of the robot device according to Embodiment 2. This is a diagram illustrating an example of the time for scenario operation and correction operation in a series of operations. This is a diagram illustrating another example of robot device control according to Embodiment 2. This is a diagram illustrating another example of robot device control according to Embodiment 2. 【0010】 Embodiments will be described below with reference to the drawings. The configurations of the embodiments are illustrative, and the configuration of the invention is not limited to the specific configurations of the disclosed embodiments. In carrying out the invention, specific configurations may be adopted as appropriate depending on the embodiment. 【0011】[Embodiment 1] (Configuration Example) Figure 1 is a block diagram showing an example configuration of the information processing system of Embodiment 1, and Figure 2 is a schematic diagram showing an example configuration of the information system of Embodiment 1. 【0012】 As shown in Figures 1 and 2, the information processing system 10 includes a control device 100, a terminal device 200, a robot device 300 to be controlled, and a detection device 400. The control device 100 includes a control unit 102, a storage unit 104, and a communication unit 106. The terminal device 200 includes an input unit 202 and a display unit 204. The control device 100 is connected to the terminal device 200, the robot device 300, and the detection device 400 in a communication manner. 【0013】 The information processing system 10, in the control device 100, creates a scenario to be executed by the robot device 300, which acts on the target device, based on user input to terminal devices 200, etc. In other words, the information processing system 10 creates a scenario to be executed by the robot device 300, which operates the target device. The scenario is a task program that shows the procedure for operation by the robot device 300. The control device 100 operates the robot device 300 based on the created scenario. 【0014】 The control device 100 accepts user input for correction of operation from the terminal device 200 while the robot device 300 is operating based on a scenario, and modifies the scenario. Examples of processes performed by the robot device 300 based on the scenario created by the control device 100 include cell production (cell culture) in life sciences. In Embodiment 1, as an example of cell culture processing, the robot device 300 uses an instrument to be operated to aspirate a liquid sample, move it, and discharge the liquid to a predetermined position. The sample includes, for example, chemical substances, bacteria, cells, and other living organisms. 【0015】The control device 100 includes a control unit 102, a storage unit 104, and a communication unit 106. The control unit 102 controls the operation of terminal devices 200, robot devices 300, detection devices 400, etc. The storage unit 104 stores various data, programs, etc., used by the control device 100. The communication unit 106 is a communication interface for sending and receiving data with terminal devices 200, robot devices 300, detection devices 400, etc. 【0016】 The terminal device 200 includes an input unit 202 and a display unit 204 as input / output means for receiving information input from the user and outputting information to the user. The terminal device 200 can be implemented as, for example, a tablet terminal, a personal computer, or a smartphone. 【0017】 The input unit 202 is an input means that accepts information input from the user. The input unit 202 can be composed of, for example, a keyboard, a pointing device, a wireless remote control, a touch panel, a camera, a microphone, a joystick (also called a jog), a foot pedal, etc. In this example, the input unit 202 is assumed to be a touch panel. The touch panel can accept operations such as touch (tap) and swipe (input operations by physical contact) from the user. Touch is an operation in which a finger or the like is brought into contact with the touch panel and then released. Swipe is an operation in which a finger or the like is moved while in contact with the touch panel. Touch and swipe on a touch panel can be substituted with clicks and drags on a pointing device. Furthermore, touch and swipe on a touch panel may be substituted with other input operations on other input means. 【0018】 The display unit 204 is an output means that outputs information to the user, etc., and is a display means having a display screen. The display unit 204 is a display that displays text information, images, etc. In this example, the display screen of the display unit 204 is a touch panel and also serves as the input unit 202. The display unit 204 may also include a speaker that outputs sound, etc. 【0019】The robotic device 300 is a device that performs predetermined operations based on instructions from the control device 100. The robotic device 300 performs operations such as grasping, moving, and releasing the instrument to be operated. In Embodiment 1, the instrument to be operated is, for example, a well plate, petri dish, test tube, beaker, flask, container, pipette, etc. Of course, the instrument to be operated is not limited to these. The robotic device 300 operates the instrument to perform predetermined processing on the sample to be processed. For example, the robotic device 300 can operate a pipette, which is an example of an instrument to be operated, to aspirate a liquid such as a sample from one well of a well plate and discharge it into another well of the well plate. It should be noted that the well plate, etc., which contains the liquid such as a sample, can also be said to be the object to be processed using an instrument such as a pipette. 【0020】 The detection device 400 monitors the operation of the robot device 300. The detection device 400 includes sensors that detect the operation of the robot device 300 by visual means or the like. The detection device 400 includes, for example, a camera that photographs an area including the robot device 300 and the equipment to be operated by the robot device 300. The detection results from the detection device 400, such as the image captured by the camera, are output to the user from the terminal device 200. The detection device 400 may also include sensors that detect the state of the sample being processed. 【0021】 Furthermore, as shown in Figure 2 as an example, the robot device 300 has a base placed on the floor. The robot device 300 also includes a rotatable joint connected to the base and an arm connected to the joint. Further joints and arms are connected to the end of the arm. A gripping part for gripping the instrument 610 is also connected to the tip of the arm. The robot device 300 moves and operates the instrument 610 using the joints, arm, gripping part, etc. The configuration of the robot device 300 is not particularly limited and is sufficient as long as it can hold various instruments 610 and operate the instruments 610 appropriately. 【0022】Multiple instruments 610, which are the targets of the robotic device 300, are placed on a table 620. The multiple instruments 610 include test tubes 611, pipettes 612, containers 613, well plates 614, flasks 615, etc. The test tubes 611, pipettes 612, and containers 613 included in the instruments 610 are each operated by the robotic device 300. The robotic device 300, the multiple instruments 610, etc. are photographed by a detection device 400, which is a camera. The images captured by the camera of the detection device 400 are displayed on the display unit 204 of the terminal device 200, for example, via the control device 100. 【0023】 These robotic devices 300, tables 620, instruments 610, etc., are placed in a so-called aseptic area. By placing the instruments 610, etc., in the aseptic area, contamination of samples with foreign matter can be suppressed. Other devices, such as devices for transporting the instruments 610, etc., may also be placed in the aseptic area. The terminal device 200 is located outside the aseptic area and can be operated by the user. The information processing system 10 can realize the aseptic area as a cleanliness control area that does not require human entry. 【0024】 Incidentally, the control device 100 can be implemented using a dedicated or general-purpose computer such as a PC (Personal Computer) or workstation (WS), or an electronic device equipped with a computer. Alternatively, the terminal device 200 can be implemented using a dedicated or general-purpose computer such as a PC, smartphone, mobile phone, tablet terminal, or PDA (Personal Digital Assistant), or an electronic device equipped with a computer. 【0025】Figure 3 shows an example of the hardware configuration of an information processing device. The information processing device shown in Figure 3 has the configuration of a typical computer. The control device 100 and terminal device 200 are implemented by the information processing device 90 shown in Figure 3. The information processing device 90 shown in Figure 3 has a processor 91, memory 92, storage unit 93, input unit 94, output unit 95, and communication control unit 96. These are connected to each other by a bus. The memory 92 and storage unit 93 are computer-readable recording media. The hardware configuration of the computer that implements the control device 100 and terminal device 200 is not limited to the example shown in Figure 3, and components may be omitted, replaced, or added as appropriate. 【0026】 The information processing device 90 can realize functions that match a predetermined purpose by having the processor 91 load a program stored on a recording medium into the working area of ​​the memory 92 and execute it, thereby controlling each component through the execution of the program. The processor 91 is, for example, a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). The memory 92 includes, for example, RAM (Random Access Memory) or ROM (Read Only Memory). The memory 92 is also called the main memory. 【0027】 The storage unit 93 may be, for example, an EPROM (Erasable Programmable ROM), a hard disk drive (HDD), or a solid state drive (SSD). The storage unit 93 may also include removable media, i.e., portable recording media. Removable media may be, for example, a USB (Universal Serial Bus) memory, or a disk recording medium such as a CD (Compact Disc) or DVD (Digital Versatile Disc). The storage unit 93 is also called a secondary storage device. 【0028】The storage unit 93 stores the operating system (OS), various programs, various tables, various data, etc. The information stored in the storage unit 93 may also be stored in the memory 92. Conversely, the information stored in the memory 92 may also be stored in the storage unit 93. 【0029】 An operating system is software that acts as an intermediary between software and hardware, manages memory space, manages files, and manages processes and tasks. The operating system includes a communication interface. The communication interface is a program that exchanges data with other external devices connected via the communication control unit 96. These external devices include, for example, other computers and external storage devices. 【0030】 The input unit 94 includes a keyboard, pointing device, wireless remote control, touch panel, etc. The input unit 94 may also include video or image input devices such as a camera, and audio input devices such as a microphone. 【0031】 The output unit 95 includes display devices such as LCDs (Liquid Crystal Displays), EL (Electroluminescence) panels, CRT (Cathode Ray Tube) displays, and PDPs (Plasma Display Panels), as well as output devices such as printers. The output unit 95 may also include audio output devices such as speakers. 【0032】 The communication control unit 96 connects to other devices and controls communication between the information processing device 90 and other devices. The communication control unit 96 is, for example, a LAN (Local Area Network) interface board, a wireless communication circuit for wireless communication, or a communication circuit for wired communication. The LAN interface board and wireless communication circuit are connected to a network such as the Internet. 【0033】The computer that implements the control device 100 performs the functions of the control unit 102 and the communication unit 106 by having the processor load a program stored in the auxiliary storage device into the main memory and execute it. On the other hand, the storage unit 104 is provided in the storage area of ​​the main memory or the auxiliary storage device. 【0034】 (Example of Operation) Next, an example of the operation of the information processing system 10 will be described. The control device 100 of the information processing system 10 creates a scenario to be executed by the robot device 300 that applies an action to the target of operation, based on user input to the terminal device 200, etc. The scenario is a task program that shows the procedure of operation by the robot device 300, etc., which is the target of control of the control device 100. The scenario (task program) created by the control device 100 defines the operation of the target device, including parameters such as the position, trajectory, and speed of the device to be controlled. For example, this scenario (task program) defines the operation of the robot device 300 so that the position, trajectory, and speed of the instrument that is the target of operation of the robot device 300 are appropriate. The scenario is, for example, the operation of the robot device 300 in processing in cell manufacturing. In addition, when the robot device 300 is operating based on the created scenario, the control device 100 of the information processing system 10 receives input from the user to correct the operation to the terminal device 200 and modifies the scenario. The scenario defines the overall operation of the target device that will not be changed later. As will be explained in more detail later, the detailed settings of the operation can be changed by manual operation mode during the execution of the scenario. By limiting the settings that can be changed, the control device 100 can reduce the tasks related to manual operation mode. 【0035】 <Scenario Creation> Next, the creation of a scenario by the control device 100 will be explained. Once a scenario is created, the control device 100 can operate the robot device 300 according to that scenario. 【0036】Figure 4 shows an example of the operation flow for scenario creation by the control device 100. A scenario created by the control device 100 is, for example, to move a well plate 614, which is an example of an instrument, to a predetermined position using a robotic device 300, to aspirate the liquid contained in the wells of the well plate 614 with a pipette 612, to move the pipette 612, which is an example of an instrument, and to dispense the liquid from the pipette. 【0037】 In S101, the control unit 102 of the control device 100 receives input for the devices to be operated during scenario execution. The devices to be operated during scenario execution are, for example, robotic devices 300 that move well plates 614 and pipettes 612. The devices to be operated during scenario execution may also be actuators, etc. The control unit 102 displays a screen on the display unit 204 of the terminal device 200 that accepts input for the devices to be operated during scenario execution. For example, icons indicating devices that can be operated during scenario execution are displayed on this screen. The input unit 202 of the terminal device 200 also accepts input from the user for the devices to be operated during scenario execution (selection of icons indicating devices). The user inputs the devices to be operated by looking at the screen displayed on the display unit 204 of the terminal device 200 and touching the icons indicating the devices to be operated. The user inputs all devices to be operated during scenario execution in the order in which they will be operated. If one device is to be operated multiple times in one scenario, that one device may be selected and input multiple times. The control unit 102 receives device information from the terminal device 200 via the input unit 202. The control unit 102 stores the input device information in the storage unit 104. The control unit 102 also displays the input devices in the order they were entered on the screen displayed on the display unit 204 of the terminal device 200. The user can confirm the input devices and the order in which they should be operated by looking at this screen. 【0038】In S102, the control unit 102 performs grid adjustment processing. The grid is a frame of lines that serves as a guide when specifying the start and end positions of an operation. The width (spacing between adjacent vertical lines) and height (spacing between adjacent horizontal lines) of the grid can be adjusted. The control unit 102 displays the grid, along with the device to be operated, on the display unit 204 of the terminal device 200. The control unit 102 displays a screen on the display unit 204 of the terminal device 200 that accepts input for the grid width and height. The grid width and height can be specified, for example, by numerical values. Alternatively, the control unit 102 may display an input bar (slider bar) on the screen for the grid width and height, allowing the user to input by operating the input bar. The control unit 102 may also display icons on the display unit 204 of the terminal device 200 for adjusting the grid width and height. These icons may be, for example, a width expansion icon, a width reduction icon, a height expansion icon, and a height reduction icon. The control unit 102 determines which icon has been selected via the input unit 202. 【0039】 If the width expansion icon is selected, the control unit 102 confirms that the width of the displayed grid will be expanded by a predetermined value. If the width reduction icon is selected, the control unit 102 confirms that the width of the displayed grid will be reduced by a predetermined value. If the height expansion icon is selected, the control unit 102 confirms that the height of the displayed grid will be expanded by a predetermined value. If the height reduction icon is selected, the control unit 102 confirms that the width of the displayed grid will be reduced by a predetermined value. In addition, if other inputs are received, the control unit 102 may confirm that the width and height of the grid will be changed based on the inputs. The control unit 102 updates the grid displayed on the screen of the display unit 204 with the confirmed grid width and height. The start position, end position, etc. of the operation are specified by the center position of the grid frame. If you want to set the start position, etc. more precisely, you can set the desired start position, etc. by reducing the spacing between the grid frames. Also, if you do not need to set the start position, etc. precisely, you can make it easier to set the start position, etc. by increasing the spacing between the grid frames. 【0040】In S103, the control unit 102 of the control device 100 receives the input of the device's movement path, which was input in S101. In S103, it can also be said that the control unit 102 receives the input of the movement path of the instrument held by the device, which was input in S101. The control unit 102 displays a screen on the display unit 204 of the terminal device 200 that accepts the input of the device's movement path. The movement path can be specified, for example, by the starting position of the moving device (the starting point of the device's operation) and the ending position (the ending point of the device's operation). The movement path may include intermediate positions between the starting position and the ending position. The screen displays, for example, input fields for inputting the coordinates of each position. The input unit 202 accepts the input of coordinates for the input fields. The control unit 102 may also have the camera of the detection device 400 photograph the area around the device to be operated and display the captured image on the screen. The input unit 202 accepts touch input on the screen as coordinate input. The input unit 202 may also accept input of the movement path by a swipe operation that traces from the starting position to the ending position. The control unit 102 converts the touch input received by the input unit 202 into actual coordinates. The movement path also includes specific operations for each device. For example, if the instrument being operated is a pipette, it includes operations such as liquid aspiration and liquid dispensing. The input unit 202 accepts input of information on these specific operations for each device as input for the movement path. The control unit 102 accepts input for the movement path for all devices to be moved. The control unit 102 stores the input movement path information in the storage unit 104. The control unit 102 also displays the input movement path for each device on the screen displayed on the display unit 204 of the terminal device 200. The user can check the input movement path for each device by looking at this screen. 【0041】In S104, the control unit 102 of the control device 100 receives inputs of speed, acceleration, and waiting time for the movement path, which were input in S103. The control unit 102 displays a screen on the display unit 204 of the terminal device 200 that accepts inputs of the movement path, speed, acceleration, and waiting time. Speed, acceleration, and waiting time can be specified, for example, by numerical values. The control unit 102 may also display multiple options as icons on the screen for speed, acceleration, and waiting time, allowing the user to select one of the icons. Alternatively, the control unit 102 may display input bars (slider bars) on the screen for speed, acceleration, and waiting time, allowing the user to input by operating the input bars. The input unit 202 accepts inputs of speed, acceleration, and waiting time from the user. The control unit 102 accepts inputs of speed, acceleration, and waiting time for all devices to be moved. The control unit 102 stores the input speed, acceleration, and waiting time information in the storage unit 104. If no input is received for speed, acceleration, or waiting time, the control unit 102 may store predetermined default values ​​as speed, acceleration, and waiting time in the storage unit 104. The control unit 102 also displays the input speed, acceleration, and waiting time for each device on the screen displayed on the display unit 204 of the terminal device 200. The user can check the input speed, acceleration, and waiting time for each device by looking at this screen. 【0042】In S105, the control unit 102 of the control device 100 receives an input for a repetitive operation. The input for a repetitive operation is an input of the number of repetitions when repeating the operations of the devices input in S101 - S104. The control unit 102 displays, on the display unit 204 of the terminal device 200, a screen for receiving an input for a repetitive operation. The input for a repetitive operation is specified by which operations of which devices are to be repeated and for how many times. The input unit 202 receives an input for a repetitive operation by the user. The control unit 102 stores the input information on the repetitive operation in the storage unit 104. By receiving an input for a repetitive operation, it is possible to suppress performing the input of the same operation of the same device multiple times. Also, the control unit 102 displays the input repetitive operation on the screen displayed on the display unit 204 of the terminal device 200. The user can confirm the input repetitive operation by viewing the screen. 【0043】 In S106, the control unit 102 stores, as one scenario, the operations of each device input in S101 - S105 and stored in the storage unit 104 in the storage unit 104. At this time, the control unit 102 may receive an input of a scenario name at the input unit 202. The control unit 102 stores the input scenario name in the storage unit 104 in association with the scenario. Thereby, a scenario is created. 【0044】 〈Scenario Execution, Correction〉 Here, the execution and correction of the scenario created by the control device 100 will be described. When a scenario is created, the control device 100 can operate the robot device 300 according to the scenario stored in the storage unit 104. Also, the control device 100 can receive correction of the scenario during the execution of the scenario. Here, it is assumed that the control device 100 can receive speed correction, path correction, and position correction. 【0045】 《Overall Operation Flow》 FIG. 5 is a diagram showing an example of the overall operation flow of the execution and correction of a scenario by the control device 100. Here, the scenario created by the operation flow of FIG. 4 is executed. 【0046】In S201, the control unit 102 of the control device 100 extracts the scenario to be executed stored in the storage unit 104, and operates the robot device 300 or the like according to the scenario. That is, at this time, the scenario is being executed in the automatic operation mode. In addition, the control unit 102 photographs the periphery of the robot device 300 operating according to the scenario with the camera of the detection device 400. The control unit 102 displays the image photographed by the camera of the detection device 400 on the display unit 204 of the terminal device 200. The user can confirm the operation of the robot device 300 or the like based on the image displayed on the display unit 204. 【0047】 In S202, the control unit 102 determines whether or not a touch input has been detected by the input unit 202 of the terminal device 200. The touch input is detected when a user's finger or the like touches the touch panel of the input unit 202. The user touching the touch panel of the input unit 202 is a signal that the user hopes to correct the scenario being executed. That is, when the user hopes to correct the scenario for the operation of the device in operation, the user touches the touch panel of the input unit 202 with a finger or the like. Here, the position where the touch input by the user is detected may be any position on the touch panel of the input unit 202. Also, the position where the touch input by the user is detected may be limited to a specific area of the touch panel. If a touch input is detected (S202; YES), the process proceeds to S203. If no touch input is detected (S202; NO), the process proceeds to S208. 【0048】 In S203, the control unit 102 determines whether or not to switch to the manual operation mode. The manual operation mode is a mode for correcting the scenario. The details of S203 will be described in the operation flow of FIG. 6. 【0049】 In S204, the control unit 102 determines whether or not to switch to the manual operation mode based on the determination result in S203. If the determination result in S203 requires switching (S204; YES), the process proceeds to S205. If the determination result in S203 does not require switching (S204; NO), the process proceeds to S208. 【0050】In S205, the control unit 102 switches to manual operation mode and performs a predetermined process. Details of S205 will be explained in the operation flow of Figure 7. 【0051】 In S206, the control unit 102 determines whether a scenario change was made in S205. If a scenario change was made (S206; YES), the process proceeds to S207. If no scenario change was made (S206; NO), the process proceeds to S208. 【0052】 In S207, the control unit 102 returns the position of the robot device 300, etc., to its initial position (at the start of the scenario). The control unit 102 operates the robot device 300, etc., based on the modified scenario stored in the memory unit 104. After that, the process returns to S201. Even after that, the control unit 102 operates the robot device 300, etc., based on the modified scenario. 【0053】 In S208, the control unit 102 determines whether the operation according to the currently running scenario has finished. If the operation according to the currently running scenario has finished (S208; YES), the processing of the operation flow in Figure 5 ends. If the operation according to the currently running scenario has not finished (S208; NO), the process returns to S201. This allows the user to correct the scenario while it is running. 【0054】 《Operation Flow for Manual Operation Mode Switching Determination》 Figure 6 shows an example of the operation flow for manual operation mode switching determination in S203. In S301, the control unit 102 monitors the touch input in the input unit 202 of the terminal device 200. That is, the control unit 102 monitors whether or not touch input is being performed in the input unit 202. 【0055】 In S302, the control unit 102 determines whether a predetermined time (for example, 100 ms) has elapsed since the touch input was detected in S202. This predetermined time is a threshold value used to determine whether the user's touch input was intended to switch to manual operation mode. If the predetermined time has elapsed (S302; YES), the process proceeds to S304. If the predetermined time has not elapsed (S302; NO), the process proceeds to S303. 【0056】 In S303, the control unit 102 determines whether or not the touch has been released. In other words, the control unit 102 determines whether or not touch input has been performed based on the monitoring in S301. If no touch input has been performed, i.e., if the touch has been released (S303; YES), the process proceeds to S305, and the control unit 102 determines that the user does not wish to switch to manual operation mode. This is because the time the user has been touching the input unit 202 is short (less than the predetermined time). If touch input has been performed, i.e., if the touch has not been released (S303; NO), the process returns to S301. 【0057】 In S304, the control unit 102 determines that it is necessary to switch to manual operation mode. This is because touch input has been detected for a predetermined time or longer, and it is assumed that the user wishes to switch to manual operation mode. In S305, the control unit 102 determines that it is not necessary to switch to manual operation mode. This is because touch input has not been detected before the predetermined time has elapsed, and it is assumed that the user does not wish to switch to manual operation mode. This prevents the system from switching to manual operation mode if the user accidentally touches the touch panel of the input unit 202. 【0058】 《Operation Flow of Manual Operation Mode》 Figure 7 shows an example of the operation flow of manual operation mode in S205. In S401, the control unit 102 displays on the display unit 204 of the terminal device 200 that the mode in the current scenario execution is manual operation mode. For example, the control unit 102 displays the text information "Manual Operation Mode" on the display unit 204. The user can confirm that the current mode is "Manual Operation Mode" by looking at this display. 【0059】 In S402, the control unit 102 instructs the robot device 300, etc., to stop its operation. As a result, the operating speed of the robot device 300, etc., slows down. For example, if the robot device 300 receives another instruction before stopping its operation, the robot device 300 may continue to operate in accordance with that other instruction before stopping its operation. 【0060】In S403, the control unit 102 performs a separation determination. The separation determination determines whether or not the user is operating the touch panel of the input unit 202. For example, it determines whether or not the time during which the touch panel has not been operated is longer than a predetermined time. If the user is not operating the touch panel, the manual operation mode is terminated. Details of S403 will be explained in the operation flow of Figure 9. 【0061】 In S404, the control unit 102 determines whether to continue the manual operation mode based on the determination result in S403. If the determination result in S403 is no separation (S404; NO), the process proceeds to S405. If the determination result in S403 is separation (S404; YES), the process ends. In this case, the user is deemed not to intend to continue manual operation (continue correcting the scenario). If the operating speed of the controlled device is reduced in S407, the control unit 102 may return the operating speed of the controlled device to the speed at which it was operating up to the stop instruction in S402. When returning the operating speed, the control unit 102 may return it with maximum acceleration or gradually. 【0062】In S405, the control unit 102 determines whether or not an event has been detected by the input unit 202. Here, an event is the selection of a parameter that can be corrected (changed) at the present time (e.g., speed, trajectory (path), position, timing). The control unit 102 displays icons on the display unit 204 that indicate the parameters that can be corrected at the present time. When the control unit 102 detects the selection of any icon by the input unit 202, it determines that an event has been detected. Event detection is not limited to the selection of an icon and may be performed by other methods (e.g., character input, predetermined operation). Here, the correctable parameters are assumed to be "speed", "trajectory", "position", and "timing". The correctable parameters may be other. The correctable parameters may be pre-set for each device being controlled. If the event "speed" is detected (S405; YES), the process proceeds to S406. If the event "trajectory" is detected (S405; YES), the process proceeds to S407, and then to S408. If the event "Position" is detected (S405; YES), the process proceeds to S407, and then to S409. If the event "Timing" is detected (S405; YES), the process proceeds to S407, and then to S410. If no events are detected (S405; NO), the process ends. If the operating speed of the controlled device is reduced in S407, the control unit 102 may return the operating speed of the controlled device to the speed at which it was operating up to the stop instruction in S402. When returning the speed, the control unit 102 may return it with maximum acceleration or gradually. 【0063】Figure 8 shows an example of a screen displayed on the display unit 204. The screen in Figure 8 displays a speed correction icon 30 indicating that speed correction is possible, a trajectory correction icon 40 indicating that trajectory correction is possible, a position correction icon 50 indicating that position correction is possible, and a timing correction icon 60 indicating that timing correction is possible. The speed correction icon 30 includes a 50% icon 31 to set the operating speed of the device to 50%, a 100% icon 32 to set the operating speed of the device to 100%, and a 200% icon 33 to set the operating speed of the device to 200%. If the 50% icon 31, 100% icon 32, or 200% icon 33 of the speed correction icon 30 is selected, it is determined that the event "speed" has been detected. If the trajectory correction icon 40 is selected, it is determined that the event "trajectory" has been detected. If the position correction icon 50 is selected, it is determined that the event "position" has been detected. If the timing correction icon 60 is selected, it is determined that the event "timing" has been detected. Furthermore, the screen in Figure 8 displays the pipette 612, the well 631 of the well plate 614, and the liquid 650 contained in the well 631, all of which are the objects of operation as captured by the camera of the detection device 400. In addition, the screen in Figure 8 displays the text information "Manual Operation Mode" to indicate that manual operation mode is in progress. 【0064】In S406, the control unit 102 performs speed correction processing. Details of S406 will be explained in the operation flow of Figure 10. After that, the process proceeds to S403. In S407, the control unit 102 restarts the operation of the robot device 300, etc., which was instructed to stop in S402, at a speed less than the speed of the operation that was being performed up to the stop instruction in S402, but greater than or equal to a predetermined value. If an event other than "speed" is detected, the control unit 102 restarts the operation of the robot device 300, etc., at a speed of 40% or more but less than 60% of the speed of the operation that was being performed up to the stop instruction in S402 (original speed). In other words, even if an event other than "speed" is detected in S405, the operating speed of the controlled device is reduced. By reducing the operating speed of the controlled device, it becomes easier for the user to correct the scenario. When restarting the operation, it is desirable that the operating speed of the controlled device be changed to the specified speed with maximum acceleration. By changing the operating speed of the controlled device to a specified speed with maximum acceleration, the control device 100 can smoothly transition to manual operation mode without the user being aware of the mode switch. 【0065】 In S408, the control unit 102 performs trajectory correction processing. Details of S408 will be explained in the operation flow of Figure 11. After that, the process proceeds to S403. In S409, the control unit 102 performs position correction processing. Details of S409 will be explained in the operation flow of Figure 14. After that, the process proceeds to S403. In S410, the control unit 102 performs timing correction processing. Details of S410 will be explained in the operation flow of Figure 17. After that, the process proceeds to S403. 【0066】《Operation Flow for Determination of Disconnection》 Figure 9 shows an example of the operation flow for determination of disconnection in S403. In S501, the control unit 102 determines whether or not the input unit 202 is disconnected from the touch. Disconnection from the touch means that the user's finger or the like is not touching the touch panel of the input unit 202. If the user's finger or the like is touching the touch panel, it is not disconnection from the touch. If it is not disconnection from the touch (the user's finger or the like is touching the touch panel of the input unit 202) (S501; NO), the process proceeds to S502. If it is disconnection from the touch (S501; YES), the process proceeds to S503. 【0067】 In S502, the control unit 102 determines that there is no disconnection. Because touch input has been detected, it is assumed that the user wishes to continue in manual operation mode. In S503, the control unit 102 monitors the touch input at the input unit 202 of the terminal device 200. That is, the control unit 102 monitors whether or not touch input is being performed at the input unit 202. 【0068】 In S504, the control unit 102 determines whether a predetermined time (for example, 100 ms) has elapsed since the touch release was detected in S501. This predetermined time is a threshold value used to determine whether the user's touch release was intended to continue the manual operation mode, and can be set arbitrarily. If the predetermined time has elapsed (S504; YES), the process proceeds to S505. If the predetermined time has not elapsed (S504; NO), the process proceeds to S506. 【0069】In S505, the control unit 102 determines that the touch has been lifted. Since no touch input is detected for a predetermined time or longer, it is assumed that the user does not wish to continue in manual operation mode. In S506, the control unit 102 determines whether or not a touch input has been made. If a touch input has been made, it is assumed that the user wishes to continue in manual operation mode. This is because the time the user lifted their finger or other object from the touch panel of the input unit 202 (touch release time) was short (less than the predetermined time). If a touch input has been made (S506; YES), the process proceeds to S507. If no touch input has been made (S506; NO), the process proceeds to S503. In S507, the control unit 102 determines that there has been no lifting. Since touch input has been detected, it is assumed that the user wishes to continue in manual operation mode. 【0070】《Operation Flow of Speed ​​Correction Processing》 Figure 10 shows an example of the operation flow of the speed correction processing in S406. In S601, the control unit 102 determines the selected speed. The control unit 102 determines which speed was selected in the event "speed" via the input unit 202. For example, as shown in Figure 8, if the 50% icon 31, 100% icon 32, and 200% icon 33 are displayed on the display unit 204, the control unit 102 determines which icon was selected and input to the input unit 202. If the 50% icon 31 is selected, the control unit 102 confirms that the operating speed of the device that was instructed to stop in S402 (for example, the robot device 300) will be set to 50% of the speed before the stop instruction. Also, if the 100% icon 32 is selected, the control unit 102 confirms that the operating speed of the device that was instructed to stop in S402 will be set to 100% of the speed before the stop instruction. In other words, the control unit 102 confirms that the speed will not be changed. Furthermore, if the 200% icon 33 is selected, the control unit 102 confirms that the speed of the device instructed to stop in S402 will be set to 200% of the speed before the stop instruction. Speeds of 50% of the speed before the stop instruction and 200% of the speed before the stop instruction are examples of changeable speeds. Also, if an icon indicating another speed is selected, the control unit 102 confirms that the speed will be changed to the speed based on the selected icon. By displaying icons indicating changeable speeds on the display unit 204 and allowing input from the input unit 202, the speed can be easily changed. 【0071】 In S602, the control unit 102 determines whether the speed of the device determined in S601 has changed from the speed before the stop instruction. If the speed has not changed (S602; NO), the process ends. If the speed has changed (S602; YES), the process proceeds to S603. 【0072】In S603, the control unit 102 displays the changed speed on the display unit 204. The user can recognize the changed speed from the display on the display unit 204. In S604, the control unit 102 stores the current scenario in the storage unit 104 as a backup. For example, the control unit 102 changes the scenario name of the current scenario so that it can be recognized as a backup, and stores it in the storage unit 104. In S605, the control unit 102 creates a new scenario that reflects the changed speed in the current scenario and stores it in the storage unit 104. This creates the modified scenario. 【0073】 《Operation Flow of Trajectory Correction Processing》 Figure 11 shows an example of the operation flow of the trajectory correction processing in S408. In S701, the control unit 102 determines whether or not there has been a touch on the input unit 202. The control unit 102 determines, for example, whether or not there has been a touch on the trajectory line (selection of the trajectory line), which will be described later. This determination can also be rephrased as determining whether or not there has been a touch release. A touch release means that the user's finger, etc., is not touching the touch panel of the input unit 202. If the user's finger, etc., is touching the touch panel, it is not a touch release. If there is no touch release (the user's finger, etc., is touching the touch panel of the input unit 202) (S701; YES), it is assumed that the user wishes to continue trajectory correction, and the process proceeds to S702. If there has been a touch release, for example, if there has been no touch on the trajectory line for a predetermined time or longer (S701; NO), it is assumed that the user does not wish to continue trajectory correction, and the process proceeds to S707. 【0074】 Figure 12 shows an example of a screen displayed during trajectory correction processing. The screen in Figure 12 is displayed on the display unit 204 of the terminal device 200 during trajectory correction processing. Similar to Figure 8, the screen in Figure 12 displays the pipette 612, the well 631 of the well plate 614, and the liquid 650 contained in the well 631, which are the objects of operation and have been photographed by the camera of the detection device 400. The screen also shows the first path line 661, which is the currently selected trajectory (path line) of the pipette 612, as a solid arrow. The first path line 661, which is the currently selected trajectory (path line), is displayed in black, for example. 【0075】 The currently selected trajectory (path line) before correction is the current trajectory (path line) of the pipette 612 being manipulated. The base of the arrow indicates the starting position of the movement path of the pipette 612 being manipulated, and the tip of the arrow indicates the ending position of the movement path of the pipette 612 being manipulated. The movement path of the pipette 612 follows the arrow. The same applies to the other arrows. The second path line 662 and the third path line 663 indicate correctable trajectories (path lines). The second path line 662 and the third path line 663, which are correctable trajectories (path lines), are shown as dotted lines. In addition, the second path line 662 and the third path line 663, which are correctable trajectories (path lines), are displayed in red, for example. Displaying the selected path line and the correctable path lines in different colors makes it easier to distinguish between these path lines. 【0076】 In S702, the control unit 102 causes the currently selected route displayed on the display unit 204 to blink. By making the currently selected route blink, the user can more easily recognize the currently displayed route. In S703, the control unit 102 displays correctable route lines around the currently selected route line displayed on the display unit 204. Correctable route lines are candidates for route lines that can be changed. 【0077】 In S704, the control unit 102 determines whether a swipe operation has been detected in the input unit 202 from the position of the currently selected path line (e.g., the first path line 661) displayed on the display unit 204 to the position of a correctable path line (e.g., the second path line 662). This operation indicates a change in the path line of the object being operated by the robot device 300 (e.g., a pipette 612) (e.g., a change from the first path line 661 to the second path line 662). If the swipe operation is detected (S704; YES), the process proceeds to S705. If the swipe operation is not detected (S704; NO), the process proceeds to S701. 【0078】In S705, the control unit 102 changes the correctable path line (e.g., the second path line 662) at the endpoint of the swipe operation detected in S704 to a solid line and makes it blink on the display unit 204. Here, this path line becomes the currently selected path line. By accepting changes through a swipe operation, the user can easily change the trajectory of the controlled device (the instrument being operated). In S706, the control unit 102 displays correctable path lines (dotted path lines) around the currently selected path line (solid path line) displayed on the display unit 204. The correctable path lines are candidates for changeable path lines. 【0079】 Figure 13 shows an example of a screen displayed during trajectory correction processing. The screen in Figure 13 is displayed on the display unit 204 of the terminal device 200 during trajectory correction processing. In the screen of Figure 13, similar to Figure 12, the pipette 612, the well 631 of the well plate 614, and the liquid 650 contained in the well 631, which are the objects of operation and have been photographed by the camera of the detection device 400, are displayed. In addition, the second path line 662, which is the currently selected trajectory (path line) of the pipette 612, is indicated by a solid arrow on this screen. The second path line 662, which is the currently selected trajectory (path line), is displayed in black, for example. The first path line 661 and the fourth path line 664 indicate trajectories (path lines) that can be corrected. The third path line 663 is not displayed here. The first path line 661 and the fourth path line 664, which are correctable trajectories (path lines), are indicated by dotted lines. The first and fourth paths 661 and 664, which are correctable paths (route lines), are displayed in red, for example. Displaying the selected path line and the correctable path lines in different colors makes it easier to distinguish between them. In addition, displaying the correctable path line (dotted line) together with the currently selected path line (solid line) makes it easier for the user to recognize the difference between the currently selected path and the modified path. 【0080】In S707, the control unit 102 hides the correctable route lines displayed on the display unit 204. That is, on the screen displayed on the display unit 204, only the currently selected route line is displayed. In S708, if a route line is being corrected and the route line being corrected is blinking, the control unit 102 stops the blinking display of the currently selected route line on the display unit 204. That is, the selected route line is displayed normally on the display unit 204. This route line becomes the modified route line. In other words, the control unit 102 confirms the currently selected route line as the modified (corrected) route line. 【0081】 In S709, the control unit 102 stores the current scenario in the storage unit 104 as a backup. The control unit 102, for example, changes the scenario name of the current scenario so that it can be recognized as a backup, and stores it in the storage unit 104. In S710, the control unit 102 creates a new scenario that reflects the changed trajectory (path) of the current scenario and stores it in the storage unit 104. This creates the modified scenario. 【0082】 《Operation Flow of Position Correction Processing》 Figure 14 shows an example of the operation flow of the position correction processing in S409. In S801, the control unit 102 displays the start position, end position, and grid of the device to be controlled on the display unit 204. The grid is a frame of lines that serves as a guide when specifying the start position and end position. The width (spacing between adjacent vertical lines) and height (spacing between adjacent horizontal lines) of the grid can be changed during the scenario creation stage. 【0083】Figure 15 shows an example of a screen displayed on the display unit 204. The screen in Figure 15 displays the pipette 612, the well 631 of the well plate 614, and the liquid 650 contained in the well 631, which are the objects of operation, as captured by the camera of the detection device 400. Furthermore, the screen in Figure 15 displays the starting position 671 and ending position 672 of the pipette 612, as well as a grid 670 consisting of multiple vertical and horizontal lines. The grid 670 is displayed according to the positions of the pipette 612, well plate 614, etc., detected from the image captured by the camera of the detection device 400. Alternatively, the pipette 612, well plate 614, etc., may be displayed as illustrations on the screen of the display unit 204, and the grid 670 may be displayed according to these illustrations. 【0084】 In S802, the control unit 102 determines whether or not an event has been detected by the input unit 202. Here, an event is the selection of a correctable parameter (start position, end position). The control unit 102 displays icons on the display unit 204 that indicate the currently correctable parameters. When the control unit 102 detects the selection of any of these icons by the input unit 202, it determines that an event has been detected. Here, the correctable parameters are assumed to be "position (start position, end position)". More specifically, the correctable parameters are the start position and end position of the pipette 612 that is being operated on. However, the correctable parameters are not limited to the start position and end position of the pipette 612; for example, they may be the start position and end position of the robot device 300 (tip of the arm) that is being controlled. 【0085】 If the event "Location" is detected (S802; YES), the process proceeds to S803. In the example in Figure 15, it is assumed that the event "Location" is detected when the starting position 671 and ending position 672 are selected. If no event is detected (S802; NO), the process proceeds to S804. 【0086】In S803, the control unit 102 performs position correction processing. Details of S803 will be explained in the operation flow of Figure 16. After that, the process proceeds to S804. In S804, the control unit 102 determines whether or not the input unit 202 is in a touch-away state. A touch-away state means that the user's finger or other object is not touching the touch panel of the input unit 202. If the user's finger or other object is touching the touch panel, it is not a touch-away state. If position correction is performed in S803, S804 determines whether or not the user's finger or other object has moved away from the touch panel. It can also be said that in S804, for example, after the start position 671, end position 672, etc. are displayed on the display unit 204, it is determined whether or not there has been a touch (selection) of the start position 671 or end position 672 within a predetermined time. 【0087】 In S804, if it is determined that the touch has not been released (the user's finger, etc., is touching the touch panel of the input unit 202) (S804; NO), the process proceeds to S801. If it is determined that the touch has been released (S804; YES), the process proceeds to S805. In S805, the control unit 102 hides the grid displayed on the display unit 204. That is, the start and end positions of the pipette 612, which is the object of the operation, are displayed on the display unit 204. 【0088】 At this time, as will be described later, if the starting point or ending point to be corrected by the position correction process in S803 is blinking, in S806 the control unit 102 stops the blinking display of the currently selected starting point or ending point on the display unit 204. That is, the starting point and ending point are displayed normally on the screen displayed on the display unit 204. These starting point and ending point become the changed starting point and ending point. In other words, the control unit 102 confirms the currently displayed starting point and ending point as the changed (corrected) starting point and ending point. 【0089】In S807, the control unit 102 stores the current scenario in the storage unit 104 as a backup. The control unit 102, for example, changes the scenario name of the current scenario so that it can be recognized as a backup, and stores it in the storage unit 104. In S808, the control unit 102 creates a new scenario that reflects the changed start and end positions of the current scenario, and stores it in the storage unit 104. This creates the modified scenario. 【0090】 《Operation Flow for Position Correction》 Figure 16 shows an example of the operation flow for position correction in S803. In S901, the selected point to be corrected is determined. The control unit 102 determines, via the input unit 202, which point (starting point 671 or ending point 672) was selected in the event "position". For example, as shown in Figure 15, if the starting point 671 and ending point 672 are displayed on the display unit 204, the control unit 102 determines which position was selected and input to the input unit 202. If the starting point 671 is selected, the control unit 102 makes the starting point 671 blink on the display unit 204. If the ending point 672 is selected, the control unit 102 makes the ending point 672 blink on the display unit 204. The blinking point (starting point 671 or ending point 672) is the point to be corrected. 【0091】 In S902, the control unit 102 determines whether a swipe operation has been detected in the input unit 202 from the position (center of the grid frame) of the point to be corrected (start position 671 or end position 672) displayed on the display unit 204 to the center of another grid frame. In the swipe operation, the point to be corrected is the starting position of the swipe operation, and the center of the other grid frame is the ending position of the swipe operation. The center of the other grid frame is a different position from the starting position 671 and the end position 672. The operation indicates a change in the position (start position 671 or end position 672) of the instrument being operated on (e.g., pipette 612). If the swipe operation is detected (S902; YES), the process proceeds to S903. If the swipe operation is not detected (S902; NO), the process ends. 【0092】 In S903, the control unit 102 determines on the display unit 204 whether the position of the endpoint of the swipe operation detected in S902 is within a predetermined range. The area that can be set as the starting position 671 or the endpoint position 672 is predetermined as a range from the current position (point). Therefore, if the destination position (point) is not within the predetermined range, the control unit 102 does not change the starting position 671 or the endpoint position 672. If the position of the endpoint of the swipe operation detected in S902 is within the predetermined range (S903; YES), the process proceeds to S904. If the position of the endpoint of the swipe operation detected in S902 is not within the predetermined range (S903; NO), the process proceeds to S905. 【0093】 In S904, the control unit 102 changes the display position of the point to be corrected to the position of the endpoint of the swipe operation detected in S902. The control unit 102 sets this position as the changed position of the point to be corrected. The display unit 204 displays the start and end positions, allowing the user to easily recognize them. Furthermore, by accepting changes to the start or end position via a swipe operation, the user can easily change the start or end position of the instrument or device being operated on or controlled. 【0094】 In S905, the control unit 102 displays on the display unit 204 that the position to be corrected is outside a predetermined range. The user recognizes from this display that the position to be corrected is outside a predetermined range (the area that can be set as the starting point position 671 or the ending point position 672). 【0095】《Operation Flow of Timing Correction Processing》 Figure 17 shows an example of the operation flow of the timing correction processing in S410. In S1001, the control unit 102 displays a screen on the display unit 204 that accepts timing correction for operations in the operation of the instrument being operated (which can also be called the operation of the device being controlled) for which timing correction is possible. Whether or not an operation is subject to timing correction is set in advance for each operation. For example, if the dispensing start timing, movement start timing, and dispensing end timing of the pipette 612 being operated can be changed, the control unit 102 displays, for example, icons for each timing adjustment on the display unit 204. 【0096】 Figure 18 shows an example of a screen displayed on the display unit 204. Figure 18 shows an example of a screen that accepts timing corrections. Here, it is assumed that the dispensing start timing, movement start timing, and dispensing end timing of the pipette 612 can be changed. The screen in Figure 18 displays icons for timing adjustment. The timing adjustment icons include a first dispensing start icon 681, a second dispensing start icon 682, a first movement icon 683, a second movement icon 684, a first dispensing end icon 685, and a second dispensing end icon 686. 【0097】 The first dispensing start icon 681 is an icon that instructs to start dispensing by pipette 612 a predetermined time earlier. The second dispensing start icon 682 is an icon that instructs to start dispensing by pipette 612 a predetermined time later. The first movement icon 683 is an icon that instructs to start moving pipette 612 a predetermined time earlier. The second movement icon 684 is an icon that instructs to start moving pipette 612 a predetermined time later. The first dispensing end icon 685 is an icon that instructs to end dispensing by pipette 612 a predetermined time earlier. The second dispensing end icon 686 is an icon that instructs to end dispensing by pipette 612 a predetermined time later. 【0098】The screen in Figure 18 displays the pipette 612, the well 631 of the well plate 614, and the liquid 650 contained in the well 631, all of which are the objects of the operation as captured by the camera of the detection device 400. Furthermore, the screen in Figure 18 also displays the pathway 665 of the pipette 612, which is the object of the operation. 【0099】 In S1002, the control unit 102 determines whether or not the input unit 202 has detected a timing correction input. For example, when a screen like the one in Figure 18 is displayed, timing correction input is made by selecting each icon. If a timing correction input is detected (S1002; YES), the process proceeds to S1003. If a timing correction input is not detected (S1002; NO), the process ends. 【0100】 In S1003, the control unit 102 changes the timing of the operation of the pipette 612, which is the target of the operation, that is, the timing of the operation of the robot device 300, which is the target of the control, based on the information detected in S1002. If the selection of the first dispensing start icon 681 is detected, the timing of the start of dispensing by the pipette 612 is advanced by a predetermined time. In other words, the timing of the operation of the robot device 300 is adjusted so that the timing of the start of dispensing by the pipette 612 is advanced by a predetermined time. If the selection of the second dispensing start icon 682 is detected, the timing of the start of dispensing by the pipette 612 is delayed by a predetermined time. If the selection of the first movement icon 683 is detected, the timing of the start of movement of the pipette 612 is advanced by a predetermined time. If the selection of the second movement icon 684 is detected, the timing of the start of movement of the pipette 612 is delayed by a predetermined time. If the selection of the first dispensing end icon 685 is detected, the timing of the end of dispensing by the pipette 612 is advanced by a predetermined time. If the selection of the second dispensing completion icon 686 is detected, the timing of the end of dispensing by the pipette 612 is delayed by a predetermined time. 【0101】In S1004, the control unit 102 stores the current scenario in the storage unit 104 as a backup. The control unit 102, for example, changes the scenario name of the current scenario so that it can be recognized as a backup, and stores it in the storage unit 104. In S1005, the control unit 102 creates a new scenario that reflects the timing of the changes made to the current scenario, and stores it in the storage unit 104. This creates the modified scenario. 【0102】 The control device 100 can smoothly connect each operation of the pipette 612 by correcting the timing described above. Furthermore, by correcting the timing for each operation of the pipette 612, such as the start of dispensing, the start of movement, and the end of dispensing, the control device 100 can bring the operation of the device that is the target of control based on the scenario (which can also be called the operation of the instrument that is the target of operation) closer to human operation (the operation desired by the user). 【0103】 <Icon Creation> The control unit 102 of the control device 100 counts the number of times each correction (speed correction, trajectory correction, position correction, etc.) is performed in manual operation mode. For example, each time a scenario backup is stored in the storage unit 104 (S604, S709, S807, S1004), the control unit 102 counts the number of times each correction is performed. If the number of times any correction is performed exceeds a predetermined number, the control unit 102 performs a trend analysis for that correction. The predetermined number is, for example, 20 times. For example, the case where the number of times position correction is performed exceeds a predetermined number will be explained. 【0104】 At this time, the control unit 102 extracts all scenarios stored in the memory unit 104 when position correction was performed. The control unit 102 analyzes the user's correction tendencies from the extracted scenarios using AI (Artificial Intelligence) or the like. For example, suppose the analysis results in a high number of position corrections (e.g., more than a predetermined number of times) where the starting position is moved 5 cm to the right. The predetermined number of times is, for example, 5 times. Alternatively, it may be more than a predetermined percentage (the ratio of the number of such corrections to the total number of corrections). This position correction is an example of frequent correction. 【0105】 At this time, the control unit 102 generates a new icon for executing the position correction and displays it on the display unit 204 along with the speed correction icon 30, trajectory correction icon 40, position correction icon 50, timing correction icon 60, etc., as shown in Figure 8. When the new icon is selected as an event in manual operation mode (S405 in Figure 7), the control unit 102 generates a scenario that reflects the position correction (modified to perform the corrected operation) and stores it in the memory unit 104. The same applies to other corrections (speed correction, trajectory correction). This makes it easy for the user to perform frequently performed correction processes. 【0106】 (Operation and Effects of the Embodiment) The control device 100 of the information processing system 10 creates a scenario to be executed by the robot device 300 that applies an action to the object of processing, based on user input to a terminal device 200, etc. The scenario is a task program that shows the procedure of operation by the robot device 300, etc., which is the object of control. The scenario (task program) created by the control device 100 defines the operation of the object of control, including parameters such as the position, trajectory, and speed of the object of control. In other words, the scenario created by the control device 100 defines the operation of the device that is the object of operation, including parameters such as the position, trajectory, and speed of the device that is the object of operation. 【0107】 Furthermore, the control device 100 of the information processing system 10 receives input from the user to correct the operation of the robot device 300 based on the created scenario, and corrects (changes) the scenario. When the control device 100 receives touch input from the input unit 202 during the execution of the scenario, it switches to manual operation mode, instructs the control device to stop operating based on the scenario, and accepts input for parameter correction. By reducing the operating speed of the controlled device, it becomes easier for the user to correct the scenario parameters. At this time, the speed of the controlled device is changed with maximum acceleration. This makes the transition to manual operation mode smooth while suppressing the user's awareness of the mode transition. 【0108】 Furthermore, the control device 100 can continuously correct the scenario while it is running. Also, when the control device 100 switches to manual operation mode, it displays the parameters that can be changed during the operation of the device on the display unit 204. The user can appropriately change the parameters of the scenario (task program) by selecting the changeable parameters and inputting them into the input unit 202. 【0109】 The control device 100 allows the user to easily change the current scenario to a new scenario that performs the actions desired by the user by having the user correct (change) the scenario parameters while the scenario is running. The user can correct the scenario parameters while viewing the operation of the device based on the scenario on the video captured by the camera of the detection device 400 displayed on the display unit 204. By being able to correct (change) the scenario parameters while the scenario is running, the scenario can be corrected efficiently. 【0110】 The control unit 102 analyzes the trend of any correction if the number of times the correction has been performed exceeds a predetermined number. Based on the analysis, the control unit 102 generates a new icon for frequently performed corrections and displays it on the display unit 204 along with existing icons such as the speed correction icon 30. In manual operation mode, if the new icon is selected as an event, the control unit 102 generates a scenario in which the correction is performed and stores it in the memory unit 104. This makes it easy for the user to perform frequently performed correction processes. 【0111】 <Computer-Readable Recording Medium> A program that enables a computer or other machine or device (hereinafter referred to as "computer, etc.") to perform any of the above functions can be recorded on a recording medium that is readable by a computer, etc. Then, by having the computer, etc. read and execute the program on this recording medium, the function can be provided. 【0112】Here, a recording medium readable by a computer refers to a recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical means, and can be read by a computer. Such a recording medium may contain computer components such as a CPU and memory, and the CPU may be used to execute programs. 【0113】 Furthermore, among such recording media, those that can be removed from computers, etc., include, for example, flexible disks, magneto-optical disks, CD-ROMs, CD-R / Ws, DVDs, DATs, 8mm tapes, and memory cards. In addition, recording media that are fixed to computers, etc., include hard disks and ROMs. 【0114】 [Embodiment 2] (Overview of the Information System) The information processing system according to the present invention comprises an operating device (also called an operating unit) which is the object to be controlled, and a control device which controls the operation of the operating device. The system processes information for controlling the operation of the operating device based on user input and detection results from various sensors. More specifically, the operating device, which is the object to be controlled, is a device that holds a processing instrument for processing an object (object to be processed) as the object to be operated, and operates the held processing instrument (object to be operated). This operating device can also be called an automation device for automating processing of an object. The information processing system comprising the operating device and the control device can also be called an automation system. 【0115】 As described in Embodiment 1, the information processing system according to the present invention is used, for example, in fields such as cell manufacturing (cell culture) in life sciences. However, the application fields of the information processing system according to the present invention are not particularly limited. The information processing system according to the present invention can also be applied to fields other than life sciences. 【0116】As schematically shown in Figure 19, the robot device 300, which is an example of an operating device included in the information processing system according to Embodiment 2, performs an operation to peel cultured adherent cells from the petri dish 850 using a scraper 800 as a peeling step, which is one of the cell culture steps (hereinafter also referred to as scraping operation). More specifically, the robot device 300 operates the scraper 800, which is a processing instrument to be operated on, and moves the scraper 800 along the surface of the petri dish 850 while keeping it in contact with the surface (bottom surface) 851 of the petri dish 850, thereby performing a scraping operation to peel adherent cells from the petri dish 850. In this scraping operation, the petri dish 850 that the scraper 800 contacts is considered to be included in the above-mentioned object together with the adherent cells. 【0117】 Furthermore, the robot device 300 according to Embodiment 2 is configured to hold the petri dish 850 so as to be operable independently of the scraper 800, and to be able to adjust the position and orientation of the petri dish 850 as appropriate. The robot device 300 according to Embodiment 2 has an arm 310 that holds the scraper 800, which is a processing instrument, and an arm 320 that holds the petri dish 850, which is the object to be processed, and is configured so that these pair of arms 310 and 320 can be operated independently. The information processing system 10 may also include a plurality of robot devices 300 that individually hold the scraper 800 and the petri dish 850. In any case, the information processing system 10 according to Embodiment 2 is configured so that scraping can be performed by moving the scraper 800, which is the object to be operated, relative to the petri dish 850 using the robot device 300. During scraping, the robot device 300 may move or change the orientation of the petri dish 850. In this case, the Petri dish 850 used for cell culture is included in the object as described above, but it also corresponds to an object manipulated by the robotic device 300. Note that the Petri dish 850 does not necessarily have to be held by the robotic device 300; for example, it may be fixed in a predetermined position such as on the table 620. 【0118】The configuration of the information processing system 10 according to this second embodiment is the same as that of the information processing system according to the first embodiment, and includes a control device 100, a terminal device 200, a robot device 300 to be controlled, and a detection device 400. The control device 100 includes a control unit 102, a storage unit 104, and a communication unit 106. The terminal device 200 includes an input unit 202 and a display unit 204. The control device 100 is connected to the terminal device 200, the robot device 300, and the detection device 400 in a communicative manner (see Figures 1 and 2). 【0119】 Furthermore, as described in Embodiment 1, in the information processing system 10, for example, the control device 100 creates a scenario to be executed by the robot device 300, which is the control target of the control device 100, based on user input to the terminal device 200, and controls the operation of the robot device 300 based on the scenario. In addition, when the control device 100 is controlling the operation of the robot device 300 based on the scenario, if it receives input from the user to correct the operation to the terminal device 200, it modifies the scenario. 【0120】Here, the cell culture process includes multiple steps, including a detachment process, and user intervention, specifically correction of the robotic device 300's operation, may be required in each step. In other words, real-time correction of the scenario by the user may be required during scenario execution. For example, the processing operations in the detachment process are difficult to automate, and user correction of the robotic device 300's operation (real-time scenario correction) is often necessary. The time required for the user to correct the robotic device 300's operation often gradually decreases with repeated corrections. However, depending on the content of the process, the learning effect from repeated corrections may be small, and the time required for correction may not be shortened even with repeated corrections. For example, when correcting the operation of the robotic device 300 based on positional information, the correction is performed through trial and error, and the correction time tends not to be shortened. Also, for example, in the detachment process, positioning is required according to the situation at the time, so the correction time tends not to be shortened even with repeated corrections. In contrast, the information processing system 10 according to Embodiment 2, as will be explained in detail below, corrects the operation of the robot device 300 based on pressure information instead of position information, thereby making it easier to shorten the correction time. 【0121】 (Control method for the robot device) The information processing system 10 according to Embodiment 2 differs from Embodiment 1 in its method of controlling the robot device 300 by the control device 100. For example, when performing a scraping operation, the control device 100 according to Embodiment 2 basically controls the robot device 300 based on a created scenario. However, for the operation in the direction normal to the surface (bottom surface) 851 of the petri dish 850 that the scraper 800 contacts (hereinafter also referred to as the direction normal to the ground surface), PID control using feedback is performed based on the detection result by the force sensor 350 provided on the robot device 300. 【0122】Here, the scraping operation is performed, for example, by moving the scraper 800 while the petri dish 850 is fixed in a predetermined position. The force sensor 350 is provided, for example, on the arm 310 of the robot device 300 that holds the scraper 800, as shown in an example in Figure 19. The force sensor 350 detects the pressure generated when the scraper 800 comes into contact with the surface (bottom) 851 of the petri dish 850. In other words, the force sensor 350 detects the reaction force received from the petri dish 850 when the scraper 800, held by the arm 310, comes into contact with the petri dish 850. It can also be said that the force sensor 350 detects the pressure received by the adherent cells as the target object when the adherent cells cultured in the petri dish 850 are peeled off by the scraper 800. 【0123】 The force sensor 350 is an existing design, so a detailed explanation will be omitted. However, it is composed of multiple sensors, such as strain sensors, that convert pressure into electrical signals, allowing for quantitative measurement of pressure. For example, the force sensor 350 can be a sensor (6-axis force sensor) that can detect force (pressure) in the three axial directions (x, y, and z axes) and moments around these three axes. Using a 6-axis force sensor as the force sensor 350 makes it easier to more accurately detect the pressure generated when the processing device comes into contact with the object. However, the force sensor 350 does not necessarily have to be a 6-axis force sensor; for example, a 3-axis force sensor that can detect force in three axial directions can also be used. 【0124】The control device 100 then acquires the normal pressure Pn generated in the direction normal to the ground surface, as shown in Figure 20, based on the detection result (force value) from the force sensor 350, and controls the operation (movement) of the robot device 300 in the direction normal to the ground surface based on the acquired normal pressure Pn. In other words, the control device 100 controls the displacement of the robot device 300 in the direction normal to the ground surface by PID control using feedback, with the component in the direction normal to the ground surface of the force vector sensed by the force sensor 350 as the detection parameter. As an example, the control device 100 controls the operation of the robot device 300 in the direction normal to the ground surface so that the value of the normal pressure Pn becomes a constant value. This makes it easier to set the pressure at which the processing tool comes into contact with the object to the user's desired pressure in various processing operations, such as scraping. 【0125】 Furthermore, in order to ensure the smooth operation of the robot device 300, it is preferable that only the displacement of the robot device 300 in the direction normal to the ground surface is controlled based on the detection results of the force sensor 350, and that other operations of the robot device 300 be performed as prescribed, regardless of whether it is during scenario execution or correction intervention. In addition, the direction of operation of the robot device 300 controlled by the control device 100 based on normal pressure does not necessarily have to be in the direction normal to the ground surface. The direction of operation of the robot device 300 controlled by the control device 100 based on normal pressure can be any predetermined direction, for example, a direction inclined at a predetermined angle with respect to the direction normal to the ground surface. 【0126】 (Control Flow of the Robot Device) Next, with reference to Figure 21, an example of controlling the robot device based on the detection results of the force sensor will be described. Figure 21 is a flowchart illustrating an example of controlling the robot device during scraping. 【0127】For example, once the petri dish 850 is fixed in a predetermined position and the scraper 800 is positioned at the starting point of the scraping operation, and the scraping operation begins, as shown in Figure 21, in step S1101, the control unit 102 first specifies the end point of the scraping operation based on a pre-created scenario and outputs a movement command to the robot device 300. Upon receiving the movement command, the robot device 300 moves the scraper 800 along the path based on the scenario. While the scraper 800 is moving by the robot device 300, the pressure generated by the contact between the scraper 800 and the petri dish 850 is detected by the force sensor 350. Pressure detection by the force sensor 350 is performed at predetermined intervals until the scraping operation is completed. 【0128】 Next, in step S1102, the control unit 102 acquires the pressure (force value) detected by the force sensor 350. More specifically, at the contact point between the scraper 800 and the petri dish 850, the pressure generated by the contact between the scraper 800 and the petri dish 850 is detected by the force sensor 350. The control unit 102 acquires the detection result from the force sensor 350 as a force value. 【0129】 Here, since the force sensor 350 is attached to the arm 310 of the robot device 300, an inertial force is generated in conjunction with the movement of the arm 310. This inertial force may cause errors in the force values ​​detected by the force sensor 350. Therefore, it is desirable to suppress the generation of errors due to inertial force when detecting pressure with the force sensor 350. For example, when creating a scenario, the inertial force due to the arm's basic movement can be measured and quantified in advance, and when executing the scraping operation, the difference caused by the inertial force can be removed from the force values ​​through calculation. Also, the inertial force acts in the opposite direction to the arm's acceleration. For this reason, for example, the acceleration can be dynamically calculated by differentiating the arm's velocity, and the force values ​​can be corrected in the direction of acceleration to reduce errors due to inertial force. Furthermore, since the inertial force is a force that acts due to the arm's acceleration, the acceleration and deceleration of the arm can be limited to a predetermined value or less while the force sensor 350 is detecting pressure. 【0130】《Identification of Force Sensor Values》 In this example, the force sensor values, which are the detection results of the force sensor 350 acquired by the control unit 102, are defined as spherical coordinates (polar coordinates) in the local coordinate system. 【0131】 The position of the robot device 300 is defined, for example, in a world coordinate system with the base as the origin. In contrast, the direction and magnitude of the pressure detected by the force sensor 350 are specified in a local coordinate system with the orientation of the object, the petri dish 850, as the reference. For example, it is specified in local coordinates with the point of contact between the scraper 800 and the petri dish 850 as the origin. For example, as shown in Figure 22, if the petri dish 850 is positioned at an angle in the world coordinate system, the orientation of the surface 851 of the petri dish 850 is defined in the xy-plane of the local coordinate system, and the direction of the normal to the ground surface is defined as the z-axis direction of the local coordinate system. Therefore, the normal pressure Pn generated in the direction of the normal to the ground surface is expressed as a vector component (force vector) along the z-axis direction. In Figure 22, each axis of the world coordinate system is shown with uppercase XYZ, and each axis of the local coordinate system is shown with lowercase xyz. 【0132】 Furthermore, it is preferable that the pressure detected by the force sensor 350 be defined as a spherical coordinate system (polar coordinate system) in such a local coordinate system. That is, it is preferable that the normal pressure Pn be defined as a spherical coordinate system in the local coordinate system. As shown in Figure 23, a point in a three-dimensional Cartesian coordinate system can be converted to spherical coordinates (polar coordinates) using the radial radius r, polar angle θ, and azimuth angle φ. 【0133】For example, as in this embodiment, if the world coordinates of the contact point of the petri dish 850 can be output, such as when the petri dish 850, which is the object of contact with the scraper 800, is fixed in a predetermined position while being held by the robot device 300, then, as shown in Figure 23, the degree to which the surface of the petri dish 850, which is the object of contact with the scraper 800, is inclined with respect to the XY plane and Z axis of the world coordinate system is converted into a specification using the polar angle θ and azimuth angle φ. This makes it possible to define the local coordinate system, which is based on the orientation of the petri dish 850, as a spherical coordinate system (polar coordinate system) based on the XYZ axes of the world coordinate system. By defining the pressure (force value) detected by the force sensor 350 in this way as a spherical coordinate system, it becomes easier to extract the normal pressure more accurately. 【0134】 Furthermore, while it is possible to detect the coordinates of the contact points of the petri dish 850, such as when the petri dish 850 held by the robot device 300 is moved during the scraping operation, if the coordinates of the petri dish 850 change over time, the local coordinates can be defined in the following way, for example. When creating the scenario, the trajectory of the scraper 800 while it is in contact with the curved surface of the petri dish 850 is stored. That is, the trajectory of the arm 310 that holds the scraper 800 is stored. Then, when executing the scraping operation, the orientation of the surface of the petri dish 850 is constantly calculated from the stored trajectory, and the direction of the normal to the contact surface is determined based on the calculation result. For example, as shown in Figure 24, if the surface of the petri dish 850 is curved, the direction of the normal to the contact surface may be changed depending on the contact point between the scraper 800 and the petri dish 850. In the example shown in Figure 24, the direction of the normal to the contact surface at contact point P1 is defined as the z1 direction, and the direction of the normal to the contact surface at contact point P2 is defined as the z2 direction. In this way, by appropriately determining the direction of the normal to the contact surface through calculation, the direction of the contact surface can be defined more accurately. In other words, the local coordinate system, with the point of contact between the scraper 800 and the petri dish 850 as the origin, can be defined more accurately as a spherical coordinate system (polar coordinate system) based on the XYZ axes of the world coordinate system. 【0135】Even if the surface of the petri dish 850 is curved, the surface of the petri dish 850 may be considered as a uniform flat surface, and the direction of the normal to the contact surface may be defined as a constant direction regardless of the contact position of the scraper 800 with respect to the petri dish 850. 【0136】 Furthermore, if the world coordinates of the contact points of the petri dish 850 cannot be output, such as when the petri dish 850 is fixed to a table, objects with polar angles θ and azimuth angles φ that are placed at multiple locations in the world coordinate system are defined during the scenario creation stage. During the scraping operation, the robot device 300 determines and selects the object that the scraper 800 will contact from among the predefined multiple objects. This allows the local coordinate system to be defined based on the world coordinates of the selected object. 【0137】 Next, in step S1103, the control unit 102 converts the acquired force values ​​into world coordinate notation by spherical coordinate (polar coordinate) calculation. As described above, the force values, which are the detection results of the force sensor 350 acquired by the control unit 102, are defined as spherical coordinates in the local coordinate system. In step S1103, the force values ​​defined as spherical coordinates in the local coordinate system are converted into world coordinate notation. 【0138】 Next, in step S1104, the control unit 102 derives detection parameters using intrinsic Euler angles. More specifically, in a local coordinate system based on the petri dish 850, the x, y, and z axis components of the force sensor 350 are identified, and the normal pressure, which is the force component in the direction normal to the ground surface, is extracted from the identified x, y, and z axis components. That is, the normal pressure, which is the force component in the direction normal to the ground surface, is extracted as a detection parameter used in PID control. 【0139】The local coordinates, based on the petri dish 850 which is the object of contact, are obtained by rotating the world coordinates by a predetermined angle around the z axis (azimuth angle φ) and then by a predetermined angle around the y axis (polar angle θ). Therefore, as shown in equation (1) below, by applying the intrinsic Euler angles around the z axis and then around the y axis to the matrix of force components expressed in world coordinates, the force components can be represented as the local coordinates of the petri dish 850 which is the object of contact. In this embodiment, since the direction of the normal pressure, which is a detection parameter used in PID control, coincides with the Z axis of the local coordinate system, the z-axis component of the force value detected by the force sensor 350, which has been converted to local coordinates, becomes the detection parameter. 【0140】 【0141】 Next, in step S1105, the control unit 102 determines whether the measured value of the normal pressure, which is the derived detection parameter, is equal to the target value (also called a specified value) set during scenario creation. If the measured value of the detection parameter is equal to the target value (S1105; YES), there is no need to adjust the operation of the robot device 300, so the process returns to step S1102, and force values ​​are acquired from the force sensor 350 at a predetermined timing. The detection interval for pressure (force values) by the force sensor 350 can be set arbitrarily, but for example, it is set to about 50 ms or more. On the other hand, if the measured value of the detection parameter is not equal to the target value (S1105; NO), the process proceeds to step S1106, where the control unit 102 derives the control parameters necessary for executing PID control based on the deviation (difference) between the measured value of the detection parameter and the target value. In other words, in the information processing system 10 according to Embodiment 2, the control parameters necessary for executing PID control are the displacement in the direction normal to the ground surface at time t, and are derived based on the normal pressure, which is the detection parameter. 【0142】Next, in step S1107, the control unit 102 converts the derived control parameters into world coordinate notation. As described above, the control parameters are derived based on the measured value of the normal pressure, which is a detected parameter, and the target value (target pressure), and the normal pressure is expressed as spherical coordinates in the local coordinate system. For this reason, the control parameters are also expressed as spherical coordinates in the local coordinate system. Therefore, in step S1107, the control parameters expressed as spherical coordinates in the local coordinate system are converted into world coordinate notation. Next, in step S1108, the control unit 102 calculates the world coordinates of the destination of the robot device 300 from the converted world coordinate notation of the control parameters. 【0143】 Next, in step S1109, the control unit 102 moves the robot device 300 to the coordinates (world coordinates) specified by the inverse motion function. In other words, the control unit 102 specifies the world coordinates to which the robot device 300 is to be moved, calculated in step S1107, as an argument, and applies the inverse motion function to move the arm 310 of the robot device 300 to a predetermined position in the direction normal to the surface of the petri dish 850, which is the object that the scraper 800 will contact. 【0144】 Next, the process proceeds to step S1110. If scraping is not yet complete (S1110; NO), steps S1102 to S1109 are repeated. The robotic device 300 then positions the scraper 800 at the end point of the scraping operation. Once the scraping operation is complete (S1110; YES), the series of processes is terminated. 【0145】As described above, in the information processing system 10 according to Embodiment 2, a force sensor 350 provided on the robot device 300 detects the pressure generated when the scraper 800, which is a processing tool, comes into contact with the petri dish 850, which is the object to be processed. The control device 100 obtains the normal pressure generated in the direction normal to the ground surface from the detection result by the force sensor 350, and controls the operation of the robot device 300 based on the obtained normal pressure. This makes it easier to appropriately adjust the pressure generated when the processing tool comes into contact with the object to be processed. Therefore, it becomes easier to make the operation of the robot device 300 based on a scenario closer to human actions (actions desired by the user). 【0146】 In particular, in this embodiment, since the control device 100 adjusts the amount of movement of the robot device 300 in the direction normal to the ground surface based on the normal pressure, in other words, since the direction of movement of the robot device 300 controlled by the control unit 102 based on the normal pressure coincides with the direction of the normal pressure, it becomes easier to adjust the pressure generated when the processing device comes into contact with the object in a shorter time and more appropriately. As a result, it is possible to reduce the impact on the operation of the automated machine and make it easier for the user to correct the operation of the machine. 【0147】 Furthermore, by having the control device 100 control the operation of the robot device 300 based on the normal pressure, the time required for correction operations is easily shortened. Consequently, as shown in Figure 26, correction operations in multiple independent series of operations can be performed in parallel, resulting in improved work efficiency. As shown in Figure 26, in the comparative example where feedback control (force feedback control) based on the detection result (pressure information) of the force sensor 350 is not performed, the correction operation time may be long in a series of operations such as a culture process. As described above, when the operation of the robot device 300 is corrected based on position information, the correction operation time may be long. For this reason, in the comparative example, when multiple series of operations are performed in parallel, the timing of each correction operation in multiple series of operations tends to overlap. Therefore, it is difficult for the user to perform multiple series of operations in parallel, and it is necessary to perform multiple series of operations individually and sequentially. 【0148】In contrast, in the embodiment where force feedback control is performed, as shown in Figure 26, the time for each correction operation in a series of operations is shortened, and the time for the scenario operation is lengthened. Therefore, in the embodiment, even if multiple series of operations are performed in parallel, the timing of the execution of each correction operation in multiple series of operations is unlikely to overlap. For example, during the scenario operation time of the first series of operations, correction operations in other series of operations (second series of operations to the nth series of operations) performed in parallel with the first series of operations can be executed. In this way, in the embodiment, correction operations in multiple series of operations can be executed in parallel, thereby improving work efficiency. 【0149】 In this embodiment, the force sensor 350 is provided on the arm 310 of the robot device 300 that holds the scraper 800, which is a processing tool. However, the arrangement of the force sensor 350 is not limited to this. For example, as shown in Figure 25, the force sensor 350 may be provided on the arm 320 of the robot device 300 that holds the petri dish 850, which is the object that the processing tool will come into contact with. Of course, the force sensor 350 may also be provided on both the arm 310 and the arm 320 of the robot device 300. Even with such a configuration, adjusting the movement of the robot device 300 in the direction normal to the ground surface based on the normal pressure makes it easier to appropriately adjust the pressure generated when the processing tool comes into contact with the object. 【0150】 Furthermore, although this embodiment describes an example in which scraping is performed by moving the arm 310 that holds the scraper 800, scraping can also be performed by moving the arm 320 that holds the petri dish 850. In this case, it is preferable that the force sensor 350 is provided at least on the arm 320 that holds the petri dish 850. With such a configuration, by adjusting the movement of the robot device 300 in the direction normal to the ground surface based on the normal pressure, it becomes easier to appropriately adjust the pressure generated when the processing instrument comes into contact with the object. 【0151】Incidentally, the information processing system 10 according to Embodiment 2 can also be switched to manual operation mode, as described in Embodiment 1. Specifically, the storage unit 104 of the information processing system 10 according to Embodiment 2 stores a scenario (task program) that defines the operation of the robot device 300, including the normal pressure parameter. When the control unit 102 is controlling the operation of the robot device 300 based on the scenario, it issues an instruction to stop the operation of the robot device 300 based on the scenario based on the user's request, switches to manual operation mode, and corrects the parameters, including the target value of the normal pressure defined in the scenario. 【0152】 The display unit 204 displays various parameters, such as the measured and target values ​​of the normal pressure, along with images of the scraper 800, petri dish 850, and robotic device 300. The user can recognize the measured and target values ​​of the normal pressure and other parameters from the display unit 204 and change the parameters as needed. In this example, the user can change the target value of the normal pressure by operating the jog dial, foot pedal, etc., which constitute the input unit 202. 【0153】In this embodiment, the operation of the robot device 300 in the direction normal to the ground surface is controlled based solely on normal pressure. That is, the amount of movement of the robot device 300 in the direction normal to the ground surface is adjusted based solely on normal pressure. Therefore, by changing the target value of the normal pressure, the adjustment of the amount of movement by the robot device 300 is reflected in a relatively short time. For example, it becomes easier to respond to user requests such as wanting to change the normal pressure to a high value for just a moment. It becomes possible to make the operation at that moment the appropriate operation as envisioned by the user. In other words, it becomes easier for the user to correct the operation of the machine while reducing the impact on the operation of the automated machine. In particular, by inputting correction of operation by operating the input unit 202 such as a jog or foot pedal, the operation of the robot device 300 can be corrected in a short time (instantaneously). In other words, by operating the input unit 202 such as a jog or foot pedal, fine operations can be transmitted to the robot device 300 in real time. Note that the control of the operation of the robot device 300 in the direction normal to the ground surface does not necessarily have to be based solely on normal pressure, but may be based on multiple parameters including normal pressure. 【0154】 Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above and includes various modifications. The present invention is not necessarily limited to having all the configurations described above. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations. 【0155】For example, in Embodiment 2, the control (including correction) of a robot device (operating device) based on normal pressure was described using the adherent cell detachment process as an example. However, the control of the robot device according to Embodiment 2 can, of course, be applied to other processes. The control of the robot device according to Embodiment 2 can, for example, be applied to the sample removal process. The sample removal process referred to here is the process of removing the sample 862 from the test tube 861 by pressing the instrument 860 against the inner wall surface of the test tube 861 and pulling it out, as shown in Figure 27. Although not shown in the figure, the instrument 860 is held by the robot device 300 equipped with the force sensor 350 described above. In this case, the pressure generated when the instrument 860 contacts the inner wall surface of the test tube 861 is detected by the force sensor 350. The control device 100 then acquires the normal pressure Pn1 generated in the direction normal to the inner wall surface of the test tube 861 (direction normal to the ground surface) based on the detection result (force value) from the force sensor 350, and controls the operation (movement) of the robot device 300 based on the acquired normal pressure Pn1. 【0156】 Furthermore, the control of the robot device 300 according to Embodiment 2 can also be applied, for example, to the process of opening and closing a test tube lid. The opening and closing process referred to here is the process of gripping the lid 863 of the test tube 861 with the gripping mechanisms 331 and 332 provided by the robot device 300 and rotating the lid 863 to open and close it, as shown in Figure 28. In this case, the pressure generated when the gripping mechanisms 331 and 332 grip the lid 863 of the test tube 861 is detected by the force sensor 350. The control device 100 then acquires the normal pressure Pn2 generated in the direction normal to the surface of the lid 863 of the test tube 861 (direction normal to the ground surface) based on the detection result from the force sensor 350, and controls the operation (movement) of the robot device 300 based on the acquired normal pressure Pn2. 【0157】10: Information processing system 30: Speed ​​correction icon 31: 50% icon 32: 100% icon 33: 200% icon 40: Trajectory correction icon 50: Position correction icon 60: Timing correction icon 90: Information processing device 91: Processor 92: Memory 93: Storage unit 94: Input unit 95: Output unit 96: Communication control unit 100: Control device 102: Control unit 104: Storage unit 106: Communication unit 200: Terminal device 202: Input unit 204: Display unit 300: Robot device 310, 320: Arm 350: Force sensor 400: Detection device 610: Instrument 611: Test tube 612: Pipette 613: Container 614: Well plate 615: Flask 620: Table 631: Well 650: Liquid 661: First path line 662: Second path line 663: Third path line 664: Fourth path line 670: Grid 671: Starting position 672: Ending position 681: First dispensing start icon 682: Second dispensing start icon 683: First movement icon 684: Second movement icon 685: First dispensing end icon 686: Second dispensing end icon 800: Scraper 850: Petri dish

Claims

1. An information processing system comprising: an operating device for holding a processing instrument for processing an object and for operating the processing instrument; and a control device for controlling the operation of the operating device, wherein the operating device is provided with a force sensor for detecting the pressure generated when the processing instrument comes into contact with the object as a result of the operation of the operating device, and the control device obtains a normal pressure generated in the direction normal to the surface of the object from the detection result of the force sensor and controls the operation of the operating device based on the normal pressure.

2. An information processing system according to claim 1, wherein the control device adjusts the amount of movement of the operating device in the normal direction.

3. An information processing system according to claim 1, wherein the control device controls the operation of the operating device so that the normal pressure becomes a target value set by the user.

4. An information processing system according to claim 3, wherein the control device identifies the normal pressure in a local coordinate system with respect to the object, converts the identified normal pressure into world coordinates with respect to the operating device, and controls the operation of the operating device based on the normal pressure converted into world coordinates.

5. An information processing system according to claim 4, wherein the control device converts the normal pressure identified in the local coordinate system into world coordinates using an inverse motion function.

6. An information processing system according to claim 1, wherein the operating device comprises a pair of arms that independently hold the processing instrument and the object, respectively, and the force sensor is provided on at least one of the pair of arms.

7. An information processing system according to claim 1, comprising a storage unit for storing a task program that defines the operation of the operating device, including the parameter of the normal pressure, wherein the control device, while controlling the operation of the operating device based on the task program, issues an instruction to stop the operation of the operating device based on the task program based on a user request, and corrects the parameter including the target value of the normal pressure defined in the task program.

8. An information processing system according to claim 7, comprising: a detection device including a camera; a terminal device having a display unit for displaying an image captured by the camera and an input unit for receiving input from the user, wherein the display unit displays parameters including a target value for the normal pressure, and the input unit accepts changes to the parameters including the target value for the normal pressure.

9. An information processing method for controlling the operation of an operating device that holds a processing instrument for processing an object and operates the processing instrument, wherein the operating device is equipped with a force sensor that detects the pressure generated when the processing instrument comes into contact with the object as a result of the operation of the operating device, and the normal pressure generated in the direction normal to the surface of the object is obtained from the detection result of the force sensor, and the operation of the operating device is controlled based on the normal pressure.

10. An information processing method according to claim 9, wherein the amount of movement of the operating device in the direction normal to the surface of the object is controlled based on the normal pressure.

Images