Designated device, robot system, designation method, and program

Abstract

This designation device, in a robot system for moving an object to be moved in accordance with a prescribed algorithm that corresponds to a work target, comprises a reception means for accepting input designating at least a portion of the external form of the object to be moved, and a control means for causing a display device to display external form accepted by the reception means and a two-dimensional image including the object to be moved.

Description

Technical Field

[0001] The present disclosure relates to a designated device, a robot system, a designation method, and program .

Background Art

[0002] Robots are used in various fields such as logistics. As a related technology, Patent Document 1 discloses a technology related to a robot system that can easily teach a desired operation.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, generally, in the case of a robot that grasps an object to be moved and places it at a destination, the state (position and posture) of the object before movement is often recognized using an automatic recognition system that uses an expensive camera called an industrial camera. However, for example, when the object to be moved is in contact with a plurality of objects, when there are both solid and soft objects among the objects to be moved, when lighting is reflected on the object to be moved, when the object to be moved has gloss, when the object to be moved is transparent, when the object to be moved is wrapped with a cushioning material, etc., even when using an industrial camera, it may be difficult to appropriately recognize individual objects.

[0005] Each aspect of the present disclosure aims to provide a designated device, a robot system, a designation method, and program that can solve the above problems.

Means for Solving the Problems

[0006] According to one aspect of this disclosure, the designated device is A robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective includes a receiving means that receives an input specifying at least a part of the external shape of the object to be moved, and a control means that displays a two-dimensional image including the object to be moved and the external shape received by the receiving means on a display device, wherein the receiving means receives an input that modifies the shape generated based on data held by an external system that manages the object to be moved. . According to another aspect of the present disclosure, the designated device comprises a receiving means for receiving input specifying at least a portion of the external shape of an object to be moved in a robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective, and a control means for displaying a two-dimensional image including the object to be moved and the external shape received by the receiving means on a display device, wherein the receiving means receives input that modifies the shape generated by an external system that photographs the object to be moved.

[0007] According to another aspect of the present disclosure, the robot system comprises the designated device, a robot capable of grasping an object to be moved, and a control device that causes the robot to grasp the object to be moved based on the external shape of the object to be moved received by the designated device.

[0008] According to another aspect of this disclosure, the method of designation is: A robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective includes: receiving an input specifying at least a part of the external shape of the object to be moved; receiving an input to change the shape generated based on data held by an external system that manages the object to be moved; and displaying a two-dimensional image including the object to be moved and the received external shape on a display device. . According to another aspect of the present disclosure, a designation method in a robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective, includes a computer receiving an input specifying at least a portion of the external shape of the object to be moved, receiving an input that modifies the shape generated by an external system that photographs the object to be moved, and displaying a two-dimensional image including the object to be moved and the received external shape on a display device.

[0009] According to another aspect of this disclosure, the program is In a robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective, the computer is made to perform the following actions: receive an input specifying at least a part of the external shape of the object to be moved; receive an input to change the shape generated based on data held by an external system that manages the object to be moved; and display a two-dimensional image including the object to be moved and the received external shape on a display device. . According to another aspect of the present disclosure, the program causes a computer to perform the following actions in a robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective: receiving an input specifying at least a part of the external shape of the object to be moved; receiving an input to modify the shape generated by an external system that photographs the object to be moved; and displaying a two-dimensional image including the object to be moved and the received external shape on a display device. [Effects of the Invention]

[0010] According to each aspect of this disclosure, a robotic system But You will be able to recognize your body correctly. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example of the configuration of a robot system according to the first embodiment of this disclosure. [Figure 2] This figure shows an example of the installation of a measuring device according to the first embodiment of this disclosure. [Figure 3] This figure shows an example of the configuration of a measuring device according to the first embodiment of this disclosure. [Figure 4] This figure shows an example of a region captured by the measuring device according to the first embodiment of this disclosure. [Figure 5] This figure shows an example of the configuration of the designated device according to the first embodiment of this disclosure. [Figure 6] This figure shows an example of an image displayed by the display unit according to the first embodiment of this disclosure. [Figure 7]It is a diagram showing an example of the configuration of a control device according to the first embodiment of the present disclosure. [Figure 8] It is a diagram showing an example of a data table stored in a storage unit according to the first embodiment of the present disclosure. [Figure 9] It is a diagram showing an example of the configuration of a robot according to the first embodiment of the present disclosure. [Figure 10] It is a diagram showing an example of the processing flow of a robot system according to the first embodiment of the present disclosure. [Figure 11] It is a diagram showing an example of the installation of a measuring device according to a modified example of the first embodiment of the present disclosure. [Figure 12] It is a diagram showing an example of an image displayed on a display unit according to a modified example of the first embodiment of the present disclosure. [Figure 13] It is a diagram showing an example of the configuration of a robot system according to the second embodiment of the present disclosure. [Figure 14] It is a diagram showing an example of the configuration of an automatic recognition system according to the second embodiment of the present disclosure. [Figure 15] It is a diagram showing an example of the installation of a camera according to the second embodiment of the present disclosure. [Figure 16] It is a diagram showing an example of an image displayed on a display unit according to the second embodiment of the present disclosure. [Figure 17] It is a diagram showing an example of an image displayed on a display unit according to a modified example of the second embodiment of the present disclosure. [Figure 18] It is a diagram showing an example of the configuration of a robot system according to the third embodiment of the present disclosure. [Figure 19] It is a diagram showing an example of the configuration of a WMS according to the third embodiment of the present disclosure. [Figure 20] It is a diagram showing an example of a data table stored in a storage unit according to the third embodiment of the present disclosure. [Figure 21] It is a diagram showing an example of an image displayed on a display unit according to the third embodiment of the present disclosure. [Figure 22] It is a diagram showing an example of an image displayed on a display unit according to a modified example of the third embodiment of the present disclosure. [Figure 23]This figure shows an example of the configuration of a robot system according to the fourth embodiment of this disclosure. [Figure 24] This figure shows an example of a destination determined by the control device according to the fifth embodiment of this disclosure. [Figure 25] This figure shows the minimum configuration of the designated device according to the embodiments of this disclosure. [Figure 26] This figure shows an example of the processing flow of a specified device with a minimum configuration. [Figure 27] This is a schematic block diagram showing the configuration of a computer according to at least one embodiment. [Modes for carrying out the invention]

[0012] The embodiments will be described in detail below with reference to the drawings. <First Embodiment> The robot system 1 according to the first embodiment of this disclosure is a system in which an operator can specify the state of an object before it is moved. The robot system 1 is a system that is introduced in a warehouse of a logistics center, for example, for the purpose of grasping an incoming or outgoing object and moving it to a predetermined location during receiving or shipping. For example, there is a technology called "goal-oriented task planning" that uses AI (Artificial Intelligence) technology to perform tasks that were previously performed by humans. When using this "goal-oriented task planning," the operator on site using the robot can simply instruct the robot on the work objective, and the robot will automatically perform the actions to achieve the work objective (i.e., the operator does nothing). Specifically, in the case where the robot grasps an object to be moved and places it at the destination, for example, if the robot is given the information "move three parts A to a tray" as the work objective, the robot will grasp the three parts A in order according to a predetermined algorithm corresponding to that work objective and move the object from its original position to the destination.

[0013] Robot system 1 is a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input. Robot system 1 may be a robot system that uses AI technology including temporal logic and reinforcement learning. In the first embodiment, the object M to be moved is assumed to be placed parallel to a plane P that is located almost horizontally (on the plane of a belt conveyor or tray, which will be described later).

[0014] (Robot system configuration) Figure 1 shows an example of the configuration of a robot system 1 according to a first embodiment of the present disclosure. As shown in Figure 1, the robot system 1 comprises a measuring device 10, a designating device 20, a control device 30, and a robot 40. The measuring device 10, the designating device 20, the control device 30, and the robot 40 are each connectable to one another via a network NW. The network NW in this disclosure is not limited to a communication network such as the Internet, but may be any network through which the necessary signals are transmitted and received. For example, some of the connections between the measuring device 10, the designating device 20, the control device 30, and the robot 40 may be directly connected by metal wiring, while the other connections may be made via a communication network.

[0015] (Configuration of the measuring device) Figure 2 shows an example of the installation of the measuring device 10 according to the first embodiment of the present disclosure. Figure 3 shows an example of the configuration of the measuring device 10 according to the first embodiment of the present disclosure. In the example shown in Figure 2, the measuring device 10 is installed in a fixed position that allows it to photograph the plane P of the tray T on which the object M to be moved is placed from above. That is, the cameras 101 and 102, which will be described later, are installed in fixed positions that allow them to photograph the plane P on which the object M to be moved is placed from above.

[0016] The operation of photographing plane P from above and moving the object to be moved to its destination is, in the case of incoming goods, performed by a human after unpacking the incoming containers and removing the packaging materials, and after taking out the individual products (hereinafter referred to as "loose items") from the unpacked containers, the human places the loose items on the conveyor belt in lots, and the robot system 1 sorts the loose items of each lot into trays corresponding to each lot. Examples of containers include cardboard boxes and trays. In this case, the loose items are the objects to be moved. The surface of the conveyor belt on which the loose items are placed is plane P. The trays are the destinations.

[0017] Furthermore, the process of photographing plane P from above and moving the object to be moved to its destination is an operation performed during shipping, for example, when multiple products destined for a certain location are placed into a single container. In the warehouse, incoming loose goods are stored in trays, organized by lot. Each loose item stored in the warehouse is a product, and during shipping, each tray containing the products to be shipped (i.e., loose items corresponding to multiple products) is sequentially transported to the position of robot system 1. In this case, the loose items transported to the position of robot system 1 in trays are the objects to be moved. The surface on which the loose items in the trays are placed after being transported to the position of robot system 1 is plane P. The container or other object is the destination.

[0018] Figure 2 shows a plane P, an object M to be moved placed on plane P, and a robot 40 that grasps the object M and moves it to a predetermined position. Figure 2 also shows the gripping part 402a of the robot 40, which will be described later. The measuring device 10 includes cameras 101 and 102, as shown in Figure 3. Cameras 101 and 102 may be housed in a single housing, as shown in Figure 2. Alternatively, cameras 101 and 102 may be housed in separate housings.

[0019] Camera 101 is a camera that captures a two-dimensional (2D; 2-dimensional) image that includes at least a portion of a plane P and a moving object M placed on plane P. Camera 101 transmits the information of the captured image to the designated device 20 via the network NW.

[0020] Camera 102 is a camera capable of measuring the depth of a moving object in the shooting direction. For example, camera 102 is a depth camera. A depth camera irradiates an object within its shooting area with light and measures the distance from camera 102 to the object based on the time from the irradiation of the light to the reception of the reflected light from the irradiated object (i.e., equivalent to a phase difference). In the first embodiment, the shooting area of ​​camera 102 is defined as a region R that includes at least a part of plane P and a moving object M placed on plane P. Note that the shooting area in which camera 101 captures a two-dimensional image is limited to any region R that includes at least the moving object M, and may be region R. In the following description, the shooting area in which camera 101 captures a two-dimensional image will be described as region R. Figure 4 is a diagram showing an example of region R captured by the measuring device 10 according to the first embodiment of this disclosure. As shown in Figure 4, region R includes the region in which plane P and the moving object M exist. Here, the lower left corner of region R is defined as the origin O, the horizontal axis as the X-axis, and the vertical axis as the Y-axis. The axis perpendicular to the XY plane is defined as the Z-axis. Note that the X-axis is positive when moving to the right from the origin on the paper. The Y-axis is positive when moving upwards from the origin on the paper. The Z-axis is positive when moving towards the front of the paper from the origin.

[0021] Camera 102 is installed in a fixed position. Therefore, within the shooting area, camera 102 can measure the height of the moving object M in the Z-axis direction relative to the XY plane by defining the area furthest from camera 102 as plane P, within an error range greater than or equal to the processing accuracy of plane P and less than or equal to the size of object M, and calculating the difference between the distance from camera 102 to the moving object M in the area of ​​the moving object M specified as described later, and the distance from camera 102 to plane P. For example, by identifying the area in which object M exists, camera 102 can measure the height of the moving object M in the Z-axis direction relative to the XY plane by calculating the difference between the distance from camera 102 to object M and the distance from camera 102 to plane P. Examples of methods for identifying the area in which object M exists include pre-setting the spatial area in which object M is placed and excluding information from other areas, using an automatic recognition means (for example, a means for recognizing an object based on 3D CAD (Computer Aided Design) information of the target object) to identify the position of object M by the degree of fit with the point cloud shape, or identifying the spatial area in which object M is placed from an image of object M. Examples of the camera 102 include a camera that estimates distance using a stereo camera, and a camera that estimates distance based on the time it takes for light to reflect back after shining light onto an object. The camera 102 transmits information indicating the measurement result (i.e., information indicating the height of object M) to the control device 30 via the network NW.

[0022] Alternatively, instead of using camera 102, LiDAR (Light Detection and Ranging) may be used to measure the difference between the distance from LiDAR to object M and the distance from LiDAR to plane P, and the height of the moving object M may be calculated using this difference.

[0023] (Configuration of the specified device) Figure 5 shows an example of the configuration of a designated device 20 according to the first embodiment of this disclosure. As shown in Figure 5, the designated device 20 comprises a display unit 201 (an example of a display device), a generation unit 202, a control unit 203 (an example of a control means), and a reception unit 204 (an example of a reception means). The designated device 20 is, for example, a tablet terminal having touch panel functionality.

[0024] The display unit 201, under the control of the control unit 203, displays a two-dimensional image captured by the camera 101 and an image showing the outline F of the object M to be moved, which will be described later, input from the reception unit 204. The outline F is displayed only for the object M to be grasped, among multiple or one object M to be moved. Figure 6 is a diagram showing an example of an image displayed by the display unit 201 according to the first embodiment of this disclosure. In the example shown in Figure 6, the objects M to be moved are shown as objects M1 and M2, and the outline F of object M1. In addition, the region R is shown in the example shown in Figure 6. Note that the hand shown in Figure 6 is not displayed by the display unit 201, but is shown to represent an image of an operator using their finger to indicate the outline F on the touch panel.

[0025] The generation unit 202 generates a control signal Cnt1 to display the outline F of the moving object M on the display unit 201 along with the two-dimensional image, based on the information of the two-dimensional image captured by the camera 101 and a signal indicating the outline F of the moving object M generated by the reception unit 204 in response to an operation performed by an operator who generates the outline F of the moving object, which will be described later. In this disclosure, "ZZ YY together with XX" includes performing the ZZ process on XX and YY simultaneously, and performing the ZZ process on XX and YY separately. For example, "display YY together with XX" includes performing the process of displaying XX and YY simultaneously. Also, "display YY together with XX" includes performing the process of displaying XX, then performing the process of displaying YY, and then performing the process of displaying YY, and then performing the process of displaying XX. "XX" and "YY" are arbitrary elements (e.g., arbitrary information). Also, "ZZ" is an arbitrary process. Furthermore, while we have used "XX" and "YY" as examples here, for three or more arbitrary elements, the ZZ process can be performed simultaneously, separately, or with some elements performed simultaneously and the rest separately.

[0026] If the line representing the outline F of the object to be moved is not a straight line due to the operator's operation to generate the outline F of the object to be moved, the generation unit 202 may correct it to a straight line. When the generation unit 202 corrects the line representing the outline F to a straight line, the generation unit 202 generates a control signal Cnt1 that displays the outline F corrected to a straight line. As a result, the outline F of the object to be moved that the control unit 203 displays on the display unit 201 is also displayed as a straight line. However, when corrected to a straight line, the outline F displayed on the display unit 201 does not necessarily match the actual outline of the object to be moved. If they do not match, the operator may change the slope of the line representing the outline F displayed on the display unit 201 to match the actual outline of the object to be moved displayed on the display unit 201 by performing an operation on the reception unit 204. In response to this operation, the reception unit 204 generates a signal corresponding to that operation. The generation unit 202 generates a control signal Cnt1 that matches the outline F to the outline of the actual object M being moved, based on the signal generated by the reception unit 204.

[0027] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the outline F of the moving object M input from the reception unit 204 on the display unit 201, along with the two-dimensional image captured by the camera 101.

[0028] Furthermore, if the reception unit 204 has not generated a signal indicating the outline F of the object M to be moved, and the camera 101 is capturing a two-dimensional image, the generation unit 202 generates a control signal Cnt1 to display the two-dimensional image captured by the camera 101 on the display unit 201. In this case, the control unit 203 will display the two-dimensional image captured by the camera 101 on the display unit 201 based on the control signal Cnt1 generated by the generation unit 202.

[0029] The reception unit 204 receives input from the operator specifying at least a part of the outline of the object to be moved. For example, the reception unit 204 is a touch panel and receives operations from the operator, such as using their finger or a stylus designed for touch panels, to generate the outline of the object to be moved. Examples of operations to generate the outline of the object to be moved include tracing the outline of the object with a finger or stylus, or specifying the vertices of the object with a finger or stylus. When the operator specifies the vertices of the object to be moved with a finger or stylus to the reception unit 204, the generation unit 202 generates a control signal Cnt1 that displays a line connecting the two specified vertices each time the operator specifies two vertices, and the control unit 203 can then control the display on the display unit 201 based on the control signal Cnt1 generated by the generation unit 202. This control signal Cnt1 allows the control unit 203 to display the outline F of the object M to be moved on the display unit 201.

[0030] The reception unit 204 also receives input of work objectives. Examples of work objectives include information such as the type of object M to be moved, the quantity of that object, and the destination of the object. The reception unit 204 receives input such as "move 3 units of part A to the tray" as a work objective. In this case, the reception unit 204 may identify the object M to be moved by determining that the type of object M is part A, the quantity of that object is 3, and the destination of the object is the tray. The reception unit 204 transmits the received work objective to the control device 30.

[0031] (Control device configuration) The control device 30 receives information indicating a work objective and information indicating the state of the object M to be moved before it is moved (i.e., position and orientation). The control device 30 then causes the robot 40 to grasp the object M according to the received state of the object M before it is moved, and then causes the robot 40 to execute a process based on a predetermined algorithm corresponding to the received work objective (i.e., a process to move the grasped object M to a predetermined destination). Figure 7 shows an example of the configuration of the control device 30 according to the first embodiment of this disclosure. As shown in Figure 7, the control device 30 comprises a storage unit 301, an acquisition unit 302, an identification unit 303, and a control unit 304.

[0032] The storage unit 301 stores various information necessary for the processing performed by the control device 30. An example of the information stored by the storage unit 301 is the data table TBL1, which shows the correspondence between work objectives and algorithms and is used by the identification unit 303, described later, when identifying an algorithm according to the work objective. Figure 8 is a diagram showing an example of the data table TBL1 stored by the storage unit 301 according to the first embodiment of this disclosure. As shown in Figure 8, the storage unit 301 stores work objectives and algorithms in association as the data table TBL1.

[0033] The acquisition unit 302 acquires information indicating the state of the object to be moved before it is moved. Specifically, the acquisition unit 302 receives from the measuring device 10 the measurement results measured by the camera 102, that is, information indicating the height of the object M to be moved from the plane P. The acquisition unit 302 also receives from the designating device 20 information indicating the outer shape F of the object M to be moved. The acquisition unit 302 can determine the shape of the object M to be moved from the received information indicating the height of the object M to be moved from the plane P and the received information indicating the outer shape F of the object M to be moved.

[0034] Furthermore, the acquisition unit 302 receives information indicating the work objective (i.e., information indicating the type of object to be moved, the quantity of that object, and the destination of that object) from the designation device 20.

[0035] The identification unit 303 identifies the algorithm to be used to move the object to be moved to the destination, based on the work objective received by the acquisition unit 302. For example, if the work objective received by the acquisition unit 302 is work objective 1, the identification unit 303 identifies work objective 1 from the work objectives in the data table TBL1 stored by the storage unit 301. Then, the identification unit 303 identifies algorithm 1 associated with the identified work objective 1 in the data table TBL1.

[0036] The control unit 304 controls the robot 40 by transmitting a control signal Cnt2 to the robot 40 that corresponds to the algorithm identified by the identification unit 303. The control signal Cnt2 is a control signal that causes the robot 40 to grasp the object M to be moved and move the grasped object M to the destination specified by the operator. The control signal Cnt2 may be pre-prepared for each algorithm in the data table TBL1, or it may be generated each time by the control unit 304 according to the algorithm identified by the identification unit 303.

[0037] (Robot configuration) The robot 40 grasps an object M to be moved based on a control signal Cnt2 received from the control device 30 and moves the object M to a destination specified by the operator in the designation device 20. The process of moving the object M to the destination by the robot 40 continues until the number of objects specified by the work objective has been moved to the destination. Examples of the robot 40 include a vertical articulated robot, a horizontal articulated robot, and any other type of robot. Figure 9 shows an example of the configuration of the robot 40 according to the first embodiment of this disclosure. As shown in Figure 9, the robot 40 includes a generation unit 401 and a movable device 402.

[0038] The generation unit 401 receives a control signal Cnt2 from the control device 30. Based on the received control signal Cnt2, the generation unit 401 generates a drive signal Drv to operate the movable device 402 (i.e., to cause the movable device 402 to grasp the object M to be moved and move the object M to the destination). Note that when the generation unit 401 causes the gripping unit 402a to grasp the object M to be moved, as described later, the gripping unit 402a approaches the object M from a direction perpendicular to the position of the center of gravity of the surface showing the outer shape F of the object M to be moved (in the first embodiment, since the object M to be moved is placed parallel to the plane P, this is directly above the object M). Drive signal Drv Generates.

[0039] As shown in Figure 9, the movable device 402 includes a gripping section 402a. The gripping section 402a has a mechanism for gripping the object M to be moved. Examples of mechanisms for gripping the object M to be moved include a mechanism that holds the object M between multiple (e.g., two) fingers, and a mechanism that attracts a predetermined surface of the object M. Examples of a predetermined surface include the surface with the largest area among multiple surfaces of the object M to be moved included in the image captured by the camera 101, and a surface among multiple surfaces of the object M to be moved that is closer to parallel to the plane P. The movable device 402 is a device that grips the object M to be moved with the gripping section 402a based on the drive signal Drv generated by the generation unit 401, and moves the object M to the destination. For example, the movable device 402 is a robot arm having a stepping motor. If the movable device 402 is a robot arm having a stepping motor, the stepping motor operates in response to the drive signal Drv generated by the generation unit 401, causing the movable device 402 to grasp the object M to be moved with the gripping unit 402a and move the object M to the destination.

[0040] (Processing performed by the robot system) Figure 10 is a diagram showing an example of the processing flow of the robot system 1 according to the first embodiment of this disclosure. Next, the processing performed by the robot system 1 will be described with reference to Figure 10.

[0041] Camera 101 captures a two-dimensional image that includes a portion of plane P and a moving object M placed on plane P. Camera 101 transmits the information of the captured image to the designated device 20 via the network NW.

[0042] At this point, the reception unit 204 has not yet generated a signal indicating the outline F of the object M to be moved. Therefore, the generation unit 202 generates a control signal Cnt1 to display the two-dimensional image captured by the camera 101 on the display unit 201 (step S1). Then, the control unit 203 displays the two-dimensional image captured by the camera 101 on the display unit 201 based on the control signal Cnt1 generated by the generation unit 202 (step S2). The display unit 201 displays the two-dimensional image captured by the camera 101.

[0043] Camera 102 defines plane P as the region furthest from camera 102 within the shooting area, within an error range greater than or equal to the machining accuracy of plane P and less than or equal to the size of object M. By calculating the difference between the distance from camera 102 to the object M in the region of the specified object M and the distance from camera 102 to plane P, it measures the height of the object M in the Z-axis direction relative to the XY plane. Camera 102 transmits information indicating the measurement result (i.e., information indicating the height of object M) to the control device 30 via the network NW.

[0044] Here, suppose the reception unit 204 receives input from the operator specifying at least a part of the outline of the object to be moved (step S3). For example, the reception unit 204 is a touch panel and receives operations from the operator, such as using their finger or a stylus designed for the touch panel, to generate the outline of the object to be moved.

[0045] The generation unit 202 generates a control signal Cnt1 to display the outline F of the object M to be moved on the display unit 201 along with the two-dimensional image information captured by the camera 101 and a signal indicating the outline F of the object M to be moved, generated by the reception unit 204 in response to the operation performed by the operator generating the outline F of the object to be moved (step S4).

[0046] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the outline F of the object M to be moved, which is input from the reception unit 204, on the display unit 201 along with the two-dimensional image captured by the camera 101 (step S5). The display unit 201 displays the outline F of the object M to be moved, which is input from the reception unit 204, along with the two-dimensional image captured by the camera 101.

[0047] If the line representing the outline F of the object to be moved is not a straight line due to the operation performed by the operator to generate the outline F of the object to be moved, the generation unit 202 may correct it to a straight line. If the generation unit 202 corrects the line representing the outline F to a straight line, the generation unit 202 generates a control signal Cnt1 to display the outline F corrected to a straight line. Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the outline F of the object M to be moved, which has been corrected to a straight line, on the display unit 201 along with the two-dimensional image captured by the camera 101. The display unit 201 displays the outline F of the object M to be moved, which has been corrected to a straight line, along with the two-dimensional image captured by the camera 101.

[0048] Now, let's assume that the reception unit 204 has received the input for the work objective. The reception unit 204 transmits the received work objective to the control device 30.

[0049] The acquisition unit 302 acquires information indicating the state of the object to be moved before it is moved. Specifically, the acquisition unit 302 receives from the measuring device 10 the measurement results measured by the camera 102, i.e., information indicating the height of the object M to be moved from the plane P. The acquisition unit 302 receives from the designating device 20 information indicating the outer shape F of the object M to be moved. The acquisition unit 302 receives from the designating device 20 information indicating the work objective (i.e., information indicating the type of object to be moved, the quantity of that object, and the destination of that object).

[0050] The identification unit 303 identifies the algorithm to be used to move the object to be moved to the destination, based on the work objective received by the acquisition unit 302. For example, if the work objective received by the acquisition unit 302 is work objective 1, the identification unit 303 identifies work objective 1 from the work objectives in the data table TBL1 stored by the storage unit 301. Then, the identification unit 303 identifies algorithm 1 associated with the identified work objective 1 in the data table TBL1.

[0051] The control unit 304 controls the robot 40 by transmitting a control signal Cnt2 to the robot 40 that corresponds to the algorithm identified by the identification unit 303. The control signal Cnt2 is a control signal that causes the robot 40 to grasp the object M to be moved and move the grasped object M to the destination specified by the operator. The control signal Cnt2 may be pre-prepared for each algorithm in the data table TBL1, or it may be generated each time by the control unit 304 according to the algorithm identified by the identification unit 303. The gripping unit 402a may be equipped with a contact sensor at its tip, and the control unit 304 may stop the movement of the gripping unit 402a toward the object M when it detects that the contact sensor has come into contact with the object M.

[0052] (advantage) The robot system 1 according to the first embodiment of this disclosure has been described above. In the designation device 20 of the robot system 1, the reception unit 204 receives an input specifying the outline F of the object M to be moved. The control unit 203 displays the specified outline F along with a two-dimensional image including the object M to be moved on the display unit 201 (an example of a display device).

[0053] In this way, the designation device 20 displays the outline F of the object M to be moved, which is specified by the operator via the reception unit 204, along with a two-dimensional image including the object M to be moved. Therefore, when an operator uses the designation device 20, the operator can specify the position of the outline F of the object M to be moved while confirming the positional relationship between the two-dimensional image including the object M and the outline F of the object M to be moved that the operator has specified. Furthermore, since the image displayed by the designation device 20 is two-dimensional, the operator only needs to match the outline F to the object M in that image, making the operation of specifying the outline F easy. Thus, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object. As a result, the robot system But You will be able to recognize your body correctly.

[0054] In a robot system that moves an object according to a predetermined algorithm corresponding to a work objective, given the state of the object before movement, the following is desired: It is desirable that the operator can input the correct state of the object before movement and the desired state of the object after movement, which are determined according to the algorithm, into the robot. Furthermore, it is desirable that the operator can easily specify the state of the object before or after movement. In the robot system 1 according to a modification of the first embodiment of this disclosure, the operator can easily specify the state of the object. As a result, the robot system But You will be able to recognize your body correctly.

[0055] <Modified form of the first embodiment> Next, a modified robot system 1 according to the first embodiment of this disclosure will be described. Similar to the robot system 1 according to the first embodiment shown in Figure 1, the modified robot system 1 comprises a measuring device 10, a designating device 20, a control device 30, and a robot 40.

[0056] Figure 11 shows an example of the installation of the measuring device 10 according to a modification of the first embodiment of this disclosure. As shown in Figure 11, the measuring device 10 is installed in a fixed position that allows it to photograph the plane P of the tray T on which the object M to be moved is placed from above. However, in the first embodiment, it is assumed that the object to be moved is an object placed parallel to a plane P that is located substantially horizontally, but in the modification of the first embodiment, it is assumed that the object to be moved is an object placed at an angle (i.e., inclined) to the plane P that is located substantially horizontally. Therefore, the processing performed by the designating device 20 differs mainly between the first embodiment and the modification of the first embodiment. Here, we will mainly explain the processing that differs between the designating device 20 according to the modification of the first embodiment and the designating device 20 according to the first embodiment. For processing that is not specifically explained, the outer shape F to be displayed on the display unit 201 becomes a plane Qa and an axis Qb that forms a predetermined angle with respect to that plane Qa, and considering that the object M was placed at an angle to the plane P instead of parallel to it, it is sufficient to consider it in the same way as in the first embodiment.

[0057] The designation device 20 according to a modification of the first embodiment includes a display unit 201 (an example of a display device), a generation unit 202, a control unit 203, and a reception unit 204, similar to the designation device 20 according to the first embodiment shown in Figure 5.

[0058] The display unit 201, under the control of the control unit 203, displays a two-dimensional image captured by the camera 101, and an image showing a plane Qa representing a predetermined plane of the object M to be moved, input from the reception unit 204, and an axis Qb that forms a predetermined angle with plane Qa. Plane Qa and the axis Qb that forms a predetermined angle with plane Qa are displayed only for the object M to be grasped among the objects M to be moved. Figure 12 shows an example of an image displayed by the display unit 201 according to a modification of the first embodiment of the present disclosure. In the example shown in Figure 12, the object M to be moved, plane Qa, and the axis Qb that forms a predetermined angle with plane Qa are shown. In addition, region R is shown in the example shown in Figure 12. Furthermore, under the control performed by the control unit 203, the surface Qa displayed by the display unit 201 may be deformed in shape according to the angle specified by axis Qb (as if using perspective) (for example, a tilted rectangular outline (for example, if a predetermined face of object M is rectangular, an outline showing a parallelogram or trapezoid depending on the angle specified by axis Qb) may be displayed on the two-dimensional screen). This makes it easy to make plane Qa coincide with a predetermined face of object M even when viewing object M from above (i.e., in the positive direction of the Z axis). As a result, the robot system But You will be able to recognize your body correctly.

[0059] Information from a two-dimensional image captured by camera 101 and data for displaying a surface Qa and an axis Qb that forms a predetermined angle with respect to surface Qa are prepared in advance. Based on a signal indicating surface Qa and the axis Qb that forms a predetermined angle with respect to surface Qa, which is generated in response to an operation performed by an operator to align surface Qa with a predetermined surface of the object M to be moved (described later), the generation unit 202 generates a control signal Cnt1 that displays the two-dimensional image, surface Qa, and the axis Qb that forms a predetermined angle with respect to surface Qa on the display unit 201.

[0060] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays on the display unit 201 a two-dimensional image captured by the camera 101, along with a surface Qa input from the reception unit 204 and an axis Qb that forms a predetermined angle with respect to that surface Qa.

[0061] Unlike the first embodiment, in which the receiving unit 204 does not generate a signal indicating the outer shape F of the object M to be moved, the receiving unit 204 generates a signal indicating a surface Qa and an axis Qb that forms a predetermined angle with respect to that surface Qa. Therefore, based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the surface Qa and the axis Qb that forms a predetermined angle with respect to that surface Qa on the display unit 201, along with the two-dimensional image captured by the camera 101.

[0062] The reception unit 204 receives input from the operator to operate a surface Qa that specifies a predetermined surface of object M. For example, if the angle between surface Qa and axis Qb is 90 degrees, the operator first uses their finger or a stylus designed for touch panels to align axis Qb with the direction in which the gripping unit 402a approaches the object M. The reception unit 204 receives this operation from the operator. This operation by the operator to align axis Qb with the direction in which the gripping unit 402a approaches the object M to be moved causes the predetermined surface of object M to be moved to become parallel to surface Qa. Next, the operator moves surface Qa parallel along axis Qb to align surface Qa with the predetermined surface of object M to be moved. The reception unit 204 receives this operation from the operator. In practice, the reception unit 204 may receive operations from the operator moment by moment. Each time the reception unit 204 receives an operation from the operator, the generation unit 202 generates a control signal Cnt1. Then, the control unit 203 controls the display of the display unit 201 based on the control signal Cnt1 generated by the generation unit 202.

[0063] (advantage) The robot system 1, which is a modified version of the first embodiment of the present disclosure, has been described above. In the designation device 20 of the robot system 1, the reception unit 204 receives input of a surface Qa that designates a predetermined surface of the object M to be moved. The control unit 203 displays the surface Qa received by the reception unit 204 on the display device along with a two-dimensional image including the object M to be moved.

[0064] By doing so, the designation device 20 displays a two-dimensional image of the object M to be moved, including a predetermined face, along with a face Qa that specifies that predetermined face. Therefore, when an operator uses the designation device 20, the operator can align face Qa with the predetermined face of the object M while confirming the positional relationship between the two-dimensional image including the predetermined face of the object M and face Qa. The image displayed by the designation device 20 is two-dimensional, and the operator only needs to align face Qa with the predetermined face of the object M in that image. In addition, an axis Qb that forms a predetermined angle with face Qa is also displayed, so this axis Qb serves as a guide for adjustment. Therefore, the operation performed by the operator on face Qa is easy. Thus, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object. As a result, the robot system But You will be able to recognize your body correctly.

[0065] Furthermore, once a predetermined surface of the object M to be moved is determined, the control device 30 controls the gripping unit 402a to bring the gripping unit 402a closer to that surface, facing it directly. Therefore, whether the gripping mechanism of the gripping unit 402a is a mechanism that holds the object M between multiple (for example, two) fingers, or a mechanism that attracts a predetermined surface of the object M, the gripping unit 402a can properly grip the object M.

[0066] <Second Embodiment> Next, a robot system 1 according to a second embodiment of this disclosure will be described. Figure 13 is a diagram showing an example of the configuration of the robot system 1 according to a second embodiment of this disclosure. As shown in Figure 13, the robot system 1 according to the second embodiment includes a measuring device 10, a designating device 20, a control device 30, and a robot 40, similar to the robot system 1 according to the first embodiment shown in Figure 1, and further includes an automatic recognition system 50. In the second embodiment, as in the first embodiment, the object M to be moved is assumed to be placed parallel to a plane P that is positioned almost horizontally. Here, we will mainly describe the differences in processing between the robot system 1 according to the second embodiment and the robot system 1 according to the first embodiment.

[0067] The automatic recognition system 50 is a system capable of photographing a moving object M and identifying the state (i.e., position and orientation) of the moving object M. Figure 14 shows an example of the configuration of the automatic recognition system 50 according to a second embodiment of the present disclosure. As shown in Figure 14, the automatic recognition system 50 includes a camera 501. The camera 501 is an industrial camera. The automatic recognition system 50 identifies the shape of the upper surface of the object M and the height of the object M from the plane P by photographing the object M with the camera 501. In other words, the automatic recognition system 50 can identify the shape of the upper surface of the object M and the height of the object M from the plane P, similar to the measuring device 10. The automatic recognition system 50 transmits the identified information on the shape of the upper surface of the object M and the height of the object M from the plane P to the designating device 20 and the control device 30. Figure 15 shows an example of the installation of the camera 501 according to a second embodiment of the present disclosure. As shown in Figure 15, the camera 501 photographs the moving object M from a different direction than the measuring device 10, for example. Furthermore, this automatic recognition system 50 may be implemented using existing technologies.

[0068] The control device 30, like the control device 30 according to the first embodiment shown in Figure 7, includes an acquisition unit 302, a specification unit 303, and a control unit 304. However, the control device 30 receives information on the shape of the upper surface of object M and the height of object M from plane P. The information received is equivalent to the information on the outer shape of object M and the height of object M from plane P received from the automatic recognition system 50, the measuring device 10, and the designation device 20. Unlike the control device 30 according to the first embodiment, the control device 30 usually generates a control signal Cnt2 based on the information on the shape of the upper surface of object M and the height of object M from plane P received from the automatic recognition system 50. Furthermore, the processing of the acquisition unit 302, the identification unit 303, and the control unit 304 can be considered in the same way as the processing of the acquisition unit 302, the identification unit 303, and the control unit 304 according to the first embodiment, by replacing the information on the external shape of the object M and the height of the object M from the plane P with the information on the shape of the upper surface of the object M and the height of the object M from the plane P received from the automatic recognition system 50.

[0069] Next, the designated device 20 will be described. The following description concerns the process that the designated device 20 performs when the control device 30 is unable to properly control the robot 40 using the information on the shape of the upper surface of object M and the height of object M from the plane P received by the control device 30 from the automatic recognition system 50.

[0070] The designation device 20 includes a display unit 201, a generation unit 202, a control unit 203, and a reception unit 204, similar to the designation device 20 according to the first embodiment shown in Figure 5.

[0071] The generation unit 202 generates a control signal Cnt1 that displays the shape U of the upper surface of the moving object M (corresponding to the outer shape F in the first embodiment) along with the two-dimensional image on the display unit 201, based on the information of the two-dimensional image captured by the camera 101 and the information of the shape of the upper surface of the moving object M and the height of the object M from the plane P received from the automatic recognition system 50.

[0072] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the shape U of the upper surface of the moving object M on the display unit 201, along with the two-dimensional image captured by the camera 101.

[0073] The reception unit 204 receives input from the operator specifying (in this case, modifying and specifying) the shape U of the upper surface of the object M to be moved. For example, the reception unit 204 is a touch panel and receives an operation to select the shape U of the upper surface of the object M to be moved using the operator's finger or a stylus designed for the touch panel, and to move the selected shape U to a desired position (i.e., the position of the actual upper surface of the object M to be moved as shown in the 2D image).

[0074] Furthermore, while the operator is moving the selected shape U to the desired position, the generation unit 202 generates a control signal Cnt1 corresponding to that operation. During this time, the control unit 203 displays the shape U of the upper surface of the object M to be moved on the display unit 201, along with the two-dimensional image captured by the camera 101, based on the control signal Cnt1 generated by the generation unit 202. As a result, the display unit 201 displays the shape U along with the two-dimensional image captured by the camera 101, under the control performed by the control unit 203. Figure 16 is a diagram showing an example of an image displayed by the display unit 201 according to the second embodiment of this disclosure. In the example shown in Figure 16, Transfer The objects M to be moved are shown as objects M1 and M2, and the shape U of the upper surface of object M1. In the example shown in Figure 16, a region R is also shown. Note that the hand shown in Figure 16 is not a display unit 201, but rather an illustration of an operation in which an operator moves shape U on a touch panel and indicates the position of shape U with their finger.

[0075] (advantage) The robot system 1 according to the second embodiment of this disclosure has been described above. In the designation device 20 of the robot system 1, the reception unit 204 receives an input to move the position of the shape U of the upper surface of the object M to be moved, which is displayed on the display unit 201, based on the state of the object M to be moved identified by the automatic recognition system 50 equipped with a camera 501. The generation unit 202 modifies the control signal Cnt1 based on the input to move the position of the shape U received by the reception unit 204. Based on the control signal Cnt1, the control unit 203 displays the shape U of the upper surface of the object M to be moved on the display unit 201 together with the two-dimensional image captured by the camera 101.

[0076] In this way, the designation device 20 displays the shape U of the upper surface of the object M to be moved, along with a two-dimensional image including the object M, via the reception unit 204. Therefore, when an operator uses the designation device 20, the operator can specify the position of the shape U of the upper surface of the object M to be moved while confirming the positional relationship between the two-dimensional image including the object M and the shape U of the upper surface of the object M to be moved that the operator specifies. Furthermore, the image displayed by the designation device 20 is two-dimensional, and the operator only needs to move the shape U of the upper surface of the object M to be moved in that image to the desired position. Therefore, the operation of specifying the shape U is easy for the operator. Thus, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object. As a result, the robot system But You will be able to recognize your body correctly.

[0077] <Modified form of the second embodiment> Next, a robot system 1 according to a modification of the second embodiment of this disclosure will be described. The robot system 1 according to the modification of the second embodiment includes a measuring device 10, a designating device 20, a control device 30, a robot 40, and an automatic recognition system 50, similar to the robot system 1 according to the second embodiment shown in Figure 13. In the modification of the second embodiment, similar to the modification of the first embodiment, the object M to be moved is positioned at an angle to the plane P.

[0078] In the robot system 1 according to the second embodiment, the outer shape F of the object M to be moved in the robot system 1 according to the first embodiment is replaced with the shape U of the upper surface of the object M to be moved. Similarly, in the robot system 1 according to the modified version of the second embodiment, the surface Qa and the axis Qb that forms a predetermined angle with respect to surface Qa in the modified version of the second embodiment can be replaced with a surface Va generated by the automatic recognition system 50 and an axis Vb that forms a predetermined angle with respect to surface Va (corresponding to surface Qa and the axis Qb that forms a predetermined angle with respect to surface Qa). This can be implemented by combining the modified version of the first embodiment and the processing of the robot system 1 in the second embodiment. For example, if the surface Va generated by the automatic recognition system 50 is different from the expected surface, the operator may specify the surface Qa by using their finger to indicate a predetermined surface of the object M on the touch panel, and correct the axis Vb by setting the axis Qb.

[0079] The information from the two-dimensional image captured by the camera 101 and the data for displaying the surface Va generated by the automatic recognition system 50 and the axis Vb that forms a predetermined angle with respect to that surface Va are prepared in advance. The generation unit 202 generates a control signal Cnt1 that displays the surface Va and the axis Vb that forms a predetermined angle with respect to that surface Va on the display unit 201, along with the two-dimensional image, based on the signal indicating the surface Va and the axis Vb that forms a predetermined angle with respect to that surface Va, which are generated in response to an operation performed by an operator to align the surface Va with a predetermined surface of the object M to be moved.

[0080] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays a two-dimensional image captured by the camera 101, along with a plane Va and an axis Vb that forms a predetermined angle with respect to that plane Va, on the display unit 201.

[0081] The receiving unit 204 receives the processing performed by the operator on surface Va and axis Vb that forms a predetermined angle with surface Va, which is performed on surface Qa and axis Qb that forms a predetermined angle with surface Qa, as described in the modified example of the first embodiment.

[0082] Furthermore, while the operator is moving the surface Va to the desired position, the generation unit 202 generates a control signal Cnt1 corresponding to that operation. During this time, the control unit 203 displays the surface Va and the axis Vb that forms a predetermined angle with respect to surface Va on the display unit 201, along with the two-dimensional image captured by the camera 101, based on the control signal Cnt1 generated by the generation unit 202. As a result, the display unit 201 displays the surface Va and the axis Vb that forms a predetermined angle with respect to surface Va, along with the two-dimensional image captured by the camera 101, under the control performed by the control unit 203. Figure 17 shows an example of an image displayed by the display unit 201 according to a modification of the second embodiment of this disclosure. In the example shown in Figure 17, the object M to be moved, the surface Va, and the axis Vb that forms a predetermined angle with respect to surface Va are shown. Also, in the example shown in Figure 17, a region R is shown.

[0083] (advantage) The robot system 1 according to a modification of the second embodiment of this disclosure has been described above. In the designation device 20 of the robot system 1, the reception unit 204 receives the processing performed by the operator on a surface Va and an axis Vb that forms a predetermined angle with respect to that surface Va, which was described in the modification of the first embodiment, on a surface Qa and an axis Qb that forms a predetermined angle with respect to that surface Qa. The generation unit 202 generates a control signal Cnt1 corresponding to the operation received by the reception unit 204. Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the surface Va and the axis Vb that forms a predetermined angle with respect to that surface Va on the display unit 201, along with a two-dimensional image captured by the camera 101.

[0084] In this way, the designation device 20 displays a two-dimensional image including a predetermined face of the object M to be moved, along with a face Va that specifies that predetermined face, via the reception unit 204. Therefore, when an operator uses the designation device 20, the operator can align face Va with the predetermined face of the object M to be moved while confirming the positional relationship between the two-dimensional image including the predetermined face of the object M and face Va. The image displayed by the designation device 20 is two-dimensional, and the operator only needs to align face Va with the predetermined face of the object M in that image. In addition, an axis Vb that forms a predetermined angle with face Va is also displayed, so that axis Vb serves as a guide for adjustment. Therefore, the operation performed by the operator on face Va is easy. Thus, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object. As a result, the robot system But You will be able to recognize your body correctly.

[0085] <Third Embodiment> Next, a robot system 1 according to a third embodiment of this disclosure will be described. Figure 18 is a diagram showing an example of the configuration of the robot system 1 according to a third embodiment of this disclosure. As shown in Figure 18, the robot system 1 according to the third embodiment includes a measuring device 10, a designating device 20, a control device 30, and a robot 40, similar to the robot system 1 according to the first embodiment shown in Figure 1. The robot system 1 according to the third embodiment further includes a WMS (Warehouse System) 60 (an example of an external system). In the third embodiment, as in the first embodiment, the object M to be moved is assumed to be placed parallel to a plane P that is positioned substantially horizontally.

[0086] WMS60 is a system that manages the storage status of each product stored in a warehouse or the like. Examples of storage status include the quantity and shape (including dimensions) of each product. WMS60 also has a transport mechanism that moves products to the storage location when they arrive and moves products from the storage location to the work area of ​​the robot 40 when they are shipped. Figure 19 is a diagram showing an example of the configuration of WMS60 according to the third embodiment of this disclosure. WMS60 comprises a storage unit 601, a transport mechanism 602, and a control unit 603.

[0087] The storage unit 601 stores various information necessary for the processing performed by the WMS 60. For example, the storage unit 601 stores the storage status of each product. Figure 20 shows an example of a data table TBL2 stored by the storage unit 601 according to a third embodiment of the present disclosure. As shown in Figure 20, for example, the storage unit 601 stores the type, quantity, and shape of the products (i.e., the objects M to be moved) stored in each tray T (#1, 2, 3, ...) in association with each other.

[0088] The transport mechanism 602 moves the goods to the desired position upon arrival and departure under the control of the control unit 603. In the robot system 1 according to the embodiment of this disclosure equipped with WMS 60, the robot system 1 will be described assuming that the goods have already been moved to the work area of ​​the robot 40 under the control of the control unit 603 (i.e., it is known what kind of goods and how many have been transported to the work area of ​​the robot 40).

[0089] The control unit 603 controls the operation of the transport mechanism 602. Based on this control, the control unit 603 also transmits information about the type, quantity, and shape of the goods moved to the robot 40's work area to the designated device 20.

[0090] Next, the designation device 20 will be described. The following description concerns the process by which the designation device 20 uses the storage status information of each product stored in the storage unit 601 of the WMS 60 to specify the external shape of the object M to be moved.

[0091] The designation device 20 includes a display unit 201, a generation unit 202, a control unit 203, and a reception unit 204, similar to the designation device 20 according to the first embodiment shown in Figure 5.

[0092] A shape Fa, which is a candidate for the outline F of the object M to be moved, is prepared in advance. Based on the information of the 2D image captured by the camera 101 and the information of the type, quantity, and shape of the object M to be moved received from the WMS 60, the generation unit 202 generates a control signal Cnt1 that displays the candidate shape Fa on the display unit 201 along with the 2D image.

[0093] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays a figure Fa, which is a candidate for the outline F of the object M to be moved, on the display unit 201 along with the two-dimensional image captured by the camera 101.

[0094] The reception unit 204 receives input from the operator specifying (in this case, by selecting) a shape Fa that is a candidate for the outline F. For example, the reception unit 204 is a touch panel. The reception unit 204 receives an operation in which the operator selects a shape Fa using their finger or a stylus designed for touch panels, and moves the selected shape Fa to the desired position (i.e., the position of the upper face of the actual object M that is to be moved, as shown in the 2D image).

[0095] Furthermore, while the operator is moving the selected figure Fa to the desired position, the generation unit 202 generates a control signal Cnt1 corresponding to that operation. During this time, the control unit 203 displays the figure Fa on the display unit 201 along with the two-dimensional image captured by the camera 101, based on the control signal Cnt1 generated by the generation unit 202. As a result, the display unit 201 displays the figure Fa along with the two-dimensional image captured by the camera 101, under the control performed by the control unit 203. Figure 21 is a diagram showing an example of an image displayed by the display unit 201 according to the third embodiment of this disclosure. In the example shown in Figure 21, TransferObjects M1 and M2, and the graphic Fa are shown as the objects M that are being moved. In the example shown in Figure 21, a region R is also shown. Note that the hand shown in Figure 21 is not something displayed by the display unit 201, but rather an illustration of an operation in which an operator moves graphic Fa on the touch panel and indicates the position of graphic Fa with their finger.

[0096] The designated device 20 may also display only one candidate graphic Fa on the display unit 201, and if the operator performs the operation to select that candidate at the reception unit 204, it may then display other graphic Fas on the display unit 201.

[0097] (advantage) The robot system 1 according to the third embodiment of this disclosure has been described above. In the designation device 20 of the robot system 1, the generation unit 202 generates a control signal Cnt1 that displays a figure Fa on the display unit 201 along with the two-dimensional image, based on the information of the two-dimensional image captured by the camera 101 and the information of the type, quantity, and shape of the object M to be moved received from the WMS 60. Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays a figure Fa, which is a candidate for the outer shape F of the object M to be moved, on the display unit 201 along with the two-dimensional image captured by the camera 101. The reception unit 204 receives input from an operator to specify (in this case, select and specify) a figure Fa that is a candidate for the outer shape F. For example, the reception unit 204 is a touch panel and receives an operation to select a figure Fa using the operator's finger or a pen for the touch panel and move the selected figure Fa to a desired position (i.e., the position of the upper face of the actual object M to be moved as shown in the two-dimensional image).

[0098] By doing so, the designation device 20 displays a two-dimensional image including the object M to be moved, along with a figure Fa corresponding to the object M to be moved. Therefore, when an operator uses the designation device 20, the operator can specify the position of figure Fa while confirming the positional relationship between the two-dimensional image including the object M and figure Fa. Furthermore, since the image displayed by the designation device 20 is two-dimensional, the operator only needs to move figure Fa in that image to the desired position. Therefore, the operation of specifying figure Fa is easy for the operator. Thus, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object. As a result, the robot system But You will be able to recognize your body correctly.

[0099] <Modified form of the third embodiment> Next, a robot system 1 according to a modification of the third embodiment of this disclosure will be described. The robot system 1 according to the modification of the third embodiment includes a measuring device 10, a designating device 20, a control device 30, a robot 40, and a WMS 60, similar to the robot system 1 according to the third embodiment shown in Figure 18. In the modification of the third embodiment, similar to the modification of the first embodiment, the object M to be moved is positioned at an angle to the plane P.

[0100] The designation device 20 includes a display unit 201, a generation unit 202, a control unit 203, and a reception unit 204, similar to the designation device 20 according to the first embodiment shown in Figure 5.

[0101] A plane Qa that specifies a predetermined face of the object M to be moved, and a figure Fa that is a candidate for an axis Qb that forms a predetermined angle with that face Qa, are prepared in advance. Based on the information of the two-dimensional image captured by the camera 101 and the information of the type, quantity, and shape of the object M to be moved received from the WMS 60, the generation unit 202 generates a control signal Cnt1 that displays the candidate figure Fa on the display unit 201 along with the two-dimensional image.

[0102] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays on the display unit 201 a two-dimensional image captured by the camera 101, along with a plane Qa that designates a predetermined plane of the object M to be moved, and a figure Fa that is a candidate for an axis Qb that forms a predetermined angle with plane Qa.

[0103] The reception unit 204 receives input from the operator specifying a face Qa that designates a predetermined face of the object M to be moved, and a figure Fa that is a candidate for an axis Qb that forms a predetermined angle with face Qa (in this case, by selection). For example, the reception unit 204 is a touch panel and receives an operation in which the operator selects a figure Fa using their finger or a stylus designed for touch panels, and moves the selected figure Fa to a desired position (i.e., the position of the upper face of the actual object M to be moved as shown in the 2D image).

[0104] Furthermore, while the operator is moving the selected figure Fa to the desired position, the generation unit 202 generates a control signal Cnt1 corresponding to that operation. Then, the control unit 203 displays the figure Fa on the display unit 201 along with the two-dimensional image captured by the camera 101, based on the control signal Cnt1 generated by the generation unit 202. As a result, the display unit 201 displays the figure Fa along with the two-dimensional image captured by the camera 101, under the control performed by the control unit 203. Figure 22 is a diagram showing an example of an image displayed by the display unit 201 according to a modification of the third embodiment of this disclosure. In the example shown in Figure 22, Transfer The object M being moved is shown as object M and the figure Fa. In the example shown in Figure 22, a region R is also shown. Note that the hand shown in Figure 22 is not something displayed by the display unit 201, but rather an illustration of an operation in which an operator moves figure Fa on the touch panel and indicates the position of figure Fa with their finger.

[0105] The designated device 20 may also display only one candidate graphic Fa on the display unit 201, and if the operator performs the operation to select that candidate at the reception unit 204, it may then display other graphic Fas on the display unit 201.

[0106] (advantage) The third embodiment of this disclosure is described above. Variant The robot system 1 described above. In the designation device 20 of the robot system 1, the generation unit 202 generates a control signal Cnt1 that displays the figure Fa on the display unit 201 along with the two-dimensional image, based on the information of the two-dimensional image captured by the camera 101 and the information of the type, quantity, and shape of the object M to be moved received from the WMS 60. Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the figure Fa, which is a candidate for the axis Qb that forms a predetermined angle with the surface Qa, along with the two-dimensional image captured by the camera 101, on the display unit 201. The reception unit 204 receives input from the operator specifying (in this case, selecting and specifying) the figure Fa that is a candidate for the axis Qb that forms a predetermined angle with the surface Qa. For example, the reception unit 204 is a touch panel and receives operations to select a shape Fa using the operator's finger or a stylus specifically for the touch panel, and to move the selected shape Fa to a desired position (i.e., the position of the upper face of the actual object M that is to be moved, as depicted in the 2D image).

[0107] By doing so, the designation device 20 displays a two-dimensional image including the object M to be moved, along with a pre-prepared figure Fa corresponding to the object M to be moved. Therefore, when an operator uses the designation device 20, the operator can specify the position of figure Fa while confirming the positional relationship between the two-dimensional image including the object M and figure Fa. Furthermore, since the image displayed by the designation device 20 is two-dimensional, the operator only needs to move figure Fa in that image to the desired position. Therefore, the operation of specifying figure Fa is easy for the operator. Thus, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object. As a result, the robot system But You will be able to recognize your body correctly.

[0108] <Fourth Embodiment> Next, a robot system 1 according to a fourth embodiment of this disclosure will be described. Figure 23 is a diagram showing an example of the configuration of the robot system 1 according to the fourth embodiment of this disclosure. As shown in Figure 23, the robot system 1 according to the fourth embodiment comprises a measuring device 10, a designating device 20, a control device 30, a robot 40, an automatic recognition system 50, and a WMS 60. In the fourth embodiment, as in the first embodiment, the object M to be moved is assumed to be placed parallel to a plane P that is positioned substantially horizontally.

[0109] The robot system 1 according to the fourth embodiment is a system that combines the robot system 1 according to the second embodiment and the robot system 1 according to the third embodiment.

[0110] Based on the information received from the automatic recognition system 50, the designating device 20 generates a shape U that indicates the upper surface of the object M to be moved. However, if the shape U does not result in a shape U that indicates the desired position and size representing the outer shape of the object M to be moved, the designating device 20 does not modify the shape U, but instead uses the figure Fa described in the third embodiment to specify the outer shape of the object M to be moved.

[0111] Therefore, the processing of the designated device 20 is to perform the process of causing the display unit 201 in the second embodiment to perform a display, and if the shape U is deviated from the outer shape of the object M to be moved, then the process of causing the display unit 201 in the third embodiment to perform a display.

[0112] (advantage) The robot system 1 according to the fourth embodiment of this disclosure has been described above. By combining the configuration of the robot system 1 according to the second embodiment and the configuration of the robot system 1 according to the third embodiment, it is possible to perform a process to display information on the display unit 201 in the second embodiment. Furthermore, if the shape U deviates from the outer shape of the object M to be moved, the outer shape of the object M to be moved can be correctly specified by performing a process to display information on the display unit 201 in the third embodiment. As a result, the robot system ButYou will be able to recognize your body correctly.

[0113] <Modified form of the fourth embodiment> Next, a robot system 1 according to a modification of the fourth embodiment of this disclosure will be described. The robot system 1 according to the modification of the fourth embodiment includes a measuring device 10, a designating device 20, a control device 30, a robot 40, an automatic recognition system 50, and a WMS 60, similar to the robot system 1 according to the fourth embodiment shown in Figure 23. In the modification of the fourth embodiment, similar to the modification of the first embodiment, the object M to be moved is positioned at an angle to the plane P.

[0114] The robot system 1 according to the modification of the fourth embodiment is a system that combines the robot system 1 according to the modification of the second embodiment and the robot system 1 according to the modification of the third embodiment.

[0115] (advantage) Therefore, the robot system 1 according to the modification of the fourth embodiment can be considered in the same way as the robot system 1 according to the fourth embodiment. By combining the configuration of the robot system 1 according to the modification of the second embodiment and the configuration of the robot system 1 according to the modification of the third embodiment, it is possible to perform the process of displaying on the display unit 201 in the modification of the second embodiment. Furthermore, if the figure Fa is deviated from a predetermined plane of the object M to be moved, the predetermined plane of the object M to be moved can be correctly specified by performing the process of displaying on the display unit 201 in the modification of the third embodiment. As a result, the robot system But You will be able to recognize your body correctly.

[0116] <Fifth Embodiment> Next, a robot system 1 according to a fifth embodiment of this disclosure will be described. Similar to the robot system 1 according to the first embodiment shown in Figure 1, the robot system 1 according to the fifth embodiment includes a measuring device 10, a designating device 20, a control device 30, and a robot 40. The robot system 1 according to the fifth embodiment is a system for changing the destination of the robot 40.

[0117] When the robot 40 moves the object M to be moved to a destination determined by the control device 30 according to its algorithm, under the control of the control device 30, that destination may not be the destination desired by the operator. The robot system 1 according to the fifth embodiment is a system that performs a process to change the destination to the desired destination in such cases.

[0118] The following description concerns the process in the robot system 1 of the first embodiment and its modified form described above, in which the state of the object M to be moved before movement is specified, and then the control device 30 changes the destination identified according to the algorithm.

[0119] In robot system 1, the destination is determined when the control signal Cnt2 according to the algorithm is determined. The control unit 304 outputs information indicating the destination to the designation device 20.

[0120] The generation unit 202 generates a control signal Cnt1 that displays the two-dimensional image, the destination, and the outline F that specifies the destination on the display unit 201, based on the information of the two-dimensional image captured by the camera 101, the information for specifying the outline F as described in the first embodiment, and the information indicating the destination received from the control device 30.

[0121] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the two-dimensional image captured by the camera 101, along with the destination, and the outline F on the display unit 201.

[0122] The reception unit 204 receives an operation from the operator to delete an unnecessary destination. For example, the reception unit 204 is a touch panel, and the operator selects a destination to delete using their finger or a stylus designed for the touch panel, and then confirms the deletion of the selected destination, at which point the reception unit 204 receives the operation. Upon receiving this operation, the generation unit 202 generates a control signal Cnt1 that prevents the destination designated for deletion from being displayed. This control signal Cnt1 then deletes the destination.

[0123] Furthermore, the reception unit 204 receives an operation to move the outline F to a desired position (i.e., a desired destination). The reception unit 204 also receives input from the operator specifying the outline F (in this case, by selecting it). For example, the reception unit 204 is a touch panel and receives an operation to select the outline F using the operator's finger or a stylus designed for touch panels, and to move the selected outline F to a desired position (i.e., a desired destination).

[0124] Furthermore, while the operator is performing operations to move the selected outline F to the desired position or to delete the destination, the generation unit 202 generates a control signal Cnt1 corresponding to the operation. Then, the control unit 203 displays the two-dimensional image captured by the camera 101 and the desired destination outline F on the display unit 201 based on the control signal Cnt1 generated by the generation unit 202. As a result, the display unit 201 displays the desired destination outline F along with the two-dimensional image captured by the camera 101, under the control performed by the control unit 203.

[0125] Figure 24 shows an example of a destination determined by the control device 30 according to the fifth embodiment of this disclosure. As shown in Figure 24, the destination may be a mixture of areas where object M is in close contact and areas where object M is not present. In this case as well, the destination can be changed to a destination desired by the operator by the above-described process.

[0126] (advantage) The robot system 1 according to the fifth embodiment of this disclosure has been described above. As described above, the technique for specifying the state of the object M to be moved can also be used for specifying the destination. Therefore, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object after movement. As a result, the robot system But You will be able to recognize your body correctly.

[0127] <Sixth Embodiment> Next, a robot system 1 according to the sixth embodiment of this disclosure will be described. Similar to the robot system 1 according to the third embodiment shown in Figure 18, the robot system 1 according to the sixth embodiment includes a measuring device 10, a designating device 20, a control device 30, a robot 40, and a WMS 60. The robot system 1 according to the sixth embodiment is a system for changing the destination of the robot 40.

[0128] When the robot 40 moves the object M to be moved to a destination determined by the control device 30 according to an algorithm, under the control of the control device 30, that destination may not be the destination desired by the operator. The robot system 1 according to the sixth embodiment is a system that performs a process to change the destination to the desired destination in such cases.

[0129] The following description concerns the processing in the robot system 1 equipped with WMS60, which is one of the robot systems 1 of the first to fourth embodiments and their variations described above. Specifically, it is the process of changing the destination identified by the control device 30 according to the algorithm after specifying the state of the object M to be moved before movement.

[0130] In robot system 1, the destination is determined when the control signal Cnt2 according to the algorithm is determined. The control unit 304 outputs information indicating the destination to the designation device 20.

[0131] The generation unit 202 generates a control signal Cnt1 that displays a two-dimensional image, the destination, and a graphic Fa on the display unit 201, based on the information of the two-dimensional image captured by the camera 101, the information of the type, quantity, and shape of the object M to be moved received from the WMS 60, and the information indicating the destination received from the control device 30.

[0132] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the graphic Fa along with the two-dimensional image captured by the camera 101 and the destination on the display unit 201.

[0133] The reception unit 204 receives an operation from the operator to delete an unnecessary destination. For example, the reception unit 204 is a touch panel, and the operator selects a destination to delete using their finger or a stylus designed for the touch panel, and then confirms the deletion of the selected destination, at which point the reception unit 204 receives the operation. Upon receiving this operation, the generation unit 202 generates a control signal Cnt1 that prevents the destination designated for deletion from being displayed. This control signal Cnt1 then deletes the destination.

[0134] Furthermore, the reception unit 204 receives an operation to move the figure Fa to a desired position (i.e., a desired destination). The reception unit 204 also receives input from the operator specifying (in this case, selecting and specifying) the figure Fa. For example, the reception unit 204 is a touch panel and receives an operation to select the figure Fa using the operator's finger or a stylus designed for touch panels, and to move the selected figure Fa to a desired position (i.e., a desired destination).

[0135] Furthermore, while the operator is performing operations to move the selected shape Fa to the desired position or to delete the destination, the generation unit 202 generates a control signal Cnt1 corresponding to the operation. Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the two-dimensional image captured by the camera 101 and the desired destination shape Fa on the display unit 201. As a result, the display unit 201 displays the desired destination shape Fa along with the two-dimensional image captured by the camera 101, under the control performed by the control unit 203.

[0136] In the sixth embodiment of this disclosure, as shown in Figure 24, the destination may be a mixture of areas where object M is in close contact and areas where object M is not present. In this case as well, the destination can be changed to a destination desired by the operator by the above-described process.

[0137] (advantage) The robot system 1 according to the sixth embodiment of this disclosure has been described above. As described above, the technique for specifying the state of the object M to be moved can also be used for specifying the destination. Therefore, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object after movement. As a result, the robot system But You will be able to recognize your body correctly.

[0138] <Seventh Embodiment> Next, a robot system 1 according to the seventh embodiment of this disclosure will be described. Similar to the robot system 1 according to the second embodiment shown in Figure 13, the robot system 1 according to the seventh embodiment comprises a measuring device 10, a designating device 20, a control device 30, and a robot 40, and further comprises an automatic recognition system 50.

[0139] The robot system 1 is equipped with an automatic recognition system 50. When the robot 40 moves an object M to be moved to a destination determined by the control device 30 according to an algorithm, the automatic recognition system 50 generates information indicating that destination. The robot system 1 according to the seventh embodiment is a system that, in such a case, performs a process to change the destination to a desired destination.

[0140] The automatic recognition system 50 outputs information indicating the generated destination to the designation device 20. The generation unit 202 generates a control signal Cnt1 that displays the two-dimensional image, the destination, and the outline F specifying the destination on the display unit 201, based on the information of the two-dimensional image captured by the camera 101, the information for specifying the outline F described in the first embodiment, and the information indicating the destination received from the automatic recognition system 50.

[0141] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the two-dimensional image captured by the camera 101, along with the destination, and the outline F on the display unit 201.

[0142] The reception unit 204 receives an operation from the operator to delete an unnecessary destination. For example, the reception unit 204 is a touch panel, and the operator selects a destination to delete using their finger or a stylus designed for the touch panel, and then confirms the deletion of the selected destination, at which point the reception unit 204 receives the operation. Upon receiving this operation, the generation unit 202 generates a control signal Cnt1 that prevents the destination designated for deletion from being displayed. This control signal Cnt1 then deletes the destination.

[0143] Furthermore, the reception unit 204 receives an operation to move the outline F to a desired position (i.e., a desired destination). The reception unit 204 also receives input from the operator specifying the outline F (in this case, by selecting it). For example, the reception unit 204 is a touch panel and receives an operation to select the outline F using the operator's finger or a stylus designed for touch panels, and to move the selected outline F to a desired position (i.e., a desired destination).

[0144] Furthermore, while the operator is performing operations to move the selected outline F to the desired position or to delete the destination, the generation unit 202 generates a control signal Cnt1 corresponding to the operation. Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the two-dimensional image captured by the camera 101 and the desired destination outline F on the display unit 201. As a result, the display unit 201 displays the desired destination outline F along with the two-dimensional image captured by the camera 101, under the control performed by the control unit 203.

[0145] (advantage) The robot system 1 according to the seventh embodiment of this disclosure has been described above. As described above, the technique for specifying the state of the object M to be moved can also be used for specifying the destination. Therefore, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object after movement. As a result, the robot system But You will be able to recognize your body correctly.

[0146] <Eighth Embodiment> Next, a robot system 1 according to the eighth embodiment of this disclosure will be described. Similar to the robot system 1 of the fourth embodiment shown in Figure 23, the robot system 1 according to the eighth embodiment includes a measuring device 10, a designating device 20, a control device 30, a robot 40, an automatic recognition system 50, and a WMS 60.

[0147] The robot system 1 is equipped with an automatic recognition system 50, and under the control of the control device 30, when the robot 40 moves the object M to be moved to a destination determined by the control device 30 according to an algorithm, the automatic recognition system 50 generates information indicating that destination. The robot system 1 according to the eighth embodiment is a system that, in such a case, performs a process to change the destination to a desired destination.

[0148] The automatic recognition system 50 outputs information indicating the generated destination to the designated device 20. The generation unit 202 generates a control signal Cnt1 that displays the graphic Fa along with the 2D image and destination on the display unit 201, based on the information of the 2D image captured by the camera 101, the information of the type, quantity, and shape of the object M to be moved received from the WMS 60, and the information indicating the destination received from the automatic recognition system 50.

[0149] Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the graphic Fa along with the two-dimensional image captured by the camera 101 and the destination on the display unit 201.

[0150] The reception unit 204 receives an operation from the operator to delete an unnecessary destination. For example, the reception unit 204 is a touch panel, and the operator selects a destination to delete using their finger or a stylus designed for the touch panel, and then confirms the deletion of the selected destination, at which point the reception unit 204 receives the operation. Upon receiving this operation, the generation unit 202 generates a control signal Cnt1 that prevents the destination designated for deletion from being displayed. This control signal Cnt1 then deletes the destination.

[0151] Furthermore, the reception unit 204 receives an operation to move the figure Fa to a desired position (i.e., a desired destination). The reception unit 204 also receives input from the operator specifying (in this case, selecting and specifying) the figure Fa. For example, the reception unit 204 is a touch panel and receives an operation to select the figure Fa using the operator's finger or a stylus designed for touch panels, and to move the selected figure Fa to a desired position (i.e., a desired destination).

[0152] Furthermore, while the operator is performing operations to move the selected shape Fa to the desired position or to delete the destination, the generation unit 202 generates a control signal Cnt1 corresponding to the operation. Based on the control signal Cnt1 generated by the generation unit 202, the control unit 203 displays the two-dimensional image captured by the camera 101 and the desired destination shape Fa on the display unit 201. As a result, the display unit 201 displays the desired destination shape Fa along with the two-dimensional image captured by the camera 101, under the control performed by the control unit 203.

[0153] (advantage) The robot system 1 according to the eighth embodiment of this disclosure has been described above. As described above, the technique for specifying the state of the object M to be moved can also be used for specifying the destination. Therefore, in a robot system that moves an object according to a predetermined algorithm corresponding to the work objective when the state of the object before movement is input from the designation device 20, the operator can easily specify the state of the object after movement. As a result, the robot system But You will be able to recognize your body correctly.

[0154] A minimal configuration of the designation device 20 according to an embodiment of the present disclosure will be described. Figure 25 is a diagram showing the minimal configuration of the designation device 20 according to an embodiment of the present disclosure. The minimal configuration of the designation device 20 according to an embodiment of the present disclosure is a designation device for a robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective, and as shown in Figure 25, comprises a reception unit 204 (an example of reception means) and a control unit 203 (an example of control means). The reception unit 204 receives an input specifying at least a part of the external shape of the object to be moved. The control unit 203 displays the external shape received by the reception unit on a display device, along with a two-dimensional image including the object to be moved. The reception unit 204 can be implemented, for example, using the functions of the reception unit 204 provided in the designation device 20 according to a modification of the first embodiment. The control unit 203 can also be implemented, for example, using the functions of the control unit 203 provided in the designation device 20 according to a modification of the first embodiment.

[0155] Next, we will explain the processing of the minimum configuration of the designated device 20. Figure 26 is a diagram showing an example of the processing flow of the minimum configuration of the designated device 20. Here, we will explain the processing of the minimum configuration of the designated device 20 with reference to Figure 26.

[0156] In a robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective, the designation device 20 has a receiving unit 204 that receives input specifying at least a part of the external shape of the object to be moved (step S11). The control unit 203 displays the external shape received by the receiving unit on a display device along with a two-dimensional image including the object to be moved (step S12). In this way, the designation device 20 allows the operator to easily specify the state of the object in a robot system that moves the object according to a predetermined algorithm corresponding to a work objective when the state of the object before movement is input. But You will be able to recognize your body correctly.

[0157] In addition, the order of processing in the embodiments of this disclosure may be changed, as long as appropriate processing is performed.

[0158] Although embodiments of this disclosure have been described, the robot system 1, measuring device 10, designating device 20, control device 30, robot 40, automatic recognition system 50, WMS 60, and other control devices described above may have a computer device inside. The process described above is stored in the form of a program on a computer-readable recording medium, and the above process is performed when the computer reads and executes this program. A specific example of a computer is shown below.

[0159] Figure 27 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. As shown in Figure 27, the computer 5 comprises a CPU 6 (including a vector processor), main memory 7, storage 8, and interface 9. For example, each of the robot system 1, measuring device 10, designating device 20, control device 30, robot 40, automatic recognition system 50, WMS 60, and other control devices described above is implemented in the computer 5. The operation of each of the above-described processing units is stored in the storage 8 in the form of a program. The CPU 6 reads the program from the storage 8 and loads it into the main memory 7, and executes the above-described processing according to the program. The CPU 6 also allocates storage areas in the main memory 7 corresponding to each of the above-described storage units according to the program.

[0160] Examples of storage 8 include HDDs (Hard Disk Drives), SSDs (Solid State Drives), magnetic disks, magneto-optical disks, CD-ROMs (Compact Disc Read Only Memory), DVD-ROMs (Digital Versatile Disc Read Only Memory), and semiconductor memory. Storage 8 may be an internal medium directly connected to the bus of computer 5, or an external medium connected to computer 5 via interface 9 or a communication line. Furthermore, if this program is distributed to computer 5 via a communication line, computer 5, upon receiving the program, may expand it into main memory 7 and execute the above processing. In at least one embodiment, storage 8 is a tangible storage medium that is not temporary.

[0161] Furthermore, the above program may implement some of the functions described above. Moreover, the above program may be a file that can implement the above functions in combination with a program already recorded on the computer device, a so-called differential file (differential program).

[0162] While several embodiments of this disclosure have been described, these embodiments are illustrative and do not limit the scope of the disclosure. These embodiments may be modified in various ways, without departing from the gist of the disclosure. [Industrial applicability]

[0163] According to each aspect of this disclosure, in a robot system that moves an object according to a predetermined algorithm corresponding to a work objective when the state of the object before movement is input, the operator can easily specify the state of the object. [Explanation of symbols]

[0164] 1. Robot System 5. Computers 6..CPU 7. Main Memory 8. Storage 9. Interface 10. Measuring device 20...designated device 30.. Control device 40... Robots 50. Automatic Identification System 60···WMS 101, 102, 501... Camera 201...Display section 202, 401...Generation section 203, 304, 603... Control Unit 204...Reception Department 301, 601...Storage section 302...Acquisition section 303...Specific section 402...Movable device 402a...Gripping part F...Outline M, M1, M2... Objects to be moved NW... Network P...Plane R...Photography area T...tray

Claims

1. A robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective includes a receiving means that receives an input specifying at least a part of the external shape of the object to be moved, A control means that causes a display device to display a two-dimensional image including the object to be moved and the external shape received by the receiving means, Equipped with, The aforementioned receiving means is The system receives an input to modify the shape of the object being moved, which is generated based on data held by an external system that manages the object being moved. Designated device.

2. A robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective, comprising a receiving means for receiving an input that specifies at least a part of the external shape of the object to be moved, A control means that causes a display device to display a two-dimensional image including the object to be moved and the external shape received by the receiving means, Equipped with, The aforementioned receiving means is The system receives an input that modifies the shape generated by an external system that photographs the object being moved. Designated device.

3. The aforementioned receiving means is The system receives input of the outline of the object to be moved, which is determined by tracing the outline of the object. The designated device according to claim 1.

4. The aforementioned receiving means is The system receives input of the outline of the shape determined by specifying the vertices of the object to be moved. The designated device according to claim 1.

5. A designated device according to any one of claims 1 to 4, A robot capable of grasping an object being moved, A control device that causes the robot to grasp the object to be moved based on the external shape of the object to be moved received by the designated device, A robotic system equipped with the following features.

6. In a robot system that moves an object to be moved according to a predetermined algorithm corresponding to the work objective, the computer, The system receives an input specifying at least a part of the external shape of the object to be moved, The system receives an input to modify the shape of the object being moved, which is generated based on data held by an external system that manages the object being moved. Displaying a two-dimensional image including the object to be moved and the received outline on a display device, A specification method that includes this.

7. A robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective, wherein a computer, The system receives an input specifying at least a part of the external shape of the object to be moved, The system receives an input that modifies the shape generated by an external system that photographs the object being moved, Displaying a two-dimensional image including the object to be moved and the received outline on a display device, A specification method that includes this.

8. In a robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective, the system receives an input specifying at least a part of the external shape of the object to be moved, The system receives an input to modify the shape of the object being moved, which is generated based on data held by an external system that manages the object being moved. Displaying a two-dimensional image including the object to be moved and the received outline on a display device, A program that causes a computer to execute something.

9. A robot system that moves an object to be moved according to a predetermined algorithm corresponding to a work objective, the system receiving an input that specifies at least a part of the external shape of the object to be moved, The system receives an input that modifies the shape generated by an external system that photographs the object being moved, Displaying a two-dimensional image including the object to be moved and the received outline on a display device, A program that causes a computer to execute something.

Images