Movement information estimation device and robot system

Abstract

A movement information estimation device is provided that can easily estimate the movement status of a passage along which a moving object moves. [Solution] The movement information estimation device 100 includes an acquisition unit 110 that acquires moving body information 501 indicating the movement direction or speed of a first moving body 510 on a moving passage 400, and an estimation unit 120 that uses the moving body information 501 to estimate the movement direction or speed of the passage 400.

Description

[Technical Field]

[0001] The present invention relates to a movement information estimation device and a robot system. [Background technology]

[0002] Patent Document 1 discloses a calibration method including the steps of provisionally setting conveyor tracking information, acquiring a reference image, a first image, and a second image by photographing a reference object placed on a conveyor at three different photographing timings while the conveyor is moving at a constant speed, determining measurement coordinate values ​​of the reference position, the first position, and the second position in the robot coordinate system by analyzing the three images, and correcting the provisionally set value of the conveyor tracking information so that the position and operation of the conveyor represented by the conveyor tracking information are consistent with various measurement results. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-11467 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional method disclosed in Patent Document 1, images are taken at multiple locations to capture three images (a reference image, a first image, and a second image) at three different capture timings. For this reason, Patent Document 1 uses multiple cameras, such as a conveyor camera (first camera) fixed to the conveyor and a mobile camera (second camera) attached to the robot arm of the robot, resulting in a complex configuration. Because of this complex configuration, the conventional method disclosed in Patent Document 1 may have difficulty in grasping the movement status of a passageway, such as a conveyor, when attempting to grasp the movement status of the passageway.

[0005] The present invention was made by the inventor of the present application by focusing on the above-mentioned problem, and aims to provide a movement information estimation device and a robot system that can easily estimate the movement status of a passage through which a moving object moves. [Means for solving the problem]

[0006] A movement information estimation device according to one aspect of the present invention includes an acquisition unit that acquires moving body information indicating the movement direction or speed of a first moving body on a moving passage, and an estimation unit that uses the moving body information to estimate the movement direction or speed of the passage.

[0007] A robot system according to one aspect of the present invention includes the above-described movement information estimation device and a robot that performs work on a second moving object on the passageway using the movement direction or speed of the passageway estimated by the movement information estimation device. [Effects of the Invention]

[0008] According to the movement information estimation device and the like of the present invention, the movement status of a passage along which a moving object moves can be easily estimated. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a perspective view showing an appearance of a robot system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing a functional configuration of a movement information estimation device according to an embodiment. [Figure 3] 3A and 3B are diagrams illustrating moving object information acquired by a movement information estimation device according to an embodiment. [Figure 4] 10 is a flowchart illustrating a process performed by the movement information estimation device according to the embodiment. [Figure 5] 10A and 10B are diagrams illustrating a state in which moving object information acquired by the movement information estimation device according to the embodiment has changed. [Figure 6] 10A and 10B are diagrams illustrating a process in which the movement information estimation device according to the embodiment estimates the position of a second moving object. DETAILED DESCRIPTION OF THE INVENTION

[0010] A movement information estimation device according to one aspect of the present invention includes an acquisition unit that acquires moving body information indicating the movement direction or speed of a first moving body on a moving passage, and an estimation unit that uses the moving body information to estimate the movement direction or speed of the passage.

[0011] According to this, the movement information estimation device acquires moving object information indicating the movement direction or speed of a first moving object on a passageway along which the first moving object is moving, and estimates the movement direction or speed of the passageway using the moving object information. This allows the movement information estimation device to estimate the movement direction or speed of the passageway using simple information, namely the moving object information indicating the movement direction or speed of the first moving object. This makes it possible for the movement information estimation device to easily estimate the movement status of the passageway along which the moving object is moving.

[0012] The acquisition unit may acquire the moving body information indicating both the direction and speed of movement of the first moving body, and the estimation unit may use the moving body information to estimate both the direction and speed of movement of the passage.

[0013] According to this, the movement information estimation device acquires moving object information indicating both the movement direction and speed of the first moving object, and estimates both the movement direction and speed of the passage using the moving object information. In this way, the movement information estimation device can more accurately estimate the movement status of the passage by estimating both the movement direction and speed of the passage from both the movement direction and speed of the first moving object.

[0014] The acquisition unit may acquire the moving object information about a plurality of first moving objects on the passage.

[0015] According to this, the movement information estimation device can obtain the movement information about the multiple first moving bodies, and can use the movement information about the multiple first moving bodies to more accurately estimate the movement status of the passage.

[0016] The estimation unit may calculate an average value of information about the plurality of first moving bodies included in the moving body information, and estimate the direction or speed of movement along the passage using the average value.

[0017] According to this, the movement information estimation device calculates an average value of information about a plurality of first moving bodies, and by using this average value, the estimation accuracy of the movement status of the passage is stabilized.

[0018] The estimation unit may further estimate an attitude of the passageway using the moving object information.

[0019] According to this, the movement information estimation device can further estimate the posture of the passage using the moving body information, and can easily estimate the movement status of the passage by adding the posture of the passage to the movement direction or speed of the passage.

[0020] The estimation unit may further estimate a position of a second moving object on the passageway using the moving object information.

[0021] According to this, the movement information estimation device can estimate the position of the second moving object on the passageway by using the moving object information of the first moving object. For example, the movement information estimation device can estimate the position of the second moving object from the movement direction and speed of the first moving object on the passageway estimated by using the moving object information of the first moving object.

[0022] The estimation unit may correct the estimated value of the position of the second moving object when the moving object information changes while the second moving object is moving.

[0023] According to this, when the moving body information changes while the second moving body is moving, the moving information estimation device can correct the estimated value of the position of the second moving body, thereby preventing a decrease in the estimation accuracy of the position of the second moving body.

[0024] The information processing device may further include a control unit that executes emergency processing, which is processing to be performed in an emergency, when the mobile object information changes by a predetermined value or more while the second mobile object is moving.

[0025] According to this, the movement information estimation device can protect facilities and ensure safety by executing emergency processing when the moving body information changes by more than a predetermined value while the second moving body is moving.

[0026] The passageway may be a portable object.

[0027] According to this, since the passage is portable, there is a risk that the passage may move or tilt, and the movement status of the passage may change. However, even if the movement status of the passage changes, the movement information estimation device can easily estimate the movement status of the passage using the moving object information.

[0028] A robot system according to one aspect of the present invention includes the above-described movement information estimation device and a robot that performs work on a second moving object on the passageway using the movement direction or speed of the passageway estimated by the movement information estimation device.

[0029] According to this, in the robot system, the movement information estimation device can easily estimate the movement status (movement direction or speed) of the passage as described above, and the robot can easily perform work on the second moving body using the movement status of the passage estimated by the movement information estimation device.

[0030] The present invention can be realized not only as such a movement information estimation device and robot system, but also as a movement information estimation method or a robot system control method including characteristic processing steps performed by the movement information estimation device or robot system. The present invention can also be realized as a program that causes a computer to execute the movement information estimation method or the robot system control method, or as a computer-readable recording medium such as a CD-ROM (Compact Disc-Read Only Memory) on which the program is recorded. Such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet. The present invention can also be realized as an integrated circuit including a processing unit that performs the movement information estimation method or the robot system control method.

[0031] Hereinafter, a movement information estimation device and a robot system according to embodiments of the present invention (including variations thereof) will be described with reference to the drawings. The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, the arrangement and connection of components, steps in methods, and the order of steps shown in the following embodiments are examples only and are not intended to limit the present invention. In each drawing, dimensions and the like are not strictly depicted. In each drawing, the same or similar components are designated by the same reference numerals.

[0032] In the following description and drawings, two intersecting directions in a horizontal plane are defined as the X-axis direction and the Y-axis direction, and the vertical direction (up and down direction) is defined as the Z-axis direction. These X-axis, Y-axis, and Z-axis directions intersect with each other (or are perpendicular in this embodiment). In the following description, the positive X-axis direction refers to the direction of the X-axis arrow, and the negative X-axis direction refers to the direction opposite to the positive X-axis direction. When simply referring to the X-axis direction, it refers to both or either of the positive X-axis direction and the negative X-axis direction. The same applies to the Y-axis and the Z-axis. Expressions indicating relative directions or attitudes, such as parallel and perpendicular, also include cases where the directions or attitudes are not strictly those of the same kind. Two directions being parallel (or perpendicular) not only mean that the two directions are completely parallel (or perpendicular), but also mean that the two directions are substantially parallel (or perpendicular), i.e., include a difference of, for example, several percent.

[0033] (Embodiment) 1. General Description of Robot System 10 First, a general description of a robot system 10 according to this embodiment will be given with reference to Fig. 1. Fig. 1 is a perspective view showing the appearance of a robot system 10 according to this embodiment.

[0034] As shown in FIG. 1 , the robot system 10 is a system that performs an operation on a moving body 500 on an aisle 400. Specifically, the robot system 10 holds an object (not shown), releases the held object, and serves the object to the moving body 500 moving along the aisle 400. An example of the object is food ingredients (toppings) such as prepared food. For example, the robot system 10 holds the food ingredients, releases them to the moving body 500, and serves (deposits) the food ingredients on the moving body 500. In this case, the moving body 500 is a container for each prepared food, a container for a lunch box, a plate, or the like. For example, multiple robot systems 10 are lined up, and various food ingredients are sequentially released to the moving body 500, thereby placing various food ingredients within the moving body 500. The object is not limited to food ingredients and may be any object that can be supplied to the moving body 500 by the robot system 10. The moving body 500 is also not limited to the above-mentioned containers, etc., and may be any object that can be placed on the aisle 400.

[0035] The robot system 10 includes a movement information estimation device 100, a robot 200, a platform 210, a camera 300, and a support member 310. The robot system 10 is disposed to the side of an aisle 400 (in the negative Y-axis direction), and a plurality of moving objects 500 are placed on the aisle 400. In this embodiment, the robot system 10 is defined without including the aisle 400 (the robot system 10 is defined as not including the aisle 400), but the robot system 10 may be defined including the aisle 400 (the robot system 10 is defined as including the aisle 400). In addition to the above configuration, the robot system 10 may also include an object storage unit such as a tray or a tray for storing objects held by the robot 200, an object supply unit for supplying objects to the robot 200, and the like.

[0036] The movement information estimation device 100 is a device that estimates the movement status (movement direction, speed, etc.) of the passage 400 and controls the operation of the robot 200. The movement information estimation device 100 is a computer that includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), input units (keyboard, touch panel, mouse, microphone, etc.), output units (liquid crystal display, speaker, etc.), a communication unit that communicates via a network, and drives, and executes various processes according to a program. The movement information estimation device 100 may be realized by a general-purpose computer system such as a personal computer executing a program, or may be realized by a dedicated computer system such as a programmable controller.

[0037] The movement information estimation device 100 is connected to the robot 200, the camera 300, etc. by wire or wirelessly, acquires information from the camera 300, and controls the operation of the robot 200 using the information. In this embodiment, the movement information estimation device 100 is housed in a pedestal 210. Note that the movement information estimation device 100 may be disposed outside the pedestal 210 or may be disposed at a location far from the robot 200, the camera 300, etc., and the location of the movement information estimation device 100 is not particularly limited. A detailed description of the configuration of the movement information estimation device 100 will be given later.

[0038] The robot 200 is an articulated robot and is installed (fixed) on a stand 210. Specifically, the robot 200 includes a robot arm 201 and a hand 202, and holds and releases an object. The robot arm 201 is an articulated arm that moves the hand 202 to a desired position within a movable range. The hand 202 is a part that holds and releases an object. The hand 202 may hold the object by grasping it, by suction, by scooping it, by piercing it, by containing it, or by any other method.

[0039] Specifically, the robot 200 performs a task on the moving object 500 on the passage 400. In this embodiment, the robot 200 performs a task on the moving object 500 on the passage 400 by using the movement status (movement direction, speed, etc.) of the passage 400 estimated by the movement information estimation device 100. That is, after holding an object, the robot 200 releases the object to the moving object 500 moving on the passage 400 by using the movement status (movement direction, speed, etc.) of the passage 400 estimated by the movement information estimation device 100.

[0040] The cradle 210 is a box-shaped base that supports the robot 200, and the robot 200 is attached and fixed on the cradle 210. The cradle 210 also serves as a base for the support member 310, and the support member 310 is also attached and fixed on the cradle 210. The cradle 210 is a case made of metal or the like, and houses the movement information estimation device 100 inside. An openable door is provided on a side wall of the cradle 210 (such as the front wall in the negative Y-axis direction), and various tasks such as maintenance or replacement of the movement information estimation device 100 can be performed through the door. The cradle 210 is a portable object that is configured to be movable (portable). Specifically, casters or the like may be arranged on the bottom surface of the cradle 210. Therefore, the robot system 10 is a portable object that is configured to be movable (portable).

[0041] The camera 300 is positioned above the passage 400 (in the positive direction of the Z axis) and is an imaging device capable of capturing images of the passage 400 and a moving object 500 on the passage 400. The camera 300 is supported by a support member 310 at a position a predetermined distance from the passage 400. The camera 300 is positioned facing directly below (in the negative direction of the Z axis) or diagonally downward (in a direction inclined from the negative direction of the Z axis), and captures a desired area in the passage 400 by capturing an image directly below or diagonally downward. The camera 300 can also measure the distance to the object being imaged. In other words, the camera 300 can detect the coordinates of the object in three-dimensional space (X, Y, Z coordinates). A publicly known 3D camera can be used as the camera 300. Various 3D camera systems exist, including stereo, ToF, and structured illumination systems, and any of these systems can be used. The camera 300 is connected to the movement information estimation device 100 by wire or wirelessly, and transmits to the movement information estimation device 100 captured images, position information (coordinates) of the captured object, and the like.

[0042] The support member 310 is an inverted L-shaped member (rod-shaped member) made of metal or the like that supports the camera 300. The end of the support member 310 facing the negative Z-axis is attached and fixed to the mount 210, and the support member 310 extends from this end in the positive Z-axis direction, bends at the end facing the positive Z-axis in the positive Y-axis direction, and continues to extend in the positive Y-axis direction. The camera 300 is attached and fixed to the center of the portion extending in the positive Y-axis direction. In this way, the support member 310 fixes the camera 300 to the mount 210.

[0043] The passage 400 is a transport device that transports the moving objects 500 on the passage 400. Specifically, the passage 400 is a belt conveyor that transports (carries) the moving objects 500, and extends in the X-axis direction. As a result, the moving objects 500 move in the X-axis direction (in this embodiment, the positive X-axis direction). In other words, as the upper surface of the passage 400 moves in a movement direction F (positive X-axis direction), the multiple moving objects 500 on the passage 400 move in a movement direction f (positive X-axis direction). The movement direction F of the passage 400 and the movement direction f of the moving objects 500 are parallel to each other. In this embodiment, the passage 400 is a portable object that is configured to be movable (portable). Specifically, it is conceivable to arrange casters or the like on legs that support the passage 400 (belt conveyor).

[0044] In a process (previous process) upstream (negative X-axis direction) of the robot 200, a person or another robot may place the moving body 500 in the passage 400 or perform work on the moving body 500 or the passage 400. As a result, the passage 400 may transport the moving body 500 in a state where the position or angle of the moving body 500 is deviated from the normal position or angle.

[0045] [2. Description of the Configuration of the Movement Information Estimation Device 100] Next, the configuration of the movement information estimation device 100 will be described in detail below. Fig. 2 is a block diagram showing the functional configuration of the movement information estimation device 100 according to this embodiment. Fig. 3 is a diagram explaining moving object information 501 acquired by the movement information estimation device 100 according to this embodiment. Fig. 3 is a top view of the passage 400 as seen from above (positive direction of the Z axis), and shows the passage 400, multiple moving objects 500 (first moving object 510, second moving object 520) on the passage 400, and the shooting area 301 of the camera 300.

[0046] As shown in Fig. 2, the movement information estimation device 100 includes an acquisition unit 110, an estimation unit 120, a control unit 130, and a storage unit 140. Also, as shown in Fig. 3, hereinafter, the area photographed by the camera 300 (area that can be photographed) will be referred to as a photographing area 301. Of the multiple moving objects 500 on the passage 400, the moving objects 510 (511 to 513) will also be referred to as first moving objects 510 (511 to 513), and the moving objects 520 (521 to 523) will also be referred to as second moving objects 520 (521 to 523). The positions to which the first moving objects 511 to 513 will move after a predetermined time has elapsed are indicated by first moving objects 511a to 513a.

[0047] 2.1 Description of the Acquisition Unit 110 The acquisition unit 110 acquires moving object information 501 indicating the moving direction or speed of the first moving object 510 on the passage 400 along which the first moving object 510 moves. In this embodiment, the acquisition unit 110 acquires moving object information 501 indicating both the moving direction and speed of the first moving object 510. In other words, the moving object information 501 is information including information indicating the moving direction of the first moving object 510 or information indicating the speed of the first moving object 510, but in this embodiment, it is information including both information indicating the moving direction of the first moving object 510 and information indicating the speed of the first moving object 510.

[0048] Specifically, the acquisition unit 110 acquires information indicating the position of the first moving body 510 before and after a predetermined time has elapsed from the camera 300. For example, the acquisition unit 110 acquires from the camera 300 first coordinates, which are the coordinates (XYZ coordinates) in three-dimensional space of the first moving body 510 at a first time, and second coordinates, which are the coordinates (XYZ coordinates) in three-dimensional space of the first moving body 510 at a second time that is a predetermined time after the first time. Explaining with reference to FIG. 3, the acquisition unit 110 acquires, for example, a first coordinate P1 that is the center position or the like of the first moving body 511 at the first time, and a second coordinate P2 that is the center position or the like of the first moving body 511a to which the first moving body 511 has moved at the second time.

[0049] Then, the acquisition unit 110 calculates the moving direction (XYZ components) and speed (XYZ components) of the first moving object 510 in three-dimensional space using the first coordinates and second coordinates (and the first time and second time). As a result, the acquisition unit 110 acquires moving object information 501 indicating both the moving direction (XYZ components) and speed (XYZ components) of the first moving object 510 in three-dimensional space. In other words, the moving object information 501 is information including both information indicating the moving direction (XYZ components) of the first moving object 510 in three-dimensional space and information indicating the speed (XYZ components) of the first moving object 510 in three-dimensional space. Explaining with reference to FIG. 3, the acquisition unit 110 acquires the moving object information 501 including both the moving direction f1 and speed v1 of the first moving object 511 by calculating the moving direction f1 and speed v1 of the first moving object 511 using the first coordinates P1 and second coordinates P2 of the first moving object 511, etc.

[0050] In this embodiment, the acquisition unit 110 acquires moving object information 501 for a plurality of first moving objects 510 on the passage 400. That is, as shown in FIG. 3 , the acquisition unit 110 acquires first coordinates and second coordinates for each of the first moving objects 512 and 513, similar to the first moving object 511 described above, and calculates the moving direction and speed, thereby acquiring the moving object information 501. Specifically, the moving object information 501 includes, in addition to the moving direction f1 and speed v1 of the first moving object 511, the moving direction f2 and speed v2 of the first moving object 512, and the moving direction f3 and speed v3 of the first moving object 513. In this manner, in this embodiment, the acquisition unit 110 acquires moving object information 501 for the first moving objects 511 to 513, which are the three first moving objects 510 on the passage 400. Note that the acquisition unit 110 may acquire moving object information 501 for two or four or more first moving objects 510 on the passage 400.

[0051] 2.2 Description of the Estimation Unit 120 The estimation unit 120 estimates the movement direction or speed of the passage 400 using the moving object information 501. In this embodiment, the estimation unit 120 estimates both the movement direction and speed of the passage 400 using the moving object information 501. Specifically, the estimation unit 120 estimates the movement direction (XYZ components) and speed (XYZ components) of the passage 400 in three-dimensional space using the moving object information 501. As shown in FIG. 3 , the estimation unit 120 estimates the movement direction F (XYZ components) and speed V (XYZ components) of the passage 400 using the moving object information 501.

[0052] As described above, the acquisition unit 110 acquires the moving object information 501 about the plurality of first moving objects 510 on the passage 400. For this reason, the estimation unit 120 calculates an average value of the information about the plurality of first moving objects 510 included in the moving object information 501, and estimates the moving direction or speed of the passage 400 (in this embodiment, the moving direction and speed) using the average value.

[0053] Explaining this with reference to FIG. 3 , the estimation unit 120 calculates the average values ​​of the moving direction f1 and speed v1 of the first moving object 511, the moving direction f2 and speed v2 of the first moving object 512, and the moving direction f3 and speed v3 of the first moving object 513, which are included in the moving object information 501. That is, the estimation unit 120 calculates the average value of the moving directions f1, f2, and f3, and calculates the average value of the speeds v1, v2, and v3. Then, the estimation unit 120 estimates the average value of the moving directions f1, f2, and f3 as the moving direction F of the passage 400, and estimates the average value of the speeds v1, v2, and v3 as the speed V of the passage 400. When the moving directions and speeds of the multiple first moving objects 510 are expressed as vectors, the estimation unit 120 calculates a composite value (composite vector) of the multiple vectors as the average value, and estimates the calculated composite vector as the moving direction and speed of the passage 400.

[0054] The estimation unit 120 further estimates the orientation of the passage 400 using the moving object information 501. The orientation of the passage 400 refers to the position and inclination of the passage 400 (for example, the coordinates of one or more points in the passage 400 in three-dimensional space and the inclination of the passage 400). Specifically, the estimation unit 120 estimates the position of the passage 400 (the coordinates of one or more points in the passage 400) from the position (coordinates) of the first moving object 510 used when acquiring the moving object information 501. The estimation unit 120 also estimates the inclination of the passage 400 from the moving direction of the passage 400 obtained using the moving object information 501. Explaining with reference to FIG. 3 , the estimation unit 120 estimates the orientation of the passage 400 using, for example, the first coordinates P1 (XYZ coordinates) of the first moving object 511 and the moving direction F (XYZ components) that is the inclination of the passage 400.

[0055] The estimation unit 120 further estimates the position of the second moving object 520 on the passage 400 using the moving object information 501. Specifically, the estimation unit 120 estimates the position of the second moving object 520 moving on the passage 400 from the movement direction and speed of the passage 400 estimated using the moving object information 501. In other words, the estimation unit 120 estimates the position of the second moving object 520 moving on the passage 400 after the first moving object 510 by estimating the movement direction and speed of the passage 400 using the movement direction and speed of the first moving object 510. The estimation unit 120 estimates the position (XYZ coordinates) of the second moving object 520 at the time when the robot 200 will perform an operation on the second moving object 520. To explain using Figure 3, for example, when a second moving body 520 (521 to 523) enters the shooting area 301 of the camera 300, the estimation unit 120 estimates the destination position of the second moving body 520 (521 to 523), assuming that the second moving body 520 (521 to 523) moves in the movement direction F and at the speed V of the passage 400.

[0056] Furthermore, if the moving body information 501 changes while the second moving body 520 is moving, the estimation unit 120 corrects the estimated value of the position of the second moving body 520.

[0057] 2.3 Description of the control unit 130 The control unit 130 executes emergency processing, which is processing in an emergency, when the mobile object information 501 changes by a predetermined value or more while the second mobile object 520 is moving. That is, the control unit 130 determines whether the mobile object information 501 has changed by a predetermined value or more while the second mobile object 520 is moving, and executes emergency processing if it determines that the mobile object information 501 has changed. Emergency processing is processing that should be performed in an emergency, such as issuing an alarm or making an emergency shutdown of equipment. The predetermined value is an upper limit (allowable value) for the amount of change in data included in the mobile object information 501 (such as the direction and speed of movement of the first mobile object 510), and is set by the user as a numerical value at which emergency processing should be performed.

[0058] Furthermore, the control unit 130 controls the robot 200 to cause the robot 200 to perform a task on the second moving object 520. That is, under the control of the control unit 130, the robot 200 performs a task on the second moving object 520 on the passage 400 using the movement direction or speed of the passage 400 estimated by the movement information estimation device 100. Furthermore, the control unit 130 performs various controls on other parts of the robot system 10.

[0059] 2.4 Description of the storage unit 140 The storage unit 140 is configured with a hard disk or a dynamic random access memory (DRAM), and is a memory that stores data and the like for controlling various operations in the robot system 10. Specifically, the storage unit 140 stores processing data 141. The processing data 141 contains the moving body information 501 acquired by the acquisition unit 110 and other information (such as data for controlling the operation of the robot 200).

[0060] [3. Description of Processing Flow of Movement Information Estimation Device 100] Next, the processing performed by the movement information estimation device 100 will be described. Fig. 4 is a flowchart showing the processing performed by the movement information estimation device 100 according to this embodiment. Fig. 5 is a diagram showing a state in which the moving object information 501 acquired by the movement information estimation device 100 according to this embodiment has changed. Fig. 6 is a diagram explaining the processing in which the movement information estimation device 100 according to this embodiment estimates the position of the second moving object 520. Figs. 5 and 6 correspond to Fig. 3.

[0061] As shown in Fig. 4, first, the acquisition unit 110 acquires information indicating the position of the first moving object 510 (step S11). Specifically, the acquisition unit 110 acquires information (XYZ coordinates) indicating the position of the first moving object 510 before and after a predetermined time has elapsed from the camera 300. As shown in Fig. 5, when the passage 400 is tilted, the acquisition unit 110 acquires information (XYZ coordinates) indicating the position of the first moving object 510 before and after a predetermined time has elapsed in a state in which the passage 400 is tilted. The specific process by which the acquisition unit 110 acquires the information indicating the position of the first moving object 510 is as described above.

[0062] Next, the acquisition unit 110 acquires moving object information 501 indicating the moving direction or speed of the first moving object 510 on the passage 400 along which the first moving object 510 is moving (step S12). In this embodiment, the acquisition unit 110 acquires moving object information 501 indicating both the moving direction and speed of the first moving object 510. The acquisition unit 110 also acquires moving object information 501 for multiple first moving objects 510 on the passage 400. The specific process by which the acquisition unit 110 acquires the moving object information 501 is as described above. The acquisition unit 110 writes the acquired moving object information 501 into processing data 141 stored in the storage unit 140.

[0063] Next, the estimation unit 120 calculates the average value of information about the multiple first moving bodies 510 included in the moving body information 501 (step S13). Specifically, the estimation unit 120 reads the moving body information 501 from the processing data 141 stored in the memory unit 140, and calculates the average value of information about the multiple first moving bodies 510 included in the moving body information 501. The specific process by which the estimation unit 120 calculates the average value is as described above. The estimation unit 120 writes the calculated average value to the processing data 141 stored in the memory unit 140.

[0064] Next, the control unit 130 determines whether the mobile object information 501 has changed by a predetermined value or more while the second mobile object 520 is moving (step S14). Specifically, the control unit 130 reads the mobile object information 501 from the processing data 141 stored in the storage unit 140, and determines whether the mobile object information 501 has changed by a predetermined value or more. The predetermined value may be set in advance, or may be stored in the processing data 141 of the storage unit 140 and read out from the processing data 141 by the control unit 130.

[0065] When the control unit 130 determines that the mobile object information 501 has changed by a predetermined value or more while the second mobile object 520 is moving (YES in step S14), the control unit 130 executes emergency processing, which is processing in an emergency (step S15). In this case, the control unit 130 issues an alarm or performs an emergency shutdown of the equipment. For example, if the mobile object information 501 has changed by a predetermined value or more when the state shown in FIG. 3 changes to the state shown in FIG. 5, the control unit 130 determines that the mobile object information 501 has changed by a predetermined value or more and executes emergency processing.

[0066] If the control unit 130 determines that the moving body information 501 has not changed by more than a predetermined value while the second moving body 520 is moving (NO in step S14), the control unit 130 performs the following process. For example, even when the state changes from that shown in Fig. 3 to that shown in Fig. 5, if the moving body information 501 has not changed by more than a predetermined value, the control unit 130 determines that the moving body information 501 has not changed by more than a predetermined value, and performs the following process.

[0067] First, the estimation unit 120 estimates the movement direction or speed of the passage 400 (step S16). In this embodiment, the estimation unit 120 estimates both the movement direction and speed of the passage 400 using the moving object information 501. Specifically, the estimation unit 120 estimates the movement direction or speed (movement direction and speed) of the passage 400 using the average value calculated in step S13. The estimation unit 120 reads the average value calculated in step S13 from the processing data 141 stored in the storage unit 140, and estimates the movement direction and speed of the passage 400 using the average value. The specific process by which the estimation unit 120 estimates the movement direction and speed of the passage 400 is as described above. In the state shown in FIG. 5, the estimation unit 120 estimates the movement direction and speed of the passage 400 as the movement direction F (XYZ components) and speed V (XYZ components) shown in FIG. 5. The estimation unit 120 writes the estimated direction and speed of movement of the passage 400 into the processing data 141 stored in the storage unit 140.

[0068] Next, the estimation unit 120 estimates the attitude of the passage 400 (step S17). Specifically, the estimation unit 120 estimates the attitude of the passage 400 by using the moving object information 501. The estimation unit 120 reads the position of the first moving object 510, the moving direction of the passage 400, the moving object information 501, etc. from the processing data 141 stored in the memory unit 140, and estimates the attitude of the passage 400. The specific process by which the estimation unit 120 estimates the attitude of the passage 400 is as described above. The estimation unit 120 writes the estimated attitude of the passage 400 to the processing data 141 stored in the memory unit 140.

[0069] Next, the estimation unit 120 estimates the position of the second moving object 520 on the passage 400 (step S18). Specifically, the estimation unit 120 estimates the position (XYZ coordinates) of the second moving object 520 on the passage 400 using the moving object information 501. The estimation unit 120 reads out the movement direction and speed of the passage 400 from the processing data 141 stored in the memory unit 140, and estimates the position of the second moving object 520. The specific process by which the estimation unit 120 estimates the position of the second moving object 520 is as described above.

[0070] In the state shown in Fig. 5, in step S16, the estimation unit 120 estimates the movement direction and speed of the passage 400 as the movement direction F (XYZ components) and speed V (XYZ components) shown in Fig. 5. Therefore, as shown in Fig. 6, the estimation unit 120 estimates the movement direction and speed of the second moving object 520 as the same movement direction F and speed V as the passage 400, thereby estimating the position (XYZ coordinates) of the second moving object 520. Specifically, the estimation unit 120 estimates that the second moving object 520 (521 to 523) will move in the movement direction F and at the speed V to the position of the second moving object 520a (521a to 523a).

[0071] 6, when a second moving body 521 enters the shooting area 301 of the camera 300, the acquisition unit 110 acquires information (XYZ coordinates) indicating the position of the second moving body 521 from the camera 300. Then, the estimation unit 120 calculates information (XYZ coordinates) indicating the position of a second moving body 521a, which is the destination of the second moving body 521, using the information indicating the position of the second moving body 521 acquired by the acquisition unit 110 and the moving direction F and speed V of the second moving body 521. The same applies to the second moving body 522 and the second moving body 523. In this way, the estimation unit 120 estimates the position of the destination second moving body 520a (521a to 523a) for each of the multiple second moving bodies 520 (521 to 523) on the passage 400. The estimation unit 120 writes the estimated position of the second moving body 520a into the processing data 141 stored in the storage unit 140.

[0072] If the moving body information 501 changes (changes smaller than the predetermined value) while the second moving body 520 is moving, the estimation unit 120 corrects the estimated value of the position of the second moving body 520. For example, if the moving body information 501 changes smaller than the predetermined value when the state changes from that shown in Fig. 3 to that shown in Fig. 5, the estimation unit 120 corrects the estimated value of the position of the second moving body 520. The estimation unit 120 writes the corrected estimated value (XYZ coordinates) of the position of the second moving body 520 into the processing data 141 stored in the storage unit 140, thereby updating (correcting) the data.

[0073] Next, the control unit 130 causes the robot 200 to perform the task on the second moving body 520 (step S19). Specifically, the control unit 130 causes the robot 200 to perform the task on the second moving body 520 using the movement direction or speed (movement direction and speed in this embodiment) of the passage 400 estimated by the estimation unit 120. More specifically, the control unit 130 causes the robot 200 to perform the task on the second moving body 520 using the estimated value (XYZ coordinates) of the position of the second moving body 520 estimated by the estimation unit 120. The control unit 130 reads the estimated value of the position of the second moving body 520 from the processing data 141 stored in the memory unit 140, and causes the robot 200 to perform the task on the second moving body 520.

[0074] 6, for example, with respect to the second moving body 521, the control unit 130 moves the hand 202 of the robot 200 to the position of the second moving body 521a and causes the hand 202 to perform work on the second moving body 521. The same applies to the second moving body 522 and the second moving body 523, etc. This allows the robot 200 to perform work on the second moving body 520, such as releasing an object at an estimated value (XYZ coordinates) of the position to which the second moving body 520 will be moved.

[0075] This completes the processing performed by the movement information estimation device 100. The timing of performing each process in the movement information estimation device 100 is not particularly limited, and any of the processes may be performed before the robot 200 starts operating (before the second moving body 520 starts moving), while the robot 200 is operating (while the second moving body 520 starts moving), or periodically.

[0076] [4. Explanation of effects] As described above, the movement information estimation device 100 according to the embodiment of the present invention acquires moving object information 501 indicating the moving direction or speed of the first moving object 510 on the passage 400 along which the moving object 510 is moving, and estimates the moving direction or speed of the passage 400 using the moving object information 501. In this way, the movement information estimation device 100 can estimate the moving direction or speed of the passage 400 using simple information, namely the moving object information 501 indicating the moving direction or speed of the first moving object 510. In particular, in this embodiment, the movement information estimation device 100 can acquire necessary information such as the moving object information 501 using a single camera 300, thereby simplifying the configuration of the robot system 10. As a result, the movement information estimation device 100 can easily estimate the movement status of the passage 400 along which the moving object is moving.

[0077] Here, the reasons for estimating the movement direction or speed of the passage 400 without using the set values ​​of the movement direction or speed of the passage 400 as they are are as follows: (i) The movement direction or speed of the passage 400 may suddenly change due to controlling the movement direction or speed of the passage 400 or a person holding the passage 400 during cleaning, etc. (ii) The movement direction or speed of the passage 400 may not match the set values ​​due to a malfunction of the passage 400 itself or individual differences between the passages 400, etc. (iii) When multiple passages 400 are connected, the set values ​​of the passages 400 may be misaligned or may be changed for each passage 400. Even in such cases, the movement information estimation device 100 can easily estimate the movement status of the passage 400.

[0078] The movement information estimation device 100 acquires moving object information 501 indicating both the moving direction and speed of the first moving object 510, and estimates both the moving direction and speed of the passage 400 using the moving object information 501. In this way, the movement information estimation device 100 estimates both the moving direction and speed of the passage 400 from both the moving direction and speed of the first moving object 510, thereby enabling more accurate estimation of the movement status of the passage 400.

[0079] By acquiring the moving body information 501 for the multiple first moving bodies 510, the moving information estimation device 100 can use the moving body information 501 for the multiple first moving bodies 510 to more accurately estimate the moving status (moving direction or speed) of the passage 400.

[0080] The movement information estimation device 100 calculates an average value of information about the multiple first moving bodies 510, and by using this average value, the estimation accuracy of the movement status (movement direction or speed) of the passage 400 is stabilized. In other words, when estimation is performed from only information about one first moving body 510, if the single first moving body 510 makes an irregular movement on the passage 400, such as rolling over or tipping over, the estimation accuracy of the movement status of the passage 400 may be significantly reduced. Therefore, by using the average value of information about the multiple first moving bodies 510, it is possible to prevent a significant reduction in the estimation accuracy of the movement status of the passage 400, and the estimation accuracy is stabilized.

[0081] The movement information estimation device 100 can further estimate the attitude (position, inclination, etc.) of the passage 400 using the moving object information 501, thereby easily estimating the movement status of the passage 400 by adding the attitude of the passage 400 to the movement direction or speed of the passage 400. This allows the movement information estimation device 100 to estimate whether the passage 400 is an uphill or downhill slope, whether the passage 400 is meandering, etc.

[0082] The movement information estimation device 100 can estimate the position of the second moving body 520 on the passage 400 using the moving body information 501 of the first moving body 510. For example, the movement information estimation device 100 can estimate the position of the second moving body 520 from the movement direction and speed of the first moving body 510 on the passage 400 estimated using the moving body information 501 of the first moving body 510. For example, while the robot 200 is performing an operation on one moving body 500, the movement information estimation device 100 can estimate the position of the other moving body 500. In particular, when some operation is performed on the second moving body 520 upstream of the robot 200 (in a previous process), the need to precisely adjust the position of the second moving body 520 after the operation is reduced, and in this respect as well, the present invention can improve the efficiency of the operation.

[0083] When the moving body information 501 changes while the second moving body 520 is moving due to vibrations of the passage 400 or the like, the movement information estimation device 100 corrects the estimated value of the position of the second moving body 520, thereby preventing a decrease in the estimation accuracy of the position of the second moving body 520. In this way, the movement information estimation device 100 can detect vibrations of the passage 400 or the like and correct the estimated value of the position of the second moving body 520 (can calibrate tracking of the second moving body 520).

[0084] The movement information estimation device 100 can protect equipment and ensure safety by executing emergency processing (such as issuing an alarm or making an emergency stop) when the moving object information 501 changes by a predetermined value or more while the second moving object 520 is moving. This allows the movement information estimation device 100 to detect a sudden stop of the passage 400, and in this case, can control the robot 200 to stop working. When executing an emergency process to stop the robot 200 from working, the movement information estimation device 100 stops the robot 200 in response to a change in the moving object information 501, so there is no need to connect the passage 400 and the robot 200 to control them. There is also no need to connect an encoder or the like to the passage 400 to determine abnormalities such as a sudden acceleration or sudden stop of the passage 400.

[0085] Because the passage 400 is portable, there is a risk that the passage 400 may move or tilt, causing a change in the movement status of the passage 400. However, even if the movement status of the passage 400 changes, the movement information estimation device 100 can easily estimate the movement status of the passage 400 by using the moving object information 501. Furthermore, because the robot system 10 is also portable, the passage 400 is likely to shift position relative to the robot system 10 (robot 200, camera 300, etc.). Even in this case, the movement information estimation device 100 can easily estimate the movement status of the passage 400 by using the moving object information 501.

[0086] According to the robot system 10 of the embodiment of the present invention, the movement information estimation device 100 can easily estimate the movement status (movement direction or speed) of the passage 400 as described above, and therefore the robot 200 can easily perform work on the second moving body 520 using the movement status of the passage 400 estimated by the movement information estimation device 100.

[0087] The various effects of the movement information estimation device 100 described above can be similarly applied to the effects of the robot system 10.

[0088] [5. Explanation of Variations] Although the movement information estimation device 100 and the robot system 10 according to the present embodiment have been described above, the present invention is not limited to the above embodiment. The embodiment disclosed herein is illustrative in all respects and is not restrictive, and the scope of the present invention includes all modifications within the meaning and scope of the claims.

[0089] In the above embodiment, the passage 400 is a transport device (belt conveyor) that transports the moving object 500 on the passage 400, and is a portable object, but is not limited to this. The passage 400 may be a transport device with a structure different from that of a belt conveyor, or may not be portable.

[0090] In the above embodiment, the acquisition unit 110 of the movement information estimation device 100 acquires the moving object information 501 indicating both the moving direction and the speed of the first moving object 510. However, this is not limited to this. The moving object information 501 may be information indicating either the moving direction or the speed of the first moving object 510. That is, the acquisition unit 110 may acquire the moving object information 501 indicating the moving direction or the speed of the first moving object 510 on the passage 400 along which the first moving object 510 moves. When the acquisition unit 110 acquires the moving object information 501 indicating the moving direction of the first moving object 510, the estimation unit 120 may estimate the moving direction of the passage 400 using the moving object information 501. When the acquisition unit 110 acquires the moving object information 501 indicating the speed of the first moving object 510, the estimation unit 120 may estimate the speed of the passage 400 using the moving object information 501. Even when the acquisition unit 110 acquires the moving object information 501 indicating both the moving direction and speed of the first moving object 510, the estimation unit 120 may estimate either the moving direction or the speed of the passage 400 using the moving object information 501. That is, in the above embodiment, the estimation unit 120 estimates both the moving direction and the speed of the passage 400 using the moving object information 501, but it is sufficient if the estimation unit 120 estimates either the moving direction or the speed of the passage 400.

[0091] In the above embodiment, the acquisition unit 110 acquires the coordinates (XYZ coordinates) of the first moving object 510 in three-dimensional space, and calculates the moving direction (XYZ components) and speed (XYZ components) of the first moving object 510 in three-dimensional space to acquire the moving object information 501, but this is not limited to this. The acquisition unit 110 may acquire the coordinates (excluding the Z coordinate) of the first moving object 510 in the XY plane, and calculate the moving direction (XY components) and speed (XY components) of the first moving object 510 in the XY plane to acquire the moving object information 501. In this case, the estimation unit 120 may use the moving object information 501 to estimate the moving direction (XY components) and speed (XY components) of the passage 400 in the XY plane. Even when the acquisition unit 110 acquires the moving object information 501 in three-dimensional space, the estimation unit 120 may use the moving object information 501 to estimate the moving direction (XY components) and speed (XY components) on the XY plane of the passage 400.

[0092] In the above embodiment, the acquisition unit 110 acquires the moving object information 501 for multiple first moving objects 510 on the passage 400, but it may also acquire one piece of moving object information 501 for only one first moving object 510. In this case, the estimation unit 120 estimates the moving direction or speed of the passage 400 using the one first moving object 510.

[0093] In the above embodiment, the estimation unit 120 calculates an average value of information about the multiple first moving bodies 510 included in the moving body information 501 and estimates the movement direction or speed of the passage 400 using the average value, but this is not limited to this. The estimation unit 120 may estimate the movement direction or speed of the passage 400 using a value other than the average, such as a maximum value, median value, or minimum value of information about the multiple first moving bodies 510 included in the moving body information 501. The value that the estimation unit 120 uses can be determined appropriately by the user depending on the status of the facility, the value to be estimated, etc.

[0094] In the above embodiment, the control unit 130 executes emergency processing when the moving object information 501 changes by a predetermined value or more while the second moving object 520 is moving, but this is not limited to this. The control unit 130 may execute emergency processing under other conditions, or may not execute emergency processing under any conditions.

[0095] In the above embodiment, the robot 200 performs work on the second moving object 520 on the passage 400 using the movement direction or speed of the passage 400 estimated by the movement information estimation device 100, but this is not limited to this. A device or worker different from the robot 200 may perform work on the second moving object 520 using the movement direction or speed of the passage 400 estimated by the movement information estimation device 100, or the state of the second moving object 520 may be simply monitored, and no work may be performed on the second moving object 520. In this case, the robot 200 does not need to be deployed.

[0096] In the above embodiment, the robot system 10 (the stand 210) may be fixed to the floor surface and may not be portable.

[0097] In the above embodiment, the robot system 10 is not limited to including all of the above-described components. For example, the robot system 10 may not include the mount 210, and the robot 200 may be fixed to another member. The robot system 10 may not include the support member 310, and the camera 300 may be fixed to another member. The robot system 10 may not include the camera 300, and the movement information estimation device 100 may acquire information such as the moving object information 501 from another device.

[0098] In the above embodiment, the movement information estimation device 100 is not limited to having all of the above-mentioned processing units. For example, the movement information estimation device 100 may not have the control unit 130, and another device may control the control unit 130. The movement information estimation device 100 may not have the storage unit 140, and may exchange data with an external memory or the like.

[0099] In the above embodiment, the movement information estimation device 100 is not limited to executing all of the above steps, and the steps executed by the movement information estimation device 100 are not limited to being performed in the above order. In the above embodiment, the movement information estimation device 100 is only required to estimate the movement direction or speed of the passage 400 using the moving object information 501 in order to utilize the movement direction or speed of the passage 400 for other processing. Therefore, for example, the estimation unit 120 is not required to estimate the posture of the passage 400, nor is it required to estimate the position of the second moving object 520. Furthermore, the estimation unit 120 does not need to correct the estimated value of the position of the second moving object 520 even if the moving object information 501 changes while the second moving object 520 is moving.

[0100] Furthermore, the present invention can be realized not only as the movement information estimation device 100 and the robot system 10, but also as a movement information estimation method or a robot system control method including characteristic processing steps performed by the movement information estimation device 100 or the robot system 10. The present invention can also be realized as a program causing a computer to execute the movement information estimation method or the robot system control method. That is, each component of the movement information estimation device 100 may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Furthermore, the present invention can also be realized as any medium, such as a computer-readable non-transitory recording medium on which the program is recorded, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray (registered trademark) Disc), semiconductor memory, flash memory, magnetic storage device, optical disc, or paper tape. The program can then be distributed via the recording medium and a transmission medium such as the Internet. The present invention can also be realized as an integrated circuit including a processing unit included in the movement information estimation device 100. That is, each functional block of the movement information estimation device 100 shown in Fig. 2 may be realized as an LSI (Large Scale Integration) which is an integrated circuit. These may be individually integrated into one chip, or some or all of them may be integrated into one chip. In this way, each component of the movement information estimation device 100 may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component.

[0101] Any combination of the components included in the above-described embodiments and their modifications is also included within the scope of the present invention. [Explanation of symbols]

[0102] 10 Robot Systems 100 Movement information estimation device 110 Acquisition Department 120 Estimation part 130 control section 140 Storage section 141 Processing Data 200 robots 201 Robot Arm 202 hands 210 Mounting stand 300 cameras 301 Shooting Area 310 Support member 400 aisles 500 Mobile 501 Mobile Information 510, 511, 511a, 512, 512a, 513, 513a First mobile body (mobile body) 520, 520a, 521, 521a, 522, 522a, 523, 523a Second mobile body (mobile body)

Claims

1. an acquisition unit that acquires moving object information indicating a moving direction or a speed of a first moving object on a moving passage; an estimation unit that estimates a moving direction or a speed of the passage and an attitude of the passage using the moving object information, the acquisition unit acquires the moving body information about all of the first moving bodies that are on the passage and are included in an image capturing area captured by an image capturing device, The estimation unit calculates an average value of information about all of the first moving bodies included in the moving body information, and estimates a moving direction or speed of the passage and an attitude of the passage using the average value. Movement information estimation device.

2. an acquisition unit that acquires moving object information indicating a moving direction or a speed of a first moving object on a moving passage; an estimation unit that estimates a moving direction or a speed of the passage and an attitude of the passage using the moving object information, the acquisition unit acquires the moving body information including information indicating a moving direction of the first moving body in a three-dimensional space; The estimation unit estimates an inclination of the passage in a height direction as the posture of the passage. Movement information estimation device.

3. The acquisition unit acquires the moving object information about the first moving objects on the passage. The movement information estimation device according to claim 2 .

4. The estimation unit calculates an average value of information about the plurality of first moving bodies included in the moving body information, and estimates a moving direction or speed of the passage and an attitude of the passage using the average value. The movement information estimation device according to claim 3 .

5. the acquisition unit acquires the moving object information indicating both a moving direction and a speed of the first moving object; The estimation unit estimates both the moving direction and the speed of the moving object along the passage using the moving object information. The movement information estimation device according to any one of claims 1 to 3.

6. The estimation unit further estimates a position of a second moving object on the passageway using the moving object information. The movement information estimation device according to any one of claims 1 to 3.

7. The estimation unit corrects the estimated value of the position of the second moving object when the moving object information changes while the second moving object is moving. The movement information estimation device according to claim 6 .

8. The information processing device further includes a control unit that executes an emergency process when the mobile object information changes by a predetermined value or more while the second mobile object is moving. The movement information estimation device according to claim 6 .

9. The passageway is a portable object The movement information estimation device according to any one of claims 1 to 3.

10. A movement information estimation device according to any one of claims 1 to 3; a robot that performs an operation on a second moving object on the passageway using the movement direction or speed of the passageway estimated by the movement information estimation device; A robot system comprising:

Images