Movement information estimation device and robot system

Abstract

A movement information estimation device is provided that can easily estimate the position of a moving object moving through a passageway. [Solution] The movement information estimation device 100 includes an acquisition unit 110 that acquires edge information 401 indicating the posture of an edge 410 of a passage 400 along which a moving body 500 moves, and an estimation unit 120 that estimates the position of the moving body 500 using the edge information 401.

Description

[Technical Field]

[0001] The present invention relates to a movement information estimation device and a robot system. [Background technology]

[0002] Patent Document 1 discloses a calibration method including the steps of provisionally setting conveyor tracking information, acquiring a reference image, a first image, and a second image by photographing a reference object placed on a conveyor at three different photographing timings while the conveyor is moving at a constant speed, determining measurement coordinate values ​​of the reference position, the first position, and the second position in the robot coordinate system by analyzing the three images, and correcting the provisionally set value of the conveyor tracking information so that the position and operation of the conveyor represented by the conveyor tracking information are consistent with various measurement results. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-11467 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional method disclosed in Patent Document 1, images are taken at multiple locations to capture three images (a reference image, a first image, and a second image) at three different capture timings. For this reason, Patent Document 1 uses multiple cameras, such as a conveyor camera (first camera) fixed to the conveyor and a mobile camera (second camera) attached to the robot arm of the robot, resulting in a complex configuration. As described above, the conventional method disclosed in Patent Document 1 has a complex configuration, so when it is desired to determine the position of a moving object moving along a passage such as a conveyor, it may be difficult to determine the position of the moving object.

[0005] The present invention was made by the inventor of the present application by focusing on the above-mentioned problem, and aims to provide a movement information estimation device and a robot system that can easily estimate the position of a moving object moving through a passageway. [Means for solving the problem]

[0006] A movement information estimation device according to one aspect of the present invention includes an acquisition unit that acquires edge information indicating the posture of the edge of a passageway along which a moving body moves, and an estimation unit that estimates the position of the moving body using the edge information.

[0007] A robot system according to one aspect of the present invention includes the movement information estimation device described above, and a robot that performs work on the moving object using the position of the moving object estimated by the movement information estimation device. [Effects of the Invention]

[0008] According to the movement information estimation device and the like of the present invention, the position of a moving object moving along a passage can be easily estimated. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a perspective view showing an appearance of a robot system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing a functional configuration of a movement information estimation device according to an embodiment. [Figure 3] 10A and 10B are diagrams illustrating edge information and point information acquired by a movement information estimation device according to an embodiment. [Figure 4] 10 is a flowchart illustrating a process performed by the movement information estimation device according to the embodiment. [Figure 5] 10A and 10B are diagrams illustrating a state in which edge information and point information acquired by the movement information estimation device according to the embodiment have changed. [Figure 6] 10A and 10B are diagrams illustrating a state in which edge information and point information acquired by the movement information estimation device according to the embodiment have further changed. [Figure 7]4A and 4B are diagrams illustrating a process in which the movement information estimation device according to the embodiment estimates the position of a moving object. DETAILED DESCRIPTION OF THE INVENTION

[0010] A movement information estimation device according to one aspect of the present invention includes an acquisition unit that acquires edge information indicating the posture of the edge of a passageway along which a moving body moves, and an estimation unit that estimates the position of the moving body using the edge information.

[0011] According to this, the movement information estimation device acquires edge information indicating the posture of the edge of the passage along which the moving object moves, and estimates the position of the moving object using the edge information. This allows the movement information estimation device to estimate the position of the moving object using simple information, namely the edge information indicating the posture of the edge of the passage. Therefore, the movement information estimation device can easily estimate the position of the moving object moving along the passage.

[0012] The acquisition unit may acquire, as the edge information, information indicating a position of a predetermined point on the edge and a slope of the edge.

[0013] According to this, the movement information estimation device can obtain edge information indicating the position of a point on the edge of the passage and the inclination of the edge, and can estimate the posture of the passage from the position of a point on the edge and the inclination of the edge, thereby making it easy to estimate the position of a moving body.

[0014] The acquisition unit may further acquire point information indicating the position of a point on the passage that is different from the edge, and the estimation unit may estimate the position of the moving body using the edge information and the point information.

[0015] According to this, the movement information estimation device further acquires point information indicating the position of a point other than on the edge of the passage, and by estimating the position of the moving body using the edge information and point information, it is possible to estimate the position of the moving body while also taking into account the inclination of the passage in the width direction.

[0016] The estimation unit may use the edge information to calculate area information indicating the attitude of an area on the passage through which the moving object passes, and may use the area information to estimate the position of the moving object.

[0017] According to this, the movement information estimation device calculates area information indicating the attitude of an area on a passage through which a moving object passes, and estimates the position of the moving object using the area information. In this way, the movement information estimation device can estimate the position of the moving object from the attitude of the area on a passage through which the moving object passes.

[0018] The acquisition unit may determine the longest line included in the image of the passage as the edge and acquire the edge information.

[0019] According to this, the movement information estimation device can easily acquire edge information by determining the longest line included in the image of the passage as the edge of the passage.

[0020] The estimation unit may correct the estimated value of the position of the moving object when the edge information changes while the moving object is moving.

[0021] According to this, when the posture of the passage changes while the moving body is moving and the edge information changes, the movement information estimation device can correct the estimated value of the moving body's position, thereby preventing a decrease in the estimation accuracy of the moving body's position.

[0022] The mobile device may further include a control unit that executes emergency processing, which is processing to be performed in an emergency, when the edge information changes by a predetermined value or more while the mobile object is moving.

[0023] According to this, if the posture of the passage changes significantly while the moving body is moving and the edge information changes by more than a predetermined value, the movement information estimation device can protect equipment and ensure safety by performing emergency processing.

[0024] The passageway may be a portable object.

[0025] According to this, since the passage is portable, there is a risk that the passage may move or tilt, causing the position of the moving object to change. However, even if the position of the moving object changes, the movement information estimation device can easily estimate the position of the moving object by using edge information that indicates the posture of the edge of the passage.

[0026] The passage may be a transport device that transports the moving object on the passage.

[0027] According to this, since the passage is a conveying device and the posture of the passage is likely to change as the conveying device operates, the position of the moving object is likely to change. However, even if the position of the moving object changes, the movement information estimation device can easily estimate the position of the moving object by using edge information that indicates the posture of the edge of the passage.

[0028] A robot system according to one aspect of the present invention includes the movement information estimation device described above, and a robot that performs work on the moving object using the position of the moving object estimated by the movement information estimation device.

[0029] As a result, in the robot system, the movement information estimation device can easily estimate the position of a moving object moving through a passageway, as described above, and the robot can easily perform work on the moving object using the position of the moving object estimated by the movement information estimation device.

[0030] The present invention can be realized not only as such a movement information estimation device and robot system, but also as a movement information estimation method or a robot system control method including characteristic processing steps performed by the movement information estimation device or robot system. The present invention can also be realized as a program that causes a computer to execute the movement information estimation method or the robot system control method, or as a computer-readable recording medium such as a CD-ROM (Compact Disc-Read Only Memory) on which the program is recorded. Such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet. The present invention can also be realized as an integrated circuit including a processing unit that performs the movement information estimation method or the robot system control method.

[0031] Hereinafter, a movement information estimation device and a robot system according to embodiments of the present invention (including variations thereof) will be described with reference to the drawings. The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, the arrangement and connection of components, steps in methods, and the order of steps shown in the following embodiments are examples only and are not intended to limit the present invention. In each drawing, dimensions and the like are not strictly depicted. In each drawing, the same or similar components are designated by the same reference numerals.

[0032] In the following description and drawings, two intersecting directions in a horizontal plane are defined as the X-axis direction and the Y-axis direction, and the vertical direction (up and down direction) is defined as the Z-axis direction. These X-axis, Y-axis, and Z-axis directions intersect with each other (or are perpendicular in this embodiment). In the following description, the positive X-axis direction refers to the direction of the X-axis arrow, and the negative X-axis direction refers to the direction opposite to the positive X-axis direction. When simply referring to the X-axis direction, it refers to both or either of the positive X-axis direction and the negative X-axis direction. The same applies to the Y-axis and the Z-axis. Expressions indicating relative directions or attitudes, such as parallel and perpendicular, also include cases where the directions or attitudes are not strictly those. Two directions being parallel (or perpendicular) not only mean that the two directions are completely parallel (or perpendicular), but also mean that the two directions are substantially parallel (or perpendicular), that is, include a difference of, for example, several percent.

[0033] (Embodiment) 1. General Description of Robot System 10 First, a general description of a robot system 10 according to this embodiment will be given with reference to Fig. 1. Fig. 1 is a perspective view showing the appearance of a robot system 10 according to this embodiment.

[0034] As shown in FIG. 1 , the robot system 10 is a system that performs an operation on a moving body 500 on an aisle 400. Specifically, the robot system 10 holds an object (not shown), releases the held object, and serves the object to the moving body 500 moving along the aisle 400. An example of the object is food ingredients (toppings) such as prepared food. For example, the robot system 10 holds the food ingredients, releases them to the moving body 500, and serves (deposits) the food ingredients on the moving body 500. In this case, the moving body 500 is a container for each prepared food, a container for a lunch box, a plate, or the like. For example, multiple robot systems 10 are lined up, and various food ingredients are sequentially released to the moving body 500, thereby placing various food ingredients within the moving body 500. The object is not limited to food ingredients and may be any object that can be supplied to the moving body 500 by the robot system 10. The moving body 500 is also not limited to the above-mentioned containers, etc., and may be any object that can be placed on the aisle 400.

[0035] The robot system 10 includes a movement information estimation device 100, a robot 200, a platform 210, a camera 300, and a support member 310. The robot system 10 is disposed to the side of an aisle 400 (in the negative Y-axis direction), and a plurality of moving objects 500 are placed on the aisle 400. In this embodiment, the robot system 10 is defined without including the aisle 400 (the robot system 10 is defined as not including the aisle 400), but the robot system 10 may be defined including the aisle 400 (the robot system 10 is defined as including the aisle 400). In addition to the above configuration, the robot system 10 may also include an object storage unit such as a tray or a tray for storing objects held by the robot 200, an object supply unit for supplying objects to the robot 200, and the like.

[0036] The movement information estimation device 100 is a device that estimates the position of a moving object 500 on a passage 400 and controls the operation of a robot 200. The movement information estimation device 100 is a computer that includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), input units (keyboard, touch panel, mouse, microphone, etc.), output units (liquid crystal display, speaker, etc.), a communication unit that communicates via a network, and a drive, and executes various processes according to a program. The movement information estimation device 100 may be realized by a general-purpose computer system such as a personal computer executing a program, or may be realized by a dedicated computer system such as a programmable controller.

[0037] The movement information estimation device 100 is connected to the robot 200, the camera 300, etc. by wire or wirelessly, acquires information from the camera 300, and controls the operation of the robot 200 using the information. In this embodiment, the movement information estimation device 100 is housed in a pedestal 210. Note that the movement information estimation device 100 may be disposed outside the pedestal 210 or may be disposed at a location far from the robot 200, the camera 300, etc., and the location of the movement information estimation device 100 is not particularly limited. A detailed description of the configuration of the movement information estimation device 100 will be given later.

[0038] The robot 200 is an articulated robot and is installed (fixed) on a stand 210. Specifically, the robot 200 includes a robot arm 201 and a hand 202, and holds and releases an object. The robot arm 201 is an articulated arm that moves the hand 202 to a desired position within a movable range. The hand 202 is a part that holds and releases an object. The hand 202 may hold the object by grasping it, by suction, by scooping it, by piercing it, by containing it, or by any other method.

[0039] Specifically, the robot 200 performs a task on the moving object 500 on the passage 400. In this embodiment, the robot 200 performs the task on the moving object 500 using the position of the moving object 500 estimated by the movement information estimation device 100. That is, after holding an object, the robot 200 releases the object from the moving object 500 using the position of the moving object 500 estimated by the movement information estimation device 100.

[0040] The cradle 210 is a box-shaped base that supports the robot 200, and the robot 200 is attached and fixed on the cradle 210. The cradle 210 also serves as a base for the support member 310, and the support member 310 is also attached and fixed on the cradle 210. The cradle 210 is a case made of metal or the like, and houses the movement information estimation device 100 inside. An openable door is provided on a side wall of the cradle 210 (such as the front wall in the negative Y-axis direction), and various tasks such as maintenance or replacement of the movement information estimation device 100 can be performed through the door. The cradle 210 is a portable object that is configured to be movable (portable). Specifically, casters or the like may be arranged on the bottom surface of the cradle 210. Therefore, the robot system 10 is a portable object that is configured to be movable (portable).

[0041] The camera 300 is positioned above the passage 400 (in the positive direction of the Z axis) and is an imaging device capable of capturing images of the passage 400 and a moving object 500 on the passage 400. The camera 300 is supported by a support member 310 at a position a predetermined distance from the passage 400. The camera 300 is positioned facing directly below (in the negative direction of the Z axis) or diagonally downward (in a direction inclined from the negative direction of the Z axis), and captures a desired area in the passage 400 by capturing an image directly below or diagonally downward. The camera 300 can also measure the distance to the object being imaged. In other words, the camera 300 can detect the coordinates of the object in three-dimensional space (X, Y, Z coordinates). A publicly known 3D camera can be used as the camera 300. Various 3D camera systems exist, including stereo, ToF, and structured illumination systems, and any of these systems can be used. The camera 300 is connected to the movement information estimation device 100 by wire or wirelessly, and transmits to the movement information estimation device 100 captured images, position information (coordinates) of the captured object, and the like.

[0042] The support member 310 is an inverted L-shaped member (rod-shaped member) made of metal or the like that supports the camera 300. The end of the support member 310 facing the negative Z-axis is attached and fixed to the mount 210, and the support member 310 extends from this end in the positive Z-axis direction, bends at the end facing the positive Z-axis in the positive Y-axis direction, and continues to extend in the positive Y-axis direction. The camera 300 is attached and fixed to the center of the portion extending in the positive Y-axis direction. In this way, the support member 310 fixes the camera 300 to the mount 210.

[0043] The passage 400 is a transport device that transports the moving objects 500 on the passage 400. Specifically, the passage 400 is a belt conveyor that transports (carries) the moving objects 500, and extends in the X-axis direction. As a result, the moving objects 500 move in the X-axis direction (in this embodiment, the positive X-axis direction). In other words, as the upper surface of the passage 400 moves in a movement direction F (positive X-axis direction), the multiple moving objects 500 on the passage 400 move in a movement direction f (positive X-axis direction). The movement direction F of the passage 400 and the movement direction f of the moving objects 500 are parallel to each other. In this embodiment, the passage 400 is a portable object that is configured to be movable (portable). Specifically, it is conceivable to arrange casters or the like on legs that support the passage 400 (belt conveyor).

[0044] In a process (previous process) upstream (negative X-axis direction) of the robot 200, a person or another robot may place the moving body 500 in the passage 400 or perform work on the moving body 500 or the passage 400. As a result, the passage 400 may transport the moving body 500 in a state where the position or angle of the moving body 500 is deviated from the normal position or angle.

[0045] [2. Description of the Configuration of the Movement Information Estimation Device 100] Next, the configuration of the movement information estimation device 100 will be described in detail below. Fig. 2 is a block diagram showing the functional configuration of the movement information estimation device 100 according to this embodiment. Fig. 3 is a diagram explaining edge information 401 and point information 402 acquired by the movement information estimation device 100 according to this embodiment. Fig. 3 is a top view of a passage 400 viewed from above (the positive direction of the Z axis), and shows an edge 410 of the passage 400, points 411 and 412 on the edge 410, an edge 420, a point 421 on the edge 420, and a shooting area 301 of a camera 300.

[0046] As shown in FIG. 2, the movement information estimation device 100 includes an acquisition unit 110, an estimation unit 120, a control unit 130, and a storage unit 140. Also, as shown in FIG. 3, hereinafter, the area captured by the camera 300 (the area that can be captured) is referred to as a captured area 301. The edges on both sides of the passage 400 in the Y-axis direction in the captured area 301 are referred to as edges 410 and 420, any two points on the edge 410 are referred to as points 411 and 412, and any one point on the edge 420 is referred to as point 421. The edge 410 is the edge (outer edge) of the passage 400 in the positive Y-axis direction, and extends linearly in the X-axis direction. In FIG. 3 and the subsequent figures, the edge 410 is indicated by a thick line. The points 411 and 412 are points located at both ends of the edge 410 in the X-axis direction. Edge 420 is the edge (outer edge) of passage 400 in the negative Y-axis direction, and extends linearly in the X-axis direction. Point 421 is a point located in the center of edge 420 in the X-axis direction. Note that point 411 or point 412 may be a point located in the center of edge 410 in the X-axis direction, or point 421 may be a point located at the end of edge 420 in the X-axis direction.

[0047] 2.1 Description of the Acquisition Unit 110 The acquisition unit 110 acquires edge information 401 indicating the posture of an edge 410 of the passage 400 along which the moving object 500 moves. At this time, the acquisition unit 110 determines the longest line included in the image of the passage 400 as the edge 410, and acquires the edge information 401. Specifically, the acquisition unit 110 acquires an image of the shooting area 301 captured by the camera 300, and determines the longest line included in the image as the edge of the passage 400. In this embodiment, the acquisition unit 110 determines the line at the edge of the passage 400 in the positive Y-axis direction (a line extending in the X-axis direction) included in the image as the edge 410. Then, the acquisition unit 110 acquires the edge information 401 indicating the posture of the edge 410.

[0048] Edge information 401 is information that indicates the position of a predetermined point on edge 410 and the inclination of edge 410. In other words, acquisition unit 110 acquires information that indicates the position of a predetermined point on edge 410 and the inclination of edge 410 as edge information 401. In this embodiment, acquisition unit 110 acquires the coordinates of two predetermined points on edge 410, points 411 and 412, from camera 300. Specifically, acquisition unit 110 acquires the coordinates (XYZ coordinates) of points 411 and 412 in three-dimensional space. Acquisition unit 110 acquires the inclination of edge 410 by calculating the inclination (Rx, Ry, Rz) of edge 410 in three-dimensional space using the coordinates (XYZ coordinates) of point 411 and the coordinates (XYZ coordinates) of point 412. As a result, the acquisition unit 110 acquires, as edge information 401, information indicating the coordinates (XYZ coordinates) of point 411 or point 412 indicating the position of a predetermined point on edge 410 and the slope (Rx, Ry, Rz) of edge 410.

[0049] The acquisition unit 110 further acquires point information 402 indicating the position of a point 421 in the passage 400 that is different from the edge 410. The point information 402 is information indicating the coordinates of the point 421. In this embodiment, the acquisition unit 110 acquires, from the camera 300, information indicating the coordinates (XYZ coordinates) of the point 421 in three-dimensional space as the point information 402. The acquisition unit 110 may acquire the point information 402 by any method. For example, the acquisition unit 110 can acquire the point information 402 indicating the position of an arbitrary point 421 on the edge 420 by determining, as the edge 420, a long line (a line extending in the X-axis direction) on the edge of the passage 400 in the negative Y-axis direction, which is included in an image of the shooting area 301 captured by the camera 300. If a marking is applied to the position of the point 421 in the passage 400, the acquisition unit 110 may acquire the point information 402 indicating the position of the point 421 by identifying the marking.

[0050] 2.2 Description of the Estimation Unit 120 The estimation unit 120 estimates the position of the moving object 500 using the edge information 401 acquired by the acquisition unit 110. Specifically, the estimation unit 120 estimates the position of the moving object 500 using the edge information 401 and point information 402 acquired by the acquisition unit 110. More specifically, the estimation unit 120 uses the edge information 401 (and the point information 402) to calculate region information 403 indicating the attitude of a region on the passage 400 through which the moving object 500 passes, and estimates the position of the moving object 500 using the region information 403. The region information 403 is information indicating the position and inclination of the region on the passage 400 through which the moving object 500 passes (for example, the coordinates of one or more points within the region in three-dimensional space and the inclination of the region).

[0051] The estimation unit 120 can calculate the posture (position and inclination) of the entire passage 400 by using the edge information 401 and the point information 402. Therefore, in this embodiment, the estimation unit 120 calculates the posture (position and inclination) of the entire passage 400 as region information 403, and estimates the position (XYZ coordinates) of the moving object 500 by using the calculated region information 403. Note that the moving object 500 is moving on the passage 400. Therefore, the estimation unit 120 estimates the position (XYZ coordinates) of the moving object 500 at the time when the robot 200 is likely to perform an operation on the moving object 500.

[0052] Furthermore, the estimation unit 120 corrects the estimated value of the position of the moving body 500 when the edge information 401 changes while the moving body 500 is moving. Specifically, the estimation unit 120 corrects the estimated value of the position of the moving body 500 when at least one of the edge information 401 and the point information 402 changes while the moving body 500 is moving.

[0053] 2.3 Description of the control unit 130 The control unit 130 executes emergency processing, which is processing in an emergency, when the edge information 401 changes by a predetermined value or more while the moving object 500 is moving. Specifically, the control unit 130 executes emergency processing when at least one of the edge information 401 and the point information 402 changes by a predetermined value or more while the moving object 500 is moving. In other words, the control unit 130 determines whether at least one of the edge information 401 and the point information 402 has changed by a predetermined value or more while the moving object 500 is moving, and executes emergency processing when it determines that there has been a change.

[0054] Emergency processing is processing that should be performed in an emergency, such as issuing an alarm or urgently shutting down equipment. The above-mentioned predetermined value for edge information 401 is the amount of change in the position of point 411 or point 412 on edge 410, or the upper limit (tolerance) of the amount of change in the slope of edge 410, and is appropriately set by the user as a numerical value for which emergency processing should be performed. The above-mentioned predetermined value for point information 402 is the upper limit (tolerance) of the amount of change in the position of point 421, and is appropriately set by the user as a numerical value for which emergency processing should be performed. The above-mentioned predetermined value for edge information 401 and the above-mentioned predetermined value for point information 402 may be the same value or different values.

[0055] Furthermore, the control unit 130 controls the robot 200 to cause the robot 200 to perform a task on the moving object 500. That is, under the control of the control unit 130, the robot 200 performs a task on the moving object 500 using the position of the moving object 500 estimated by the movement information estimation device 100. Furthermore, the control unit 130 performs various controls on other parts of the robot system 10.

[0056] 2.4 Description of the storage unit 140 The storage unit 140 is configured with a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and is a memory that stores data and the like for controlling various operations in the robot system 10. Specifically, the storage unit 140 stores processing data 141. Written in the processing data 141 are edge information 401 and point information 402 acquired by the acquisition unit 110, area information 403 calculated by the estimation unit 120, and other information (such as data for controlling the operation of the robot 200).

[0057] [3. Description of Processing Flow of Movement Information Estimation Device 100] Next, the processing performed by the movement information estimation device 100 will be described. Fig. 4 is a flowchart showing the processing performed by the movement information estimation device 100 according to this embodiment. Fig. 5 is a diagram showing a state in which the edge information 401 and point information 402 acquired by the movement information estimation device 100 according to this embodiment have changed. Fig. 6 is a diagram showing a state in which the edge information 401 and point information 402 acquired by the movement information estimation device 100 according to this embodiment have further changed. Fig. 7 is a diagram explaining the processing in which the movement information estimation device 100 according to this embodiment estimates the position of a moving object 500. Figs. 5 to 7 correspond to Fig. 3.

[0058] As shown in Fig. 4, first, the acquisition unit 110 acquires an image of the passage 400 (step S11). Specifically, the acquisition unit 110 causes the camera 300 to capture an image of the photographed area 301, and acquires, from the camera 300, an image of the passage 400 included in the photographed area 301. When the passage 400 is tilted within the horizontal plane (XY plane) as shown in Fig. 5, or when the passage 400 is tilted in the width direction as well as in the horizontal plane as shown in Fig. 6, the acquisition unit 110 acquires an image in which the passage 400 is tilted.

[0059] Next, the acquisition unit 110 determines that the longest line included in the image of the passage 400 is the edge of the passage 400 (step S12). Specifically, the acquisition unit 110 analyzes the image acquired from the camera 300 and determines that the longest line included in the image is the edge of the passage 400. In this embodiment, the acquisition unit 110 determines that the line at the edge of the passage 400 in the positive Y-axis direction (a line extending in the X-axis direction) included in the image is the edge 410, and determines that the line at the edge of the passage 400 in the negative Y-axis direction (a line extending in the X-axis direction) included in the image is the edge 420. As shown in FIGS. 5 and 6, even if the passage 400 is tilted, the acquisition unit 110 can determine that the longest line included in the image of the passage 400 is the edge of the passage 400 (edge ​​410, edge 420).

[0060] Next, the acquisition unit 110 acquires edge information 401 and point information 402 (step S13). Specifically, the acquisition unit 110 acquires edge information 401 indicating the posture of edge 410 determined to be an edge of passage 400. At this time, the acquisition unit 110 acquires information indicating the position of a predetermined point on edge 410 and the inclination of edge 410 as edge information 401. As described above, the acquisition unit 110 acquires the coordinates (XYZ coordinates) of points 411 and 412 on edge 410 from camera 300 and calculates the inclination (Rx, Ry, Rz) of edge 410 to acquire edge information 401. As shown in FIGS. 5 and 6 , even when passage 400 is inclined, the acquisition unit 110 can acquire edge information 401 by acquiring the coordinates (XYZ coordinates) of points 411 and 412 on edge 410 and calculating the inclination (Rx, Ry, Rz) of edge 410.

[0061] The acquisition unit 110 also acquires point information 402 indicating the position of a point 421 on the passage 400 that is different from a point on the edge 410. Specifically, as described above, the acquisition unit 110 acquires, from the camera 300, information indicating the coordinates (XYZ coordinates) of the point 421 on the edge 420 as the point information 402. As shown in FIGS. 5 and 6 , even when the passage 400 is inclined, the acquisition unit 110 can acquire, as the point information 402, information indicating the coordinates (XYZ coordinates) of the point 421 on the edge 420. The acquisition unit 110 writes the acquired edge information 401 and point information 402 to the processing data 141 stored in the storage unit 140.

[0062] Next, the control unit 130 determines whether or not at least one of the edge information 401 and the point information 402 has changed by a predetermined value or more while the moving object 500 is moving (step S14). Specifically, the control unit 130 reads the edge information 401 and the point information 402 from the processing data 141 stored in the storage unit 140, and determines whether or not at least one of the edge information 401 and the point information 402 has changed by a predetermined value or more. The predetermined value may be determined and set in advance, or may be stored in the processing data 141 of the storage unit 140 and read out from the processing data 141 by the control unit 130.

[0063] When the control unit 130 determines that at least one of the edge information 401 and the point information 402 has changed by a predetermined value or more while the moving object 500 is moving (YES in step S14), the control unit 130 executes emergency processing, which is processing in an emergency (step S15). In this case, the control unit 130 issues an alarm or performs an emergency shutdown of the equipment. For example, if the state shown in FIG. 3 changes to the state shown in FIG. 6 and the edge information 401 or the point information 402 has changed by a predetermined value or more, the control unit 130 determines that at least one of the edge information 401 and the point information 402 has changed by a predetermined value or more, and executes emergency processing.

[0064] If the control unit 130 determines that neither the edge information 401 nor the point information 402 has changed by more than a predetermined value while the moving object 500 is moving (NO in step S14), the control unit 130 performs the following processing. For example, when the state changes from that shown in Fig. 3 to that shown in Fig. 5, if neither the edge information 401 nor the point information 402 has changed by more than a predetermined value, the control unit 130 determines that neither the edge information 401 nor the point information 402 has changed by more than a predetermined value, and performs the following processing.

[0065] First, the estimation unit 120 calculates region information 403 indicating the orientation of the region on the passage 400 through which the moving object 500 passes, using the edge information 401 and the point information 402 (step S16). Specifically, the estimation unit 120 reads the edge information 401 and the point information 402 from the processing data 141 stored in the storage unit 140, and calculates the region information 403 using the edge information 401 and the point information 402. In this embodiment, the estimation unit 120 calculates the orientation (position and tilt) of the entire passage 400 as the region information 403. Even if the passage 400 is tilted as shown in FIG. 5, the estimation unit 120 calculates the region information 403. For example, the estimation unit 120 calculates the coordinates of one or more points within the passage 400 in three-dimensional space and the tilt of the passage 400 as the region information 403. The estimation unit 120 writes the calculated region information 403 into the processing data 141 stored in the storage unit 140.

[0066] Next, the estimation unit 120 estimates the position of the moving object 500 using the region information 403 (step S17). Specifically, the estimation unit 120 reads out the region information 403 from the processing data 141 stored in the storage unit 140, and estimates the position (XYZ coordinates) of the moving object 500 using the region information 403. Even if the passage 400 is tilted as shown in FIG. 5, the estimation unit 120 estimates the position (XYZ coordinates) of the moving object 500. Specifically, when the passage 400 is tilted as shown in FIG. 5, the passage 400 moves in a movement direction F tilted from the X-axis direction, as shown in FIG. 7, and the moving object 501 also moves in a traveling direction f tilted from the X-axis direction, and moves to the position of the moving object 501a.

[0067] In this case, the estimation unit 120 knows the attitude (position and tilt) of the passage 400 from the area information 403, and therefore estimates that the moving body 501 will move in the traveling direction f to the position of the moving body 501a. Similarly, the estimation unit 120 estimates that the moving body 502 will move in the traveling direction f to the position of the moving body 502a, and that the moving body 503 will move in the traveling direction f to the position of the moving body 503a. The estimation unit 120 writes the estimated values ​​(XYZ coordinates) of the estimated position of the moving body 500 into the processing data 141 stored in the storage unit 140. In this way, the estimation unit 120 estimates the position of the moving body 500 using the edge information 401 and point information 402 acquired by the acquisition unit 110.

[0068] Note that, when at least one of the edge information 401 and the point information 402 changes (changes smaller than the predetermined value) while the moving object 500 is moving, the estimation unit 120 corrects the estimated value of the position of the moving object 500. For example, when the state shown in FIG. 3 changes to the state shown in FIG. 5, if the edge information 401 and the point information 402 change smaller than the predetermined value, the estimation unit 120 corrects the estimated value of the position of the moving object 500. The estimation unit 120 writes the corrected estimated value (XYZ coordinates) of the position of the moving object 500 into the processing data 141 stored in the storage unit 140, thereby updating (correcting) the data.

[0069] Next, the control unit 130 causes the robot 200 to perform a task on the moving body 500 using the position of the moving body 500 estimated by the estimation unit 120 (step S18). Specifically, the control unit 130 reads out the estimated values ​​(XYZ coordinates) of the position of the moving body 500 from the processing data 141 stored in the memory unit 140, and causes the robot 200 to perform a task on the moving body 500 using the estimated values ​​(XYZ coordinates) of the position of the moving body 500.

[0070] 7, for example, for the moving body 501, the control unit 130 moves the hand 202 of the robot 200 to the position of the moving body 501a and causes the hand 202 to perform an operation on the moving body 501a. The same applies to the moving bodies 502 and 503. This allows the robot 200 to perform an operation on the moving body 500, such as releasing an object at an estimated value (XYZ coordinates) of the position to which the moving body 500 will move.

[0071] This completes the processing performed by the movement information estimation device 100. The timing of performing each processing in the movement information estimation device 100 is not particularly limited, and any of the processing may be performed before the robot 200 starts operating (before the moving object 500 starts moving), while the robot 200 is operating (while the moving object 500 is moving), or periodically.

[0072] [4. Explanation of effects] As described above, the movement information estimation device 100 according to the embodiment of the present invention acquires edge information 401 indicating the posture of the edge 410 of the passage 400 along which the moving object 500 moves, and estimates the position of the moving object 500 using the edge information 401. As a result, the movement information estimation device 100 can estimate the position of the moving object 500 using simple information, such as the edge information 401 indicating the posture of the edge 410 of the passage 400. In particular, in this embodiment, the movement information estimation device 100 can acquire necessary information, such as the edge information 401, using a single camera 300, thereby simplifying the configuration of the robot system 10. Therefore, the movement information estimation device 100 can easily estimate the position of the moving object 500 moving along the passage 400. This allows the estimated position of the moving object 500 to be easily adjusted (calibrated). In particular, when some kind of work is performed on the moving object 500 upstream (in a previous process) of the robot 200, the need to precisely adjust the position of the moving object 500 after the work is reduced. Therefore, the present invention can also improve work efficiency in this respect.

[0073] The movement information estimation device 100 acquires, as edge information 401, information indicating the positions of points on an edge 410 of the passage 400 and the inclination of the edge 410. This allows the movement information estimation device 100 to estimate the posture of the passage 400 from the positions of points on the edge 410 and the inclination of the edge 410, thereby making it possible to easily estimate the position of the moving object 500.

[0074] The movement information estimation device 100 further acquires point information 402 indicating the position of a point 421 that is not on the edge 410 of the passage 400, and estimates the position of the moving object 500 using the edge information 401 and the point information 402. This allows the movement information estimation device 100 to estimate the position of the moving object 500, taking into account the inclination of the passage 400 in the width direction.

[0075] The movement information estimation device 100 calculates area information 403 indicating the attitude of an area on the passage 400 through which the moving object 500 passes, and estimates the position of the moving object 500 using the area information 403. In this way, the movement information estimation device 100 can estimate the position of the moving object 500 from the attitude of an area on the passage 400 through which the moving object 500 passes.

[0076] The movement information estimation device 100 can easily acquire edge information 401 by determining that the longest line included in the image of the passage 400 is the edge 410 of the passage 400. In this way, the movement information estimation device 100 can easily determine the edge 410 of the passage 400 by analyzing the image of the passage 400, and can easily acquire edge information 401.

[0077] When the posture of the passage 400 changes while the moving object 500 is moving due to vibrations of the passage 400 or the like, causing a change in the edge information 401, the movement information estimation device 100 corrects the estimated value of the position of the moving object 500, thereby preventing a decrease in the estimation accuracy of the position of the moving object 500. In this way, the movement information estimation device 100 can detect vibrations of the passage 400 or the like, and correct the estimated value of the position of the moving object 500 (can calibrate tracking of the moving object 500).

[0078] When the posture of the passage 400 changes significantly while the moving object 500 is moving and the edge information 401 changes by a predetermined value or more, the movement information estimation device 100 can protect the equipment and ensure safety by executing emergency processing (issuing an alarm, emergency stop, etc.). When executing emergency processing to stop the robot 200 from working, the movement information estimation device 100 stops the robot 200 from working due to a change in the posture of the passage 400, so there is no need to connect the passage 400 and the robot 200 to control them. There is also no need to connect an instrument to the passage 400 to determine abnormalities such as sudden movement or tilting of the passage 400.

[0079] Because the passage 400 is portable, there is a risk that the passage 400 may move or tilt, causing the position of the moving object 500 to change. However, even if the position of the moving object 500 changes, the movement information estimation device 100 can easily estimate the position of the moving object 500 by using edge information 401 indicating the posture of an edge 410 of the passage 400. Furthermore, because the robot system 10 is also portable, the passage 400 is likely to shift in position relative to the robot system 10 (robot 200, camera 300, etc.). Even in this case, the movement information estimation device 100 can easily estimate the position of the moving object 500 by using the edge information 401 indicating the posture of the edge 410 of the passage 400.

[0080] The passage 400 is a conveyance device, and as the conveyance device operates, the posture of the passage 400 is likely to change, and therefore the position of the moving object 500 is likely to change. However, even if the position of the moving object 500 changes, the movement information estimation device 100 can easily estimate the position of the moving object 500 by using the edge information 401 that indicates the posture of the edge 410 of the passage 400.

[0081] According to the robot system 10 of the embodiment of the present invention, the movement information estimation device 100 can easily estimate the position of the moving body 500 moving through the passage 400, as described above, and therefore the robot 200 can easily perform work on the moving body 500 using the position of the moving body 500 estimated by the movement information estimation device 100.

[0082] The various effects of the movement information estimation device 100 described above can be similarly applied to the effects of the robot system 10.

[0083] [5. Explanation of Variations] Although the movement information estimation device 100 and the robot system 10 according to the present embodiment have been described above, the present invention is not limited to the above embodiment. The embodiment disclosed herein is illustrative in all respects and is not restrictive, and the scope of the present invention includes all modifications within the meaning and scope of the claims.

[0084] In the above embodiment, the passage 400 is a conveying device (belt conveyor) that conveys the moving object 500 on the passage 400, and is a portable object, but is not limited to this. The passage 400 may be a platform such as a table that does not convey the moving object 500, and the moving object 500 may move on the passage 400 by its own force or by an external force other than the passage 400. The passage 400 may also be a non-portable passage such as a road. In this case, the moving object 500 may be a vehicle or the like that travels on the road.

[0085] In the above embodiment, the acquisition unit 110 of the movement information estimation device 100 acquires, as the edge information 401, information indicating the orientation of the edge 410 of the passage 400, but may also acquire information indicating the orientation of the edge 420. In this case, the acquisition unit 110 may acquire, as the point information 402, information indicating the position of a point on the passage 400 that is not on the edge 420.

[0086] In the above embodiment, the acquisition unit 110 acquires, as the point information 402, information indicating the position of the point 421 on the edge 420. However, this is not limited to this. The acquisition unit 110 may acquire, as the point information 402, information indicating the position of a point on the passage 400 that is not on the edge 410, and may acquire information indicating the position of any point, such as the center of the passage 400. However, by the acquisition unit 110 acquiring, as the point information 402, information indicating the position of a point on the edge 420, the estimation unit 120 can estimate (calculate) the width of the passage 400 (width in the Y-axis direction) using the edge information 401 and the point information 402. Therefore, when the acquisition unit 110 acquires, as the point information 402, information indicating the position of a point other than on the edge 420, it may also acquire information indicating the width of the passage 400 (width in the Y-axis direction).

[0087] In the above embodiment, the acquisition unit 110 acquires edge information 401 indicating the attitude of edge 410 of passage 400, but may acquire information indicating the attitude of the entire (or a part of) image capture area 301 of camera 300 including edge 410 of passage 400. In this case, the estimation unit 120 may estimate the position of the moving object 500 using the information indicating the attitude of the entire (or a part of) image capture area 301 including edge information 401.

[0088] In the above embodiment, the acquisition unit 110 determines the longest line included in the image of the passage 400 as the edge 410 and acquires the edge information 401, but this is not limiting. A marking may be provided at the position of the edge 410 in the passage 400, and the acquisition unit 110 may identify the marking to determine the edge 410 and acquire the edge information 401. Alternatively, markings may be provided at the positions of points 411 and 412 in the passage 400, and the acquisition unit 110 may identify the marking to acquire the edge information 401 obtained from the positions of points 411 and 412.

[0089] In the above embodiment, the acquisition unit 110 acquires information indicating the position of a predetermined point on the edge 410 and the inclination of the edge 410 as the edge information 401, but this is not limiting. The acquisition unit 110 may acquire information indicating the positions of two points (points 411 and 412, etc.) on the edge 410 as the edge information 401. In this case, the estimation unit 120 may calculate the inclination of the edge 410 from the coordinates of the two points, and estimate the position of the moving object 500.

[0090] In the above embodiment, the acquisition unit 110 acquires the coordinates (XYZ coordinates) of the points 411 and 412 in three-dimensional space, calculates the gradient (Rx, Ry, Rz) of the edge 410 in three-dimensional space, and acquires the edge information 401. However, this is not limiting. The acquisition unit 110 may acquire the coordinates (excluding the Z coordinate) of the points 411 and 412 in the XY plane, calculate the gradient (Rx, Ry) (excluding Rz) of the edge 410 in the XY plane, and acquire the edge information 401 in the XY plane. In this case, the acquisition unit 110 does not need to acquire the point information 402, and the estimation unit 120 may estimate the position of the moving object 500 without using the point information 402. For example, assuming that the passage 400 does not rotate (tilt in the width direction) as shown in Fig. 6 does not occur, the acquisition unit 110 may acquire width information indicating the width of the passage 400 (width in the Y-axis direction) instead of the point information 402, and the estimation unit 120 may use the edge information 401 and the width information to estimate the position of the moving object 500. In this case, the acquisition unit 110 may acquire, as the edge information 401, information indicating the tilt of the edge 410 (not including information indicating the position of a predetermined point on the edge 410).

[0091] In the above embodiment, the estimation unit 120 calculates the region information 403 using the edge information 401 and estimates the position of the moving object 500 using the region information 403, but this is not limiting. The estimation unit 120 may estimate the position of the moving object 500 using the edge information 401 without calculating the region information 403.

[0092] In the above embodiment, the control unit 130 executes emergency processing when the edge information 401 changes by a predetermined value or more while the moving object 500 is moving, but this is not limited to this. The control unit 130 may execute emergency processing under other conditions, or may not execute emergency processing under any conditions.

[0093] In the above embodiment, the robot 200 performs work on the moving body 500 using the position of the moving body 500 estimated by the movement information estimation device 100, but this is not limited to this. A device or worker different from the robot 200 may perform work on the moving body 500 using the position of the moving body 500 estimated by the movement information estimation device 100, or the state of the moving body 500 may be simply monitored and no work may be performed on the moving body 500. In this case, the robot 200 does not need to be deployed.

[0094] In the above embodiment, the robot system 10 (the stand 210) may be fixed to the floor surface and may not be portable.

[0095] In the above embodiment, the robot system 10 is not limited to including all of the above-described components. For example, the robot system 10 may not include the mount 210, and the robot 200 may be fixed to another member. The robot system 10 may not include the support member 310, and the camera 300 may be fixed to another member. The robot system 10 may not include the camera 300, and the movement information estimation device 100 may acquire information such as the edge information 401 from another device.

[0096] In the above embodiment, the movement information estimation device 100 is not limited to having all of the above-mentioned processing units. For example, the movement information estimation device 100 may not have the control unit 130, and another device may control the control unit 130. The movement information estimation device 100 may not have the storage unit 140, and may exchange data with an external memory or the like.

[0097] In the above embodiment, the movement information estimation device 100 is not limited to executing all of the steps described above, and the steps executed by the movement information estimation device 100 are not limited to being performed in the order described above.

[0098] Furthermore, the present invention can be realized not only as the movement information estimation device 100 and the robot system 10, but also as a movement information estimation method or a robot system control method including characteristic processing steps performed by the movement information estimation device 100 or the robot system 10. The present invention can also be realized as a program causing a computer to execute the movement information estimation method or the robot system control method. That is, each component of the movement information estimation device 100 may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Furthermore, the present invention can also be realized as any medium, such as a computer-readable non-transitory recording medium on which the program is recorded, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray (registered trademark) Disc), semiconductor memory, flash memory, magnetic storage device, optical disc, or paper tape. The program can then be distributed via the recording medium and a transmission medium such as the Internet. The present invention can also be realized as an integrated circuit including a processing unit included in the movement information estimation device 100. That is, each functional block of the movement information estimation device 100 shown in Fig. 2 may be realized as an LSI (Large Scale Integration) which is an integrated circuit. These may be individually integrated into one chip, or some or all of them may be integrated into one chip. In this way, each component of the movement information estimation device 100 may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component.

[0099] Any combination of the components included in the above-described embodiments and their modifications is also included within the scope of the present invention. [Explanation of symbols]

[0100] 10 Robot Systems 100 Movement information estimation device 110 Acquisition Department 120 Estimation part 130 control section 140 Storage section 141 Processing Data 200 robots 201 Robot Arm 202 hands 210 Mounting stand 300 cameras 301 Shooting Area 310 Support member 400 aisles 401 Relationship Information 402 points information 403 Area information 410, 420 Edge 411, 412, 421 points 500, 501, 501a, 502, 502a, 503, 503a Mobile

Claims

1. an acquisition unit that acquires edge information indicating the posture of an edge of a passage along which the moving object moves; an estimation unit that estimates a position of the moving object moving along the edge using the edge information, The passage is a conveying device that conveys the moving object on the passage. Movement information estimation device.

2. The acquisition unit acquires, as the edge information, information indicating a position of a predetermined point on the edge and a slope of the edge. The movement information estimation device according to claim 1 .

3. the acquisition unit further acquires point information indicating a position of a point on the passage that is different from the edge, The estimation unit estimates a position of the moving object using the edge information and the point information. The movement information estimation device according to claim 1 or 2.

4. The estimation unit Using the edge information, calculate area information indicating the attitude of an area on the passage through which the moving object passes; The location of the moving object is estimated using the area information. The movement information estimation device according to claim 1 or 2.

5. The acquisition unit determines the longest line included in the image of the passage as the edge and acquires the edge information. The movement information estimation device according to claim 1 or 2.

6. The estimation unit corrects the estimated value of the position of the moving object when the edge information changes while the moving object is moving. The movement information estimation device according to claim 1 or 2.

7. The vehicle further includes a control unit that executes emergency processing when the edge information changes by a predetermined value or more during the movement of the vehicle. The movement information estimation device according to claim 1 or 2.

8. The passageway is a portable object The movement information estimation device according to claim 1 or 2.

9. The movement information estimation device according to claim 1 or 2; a robot that performs a task on the moving object using the position of the moving object estimated by the movement information estimation device; A robot system comprising:

Images