Hook position detection device

Abstract

To detect position etc. of a hook accurately even if there is a portion concealed by a wire in a captured picture image in a crane's hook position detection device.SOLUTION: A hook position detection device 100 comprises a camera 50 which is provided on a tip end of a telescopic boom 16 of a crane 10, four markers M which are provided on an upper surface 42 of a sub-hook 40 which is hung down from the tip end of the telescopic boom 16 by an auxiliary hoist wire rope 41, and a hook position calculation unit 110 which calculates the position and orientation of the sub-hook 40 on the basis of physical relationship of three makers M which are imaged by the camera 50 among four makers M. The four markers M are disposed by distributing in physical relationship that four triangles T1, T2, T3, T4, of which the three markers M selected arbitrarily are formed as three apexes, are formed as all shapes different from each other.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a hook position detection device. [Background technology]

[0002] For example, a crane lifts a load using a hook suspended by a wire from a boom. A hook position detection device has been proposed that automatically detects the position of the hook and the rotational direction of the hook around the wire suspending the hook (hereinafter, these positions and directions are collectively referred to as "position" and the like) (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 7-179290 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology proposed in Patent Document 1 involves drawing an identification pattern, for example made up of four black circles, on the top surface (the plane facing upward) of the hook block on which the hook is attached, and placing a video camera facing vertically downward at the top of the boom. The video camera captures an image of the area vertically below, and by detecting the area of ​​the entire image captured by the video camera that contains the identification pattern drawn on the hook block, the area of ​​the hook block in real space is determined and the swaying of the load suspended by the hook is detected.

[0005] Here, the technology of Patent Document 1 detects an identification pattern in a captured image by examining the correlation with a plurality of original templates that are stored in advance in correspondence with the identification pattern, and determines the area in which the identification pattern appears based on the relationship with the original template with the highest correlation.

[0006] However, since the video camera must be installed at a position offset from the wire at the top of the boom, the image captured by the video camera always includes an image of the hanging wire. In addition, the identification pattern of the hook block suspended by the wire may be partially hidden by the wire in the image. In such cases, the technology of Patent Document 1 has the problem of being unable to accurately calculate the correlation between the identification pattern in the image and the original template.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide a hook position detection device that can accurately detect the position of a hook, etc., even if there is a part of the captured image that is hidden by a wire. [Means for solving the problem]

[0008] The present invention is a crane hook position detection device comprising an imaging device provided at the tip of a crane boom, four or more markers provided on the upper surface of a hook suspended by a wire from the tip of the boom, and a hook position calculation unit that calculates the position and orientation of the hook based on the positional relationships of three or more of the four or more markers imaged by the imaging device, wherein the four or more markers are distributed and positioned in a positional relationship such that multiple triangles formed with three arbitrarily selected markers as vertices all have different shapes. [Effects of the Invention]

[0009] The crane hook position detection device according to the present invention can accurately detect the position of the hook, etc., even if there is a portion hidden by the wire in the captured image. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing a rough terrain crane 10 (hereinafter simply referred to as the crane 10). [Figure 2]FIG. 10 is a schematic diagram showing an example of the arrangement of four markers provided on the upper surface of the sub-hook in a plan view (as viewed in a direction perpendicular to the horizontal plane). [Figure 3A] FIG. 1 is a diagram (part 1) that schematically illustrates a triangle formed with any three vertices of four markers. [Figure 3B] FIG. 2 is a diagram (part 2) that schematically illustrates a triangle formed with any three of the four markers as vertices. [Figure 3C] FIG. 3 is a diagram (part 3) that schematically illustrates a triangle formed with any three of the four markers as vertices. [Figure 3D] FIG. 4 is a diagram (part 4) that schematically illustrates a triangle formed with any three of the four markers as vertices. [Figure 4] 1 is a block diagram showing the configuration of a hook position detection device that detects the position and orientation of a sub-hook of a crane. [Figure 5] This is an example of an image of a scene vertically below taken by a camera. [Figure 6] FIG. 10 is a schematic diagram illustrating the orientation of the sub-hook when the sub-hook is rotated 60 degrees to the right (clockwise) about center C from the reference position, for example. [Figure 7] FIG. 10 is a schematic diagram equivalent to FIG. 2, showing another example of the arrangement of four markers. [Figure 8] FIG. 2 is a block diagram showing the configuration of a hook position detection device that detects the position and orientation of a sub-hook of a crane. [Figure 9] FIG. 10 is a schematic diagram showing an example of the arrangement of four markers provided on the upper surface of the sub-hook in a plan view (as viewed in a direction perpendicular to the horizontal plane). [Figure 10A] FIG. 1 is a diagram (part 1) that schematically illustrates a triangle formed with any three vertices of four markers. [Figure 10B] FIG. 2 is a diagram (part 2) that schematically illustrates a triangle formed with any three of the four markers as vertices. [Figure 10C]FIG. 3 is a diagram (part 3) that schematically illustrates a triangle formed with any three of the four markers as vertices. [Figure 10D] FIG. 4 is a diagram (part 4) that schematically illustrates a triangle formed with any three of the four markers as vertices. [Figure 11] This is an example of an image of a scene vertically below taken by a camera. DETAILED DESCRIPTION OF THE INVENTION

[0011] An embodiment of a crane hook position detection device according to the present invention will be described below with reference to the drawings.

[0012] <Embodiment 1> Figure 1 is a schematic diagram showing a rough terrain crane 10 (hereinafter simply referred to as crane 10), Figure 2 is a schematic diagram showing an example of the arrangement of four markers M provided on the upper surface 42 of the sub-hook 40 in a plan view (viewed in a direction perpendicular to the horizontal plane), and Figures 3A, 3B, 3C, and 3D are schematic diagrams showing triangles T1, T2, T3, and T4 formed with any three of the four markers M as vertices.

[0013] 4 is a block diagram showing the configuration of a hook position detection device 100 that detects the position and orientation of the sub-hook 40 of the crane 10. The hook position detection device 100 is one embodiment (embodiment 1) of the hook position detection device for a crane according to the present invention.

[0014] [Overall crane configuration] As shown in FIG. 1, the crane 10 includes a carrier 11, a swivel base 14, a cabin 20, a telescopic boom 16, a main hook 30, a sub-hook 40, a camera 50 (imaging device), and a hook position detection device 100.

[0015] The carrier 11 is a traveling body that travels the crane 10. The carrier 11 is equipped with outriggers 12 and 13 that keep the crane 10 in contact with the ground.

[0016] The swivel base 14 is a base that rotates around a vertical axis relative to the carrier 11. A cabin 20 and a telescopic boom 16 are provided on the swivel base 14, and the cabin 20 and the telescopic boom 16 rotate integrally with the swivel base 14.

[0017] The cabin 20 is a compartment in which an operator sits and performs various operations. The various operations performed by the operator in the cabin 20 include driving the carrier 11, rotating the swivel base 14, extending and lowering the telescopic boom 16, and operating the crane to raise and lower the main hook 30 and the sub-hook 40.

[0018] The cabin 20 is equipped with an AML (Automatic Moment Limiter; overload prevention device) to assist and support the various operations performed by these operators, a monitor 120 that displays images captured by the camera 50, and a hook position detection device 100, which will be described later.

[0019] The AML calculates the performance in crane operation by inputting information on factors that determine the performance in crane operation, such as the angle of elevation of the telescopic boom 16 (angle relative to the horizontal plane), the telescopic length (length from the support shaft 17 to the sheave), the rotation angle of the swivel base 14 (the rotation angle of the swivel base 14 relative to the fore-and-aft direction of the carrier 11), and the extension length of the outriggers 12 and 13.

[0020] The telescopic boom 16 is mounted on the swivel base 14 via a bracket 15, and is raised and lowered around a support shaft 17 by extension and retraction of a hoisting cylinder 18. The telescopic boom 16 is formed by combining three booms (a base boom 16A, an intermediate boom 16B, and a tip boom 16C) in a nested manner, with the intermediate boom 16B projecting from the base boom 16A and the tip boom 16C projecting from the intermediate boom 16B, thereby extending the overall length, and the intermediate boom 16B being accommodated within the base boom 16A and the tip boom 16C being accommodated within the intermediate boom 16B, thereby shortening the overall length.

[0021] A boom head 16D is provided at the tip of the tip boom 16C, and a sheave (not shown) is provided on the boom head 16D. A main hoisting wire rope 31 and an auxiliary hoisting wire rope 41 are reeled around this sheave separately and independently. A main hook 30 is suspended from the main hoisting wire rope 31, and a sub-hook 40 is suspended from the auxiliary hoisting wire rope 41. When lifting a load, the wire rope or the like suspended from the load is hooked onto the main hook 30 and the sub-hook 40, respectively.

[0022] Both the main hoisting wire rope 31 and the auxiliary hoisting wire rope 41 are wound around a winch drum (not shown) provided on the swivel base 14 at the base end of the telescopic boom 16. The crane 10 drives the winch motor to pay out the main hoisting wire rope 31 wound around the drum, thereby lowering the main hook 30, and rewinds the main hoisting wire rope 31 wound around the drum, thereby raising the main hook 30.

[0023] Similarly, the crane 10 lowers the sub-hook 40 by driving the winch motor to pay out the auxiliary hoisting wire rope 41 wound around the drum, and raises the sub-hook 40 by rewinding the auxiliary hoisting wire rope 41 wound around the drum.

[0024] As shown in Fig. 2, four markers M are provided on the upper surface 42 of the sub-hook 40. The four markers M are, for example, light-emitting diodes (LEDs), and emit light when powered on. All four markers M are identical.

[0025] Here, when distinguishing between the four markers M, they will be referred to as markers M1, M2, M3, and M4. The four markers M are distributed and arranged on the circumference of the same circle C1, which is centered at the center C of the upper surface 42 of the sub-hook 40. The circle C1 has a diameter D, for example.

[0026] Center C is the center of gravity of the sub-hook 40 on the top surface 42, and is the position where the auxiliary hoisting wire rope 41 is connected to the top surface 42 and the sub-hook 40 is supported by the auxiliary hoisting wire rope 41. Because the auxiliary hoisting wire rope 41 is connected to center C, when the sub-hook 40 is lifted by the auxiliary hoisting wire rope 41, the top surface 42 is in a position parallel to the horizontal plane.

[0027] The marker M in this embodiment is placed on a chord where the marker M1 and the marker M2 form a diameter D of the circle C1, that is, the marker M1 and the marker M2 are placed at a position that forms an angle of 180 degrees with the center C in between. In FIG. 2, the marker M2 is placed at a position that is 180 degrees clockwise from the marker M1 as the reference.

[0028] Marker M2 and marker M3 are arranged at positions that form an angle of 90 degrees with center C in between. Marker M3 is arranged at an angle of 270 degrees clockwise from marker M1 in FIG. 2.

[0029] Marker M3 and marker M4 are arranged at positions that form an angle of 60 degrees with respect to center C. Marker M4 is arranged at an angle of 330 degrees clockwise from marker M1 in FIG.

[0030] The four markers M are distributed and arranged in the above-mentioned positional relationship, so that, as shown in Figures 3A, 3B, 3C, and 3D, any three of the four markers M can be selected, and the four triangles T1, T2, T3, and T4 formed with the selected three markers M as vertices will all have different shapes.

[0031] Here, the fact that triangles T1, T2, T3, and T4 have different shapes means that even if triangles T1, T2, T3, and T4 are rotated within a plane, they do not overlap exactly with the other triangles T1, T2, T3, and T4. Therefore, for example, triangle T1' obtained by rotating triangle T1 within a plane has the same shape as triangle T1, and is not a different shape from triangle T1.

[0032] It should be noted that other variations, which will be described later, can also be applied to the distributed arrangement of the four markers M.

[0033] A camera 50 is provided on the boom head 16D and is arranged facing downward. The camera 50 is attached to the boom head 16D, for example, via a gimbal mechanism. Therefore, regardless of the angle at which the telescopic boom 16 is raised or lowered, the optical axis O of the camera 50 always faces vertically downward, and the camera 50 captures the vertically downward scene that falls within the angle of view of the lens of the camera 50. Because the main hook 30 and the sub-hook 40 are also within the angle of view of the lens, the camera 50 captures images of the main hook 30 and the sub-hook 40, as well as the load (suspended load) suspended from the main hook 30 and the load suspended from the sub-hook 40.

[0034] [Configuration of hook position detection device] As shown in FIG. 4, the hook position detection device 100 includes a camera 50, four markers M provided on the upper surface 42 of the sub-hook 40, a hook position calculation unit 110, and a monitor 120.

[0035] In the hook position detection device 100 of this embodiment, the camera 50 that photographs the suspended load also serves as a camera that photographs the marker M, but a dedicated camera for photographing the marker M may also be provided separate from the camera 50.

[0036] In addition, the hook position detection device 100 of this embodiment uses the monitor 120 as a monitor that also displays the position of the sub-hook 40, but a dedicated monitor for displaying the position of the sub-hook 40 may be provided separately from the monitor 120.

[0037] The hook position calculation unit 110 is provided inside the cabin 20. The hook position calculation unit 110 calculates the position of the sub-hook 40 and the orientation of the rotation direction around the center C based on the positional relationship of the images m of three markers M captured by the camera 50 out of four markers M provided on the top surface 42 of the sub-hook 40.

[0038] The hook position calculation unit 110 includes a storage unit 111 and a calculation unit 112. The storage unit 111 pre-stores information regarding the positional relationship in a planar view of the four markers M on the upper surface 42 of the sub-hook 40 in the above-described real space (see FIG. 2), the diameter D of the circle C1, and the specifications (specs) of the camera 50. The specifications of the camera 50 include the size and resolution of the image pickup element (CCD, CMOS, etc.) of the camera 50, the focal length and zoom magnification of the lens, and the spatial resolution. This specification information of the camera 50 provides the hook position calculation unit 110 with the correspondence between the size of the image of the subject captured by the camera 50 and the size of the subject in real space.

[0039] The hoisting angle and telescopic length of the telescopic boom 16 are also input from the AML to the hook position calculation unit 110. The hoisting angle and telescopic length of the telescopic boom 16 may also be input directly to the hook position calculation unit 110 without going through the AML.

[0040] Information indicating the state of the telescopic boom 16, such as the distance (height) from the ground to the sheave provided at the tip of the telescopic boom 16, is provided to the hook position calculation unit 110.

[0041] The calculation unit 112 calculates the position and orientation of the sub-hook 40 based on the positional relationship between the images m of the three markers M captured by the camera 50 and the positional relationship between the markers M stored in the storage unit 111.

[0042] [Function of the hook position detection device] Fig. 5 is an example of an image P of a scene vertically below captured by the camera 50. The hook position detection device 100 configured as described above operates as follows. As shown in Fig. 5, the camera 50 captures the scene vertically below as an image P of a subject. The image P shows an image m of a marker M provided on the upper surface 42 of the sub-hook 40 suspended by the auxiliary hoisting wire rope 41.

[0043] In addition to the sub-hook 40, image P also shows images of the main hook 30 and the main hoisting wire rope 31. However, displaying the images of the main hook 30 and the main hoisting wire rope 31 would make image P cluttered, and since it is not necessary to explain the images of the main hook 30 and the main hoisting wire rope 31 for the purposes of explaining this embodiment, the images of the main hook 30 and the main hoisting wire rope 31 have been omitted.

[0044] The camera 50 photographs the view vertically below, but because the optical axis O of the camera 50 is installed offset from the center of the auxiliary hoisting wire rope 41, the image P photographed by the camera 50 shows the image of the auxiliary hoisting wire rope 41 crossing the image P, as shown in Figure 5. Depending on the orientation of the sub-hook 40, this image of the auxiliary hoisting wire rope 41 may obscure the image m of one of the four markers M provided on the top surface 42.

[0045] The hook position detection device 100 calculates the position and orientation of the sub-hook 40 based on the image P captured by the camera 50. That is, the image P captured by the camera 50, for example, as shown in FIG. 5, is input to the hook position calculation unit 110. Since the image P may not include one of the images m of the four markers M as described above, the hook position calculation unit 110 calculates the position and orientation of the sub-hook 40 based on the positional relationship of the images m of at least three markers M.

[0046] The specific method by which the hook position calculation unit 110 calculates the position and orientation of the sub-hook 40 is as follows. That is, the calculation unit 112 of the hook position calculation unit 110 detects the positions of the images m of the three markers M in the image P as their respective coordinates (x, y) in the image P. The position of the image m of each marker M is set to the position of the center c of the image m of each marker M. The calculation unit 112 connects the coordinates (x, y) of the images m of two of the three markers M with a line segment. This allows the hook position calculation unit 110 to form a triangle with the coordinates (x, y) of the images m of the three markers M as its vertices.

[0047] The calculation unit 112 may or may not form the above-mentioned triangle as an internal calculation, and the following explanation will be given assuming that the above-mentioned triangle is formed as appropriate to make the explanation of the embodiment easier to understand, but even if a triangle is not formed, it can be replaced with calculation processing based on the positional relationship of the four markers M.

[0048] When any three markers M are selected from the four markers M provided on the upper surface 42 of the sub-hook 40, as shown in Figures 3A to 3D, the triangles T1 to T4 formed with the selected three markers M as vertices have different shapes, so the hook position calculation unit 110 determines which three of the four markers M form the triangle formed with the coordinates (x, y) of the images m of the three markers M as vertices.

[0049] In other words, since the memory unit 111 stores the positional relationship of the four markers M in real space, by comparing the positional relationship of the coordinates (x, y) of the images m of the three markers M calculated by the calculation unit 112 based on the image P with the positional relationship of the four markers M in real space stored in the memory unit 111, it is possible to identify which marker M the image m of the three markers M corresponds to.

[0050] As an example of a comparison based on positional relationships, the hook position calculation unit 110 compares a triangle having vertices at the coordinates of the images m of three markers M with the shapes of four triangles T1, T2, T3, and T4 having vertices of the four markers M in real space, and identifies one triangle (for example, triangle T4 (see Figure 3D)) among the four triangles T1 to T4 whose shape matches.

[0051] As a result, the hook position calculation unit 110 identifies each of the images m of the three detected markers M as the three markers M (markers M2, M3, and M4 of the matched triangle T4) corresponding to the vertices of the matched triangle (for example, triangle T4).

[0052] In addition, when the hook position calculation unit 110 compares the positional relationship of the coordinates (x, y) of the images m of the three markers M detected in the image P with the positional relationship of the four markers M in real space, it also includes comparing the positional relationship of the four markers M in real space by rotating them around the center C or converting them into an enlarged or reduced similar shape.

[0053] As described above, the calculation unit 112 can identify which of the four markers M each coordinate (x, y) is the coordinate of the image m of, based on the positional relationship (coordinates) of the images m of the three markers M, without actually forming a triangle. Specifically, the calculation unit 112 can identify which of the four markers M each coordinate (x, y) is the coordinate of the image m of, by calculating the ratio of the distances between the coordinates (corresponding to the ratio of the lengths of the sides of the triangle), etc.

[0054] The calculation unit 112 calculates the coordinates of the intersection of the perpendicular bisectors of the line segments connecting the coordinates (x, y) of the images m of the three markers M described above. Since the four markers M in real space are distributed and arranged on the circumference of the same circle C1, the intersection of the perpendicular bisectors of the sides of a triangle T having three of the four markers M as vertices is the center C of the circle C1, as shown in FIG. 3D, for example. The images m of the three markers M in the image P are also distributed and arranged on the circumference of the same circle, just like the markers M that are the subject.

[0055] Therefore, the intersection of the perpendicular bisectors of the line segments connecting the coordinates (x, y) of the images m of the three markers M calculated by the calculation unit 112 is the coordinates (xc, yc) of the center c of the circle on which the images m of the three markers M are arranged. The coordinates (xc, yc) of the center c are the image of the center C of the circle C1 on which the four markers M, the subjects corresponding to the images m, are arranged. Therefore, the calculation unit 112 can determine the correspondence between the coordinate system of the image P and the coordinate system of the real space, thereby determining the center C in the real space, that is, the coordinates (Xc, Yc, Zc) of the position of the sub-hook 40.

[0056] In this way, the calculation unit 112 calculates the distance between the center c and the image m of any one of the markers M after determining the coordinates (xc, yc) of the center c.

[0057] This distance calculated by the hook position calculation unit 110 corresponds to the radius of a circle in which the images m of the three markers M are arranged from the center c in the image P. The calculation unit 112 doubles the calculated radius of the circle to calculate the diameter d (see FIG. 5) of the circle in which the images m of the three markers M are arranged in the image P.

[0058] In the image P, the diameter d of the circle in which the images m of the three markers M are arranged becomes smaller as the sub-hook 40 moves away from the camera 50, and becomes larger as the sub-hook 40 moves closer to the camera 50.

[0059] Since the memory unit 111 stores information regarding the specifications of the camera 50, the calculation unit 112 calculates the distance h1 (see Figure 1) from the camera 50 to the center C of the sub-hook 40 in real space based on the ratio between the diameter d of the circle on which the image m of the marker M is located in the calculated image P and the diameter D of the circle C1 on which the marker M is located in real space, and the information regarding the specifications of the camera 50.

[0060] The calculation unit 112 also calculates the height h0 from the ground to the tip of the telescopic boom 16 (for example, a sheave) in real space based on the hoisting angle and telescopic length of the telescopic boom 16 input from the AML. The calculation unit 112 then subtracts the distance A (see FIG. 1) from the tip of the telescopic boom 16 to the camera 50 in real space and the distance h1 from the camera 50 to the center C of the sub-hook 40 in real space, which are stored in the memory unit 111, from the calculated height h0, to calculate the height h2 (= h0 - h1 - A) of the center C in real space from the ground.

[0061] As a result, the calculation unit 112 calculates the coordinates (Xc, Yc) in the coordinate system of the horizontal plane at height h2 (= Zc) from the ground in real space, which correspond to the two-dimensional coordinates (xc, yc) of the center c within the plane of the image P. In this way, the hook position calculation unit 110 calculates the position (center C) of the sub-hook 40 in real space as three-dimensional coordinates (Xc, Yc, Zc).

[0062] FIG. 6 is a schematic diagram illustrating the orientation of the sub-hook 40 when the sub-hook 40 is rotated by an angle of 60 degrees to the right (clockwise) about the center C from the reference position, for example.

[0063] The calculation unit 112 calculates the orientation of the sub-hook 40 around the center c within the plane of the image P. The arrangement of the four markers M on the upper surface 42 is such that four triangles T1, T2, T3, and T4 formed with any three of the four markers M as vertices have different shapes, as shown in FIGS.

[0064] When the four markers M are arranged so that the four triangles T1, T2, T3, and T4 have different shapes, even if the sub-hook 40 is rotated in a plane around the center C, the calculation unit 112 can identify the posture (orientation) of each triangle T1, T2, T3, and T4 based on the relative positional relationship of any three markers M. In other words, the calculation unit 112 can identify the orientation of each marker M rotated around the center C from the reference position, and can identify the orientation of the sub-hook 40 around the center C.

[0065] The reference position of the marker M is the position when the sub-hook 40 is in a natural state and not rotated. At this time, for example, as shown in FIG. 2, the marker M1 is located at 12 o'clock relative to the center C. Then, as shown in FIG. 6, when the sub-hook 40 rotates around the center C and the marker M1 is located at 2 o'clock relative to the center C, the orientation of the sub-hook 40 is rotated 60 degrees clockwise from the reference position. Furthermore, when the marker M1 is located at 8 o'clock relative to the center C, the orientation of the sub-hook 40 is rotated 240 degrees clockwise from the reference position.

[0066] In the above description, when the marker M1 is in the 8 o'clock direction relative to the center C, the sub-hook 40 is rotated clockwise by an angle of 240 degrees from the reference position, but this rotated orientation may also be specified as the orientation of the sub-hook 40 rotated counterclockwise by an angle of 120 degrees from the reference position. In other words, the orientation of the sub-hook 40 may be expressed in any way that specifies a direction within a range of an angle of 360 degrees around the center C.

[0067] The calculation unit 112 applies the process for identifying the orientation of the sub-hook 40 described above to the image m of the marker M in the image P to identify the orientation of the image of the sub-hook 40 in the image P. The orientation of the sub-hook 40 in the image P is applied as is as the orientation of the sub-hook 40 in real space.

[0068] The hook position calculation unit 110 outputs the calculated position and orientation of the sub-hook 40 in real space to the monitor 120. The monitor 120 displays the position and orientation of the sub-hook 40 input from the hook position calculation unit 110.

[0069] As described in detail above, the hook position detection device 100 of this embodiment 1 can accurately detect the position and orientation of the sub-hook 40 based on the image P captured by the camera 50, even if the auxiliary hoisting wire rope 41 hides one marker M in the image P.

[0070] In the hook position detection device 100 of the above-described embodiment, the hook position calculation unit 110 calculates the coordinates (xc, yc) of the center c as the intersection of the perpendicular bisectors of the line segments connecting the images m of the three markers M, and calculates twice the distance between the center c and the image m of the marker M as the diameter of the circle on which the image m of the marker M is located.

[0071] Here, as can be seen from the triangle T1 shown in Figure 3A, when two of the four markers M, M1 and M2, are positioned at an angle of 180 degrees with respect to the center C, the distance between the two markers M1 and M2 corresponds to the diameter D of the circle C1.

[0072] Therefore, when the images m of the three markers M detected in the image P include the image m1 of the marker M1 and the image m2 of the marker M2, the calculation unit 112 may calculate the diameter d of the circle in the image P by calculating the distance between the coordinates (x1, y1) of the image m1 of the marker M1 and the coordinates (x2, y2) of the image m2 of the marker M2, without performing the process of determining the intersection of the perpendicular bisectors described above.

[0073] In this case, the midpoint (midpoint) between the coordinates (x1, y1) of the image m1 of marker M1 and the coordinates (x2, y2) of the image m2 of marker M2 is the center C of the circle C1, so it can also be calculated as the coordinates (xc, yc) of the center c in the image P. Therefore, a hook position detection device 100 configured such that two of the four markers M, M1 and M2, are positioned at an angle of 180° with the center C in between can reduce the calculation load on the hook position calculation unit 110.

[0074] In the hook position detection device 100 of embodiment 1, the angles between the four markers M formed on the upper surface 42 of the sub-hook 40 and center C are 180 degrees, 90 degrees, 60 degrees, and 30 degrees, but the angles between the four markers M and center C are not limited to these angles.

[0075] Specifically, in the hook position detection device 100, the four markers M are distributed around the circumference of the same circle C1, and it is preferable that the angle between any two of the markers M about the center C is 180 degrees, and that the four triangles T1, T2, T3, and T4 formed with the three markers M obtained by selecting any three of the four markers M as vertices all have different shapes.

[0076] That is, in the hook position detection device 100, it is preferable that two of the four markers M distributed around the same circle C1 are arranged on a chord corresponding to the diameter of the circle C1. In the hook position detection device 100 in which two markers M are arranged in this manner, the distance between the coordinates (w, y) of the images m of the two markers M arranged on the chord corresponding to the diameter of the circle C1 can be calculated as the diameter d of the circle on which the images m of the markers M are arranged, thereby reducing the processing load for calculating the diameter d of the circle.

[0077] However, the hook position detection device 100 of embodiment 1 is not limited to a case in which two of the four markers M distributed around the same circle C1 are arranged on a chord corresponding to the diameter of the circle C1. That is, as shown in Fig. 7, the hook position detection device 100 only needs to set the positional relationship of the four markers M so that the four markers M are distributed around the same circle C1 and four triangles T1, T2, T3, and T4 formed with vertices formed by selecting any three markers M out of the four markers M all have different shapes.

[0078] Fig. 7 is a schematic diagram equivalent to Fig. 2, showing another example of the arrangement of four markers M. In the arrangement of the markers M shown in Fig. 7, as an example, marker M2 is positioned at an angle of 150 degrees clockwise from marker M1 about center C, marker M3 is positioned at an angle of 120 degrees clockwise from marker M2 about center C, and marker M4 is positioned at an angle of 45 degrees clockwise from marker M3 about center C. In the hook position detection device 100, the angles between each marker M about center C are not limited to the angles in the above example.

[0079] In this way, the hook position detection device 100 in which the marker M is placed can determine the coordinates of the center c of the circle in which the image m of the marker M is placed by having the calculation unit 112 determine the intersection of the perpendicular bisectors of the line segment connecting the two markers M, as described above.

[0080] In the hook position detection device 100 of the first embodiment, the number of markers M formed on the upper surface 42 of the sub-hook 40 is four, but the number of markers M may be five or more. However, when the number of markers M in the hook position detection device 100 is five or more, the hook position calculation unit 110 detects images m of four or more markers M in the image P, and the hook position calculation unit 110 only needs to calculate the coordinates (xc, yc) of the position of the center c based on the positional relationship of the images m of any three markers M among the detected images m of the four or more markers M.

[0081] Therefore, the hook position detection device 100 is most efficient in terms of balance with the calculation load when the number of markers M formed on the upper surface 42 of the sub-hook 40 is four, which is preferable.

[0082] In the hook position detection device 100 of the first embodiment, the marker M is configured using an LED, but the marker M is not limited to an LED, and the marker M may be painted on the upper surface 42, or a sheet on which the marker M is painted may be attached to the upper surface 42. However, even if the hook position calculation unit 110 has difficulty detecting the image m of the marker M based on the image P because the environment in which the crane 10 is placed is dark, such as in the evening or at night, the marker M configured using an LED can be illuminated to increase the brightness of the image m of the marker M in the image P, making it easier for the hook position calculation unit 110 to detect the image m of the marker M.

[0083] <Embodiment 2> Fig. 8 is a block diagram showing the configuration of a hook position detection device 200 that detects the position and orientation of the sub-hook 40 of the crane 10, and Fig. 9 is a schematic diagram showing an example of the arrangement of four markers N provided on the upper surface 42 of the sub-hook 40 in a plan view (as viewed in a direction perpendicular to the horizontal plane). The hook position detection device 200 is another embodiment (embodiment 2) of the hook position detection device for a crane according to the present invention.

[0084] In the hook position detection device 100 of embodiment 1, the four markers M provided on the upper surface 42 of the sub-hook 40 are arranged on the circumference of the same circle C1, and the hook position calculation unit 110 detects the position and orientation of the sub-hook 40 based on the geometric feature that the four markers M are arranged on the circumference of the same circle C1, but the hook position detection device of the present invention is not limited to one in which the four markers M are arranged on the circumference of the same circle C1.

[0085] A hook position detection device 200 of the second embodiment is provided in place of the hook position detection device 100 in the crane 10 shown in Fig. 1. As shown in Fig. 8, the hook position detection device 200 includes a camera 50, four markers N provided on the upper surface 42 of the sub-hook 40, a hook position calculation unit 210, and a monitor 120.

[0086] The camera 50 and the monitor 120 of the hook position detection device 200 are the same as those in the hook position detection device 100 of the first embodiment.

[0087] Figures 10A, 10B, 10C, and 10D are schematic diagrams showing a triangle formed with any three of the four markers N as vertices, and Figure 11 is an example of an image of a scene vertically below captured by camera 50.

[0088] Like the markers M, four markers N are provided on the upper surface 42 of the sub-hook 40, and are configured, for example, with LEDs. Unlike the markers M, the four markers N are not provided on the circumference of the same circle C1, as shown in Fig. 9. However, as shown in Figs. 10A, 10B, 10C, and 10D, the four markers N are distributed and arranged by selecting any three markers N from the four markers N and forming four triangles V1, V2, V3, and V4 with the selected three markers N as vertices, so that they all have different shapes.

[0089] The hook position calculation unit 210 includes a memory unit 211 and a calculation unit 212. The hoisting angle and telescopic length of the telescopic boom 16 are input from the AML to the hook position calculation unit 210. The hoisting angle and telescopic length of the telescopic boom 16 may also be input directly to the hook position calculation unit 210 without going through the AML.

[0090] The memory unit 211 pre-stores information regarding the positional relationship of the four markers N on the top surface 42 of the sub-hook 40 in real space in a planar view (see Figure 8), the positional relationship of the center C relative to the four markers N, and the specifications of the camera 50.

[0091] The hoisting angle and telescopic length of the telescopic boom 16 are also input from the AML to the hook position calculation unit 210. The hoisting angle and telescopic length of the telescopic boom 16 may also be input directly to the hook position calculation unit 210 without going through the AML.

[0092] The storage unit 211 selects any three markers N from the four markers N as the positional relationship of the four markers N in a planar view, and stores the shapes of four mutually different triangles V1, V2, V3, and V4 formed with the selected three markers N as vertices as patterns corresponding to the positional relationship of the four markers N. Furthermore, the storage unit 211 stores the positional relationship of the center C with respect to each of the triangles V1, V2, V3, and V4 as the positional relationship of the center C with respect to the four markers N.

[0093] The fact that triangles V1, V2, V3, and V4 have different shapes means that even when triangles V1, V2, V3, and V4 are rotated within a plane, they do not overlap exactly with the other triangles V1, V2, V3, and V4.

[0094] As shown in Figure 11, the calculation unit 212 compares and collates the shape of a triangle (a triangle with the images n of the three markers N as vertices) which is a pattern corresponding to the positional relationship of the images n of the three markers N in the image P captured by the camera 50 with triangles V1, V2, V3, and V4 which are patterns corresponding to the positional relationship of the markers N stored in the memory unit 211, and identifies one triangle from the four triangles V1, V2, V3, and V4.

[0095] In addition, the calculation unit 212 calculates the coordinates (xc, yc) of the position of the center c in the image P based on the positional relationship of the center C with respect to the identified triangle stored in the memory unit 211 and the triangle formed by the images n of the three markers N in the image P.

[0096] In addition, the calculation unit 212 calculates the distance h1 (see Figure 1) from the camera 50 to the center C of the sub-hook 40 in real space based on the area ratio between the identified triangle stored in the memory unit 211 and the triangle formed by the images n of the three markers N in the image P, and information regarding the specifications of the camera 50 stored in the memory unit 211.

[0097] The calculation unit 212 also calculates the height h0 from the ground to the tip of the telescopic boom 16 (for example, a sheave) in real space based on the hoisting angle and telescopic length of the telescopic boom 16 input from the AML. The calculation unit 212 then subtracts the distance A (see FIG. 1) from the tip of the telescopic boom 16 to the camera 50 in real space and the distance h1 from the camera 50 to the center C of the sub-hook 40 in real space, which are stored in the memory unit 211, from the calculated height h0, to calculate the height h2 (= h0 - h1 - A) of the center C in real space from the ground.

[0098] As a result, the calculation unit 212 calculates the coordinates (Xc, Yc) in the coordinate system of the horizontal plane at height h2 (= Zc) from the ground in real space, which correspond to the two-dimensional coordinates (xc, yc) of the center c within the plane of the image P. In this way, the hook position calculation unit 210 calculates the position (center C) of the sub-hook 40 in real space as three-dimensional coordinates (Xc, Yc, Zc).

[0099] The calculation unit 212 also calculates the orientation of the sub-hook 40 around the center c within the plane of the image P. Specifically, the calculation unit 212 compares and collates a triangle having the images n of the three markers N in the image P as its vertices with a specified triangle from among the triangles V1, V2, V3, and V4 stored in the storage unit 211, thereby specifying the orientation of the sub-hook 40 rotated around the center C.

[0100] The four triangles V1, V2, V3, and V4 with the marker N as their vertices are stored in the memory unit 211 in shapes corresponding to the orientation of the sub-hook 40 in its natural state when it is not rotated. Using this orientation in its natural state as a reference position, the calculation unit 212 calculates the rotation angle when one identified triangle of the four triangles V1, V2, V3, and V4 stored in the memory unit 211 is rotated so that it overlaps with a triangle in the image P with the images n of the three markers N as its vertices. The calculation unit 212 specifies this calculated rotation angle as the orientation of the sub-hook 40.

[0101] The hook position calculation unit 210 outputs the calculated position and orientation of the sub-hook 40 in real space to the monitor 120. The monitor 120 displays the position and orientation of the sub-hook 40 input from the hook position calculation unit 210.

[0102] As described in detail above, the hook position detection device 200 of this embodiment 2 can accurately detect the position and orientation of the sub-hook 40 based on the image P captured by the camera 50, even if the auxiliary hoisting wire rope 41 hides one marker N in the image P.

[0103] Furthermore, in the hook position detection device 200 of embodiment 2, it is not necessary to distribute the four markers N to be placed on the upper surface 42 of the sub-hook 40 on the circumference of a single circle. Therefore, the hook position detection device 200 can increase the degree of freedom in arranging the markers N compared to the hook position detection device 100.

[0104] Furthermore, in the hook position detection device 200 of the second embodiment, similar to the hook position detection device 100 of the first embodiment, the number of markers N formed on the upper surface 42 of the sub-hook 40 may be five or more. [Explanation of symbols]

[0105] 10 Rough terrain crane (crane) 16 Telescopic boom 40 Sub Hook 41 Auxiliary wire rope 42 Top surface 50 cameras 100 Hook position detection device 110 Hook position calculation unit M marker T1,T2,T3,T4 triangle

Claims

1. an imaging device provided at the tip of the crane boom; four or more markers provided on an upper surface of a hook suspended by a wire from the tip of the boom; a hook position calculation unit that calculates the position and orientation of the hook based on a positional relationship between three or more of the markers captured by the imaging device out of the four or more markers, A crane hook position detection device in which the four or more markers are distributed and arranged in a positional relationship such that multiple triangles formed with three arbitrarily selected markers as vertices all have different shapes.

2. The four or more markers are arranged on the upper surface of the hook, distributed on the circumference of the same circle centered at the position where the hook is supported by the wire; 2. The hook position detection device for a crane according to claim 1, wherein the hook position calculation unit includes: a memory unit that stores the positional relationships of any three of the four or more markers; and a calculation unit that calculates the position and orientation of the hook based on the positional relationships of images of the three markers captured by the imaging device and the positional relationships of the markers stored in the memory unit.

3. 3. The hook position detection device for a crane according to claim 2, wherein the calculation unit calculates the position of the hook as the intersection of perpendicular bisectors of sides of a triangle formed with the three markers imaged by the imaging device as vertices, and calculates the orientation of the hook based on the relative positional relationship of the three markers.

4. two of the four or more markers are arranged on the circumference at positions corresponding to a diameter of the circle; 3. The hook position detection device for a crane according to claim 2, wherein the calculation unit calculates the midpoint between two of the markers arranged at positions corresponding to the diameter as the position of the hook, and calculates the orientation of the hook based on the relative positional relationship of the three markers.

5. 2. The hook position detection device for a crane according to claim 1, wherein the hook position calculation unit includes: a memory unit that stores a plurality of patterns corresponding to positional relationships of any three of the four or more markers; and a calculation unit that determines the position and orientation of the hook by comparing and collating the positional relationship of the three markers imaged by the imaging device with the plurality of patterns stored in the memory unit and identifying one pattern from the plurality of patterns.

6. The hook position detection device for a crane according to any one of claims 1 to 5, wherein the marker is an LED.

Images