Optical axis deviation angle correction device, crane, and optical axis deviation angle correction method
The optical axis deviation angle correction method for camera systems simplifies the alignment process by using a camera with a built-in lens to calculate and correct deviations, resulting in a cost-effective and precise solution for camera systems.
Patent Information
- Application Number
- PCT/JP2024/018385
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-18
- Filing Date
- 2024-05-17
- Publication Date
- 2024-11-21
AI Technical Summary
Existing camera systems with optical axis misalignment require complex and expensive mechanical adjustments to correct optical axis deviation, complicating the imaging device and increasing costs.
A simple and inexpensive optical axis deviation angle correction method using a camera with a built-in lens that calculates and corrects the optical axis deviation by drawing a mark on the captured image to align it with a known target, allowing for precise adjustment of the lens position.
Enables accurate and cost-effective correction of optical axis deviation in camera systems, improving image alignment and reducing the complexity of the imaging device configuration.
Smart Images

Figure JP2024018385_21112024_PF_FP_ABST
Abstract
Description
Optical axis deviation angle correction device, crane, and optical axis deviation angle correction method
[0001] The present invention relates to an optical axis deviation angle correcting device, a crane, and an optical axis deviation angle correcting method.
[0002] Conventionally, as disclosed in Patent Document 1, if the center of an image captured by a camera is misaligned with the optical axis of the lens, a subject that appears in the center of the image captured at the telephoto end of the camera may appear off-center at the wide-angle end.
[0003] In the case of the imaging device described in Patent Document 1, the amount of deviation between the center of the captured image and the optical axis of the lens is stored in a storage device, and the stored amount of deviation is used to mechanically adjust the angle of the camera platform that supports the camera, thereby correcting the deviation of the optical axis of the lens. As a result, the subject is positioned at the center of the captured image regardless of the magnification at which the video is captured.
[0004] Japanese Patent Application Laid-Open No. 2008-311804
[0005] As described above, when correcting the misalignment of the lens optical axis by mechanically adjusting the angle of the camera platform that supports the camera, in addition to a controller that performs the camera's imaging operations and processes the captured images, a motor for adjusting the angle of the camera platform and a motor driver for controlling the motor are required, making the imaging device complex and expensive.
[0006] Therefore, the present invention provides an optical axis deviation angle correcting device, a crane, and an optical axis deviation angle correcting method that are capable of correcting the optical axis deviation angle of a lens in a camera with a simple and inexpensive configuration.
[0007] One aspect of the optical axis deviation angle correction device according to the present invention comprises: a camera having a lens that captures an image including a target whose position is known; and a controller that draws a mark indicating the position of the optical axis of the lens on the image captured by the camera, calculates a deviation angle of the optical axis of the lens with respect to the imaging direction of the camera based on the positional relationship between the mark and the target in the captured image, and corrects the position of the mark in the captured image based on the deviation angle of the optical axis so that the mark overlaps the target.
[0008] One aspect of the crane according to the present invention includes the optical axis deviation angle correction device described above.
[0009] One aspect of the optical axis deviation angle correction method according to the present invention includes the steps of: capturing an image including a target whose position is known using a camera having a lens; drawing a mark indicating the position of the optical axis of the lens on the image captured by the camera; calculating the deviation angle of the optical axis of the lens with respect to the imaging direction of the camera based on the positional relationship between the mark and the target in the captured image; and correcting the position of the mark in the captured image so that it overlaps with the target based on the deviation angle of the optical axis.
[0010] According to the present invention, the optical axis deviation angle of the lens in a camera can be corrected with a simple and inexpensive configuration.
[0011] FIG. 1 is a side view of a crane. FIG. 2 is a plan view showing the interior of the cabin. FIG. 3 is a block diagram showing the configuration of a control system. FIG. 4 is a diagram showing an image captured by a camera. FIG. 5 is a diagram showing a camera attached to a boom. FIG. 6 is a diagram showing the configuration of the camera. FIG. 7A is a diagram showing the angle of optical axis deviation of the lens in the pan direction. FIG. 7B is a diagram showing the angle of optical axis deviation of the lens in the tilt direction. FIG. 8 is a diagram showing a switch operator. FIG. 9 is a diagram showing a captured image including the grounding point of the hook. FIG. 10 is a diagram showing a captured image in which a target plate is placed on the grounding point of the hook and the hook is raised to a height where the target plate is visible. FIG. 11 is a diagram showing a captured image in which an image including a target plate is captured in the first optical axis deviation measurement mode and the center of a danger zone circle is drawn. FIG. 12 is a diagram showing a captured image in which an image including a target plate is captured in the second optical axis deviation measurement mode and the center of a danger zone circle is drawn. Fig. 13 is a diagram showing a captured image in which an image including a target plate is captured in the third optical axis deviation measurement mode and the center of a danger area circle is drawn. Fig. 14 is a diagram showing a captured image in which the danger area circle is drawn after the optical axis deviation angle is corrected. Fig. 15 is a diagram showing a captured image in which the danger area circle and performance line are drawn after the optical axis deviation angle correction mode is completed.
[0012] Next, an embodiment of the present invention will be described with reference to the accompanying drawings.
[0013] 1 and 2 is an embodiment of a crane equipped with an optical axis deviation angle correction device according to the present invention. The crane 1 includes a running body 2 and a rotating body 3. The rotating body 3 is an example of a vehicle body.
[0014] The running body 2 is equipped with a pair of left and right front wheels 21 and a pair of left and right rear wheels 22. The running body 2 is also equipped with outriggers 23 that are grounded to provide stability when transporting cargo W. The running body 2 supports a rotating body 3 so that it can rotate horizontally. The rotating body 3 is rotated about a rotation center X by an actuator equipped in the running body 2.
[0015] The revolving unit 3 has a boom 31. The boom 31 is supported at the rear of the revolving unit 3. The boom 31 supported by the revolving unit 3 extends so as to protrude forward from the rear of the revolving unit 3. The boom 31 can be rotated together with the revolving unit 3 in a revolving direction A by an actuator (see arrow A in FIG. 1). The boom 31 can be extended and retracted in a retracting direction B by an actuator (see arrow B in FIG. 1). The boom 31 can be raised and lowered in a raising and lowering direction C by an actuator (see arrow C in FIG. 1).
[0016] A wire rope 32 is stretched across the boom 31. A hook 33 is attached to the wire rope 32 hanging down from the tip of the boom 31. A winch 34 is disposed near the base end of the boom 31. The winch 34 is configured integrally with an actuator and winds in or unwinds the wire rope 32. This allows the hook 33 to be freely raised and lowered by the actuator in a lifting direction D (see arrow D in FIG. 1 ).
[0017] The revolving body 3 is provided with a cabin 35 on the side of the boom 31. Inside the cabin 35, a revolving lever 51, a telescopic lever 52, a hoisting lever 53, a winding lever 54, etc. are provided.
[0018] [Steering System] The crane 1 is equipped with a steering system 4.
[0019] As shown in Figures 1 to 3, the control system 4 mainly includes a controller 40, a receiver 43, various levers 51 to 54, a camera 55, a switch operator 58, various valves 61 to 64, a display 65, and various detection sensors 91 to 96.
[0020] The controller 40 has an information storage unit 41. The information storage unit 41 stores various programs required for controlling the crane 1. The information storage unit 41 also stores various types of data.
[0021] The controller 40 has an information processing unit 42. The information processing unit 42 converts the operation amounts of the various levers 51 to 54 into electrical signals and transmits the converted electrical signals to the various valves 61 to 64 that operate the actuators. In this way, the controller 40 realizes the operation of the boom 31 (swinging operation, extension / retraction operation, and raising / lowering operation) and the operation of the winch 34 (winding-in operation and winding-out operation).
[0022] More specifically, the boom 31 is rotatable in a rotation direction A by an actuator (see arrow A in FIG. 1). In this embodiment, the actuator that rotates the boom 31 is a rotation motor 36 (see FIG. 1). The rotation motor 36 is operated appropriately by a rotation valve 61, which is a directional control valve. The rotation valve 61 is operated based on instructions from the controller 40. The rotation angle of the boom 31 is detected by a rotation angle detection sensor 91.
[0023] The boom 31 is extendable and retractable in a telescoping direction B by an actuator (see arrow B in FIG. 1). In this embodiment, the actuator that extends and retracts the boom 31 is an extension cylinder 37 (see FIG. 1). The extension cylinder 37 is operated appropriately by an extension valve 62, which is a directional control valve. The extension valve 62 is operated based on instructions from the controller 40. The extension length of the boom 31 is detected by a boom length detection sensor 92.
[0024] The boom 31 can be raised and lowered in a hoisting direction C by an actuator (see arrow C in Figure 1). In this embodiment, the actuator that raises and lowers the boom 31 is a hoist cylinder 38 (see Figure 1). The hoist cylinder 38 is operated appropriately by a hoist valve 63, which is a directional control valve. The hoist valve 63 is operated based on instructions from the controller 40. The hoist angle of the boom 31 is detected by a boom angle detection sensor 93.
[0025] The hook 33 can be raised and lowered by an actuator in a lifting direction D (see arrow D in FIG. 1). In this embodiment, the actuator that raises and lowers the hook 33 is a winding motor 39 (see FIG. 1). The winding motor 39 is operated appropriately by a winding valve 64, which is a directional control valve. The winding valve 64 is operated based on instructions from the controller 40. The hanging length of the hook 33 is detected by a sensor (not shown).
[0026] A boom hoist cylinder pressure detection sensor 94 provided in the steering system 4 detects the pressure applied to the boom hoist cylinder 38. A lever operation detection sensor 95 detects the operation states of the various levers 51 to 54. An outrigger extension width detection sensor 96 detects the extension width of the outriggers 23.
[0027] The operation system 4 is equipped with an overload prevention device 45. The overload prevention device 45 is a safety device that prevents the working moment applied to the crane 1 from becoming excessive, thereby preventing the crane 1 from tipping over or being damaged due to an overload that exceeds the rated load.
[0028] The overload prevention device 45 receives input of the working status of the crane 1 registered by the operator, as well as the swing angle of the boom 31, the extension length of the boom 31, the hoisting angle of the boom 31, the pressure applied to the hoisting cylinder 38, the operation status of the various levers 51-54, and the extension width of the outriggers 23, which are detected by the various detection sensors 91-96. Based on the input information, the overload prevention device 45 calculates the working moment and the rated moment. The overload prevention device 45 then divides the working moment by the rated moment to obtain the moment load factor. The obtained moment load factor is input from the overload prevention device 45 to the controller 40. The obtained moment load factor may also be displayed on the display 65.
[0029] When the input moment load rate reaches 100% or more, the controller 40 stops operation of the crane 1 toward the dangerous side and issues a warning by displaying an error code on the display 65, sounding a buzzer, lighting an indicator light, etc. When the moment load rate reaches 100% or more, operation of the crane 1 toward the dangerous side is stopped, but operation toward the safe side is permitted.
[0030] The controller 40 performs overload control based on the moment load rate input from the overload prevention device 45, for example, by stopping operation of the crane 1 toward the dangerous side and issuing an alarm when the moment load rate becomes 100% or more. Also, as overload control, the controller 40 issues an alarm to call attention when the moment load rate becomes 90% or more but less than 100%. Also, as overload control, the controller 40 performs control to indicate that it is safe when the moment load rate becomes less than 90%.
[0031] As shown in Fig. 1, camera 55 is attached to the tip of boom 31. As shown in image PT in Fig. 4, camera 55 can image the construction site from above and can image hook 33 hanging down from the tip of boom 31 and cargo W suspended from hook 33. Camera 55 is attached to boom 31 in an orientation in which the imaging direction PD (see Fig. 5) is vertically downward. Camera 55 has a built-in tilt angle detector 56, which can detect the tilt angle of camera 55 with respect to the vertically downward direction.
[0032] The controller 40 includes a receiver 43. The camera 55 is wirelessly connected to the controller 40 via the receiver 43. However, the camera 55 may be wired to the controller 40. The controller 40 can acquire a captured image PT captured by the camera 55, the image including the hook 33 and the baggage W hanging from the hook 33.
[0033] The display 65 displays various images. The display 65 is attached to the front side of the driver's seat inside the cabin 35 so that the operator can view it while operating the various levers 51 to 54 (see FIG. 2). The display 65 is connected to the controller 40 via a receiver 43. The controller 40 can provide information to the operator through the display 65. Specifically, the controller 40 can display on the display 65 a captured image PT that is captured by the camera 55 and includes the hook 33 and the baggage W hanging from the hook 33.
[0034] 5, camera 55 is attached to boom 31 by a mounting fixture 57. Camera 55 is located a distance E away from wire rope 32 hanging down from boom 31 in the direction along the rotation direction A. Camera 55 is attached to boom 31 with imaging direction PD pointing vertically downward. The direction in which wire rope 32 hanging down from boom 31 extends and the imaging direction PD of camera 55 are parallel. Mounting fixture 57 may be, for example, a rotary damper.
[0035] As shown in Fig. 6, the camera 55 includes an image sensor 551 and a lens 552. The lens 552 adjusts the capture magnification of the image captured by the camera 55. The image sensor 551 captures the image whose capture magnification has been adjusted by the lens 552. The image sensor 551 captures the image in an image capturing direction PD of the camera 55, which is a direction perpendicular to the image capturing surface of the image sensor 551. The camera 55 is capable of capturing an image including a target Tg whose positional relationship with the camera 55 is known. In this case, the camera 55 can capture the image with the target Tg located on an extension of the image capturing direction PD of the camera 55.
[0036] In the camera 55, the lens 552 is assembled so that the axial direction of the optical axis OA of the lens 552 coincides with the imaging direction PD, but due to the influence of assembly errors and the like, the optical axis OA of the lens 552 after assembly may be misaligned with the imaging direction PD. In other words, the optical axis OA may have optical axis deviation angles Ψ and θ with respect to the imaging direction PD.
[0037] For example, when the optical axis deviation angles Ψ and θ are measured for multiple cameras 55, it is found that the cameras 55 have the optical axis deviation angle Ψ in the pan direction shown in Fig. 7A and the optical axis deviation angle θ in the tilt direction shown in Fig. 7B. In Fig. 7A, the horizontal axis represents the zoom magnification, which is the capture magnification of the image captured by the cameras 55, and the vertical axis represents the optical axis deviation angle Ψ. In Fig. 7B, the horizontal axis represents the zoom magnification and the vertical axis represents the optical axis deviation angle θ. Note that the pan direction is the left-right direction in the captured image PT shown in Fig. 4, and the tilt direction is the up-down direction in the captured image PT shown in Fig. 4.
[0038] 5 and 6 , when the camera 55 has an optical axis deviation angle Ψ, the amount of deviation of the optical axis OA from the imaging direction PD in the pan direction at a position a distance H away from the center 552 a of the lens 552 in the imaging direction PD of the camera 55 is a first optical axis deviation amount Lx. Also, when the camera 55 has an optical axis deviation angle θ, the amount of deviation of the optical axis OA from the imaging direction PD in the tilt direction at a position a distance H away from the center 552 a of the lens 552 in the imaging direction PD of the camera 55 is a second optical axis deviation amount Ly.
[0039] In the crane 1, when the controller 40 displays the captured image PT captured by the camera 55 on the display 65, it can draw the danger zone circle CD (see FIG. 4) and the performance line CR by superimposing them on the captured image PT.
[0040] The danger area circle CD is a drawing that shows a circular area centered on the contact point T of the hook 33 attached to the wire rope 32 hanging down from the boom 31 or the load W suspended from the hook 33 with the ground S. The danger area edge CD indicates an area into which the load W suspended from the hook 33 may fall. Therefore, by drawing the danger area circle CD on the captured image PT, it is possible to issue a warning to prohibit people from entering the danger area circle CD. The danger area circle CD is drawn at a position where the center CDp of the danger area circle CD coincides with the optical axis OA of the camera 55.
[0041] The performance line CR is a circle centered on the rotation center X, and indicates the moment load rate calculated by the overload prevention device 45. For example, the performance line CR can indicate the boundary where the moment load rate of the crane 1 is 90%. In this case, the inner area surrounded by the performance line CR indicates the area where the moment load rate is 90% or less.
[0042] The controller 40 can display the captured image PT so that the ground contact point T of the hook 33 is located on an extension of the imaging direction PD of the camera 55. In other words, because the camera 55 is located a distance E away from the wire rope 32 along the rotation direction A, the controller 40 can correct the position of the ground contact point T of the hook 33 shown in the captured image PT by a dimension corresponding to the distance E, thereby displaying the ground contact point T in the captured image PT so that it is at the position it would be when the ground contact point T is imaged from directly above the wire rope 32.
[0043] In this way, when a danger area circle CD is drawn on the captured image PT displayed so that the ground contact point T of the hook 33 attached to the wire rope 32 is located on an extension of the imaging direction PD of the camera 55, if the imaging direction PD of the camera 55 and the axial direction of the optical axis OA are aligned, the position of the ground contact point T of the hook 33 with the ground surface S will be aligned with the position of the center CDp of the danger area circle CD.
[0044] However, if the optical axis OA has optical axis deviation angles Ψ and θ with respect to the imaging direction PD, the position of the center CDp of the drawn danger area circle CD will be displayed as being misaligned from the position of the ground contact point T. If the position of the center CDp of the danger area circle CD is displayed as being misaligned from the position of the ground contact point T, the function of the danger area circle CD to call attention to the driver will be reduced, and therefore it is preferable that there is no misalignment between the position of the center CDp of the danger area circle CD and the position of the ground contact point T.
[0045] [Optical Axis Deviation Angle Correction Device] Therefore, the crane 1 is provided with an optical axis deviation angle correction device 7 for correcting the optical axis deviation angles Ψ and θ of the lens 552 in the camera 55. The optical axis deviation angle correction device 7 corrects the optical axis deviation angles Ψ and θ of the lens 552 so that the position of the ground contact point T coincides with the position of the center CDp of the danger area circle CD drawn in the captured image PT.
[0046] As shown in FIG. 3, the optical axis deviation angle correcting device 7 is composed of a controller 40, a camera 55, a display 65, a receiver 43, a switch operator 58, and the like.
[0047] 8 , the switch operator 58 is connected to the controller 40 via the receiver 43. The switch operator 58 has a setting button 581, a first preset button 582, a second preset button 583, a third preset button 584, a zoom button 585, and a joystick 586.
[0048] A method for correcting the optical axis deviation angles Ψ and θ of the lens 552 using the optical axis deviation angle correcting device 7 will be described below.
[0049] When correcting the optical axis deviation angles Ψ and θ, the controller 40 adjusts the extension length and elevation angle of the boom 31 so that the height position H of the camera 55 from the ground S is a predetermined height position, and also adjusts the hanging length of the hook 33 to a length that allows the hook 33 to contact the ground S.
[0050] 9, the controller 40 captures an image including the contact point T of the hook 33 with the camera 55 while the hook 33 is in contact with the ground S. In this case, the controller 40 draws a danger zone circle CD on the captured image PT.
[0051] The worker places the target plate TPL on the ground contact point T of the hook 33. The target plate TPL is a plate that indicates the three-dimensional position of the ground contact point T of the hook 33, and is a target that is imaged by the camera 55.
[0052] The information storage unit 41 of the controller 40 stores the attachment position of the camera 55 relative to the boom 31. The controller 40 can also grasp the three-dimensional position of the hook 33 relative to the camera 55, and can store the grasped three-dimensional position of the hook 33 relative to the camera 55 in the information storage unit 41.
[0053] The controller 40 determines the three-dimensional position of the target plate TPL located at the ground contact point T relative to the camera 55 based on the mounting position of the camera 55 relative to the boom 31, the extension / retraction length of the boom 31 detected by the boom length detection sensor 92, the hoisting angle of the boom 31 detected by the boom angle detection sensor 93, and the suspension length of the hook 33 detected by a sensor not shown. The controller 40 can store the determined three-dimensional position of the target plate TPL relative to the camera 55.
[0054] The target plate TPL, whose three-dimensional position relative to the camera 55 is grasped by the controller 40, is disposed at a distance from the camera and is a target whose positional relationship relative to the camera is known. The positional relationship of the target plate TPL relative to the camera 55 corresponds to the positional relationship of the hook 33 relative to the camera 55.
[0055] 10, the controller 40 controls the winding valve 64 to raise the hook 33, which is in the ground, by the winding motor 39. In this case, the controller 40 raises the hook 33 until the target plate TPL placed on the ground S reaches a height at which it appears in the captured image PT.
[0056] The operator presses the setting button 581 of the switch operator 58 to switch the controller 40 to the optical axis deviation angle correction mode. The operator then presses the first preset button 582. When the first preset button 582 is pressed, the controller 40 transitions to the first optical axis deviation amount measurement mode. However, the controller 40 can also automatically transition to the first optical axis deviation amount measurement mode after switching to the optical axis deviation angle correction mode.
[0057] In the first optical axis shift measurement mode, the controller 40 sets the image capture magnification zi of the camera 55 to a predetermined capture magnification, and captures an image including the target plate TPL with the camera 55. In this case, the capture magnification zi set in the first optical axis shift measurement mode is set to capture magnification z1. The captured image PT captured in the first optical axis shift measurement mode is the captured image PT1. The controller 40 can display the image capture magnification z1 used when capturing the captured image PT1 on the captured image PT1.
[0058] 11 , the controller 40 draws a center CDp of the danger area circle CD in the captured image PT1. The center CDp of the danger area circle CD drawn in the captured image PT1 is a mark indicating the position of the optical axis OA of the lens 552 in the captured image PT1. In other words, the danger area circle CD is drawn at a position where the center CDp of the danger area circle CD coincides with the position of the optical axis OA of the lens 552. Note that in the captured image PT1, the danger area circle CD itself is not drawn and only the center CDp is drawn, but it is also possible to draw the danger area circle CD in addition to the center CDp.
[0059] In the captured image PT1, the position of the target plate TPL is misaligned with the position of the center CDp. The amount of misalignment between the position of the target plate TPL and the position of the center CDp in the left-right direction in the captured image PT1 is a first misalignment amount x1, and the amount of misalignment between the position of the target plate TPL and the position of the center CDp in the up-down direction is a second misalignment amount y1. Note that the left-right direction, which is the pan direction of the camera 55, is an example of the first direction, and the up-down direction, which is the tilt direction of the camera 55, is an example of the second direction.
[0060] The center CDp drawn in the captured image PT1 can be moved by operating the joystick 586 of the switch operator 58. The operator operates the joystick 586 to move the center CDp to the position of the target plate TPL, and presses the joystick 586 after the position of the center CDp and the position of the target plate TPL are aligned.
[0061] By operating the joystick 586 to move the center CDp until it coincides with the position of the target plate TPL, and then pressing the joystick 586, the first positional deviation amount x1 and the second positional deviation amount y1 on the captured image PT1 are measured and input to the controller 40. The first positional deviation amount x1 and the second positional deviation amount y1 correspond to an example of the positional relationship between the mark and the target in the captured image PT1.
[0062] However, the switch operator 58 may be configured to include a cross key instead of the joystick 586, and the first positional deviation amount x1 and the second positional deviation amount y1 may be input to the controller 40 by operating the cross key to move the center CDp. Alternatively, the display 65 displaying the captured image PT1 may be configured as a touch panel display, and the first positional deviation amount x1 and the second positional deviation amount y1 may be input to the controller 40 by directly touching and moving an object to be moved, such as the center CDp, on the display 65. Furthermore, the first positional deviation amount x1 and the second positional deviation amount y1 may be automatically acquired by the information processing unit 42 when the controller 40 draws the center CDp and input to the information storage unit 41. The first positional deviation amount x1 and the second positional deviation amount y1 input to the controller 40 can be displayed on the captured image PT1.
[0063] When operating the joystick 586 to move the center CDp to the position of the target plate TPL, if the target plate TPL is small and difficult to see, it is possible to change the display magnification of the captured image PT1 by digital zoom, for example, by operating the zoom button 585 of the switch operator 58.
[0064] The controller 40 determines the optical axis deviation angle Ψ of the lens 552 of the camera 55 based on the input first deviation amount x1 and the positional relationship between the camera 55 and the target plate TPL. 1 That is, the controller 40 calculates the deviation angle of the optical axis of the lens with respect to the imaging direction of the camera based on the positional relationship between the mark and the target in the captured image PT1.
[0065] Specifically, when the captured image PT1 is displayed so that the target plate TPL is positioned on an extension line of the imaging direction PD of the camera 55, a first positional deviation amount x1 in the left-right direction between the target plate TPL and the center CDp of the danger area circle CD on the captured image PT1 is calculated based on the first positional deviation amount x1 in the left-right direction between the target plate TPL and the center CDp of the danger area circle CD on the captured image PT1. Furthermore, a first optical axis deviation amount Lx1 of the lens 552 in the left-right direction with respect to the target plate TPL is calculated at a position away from the camera 55 by a distance H from the center 552a of the lens 552 in the imaging direction PD of the camera 55 to the ground point T on which the target plate TPL is placed, using the first optical axis deviation amount Lx1 and the distance H, according to equation (1). 1 Calculate.
[0066]
[0067] In the formula (1), i is a natural number, and the first optical axis deviation angle Ψ 1 When calculating i=1.
[0068] Furthermore, the controller 40 calculates the optical axis deviation angle θ of the lens 552 of the camera 55 based on the input second deviation amount y1 and the positional relationship between the camera 55 and the target plate TPL. 1 Calculate.
[0069] Specifically, when the captured image PT1 is displayed so that the target plate TPL is positioned on an extension line of the imaging direction PD of the camera 55, a second positional deviation amount y1 in the up-down direction between the target plate TPL and the center CDp of the danger area circle CD in the captured image PT1 is calculated based on the second positional deviation amount y1 in the up-down direction between the target plate TPL and the center CDp of the danger area circle CD in the captured image PT1. Furthermore, a second optical axis deviation amount Ly1 of the lens 552 in the up-down direction with respect to the target plate TPL is calculated at a position away from the camera 55 by a distance H from the center 552a of the lens 552 in the imaging direction PD of the camera 55 to the ground point T on which the target plate TPL is placed, using the second optical axis deviation amount Ly1 and the distance H, according to equation (2). 1 Calculate.
[0070]
[0071] In the formula (2), i is a natural number, and the second optical axis deviation angle θ 1 When calculating i=1.
[0072] In the first optical axis deviation measurement mode, the operator operates the joystick 586 to move the center CDp until it coincides with the position of the target plate TPL, presses the joystick 586, and then presses the second preset button 583 of the switch operator 58. When the second preset button 583 is pressed, the controller 40 transitions to the second optical axis deviation measurement mode. However, the controller 40 can also automatically transition to the second optical axis deviation measurement mode after the input of the first positional deviation amount x1 and the second positional deviation amount y1 to the controller 40 in the first optical axis deviation measurement mode is completed.
[0073] In the second optical axis shift measurement mode, the controller 40 sets the image capture magnification zi of the camera 55 to a predetermined capture magnification, and captures an image including the target plate TPL with the camera 55. In this case, the capture magnification zi set in the second optical axis shift measurement mode is set to a capture magnification z2 that is different from the capture magnification z1. In this embodiment, the capture magnification z2 is set to a magnification greater than the capture magnification z1. Furthermore, the captured image PT captured in the second optical axis shift measurement mode is the captured image PT2. The controller 40 can display the capture magnification z2 of the image when the captured image PT2 is captured on the captured image PT2.
[0074] 12, the controller 40 draws a center CDp of the danger area circle CD in the captured image PT2. The center CDp of the danger area circle CD drawn in the captured image PT2 is a mark that indicates the position of the optical axis OA of the lens 552 in the captured image PT2. Note that in the captured image PT2, the danger area circle CD itself is not drawn and only the center CDp is drawn, but it is also possible to draw the danger area circle CD in addition to the center CDp.
[0075] In the captured image PT2, the position of the target plate TPL is misaligned with the position of the center CDp. The amount of misalignment between the position of the target plate TPL and the position of the center CDp in the left-right direction in the captured image PT2 is a first misalignment amount x2, and the amount of misalignment between the position of the target plate TPL and the position of the center CDp in the up-down direction is a second misalignment amount y2.
[0076] The center CDp drawn in the captured image PT2 can be moved by operating the joystick 586 of the switch operator 58. The operator operates the joystick 586 to move the center CDp to the target plate TPL, and presses the joystick 586 after the position of the center CDp and the position of the target plate TPL are aligned.
[0077] By operating the joystick 586, the center CDp is moved until it coincides with the position of the target plate TPL, and then the joystick 586 is pressed down, whereby the first positional deviation amount x2 and the second positional deviation amount y2 on the captured image PT2 are measured and input to the controller 40.
[0078] However, the first positional deviation amount x2 and the second positional deviation amount y2 may be automatically acquired by the information processing unit 42 when the controller 40 draws the center CDp, and input to the information storage unit 41. The first positional deviation amount x2 and the second positional deviation amount y2 input to the controller 40 can be displayed on the captured image PT2.
[0079] In the second optical axis deviation amount measurement mode, too, the display magnification of the captured image PT2 can be changed by digital zooming, for example, by operating the zoom button 585 of the switch operation device 58.
[0080] The controller 40 determines the optical axis deviation angle Ψ of the lens 552 of the camera 55 based on the input first deviation amount x2 and the positional relationship between the camera 55 and the target plate TPL. 2 Calculate.
[0081] Specifically, when the captured image PT2 is displayed so that the target plate TPL is positioned on an extension of the imaging direction PD of the camera 55, the first optical axis deviation amount Lx2 of the lens 552 at a position a distance H away from the camera 55 is calculated based on the first positional deviation amount x2 on the captured image PT2. Furthermore, the first optical axis deviation amount Lx2 and the distance H are used to calculate the first optical axis deviation angle Ψ of the lens 552 in the left-right direction according to equation (1). 2 In the formula (1), i is a natural number, and the first optical axis deviation angle Ψ 2 When calculating i=2.
[0082] Furthermore, the controller 40 calculates the optical axis deviation angle θ of the lens 552 of the camera 55 based on the input second deviation amount y2 and the positional relationship between the camera 55 and the target plate TPL. 2 Calculate.
[0083] Specifically, when the captured image PT2 is displayed so that the target plate TPL is positioned on an extension of the imaging direction PD of the camera 55, the second optical axis deviation amount Ly2 of the lens 552 at a position a distance H away from the camera 55 is calculated based on the second position deviation amount y2 on the captured image PT2. Furthermore, the second optical axis deviation amount Ly2 and the distance H are used to calculate the second optical axis deviation angle θ of the lens 552 in the up-down direction according to equation (2). 2 In the formula (2), i is a natural number, and the second optical axis deviation angle θ 2 When calculating i=2.
[0084] In the second optical axis deviation measurement mode, the operator operates the joystick 586 to move the center CDp until it coincides with the position of the target plate TPL, presses the joystick 586, and then presses the third preset button 584 of the switch operator 58. When the third preset button 584 is pressed, the controller 40 transitions to the third optical axis deviation measurement mode. However, the controller 40 can also automatically transition to the third optical axis deviation measurement mode after the input of the first positional deviation amount x2 and the second positional deviation amount y2 to the controller 40 in the second optical axis deviation measurement mode is completed.
[0085] In the third optical axis shift measurement mode, the controller 40 sets the image capture magnification zi of the camera 55 to a predetermined capture magnification, and captures an image including the target plate TPL with the camera 55. In this case, the capture magnification zi set in the third optical axis shift measurement mode is set to a capture magnification z3, which is different from the capture magnifications z1 and z2. In this embodiment, the capture magnification z3 is set to a magnification greater than the capture magnification z2. Furthermore, the captured image PT captured in the third optical axis shift measurement mode is a captured image PT3. The controller 40 can display the capture magnification z3 of the image when the captured image PT3 is captured on the captured image PT3.
[0086] 13, the controller 40 draws a center CDp of the danger area circle CD in the captured image PT3. The center CDp of the danger area circle CD drawn in the captured image PT3 is a mark indicating the position of the optical axis OA of the lens 552 in the captured image PT3. Note that in the captured image PT3, the danger area circle CD itself is not drawn and only the center CDp is drawn, but it is also possible to draw the danger area circle CD in addition to the center CDp.
[0087] In the captured image PT3, the position of the target plate TPL is misaligned with the position of the center CDp. The amount of misalignment between the position of the target plate TPL and the position of the center CDp in the left-right direction in the captured image PT3 is a first misalignment amount x3, and the amount of misalignment between the position of the target plate TPL and the position of the center CDp in the up-down direction is a second misalignment amount y3.
[0088] The center CDp drawn in the captured image PT3 can be moved by operating the joystick 586 of the switch operator 58. The operator operates the joystick 586 to move the center CDp to the target plate TPL, and presses the joystick 586 after the position of the center CDp and the position of the target plate TPL are aligned.
[0089] By operating the joystick 586, the center CDp is moved until it coincides with the position of the target plate TPL, and then the joystick 586 is pressed down, whereby the first positional deviation amount x3 and the second positional deviation amount y3 on the captured image PT3 are measured and input to the controller 40.
[0090] However, the first positional deviation amount x3 and the second positional deviation amount y3 may be automatically acquired by the information processing unit 42 when the controller 40 draws the center CDp, and input to the information storage unit 41. The first positional deviation amount x3 and the second positional deviation amount y3 input to the controller 40 can be displayed on the captured image PT3.
[0091] In the third optical axis deviation amount measurement mode, too, the display magnification of the captured image PT3 can be changed by digital zooming, for example, by operating the zoom button 585 of the switch operation device 58.
[0092] The controller 40 determines the optical axis deviation angle Ψ of the lens 552 of the camera 55 based on the input first deviation amount x3 and the positional relationship between the camera 55 and the target plate TPL. 3 Calculate.
[0093] Specifically, when the captured image PT3 is displayed so that the target plate TPL is positioned on an extension of the imaging direction PD of the camera 55, the first optical axis deviation amount Lx3 of the lens 552 at a position a distance H away from the camera 55 is calculated based on the first positional deviation amount x3 on the captured image PT3. Furthermore, the first optical axis deviation amount Lx3 and the distance H are used to calculate the first optical axis deviation angle Ψ of the lens 552 in the left-right direction according to equation (1). 3 In the formula (1), i is a natural number, and the first optical axis deviation angle Ψ 3 When calculating i=3.
[0094] Furthermore, the controller 40 calculates the optical axis deviation angle θ of the lens 552 of the camera 55 based on the input second deviation amount y3 and the positional relationship between the camera 55 and the target plate TPL. 3 Calculate.
[0095] Specifically, when the captured image PT3 is displayed so that the target plate TPL is positioned on an extension of the imaging direction PD of the camera 55, the second optical axis deviation amount Ly3 of the lens 552 at a position a distance H away from the camera 55 is calculated based on the second positional deviation amount y3 on the captured image PT2. Furthermore, the second optical axis deviation amount Ly3 and the distance H are used to calculate the second optical axis deviation angle θ of the lens 552 in the up-down direction according to equation (2). 3 In the formula (2), i is a natural number, and the second optical axis deviation angle θ 3 When calculating i=3.
[0096] In each of the first, second, and third optical axis deviation measurement modes, the controller 40 calculates the first optical axis deviation angle Ψ corresponding to each optical axis deviation measurement mode at the timing when the first positional deviation amount xi and the second positional deviation amount yi are input to the controller 40. 1 , Ψ 2 , Ψ 3 , and the second optical axis deviation angle θ 1 , θ 2 , θ 3 In addition, in a third optical axis deviation measurement mode after the first optical axis deviation measurement mode and the second optical axis deviation measurement mode are performed, after the first positional deviation amount xi and the second positional deviation amount yi are input to the controller 40, the controller 40 can calculate all of the first optical axis deviation angles Ψ 1 , Ψ 2 , Ψ 3 , and the second optical axis deviation angle θ 1 , θ 2 , θ 3 may be calculated.
[0097] The controller 40 calculates the first optical axis deviation angle Ψ for the captured images PT1, PT2, and PT3 captured at the capture magnifications z1, z2, and z3, respectively. 1 , Ψ 2 , Ψ 3 , and the second optical axis deviation angle θ 1 , θ 2 , θ 3 After calculating the first optical axis deviation angle Ψ 1 , Ψ 2 , Ψ 3 , and the second optical axis deviation angle θ1 , θ 2 , θ 3 Based on this, the position of the center CDp of the danger area circle CD drawn in the captured image PT is corrected.
[0098] Specifically, the controller 40 calculates the first optical axis deviation angle Ψ 1 and the first optical axis deviation angle Ψ 1 the capture magnification z1 corresponding to the calculated first optical axis deviation angle Ψ 2 and the first optical axis deviation angle Ψ 2 and the calculated first optical axis deviation angle Ψ 3 and the first optical axis deviation angle Ψ 3 The acquisition magnification z3 corresponding to each of the above is substituted into equation (3).
[0099]
[0100] In equation (3), i is a natural number, k is the first unknown, and Ψ 0 The first unknown k is a component of the optical axis deviation angle that varies depending on the capture magnification zi, and the second unknown Ψ 0 is a value indicating the optical axis deviation angle that is constant regardless of the capture magnification zi.
[0101] The controller 40 also calculates the second optical axis deviation angle θ 1 and the second optical axis deviation angle θ 1 The corresponding capture magnification z1, the calculated second optical axis deviation angle θ 2 and the second optical axis deviation angle θ 2 The corresponding capture magnification z2, the calculated second optical axis deviation angle θ 3 and the second optical axis deviation angle θ 3 The acquisition magnification z3 corresponding to each of the above is substituted into equation (4).
[0102]
[0103] In equation (4), i is a natural number, m is the third unknown, and θ is the fourth unknown. The third unknown m is a component of the optical axis deviation angle that varies depending on the capture magnification zi, and the fourth unknown θ 0 is a value indicating the optical axis deviation angle that is constant regardless of the capture magnification zi.
[0104] The controller 40 adds the first optical axis deviation angle Ψ to the equation (3). 1 and the capture magnification z1 are substituted into Equation (5), and the first optical axis deviation angle Ψ is then calculated in Equation (3). 2 and the capture magnification z2 are substituted into Equation (6), and the first optical axis deviation angle Ψ is then added to Equation (3). 3 and the capture magnification z3 are substituted to obtain Equation (7).
[0105]
[0106]
[0107]
[0108] The controller 40 calculates the first unknown k and the second unknown Ψ by the least squares method using Equation 5, Equation 6, and Equation 7, which are Equation 3 corresponding to the different acquisition magnifications z1, z2, and z3. 0 In this case, the first unknown k and the second unknown Ψ are calculated by the least squares method. 0 The first unknown k and the second unknown Ψ are obtained by calculating 0 The error can be minimized.
[0109] The controller 40 calculates the first unknown k and the second unknown Ψ 0 After calculating the first unknown k and the second unknown Ψ 0 By substituting into Equation (3), the first optical axis deviation angle Ψ corresponding to an arbitrary capture magnification zi is obtained. i That is, the controller 40 can calculate the first unknown k and the second unknown Ψ 0 By further substituting an arbitrary capture magnification zi into equation (3) into which is substituted, the first optical axis deviation angle Ψ corresponding to the substituted arbitrary capture magnification zi is obtained. i can be calculated.
[0110] Furthermore, the controller 40 adds the second optical axis deviation angle θ 1 and the capture magnification z1 are substituted to obtain Equation (8), and the second optical axis deviation angle θ 2 and the capture magnification z2 are substituted to obtain Equation (9), and the second optical axis deviation angle θ 3and the acquisition magnification z3 are substituted to obtain the formula (10).
[0111]
[0112]
[0113]
[0114] Furthermore, the controller 40 uses the formulas 8, 9, and 10, which are formulas 4 corresponding to the different acquisition magnifications z1, z2, and z3, to calculate the unknowns m and θ0 by the least squares method. In this case, the third unknown m and the fourth unknown θ are calculated by the least squares method. 0 By finding the third unknown m and the fourth unknown θ 0 The error can be minimized.
[0115] The controller 40 calculates the third unknown m and the fourth unknown θ 0 After finding the third unknown m and the fourth unknown θ 0 By substituting into equation (4), the second optical axis deviation angle θ corresponding to an arbitrary capture magnification zi can be obtained. i That is, the controller 40 can calculate the third unknown m and the fourth unknown θ 0 By further substituting an arbitrary capture magnification zi into equation (4) into which is substituted, the second optical axis deviation angle θ corresponding to the substituted arbitrary capture magnification zi is obtained. i can be calculated.
[0116] The controller 40 calculates the first unknown k and the second unknown Ψ. 0 , the third unknown m, and the fourth unknown θ 0 After obtaining the above, when the danger area circle CD is drawn on the captured image PT captured at a predetermined capture magnification zi, the first optical axis deviation angle Ψ corresponding to the capture magnification zi of the captured image PT to be displayed is i and the second optical axis deviation angle θ i The first unknown k and the second unknown Ψ 0 Equation (3) into which is substituted, and the third unknown m and the fourth unknown θ 0 is substituted into equation (4), and the calculated first optical axis deviation angle Ψ i and the second optical axis deviation angle θi The optical axis deviation angles Ψ and θ of the optical axis OA of the camera 55 are corrected using the above equation.
[0117] In this case, the optical axis deviation angles Ψ, θ of the optical axis OA are corrected by shifting the optical axis OA in the left-right direction of the captured image PT to the side opposite to the direction in which the optical axis OA is shifted with respect to the imaging direction PD, using the calculated first optical axis deviation angle Ψ i and in the up-down direction of the captured image PT, the optical axis OA is moved by the calculated second optical axis deviation angle θi in the direction opposite to the direction of deviation with respect to the imaging direction PD. In this way, the controller 40 corrects the optical axis deviation angles Ψ, θ of the optical axis OA on the captured image PT.
[0118] As shown in Figure 14, when the optical axis deviation angles Ψ and θ of the optical axis OA are corrected and then the danger area circle CD is drawn on the captured image PT, the position of the center CDp of the danger area circle CD coincides with the position of the target plate TPL.
[0119] The controller 40 compares the position of the center CDp of the danger area circle CD displayed in the captured image PT with the position of the target plate TPL, and if the amount of deviation between the position of the center CDp of the danger area circle CD and the position of the target plate TPL is within a preset threshold range, it determines that the position of the center CDp of the danger area circle CD and the position of the target plate TPL are consistent, and terminates the optical axis deviation angle correction mode.
[0120] If the amount of deviation between the position of the center CDp of the danger area circle CD and the position of the target plate TPL exceeds the range of a preset threshold, the controller 40 may not have properly corrected the optical axis deviation angles Ψ and θ, and therefore executes the first optical axis deviation amount measurement mode, the second optical axis deviation amount measurement mode, and the third optical axis deviation amount measurement mode again to correct the first optical axis deviation angle Ψ 1 , Ψ 2 , Ψ 3 , and the second optical axis deviation angle θ 1 , θ 2 , θ 3 can be recalculated, and the optical axis deviation angles Ψ and θ of the optical axis OA can be corrected again.
[0121] As shown in FIG. 15 , after the optical axis deviation angles Ψ and θ of the optical axis OA are corrected, it becomes possible to draw the danger area circle CD in the captured image PT captured at an arbitrary capture magnification zi at a position where the center CDp of the danger area circle CD coincides with the ground contact point T of the hook 33, and to draw the performance line CR with high accuracy.
[0122] In this embodiment, the first optical axis deviation angle Ψ is calculated for the captured images PT1, PT2, and PT3 captured at three different capture magnifications z1, z2, and z3. 1 , Ψ 2 , Ψ 3 , and the second optical axis deviation angle θ 1 , θ 2 , θ 3 and corrects the optical axis deviation angles Ψ and θ of the optical axis OA. i and the second optical axis deviation angle θ i It is also possible to calculate the first optical axis deviation angle Ψ and θ of the optical axis OA and correct the first optical axis deviation angle Ψ i and the second optical axis deviation angle θ i By increasing the number of types of capture magnifications zi of the captured image PT for calculating the angle Ψ, θ, the accuracy of correction of the optical axis deviation angles can be improved.
[0123] As described above, in the optical axis deviation angle correction device 7 provided in the crane 1, the controller 40 draws, on the captured image PT, the center CDp of the danger area circle CD, which is a mark indicating the position of the optical axis OA of the lens 552. Furthermore, by inputting the first positional deviation amounts xi and second positional deviation amounts yi between the center CDp of the danger area circle CD on the captured image PT and the target plate TPL indicating the ground contact point T of the hook 33, whose positional relationship with the camera 55 is known, the controller 40 calculates the optical axis deviation angles Ψ and θ of the lens 552 in the camera 55 using the positional relationship between the camera 55 and the target plate TPL and the first positional deviation amounts xi and second positional deviation amounts yi. Furthermore, the controller 40 corrects the position of the center CDp of the danger area circle CD drawn on the captured image PT based on the calculated optical axis deviation angles Ψ and θ.
[0124] That is, in the optical axis deviation angle correction device 7, the optical axis deviation angles Ψ and θ of the lens 552 are calculated using the first positional deviation amount xi and the second positional deviation amount yi between the center CDp of the danger area circle CD on the captured image PT and the target plate TPL, and the position of the center CDp of the danger area circle CD drawn on the captured image PT is corrected, so that the optical axis deviation angles Ψ and θ of the lens 552 in the camera 55 can be corrected with a simple and inexpensive configuration.
[0125] In this case, by correcting the optical axis deviation angles Ψ and θ of the lens 552 in the captured image PT that captures a video including the hook 33, the performance line CR and danger zone circle CD that indicate the range of a predetermined moment load rate in the crane 1 can be drawn with high accuracy on the captured image PT.
[0126] In particular, the controller 40 of the optical axis deviation angle correction device 7 calculates the first optical axis deviation amount Lxi (Lx1, Lx2, Lx3) of the lens 552 in the left-right direction based on first positional deviation amounts x1, x2, x3 in the left-right direction between the target plate TPL in the captured image PT and the center CDp of the danger area circle CD when the captured image PT is displayed so that the target plate TPL is located on an extension of the imaging direction PD of the camera 55. Furthermore, the controller 40 calculates the first optical axis deviation angle Ψ of the lens 552 in the left-right direction using the first optical axis deviation amount Lxi and the distance H according to equation (1). i Furthermore, the controller 40 calculates the second optical axis deviation amount Lyi (Ly1, Ly2, Ly3) of the lens 552 in the vertical direction based on the second positional deviation amounts y1, y2, y3 in the vertical direction between the target plate TPL and the center CDp of the danger area circle CD on the captured image PT. Then, the controller 40 calculates the second optical axis deviation angle θ of the lens 552 in the vertical direction using the second optical axis deviation amount Lyi and the distance H according to equation (2). i Calculate.
[0127] In this way, in the optical axis deviation angle correcting device 7, the optical axis deviation angles Ψ and θ of the lens 552 are set as follows: i and a second optical axis deviation angle θ in the vertical direction. i Therefore, the optical axis deviation angles Ψ and θ can be corrected with high precision.
[0128] Furthermore, the controller 40 of the optical axis deviation angle correcting device 7 calculates the first optical axis deviation angle Ψ for each of the captured images PT captured at three or more different capture magnifications zi. i Furthermore, the controller 40 calculates the calculated first optical axis deviation angle Ψ i and the first optical axis deviation angle Ψ i The first unknown k and the second unknown Ψ in equation (3) are substituted into equation (3) to obtain 0 The controller 40 calculates the second optical axis deviation angle θ for each of the captured images PT captured at the three or more capture magnifications zi by the least squares method using the formula (3). i The controller 40 also calculates the calculated second optical axis deviation angle θ i and the second optical axis deviation angle θ i The third unknown m and the fourth unknown θ in equation (4) are substituted into equation (4) to obtain 0 is calculated by the least squares method using equation (4) corresponding to three or more types of capture magnifications zi.
[0129] As a result, the first unknown k and the second unknown Ψ are obtained. 0 , the third unknown m, and the fourth unknown θ 0 The error can be minimized.
[0130] The controller 40 of the optical axis deviation angle correcting device 7 calculates the first unknown k and the second unknown Ψ 0 , and an arbitrary capture magnification zi are substituted into Equation (3), the first optical axis deviation angle Ψ corresponding to the substituted arbitrary capture magnification zi is obtained. i Furthermore, the controller 40 calculates the third unknown m and the fourth unknown Ψ 0 , and an arbitrary capture magnification zi are substituted into Equation (4), the second optical axis deviation angle θ corresponding to the substituted arbitrary capture magnification zi is obtained. i Calculate.
[0131] As a result, the first optical axis deviation angle Ψ corresponding to an arbitrary capture magnification zi is i and the second optical axis deviation angle θ ican be calculated with high accuracy.
[0132] Furthermore, by correcting the optical axis deviation angles Ψ and θ of the lens 552 using the optical axis deviation angle correction device 7 and aligning the ground contact point T of the hook 33 on the captured image PT with the center CDp of the danger area circle CD, the accuracy of depicting the danger area circle CD is improved. This makes it possible to more accurately grasp the position where the hook 33 will descend and the position where the load W suspended from the hook 33 will fall. As a result, it is possible to accurately alert workers around the crane 1, further improving safety.
[0133] Furthermore, the controller 40 is capable of changing the display radius of the danger area circle CD, so that the display radius of the danger area circle CD can be changed to match the size of the suspended load.
[0134] Furthermore, although commercially available cameras 55 have variations in the axial direction of the optical axis OA, by correcting the optical axis deviation angles Ψ and θ using the optical axis deviation angle correction device 7, it is possible to draw the performance line CR and the danger zone circle CD on the captured image PT with high accuracy even when using a commercially available camera 55.
[0135] Furthermore, by attaching the camera 55 to the boom 31 of the crane 1 and providing the crane 1 with the optical axis deviation angle correction device 7, and by using the hook 33 or the ground contact point T of the hook 33 as a target whose positional relationship with the camera 55 is known, the optical axis deviation angles Ψ and θ can be measured at low cost and in a short time.
[0136] In this embodiment, various operations in the optical axis deviation angle correction mode are performed by operating the switch operation device 58, but it is also possible to configure the operation to be performed on the display 65 which is configured as a touch panel.
[0137] Furthermore, in each optical axis deviation measurement mode, the controller 40 can display the capture magnification zi of the camera 55 on the captured image PT1, making it easy to detect a malfunction of the zoom function of the camera 55.
[0138] Furthermore, in this embodiment, the danger area circle CD is drawn at a height based on the ground surface S, but it may also be drawn at any height position of the hook 33. When the danger area circle CD is drawn at the height position of the hook 33, the danger area circle CD can be displayed even in a high place such as the rooftop of a building, and therefore the danger area circle CD can be used as a positioning device when unloading the luggage W.
[0139] In addition, by predicting the trajectory of the luggage W from the position of the danger area circle CD and drawing it on the captured image PT1, it is possible to check whether there are people or obstacles in the destination of the luggage W. Furthermore, by adding a sensor capable of detecting people and obstacles to the crane 1, it is possible to detect the presence of a person or obstacle within the danger area circle CD and to notify the user that a person or obstacle has entered the danger area.
[0140] The disclosures of the specification, drawings, and abstract contained in Japanese Patent Application No. 2023-82537, filed on May 18, 2023, are incorporated herein by reference in their entirety.
[0141] The present invention can be applied to various types of cranes.
[0142] 1 Crane 4 Control system 7 Optical axis deviation angle correction device 31 Boom 32 Wire rope 33 Hook 40 Controller 55 Camera 552 Lens CD Danger area circle CDp Center of (danger area circle) H Distance k First unknown Lxi First optical axis deviation amount Lyi Second optical axis deviation amount m Third unknown OA Optical axis PD Imaging direction PT Imaging image T Ground contact point Tg Target TPL Target plate xi First position deviation amount yi Second position deviation amount zi Capture magnification Ψ, θ Optical axis deviation angle Ψ0 Second unknown Ψi First optical axis deviation angle θ0 Fourth unknown θi Second optical axis deviation angle W Baggage
Claims
1. An optical axis deviation angle correction device comprising: a camera having a lens that captures an image including a target whose position is known; and a controller that draws a mark indicating the position of the optical axis of the lens on the image captured by the camera, calculates a deviation angle of the optical axis of the lens with respect to the imaging direction of the camera based on the positional relationship between the mark and the target in the captured image, and corrects the position of the mark in the captured image based on the deviation angle of the optical axis so that the mark overlaps the target.
2. The optical axis deviation angle correction device according to claim 1, wherein the controller calculates the optical axis deviation angle based on the amount of positional deviation between the mark and the target in the captured image and the positional relationship between the camera and the target.
3. The controller calculates a first optical axis shift amount Lxi of the lens with respect to the target in the first direction at a position away from the camera by a distance H from the center of the lens in the imaging direction of the camera to the target, based on a first position shift amount in a first direction between the target and the mark on the captured image when the captured image is displayed so that the target is located on an extension line of the imaging direction of the camera, and calculates a first optical axis shift angle Ψ of the lens in the first direction by using the first optical axis shift amount Lxi and the distance H, according to Formula (1). i a second optical axis shift amount Lyi of the lens with respect to the target in the second direction at a position away from the camera by the distance H based on a second position shift amount in a second direction perpendicular to the first direction between the target and the mark on the captured image when the captured image is displayed so that the target is located on an extension line of the imaging direction of the camera, and a second optical axis shift angle θ of the lens in the second direction at a position away from the camera by the distance H is calculated by using the second optical axis shift amount Lyi and the distance H according to Equation (2). i The optical axis deviation angle correcting device according to claim 2 , further comprising:
4. The camera captures images including the target at three or more different capture magnifications zi, and the controller calculates the first optical axis deviation angle Ψ for each captured image captured at each capture magnification zi. i and calculating the calculated first optical axis deviation angle Ψ i and the first optical axis deviation angle Ψ i The first unknown k and the second unknown Ψ in the formula (3) are substituted into the formula (3) to obtain 0 is calculated by the least squares method using the formula (3) corresponding to three or more kinds of acquisition magnifications zi, and the second optical axis deviation angle θ is calculated for each captured image taken at each acquisition magnification zi. i The calculated second optical axis deviation angle θ i and the second optical axis deviation angle θ i The third unknown m and the fourth unknown θ in the formula (4) are substituted into the formula (4) 0 4. The optical axis deviation angle correcting device according to claim 3, wherein: is determined by the least squares method using equation (4) corresponding to three or more types of acquisition magnifications zi.
5. The controller calculates the first unknown k and the second unknown Ψ 0 , and the arbitrary capture magnification zi are substituted into Equation (3), whereby the first optical axis deviation angle Ψ corresponding to the arbitrary capture magnification zi substituted is obtained. i The third unknown m and the fourth unknown Ψ are calculated. 0 , and the arbitrary capture magnification zi is substituted into Equation (4), whereby the second optical axis deviation angle θ corresponding to the arbitrary capture magnification zi is obtained. i The optical axis deviation angle correcting device according to claim 4 , further comprising:
6. A crane equipped with the optical axis deviation angle correction device according to claim 1.
7. A crane as claimed in claim 6, comprising a hook supported by a wire rope hanging down from a boom, the camera of said optical axis deviation angle correction device being mounted on said boom, and said target being said hook.
8. The crane described in claim 7, wherein a controller of the optical axis deviation angle correction device stores the positional relationship of the hook with respect to the camera, draws a mark indicating the position of the optical axis of the lens on an image captured by the camera, measures the amount of positional deviation between the mark and the hook on the captured image, calculates a deviation angle of the optical axis of the lens in the camera using the positional relationship between the camera and the hook and the amount of positional deviation, and corrects the position of the mark drawn on the captured image based on the calculated deviation angle of the optical axis.
9. The crane according to claim 8, wherein the controller draws a danger area circle indicating an area of the ground where a load suspended from the hook will touch if the load falls vertically, at a position where the center of the danger area circle coincides with the optical axis position of the lens.
10. A method for correcting an optical axis deviation angle, comprising the steps of: capturing an image including a target whose position is known using a camera having a lens; drawing a mark indicating the position of the optical axis of the lens on the image captured by the camera; calculating a deviation angle of the optical axis of the lens with respect to the imaging direction of the camera based on the positional relationship between the mark and the target in the captured image; and correcting the position of the mark in the captured image so that it overlaps with the target based on the deviation angle of the optical axis.
11. The optical axis deviation angle correction method according to claim 10, wherein in the step of calculating the optical axis deviation angle, a positional deviation amount between the mark and the target on the captured image is measured as the positional relationship between the mark and the target, and the optical axis deviation angle is calculated using the positional relationship between the camera and the target and the positional deviation amount.
Citation Information
Patent Citations
Imaging apparatus and method
JP2008311804A
Door opening / closing mechanism for aircraft lavatory unit
JP2023082537A
On-vehicle image pickup unit
JP2002074339A
Network camera system
JP2009302764A
Multi-lens camera and control method therefor
JP2011071604A
