Crane device

Abstract

[Problem] To provide a crane device that makes it possible to eliminate blind spots regardless of the form or specification of a device or makes it possible to eliminate blind spots according to a request from an operator. [Solution] A crane device 12 comprises a safety device 17. The safety device 17 includes a wide angle camera 18, a display 84 which is provided on a cabin, and a controller 70. The wide angle camera 18 is provided on the upper surface of a boom and captures images in a range including a region around the tip end part of the boom, a region facing the upper surface of the boom, side regions of the boom, and a rear region of the cabin. The controller 70 determines the type of a monitoring image that is displayed on the display 84, on the basis of first designation information which is stored in a memory 72 according to the specifications and second designation information which indicates the designation of a screen by an operator. The controller 70 then generates monitoring image data that indicates the monitoring image of the determined type on the basis of captured image data which has been output from the wide angle camera 18.

Description

Crane equipment

[0001] The present invention relates to a crane apparatus having a boom that can be rotated, raised, and extended.

[0002] Patent Document 1 discloses a crane apparatus including a swivel base and a boom mounted on the swivel base. The boom is capable of raising and lowering and extending. A sensor is provided at the tip of the boom as a safety device. The sensor detects obstacles approaching the tip of the boom.

[0003] WO2022-092093 publication

[0004] Crane equipment comes in a variety of types, including mobile types such as rough terrain cranes and all-terrain cranes, as well as fixed types such as tower cranes. Even for the same type of crane equipment, the size and working radius vary depending on the specifications (model), resulting in different blind spots for the operator during operation. For example, with one model, the blind spot is the area above the boom, while with another model, the blind spot is the area around the tip of the boom. In some cases, both of these areas may be blind spots. Thus, the blind spots that need to be monitored on the boom vary depending on the model of crane equipment.

[0005] To ensure work safety, measures are required that allow the operator to monitor the above-mentioned areas and eliminate blind spots. However, it is difficult to monitor blind spots using the above-mentioned sensors, which vary depending on the specifications of the crane apparatus. Moreover, the above-mentioned areas (blind spots that need to be eliminated) also vary depending on the wishes of the operator who actually operates the boom.

[0006] The present invention has been made in light of the above background, and its object is to provide a crane apparatus that can eliminate blind spots regardless of the type or specifications of the apparatus, or that can eliminate blind spots in response to the operator's request.

[0007] (1) A crane apparatus according to the present invention includes a swivel base, a cabin mounted on the swivel base and equipped with a control device, a telescopic boom provided on the swivel base so as to be able to raise and lower, and a safety device having a wide-angle camera attached to an upper surface of the boom when the boom is in a lowered position, a controller, and a display. The controller executes a generation process for generating surveillance image data based on image data generated by the wide-angle camera, and a display process for displaying a surveillance image represented by the surveillance image data on the display.

[0008] The wide-angle camera is attached to the top of the boom, so it can simultaneously capture images of the area around the boom tip, the side areas of the boom, the area facing the top of the boom, and the rear area around the base of the boom. The controller generates monitoring image data from the captured image data for an area indicated by settings pre-stored in memory or an area specified by the operator (generation process), inputs the monitoring image data to the display, and displays the monitoring image on the display (display process). Therefore, the safety device can display a monitoring image of a blind spot on the display, whether it is an area around the boom tip, a side area of ​​the boom, an area facing the top of the boom, or a rear area around the base of the boom, which is determined by the model or specifications or desired by the operator. Therefore, the crane apparatus of the present invention can eliminate blind spots regardless of the model or specifications, or can eliminate blind spots according to the operator's request.

[0009] (2) The controller may further execute a designation information acquisition process for acquiring designation information. The generation process is one or more processes indicated by the designation information among a first process of generating boom-tip monitoring image data by correcting the captured image data based on a first correction standard corresponding to the tip of the boom shown in the captured image indicated by the captured image data, a second process of generating boom-side monitoring image data by correcting the captured image data based on a predetermined second correction standard for an area shown in the captured image indicated by the captured image data that is to the side of the boom, a third process of generating boom-top-side monitoring image data by correcting the captured image data based on a predetermined third correction standard for an area shown in the captured image indicated by the captured image data that faces the top of the boom, and a fourth process of generating rearward monitoring image data by correcting the captured image data based on a fourth correction standard corresponding to the base end of the boom shown in the captured image indicated by the captured image data.

[0010] The designation information is, for example, stored in advance in a memory included in the controller, or input by the operator to the controller. The controller reads the designation information from the memory, or receives and acquires the designation information input by the operator. The controller executes one of the first to fourth processes, designated by the designation information, as a generation process to generate monitoring image data. Therefore, an appropriate monitoring image according to the type and specifications of the crane apparatus is displayed on the display, or an appropriate monitoring image designated by the operator is displayed on the display.

[0011] (3) The crane apparatus according to the present invention may further include a boom hoisting angle sensor that outputs a first detection value indicating a boom hoisting angle. The fourth process includes a process of correcting the captured image data based on the fourth correction standard and a conversion process of converting a viewpoint based on the boom hoisting angle indicated by the first detection value. The rear monitoring image indicated by the rear monitoring image data is an overhead image showing the area behind the cabin.

[0012] The position and height of the wide-angle camera relative to the swivel base change as the boom is raised or lowered. The conversion process to create a bird's-eye view image is performed based on the boom hoisting angle, which is a parameter that determines the position and height of the wide-angle camera. Therefore, regardless of the boom hoisting angle, a rear monitoring image that overlooks the area behind the cabin from a fixed position is always displayed on the display.

[0013] (4) The crane apparatus according to the present invention may further include a boom hoisting angle sensor that outputs a first detection value indicating a boom hoisting angle. The second process further includes a process of correcting the captured image data based on the second correction standard, and a rotation process of rotating a boom side monitoring image, which is a monitoring image represented by the boom side monitoring image data, by a rotation angle corresponding to the boom hoisting angle indicated by the first detection value.

[0014] The boom side monitoring image tilts depending on the boom hoisting angle. The controller rotates the boom side monitoring image depending on the boom hoisting angle to correct the tilt of the boom side monitoring image. Therefore, the crane apparatus according to the present invention makes it easier for the operator to recognize obstacles compared to a case where the tilt of the boom side monitoring image is not corrected.

[0015] (5) The crane apparatus according to the present invention may further include a boom hoisting angle sensor that outputs a first detection value indicating a boom hoisting angle. The controller executes the third process when the designation information indicates that the third process is to be executed, and executes a first enlargement process to enlarge the boom top-side monitoring image indicated by the boom top-side monitoring image data generated in the third process before executing the display process when it determines that the boom is being raised based on a change over time in the boom hoisting angle indicated by the first detection value.

[0016] When the boom is being raised, there is a risk of contact between the top of the boom and an obstacle. The controller enlarges and displays the boom top side monitoring image while the boom is being raised. Therefore, the crane apparatus according to the present invention makes it easier for the operator to recognize obstacles compared to when the boom top side monitoring image is always displayed at a constant size on the display.

[0017] (6) The crane apparatus according to the present invention may further include a swing angle sensor that outputs a second detection value indicating a swing angle of the swing base. The boom side monitoring image data includes boom left side monitoring image data indicating the left side of the boom and boom right side monitoring image data indicating the right side of the boom. The controller executes the second process based on the designation information indicating execution of the second process, executes a second enlargement process to enlarge the boom left side monitoring image indicated by the boom left side monitoring image data generated in the second process before executing the display process based on the determination of a left swing based on the second detection value, and executes a third enlargement process to enlarge the boom right side monitoring image indicated by the boom right side monitoring image data generated in the second process before executing the display process based on the determination of a right swing based on the second detection value.

[0018] When the boom is being rotated, there is a risk of contact between the side of the boom and an obstacle. The controller enlarges and displays the boom left side monitoring image when the boom is being rotated left, and enlarges and displays the boom right side monitoring image when the boom is being rotated right. Therefore, the crane apparatus according to the present invention makes it easier for the operator to recognize obstacles compared to when the boom side monitoring image is always displayed at a constant size on the display.

[0019] (7) The boom may have a plurality of cylindrical bodies including a base boom, and the wide-angle camera may be provided on the base boom.

[0020] Since the wide-angle camera is mounted on the base boom, whose position does not change with the extension and contraction of the boom, wired connection between the wide-angle camera and the controller is easier than when the wide-angle camera is mounted on another cylindrical body whose position changes with the extension and contraction of the boom.

[0021] (8) The crane apparatus according to the present invention may further include a boom length sensor that outputs a third detection value indicating an extended length of the boom. The boom has multiple cylindrical bodies including a base boom. The wide-angle camera is provided on the base boom. The first process includes a process of correcting the captured image data based on the first correction standard and a fourth enlargement process of enlarging the captured image data by a magnification factor corresponding to the third detection value.

[0022] If the wide-angle camera is mounted on the base boom, the boom tip moves away from the wide-angle camera as the boom extends. The boom tip monitoring image, which is the monitoring image shown by the boom tip monitoring image data, is enlarged by a magnification factor that corresponds to the extended length of the boom, making it easier for the operator to recognize obstacles approaching the boom tip.

[0023] (9) The wide-angle camera may have a field of view of 180 degrees or more.

[0024] A wide-angle camera with a field of view of 180 degrees or more is used.

[0025] The crane apparatus according to the present invention can eliminate blind spots regardless of the type or specifications of the apparatus, or can eliminate blind spots according to the operator's request.

[0026] FIG. 1 is a functional block diagram of the crane apparatus 12. FIG. 2 is a schematic side view of the crane vehicle 10 with the boom 22 raised. FIG. 3 is a schematic side view of the crane vehicle 10 with the boom 22 in a lowered position. FIG. 4 is a diagram showing the control device 29 inside the cabin 13. FIG. 5 is a flowchart of the monitoring image display process. FIG. 6(A) is a schematic diagram of an image captured by the wide-angle camera 18, FIG. 6(B) is a diagram showing a rear monitoring image, FIG. 6(C) is a diagram showing a left-side monitoring image of the boom, FIG. 6(D) is a diagram showing a right-side monitoring image of the boom, FIG. 6(E) is a diagram showing a boom tip monitoring image, and FIG. 6(F) is a diagram showing a boom top monitoring image.

[0027] An embodiment of the present invention will be described below. Note that the embodiment described below is merely one example of the present invention, and it goes without saying that the embodiment can be modified as appropriate without departing from the spirit of the present invention. For example, the execution order of each process described below can be modified as appropriate without departing from the spirit of the present invention. Alternatively, part of the process described below can be omitted as appropriate without departing from the spirit of the present invention.

[0028] Fig. 1 is a functional block diagram of the crane apparatus 12. Fig. 2 is a side view of the mobile crane 10 with the boom 22 in an upright position. Fig. 3 is a schematic side view of the mobile crane 10 with the boom 22 in a lowered position.

[0029] 2 and 3, the mobile crane 10 is a rough terrain crane. However, the mobile crane 10 may also be an all-terrain crane.

[0030] The crane vehicle 10 includes a running body 11, an outrigger device 14 attached to the running body 11, and a crane device 12 and a cabin 13 mounted on the running body 11. Note that the outrigger device 14 is not shown in Figure 3.

[0031] The outrigger device 14 has a plurality of jacks 24 that contact the ground to stabilize the position of the mobile crane 10. The crane device 12 is used with the jacks 24 extended and the position of the mobile crane 10 stabilized.

[0032] The crane apparatus 12 has a swivel base 21, a boom 22, a winch 23, a group of sensors 26 (see Figure 1), a group of hydraulic actuators 27 (see Figure 1), a hydraulic supply device 28 (see Figure 1), a control device 29 (see Figure 1), a hook block 32, and a safety device 17 (see Figure 1).

[0033] The swivel base 21 is rotatably held by the traveling body 11. The rotation axis of the swivel base 21 is aligned in the vertical direction (up-down direction) and is located approximately in the center of the traveling body 11.

[0034] The boom 22 is supported by the swivel base 21 so as to be able to rise and fall. That is, the boom 22 is able to rise and fall and to swing.

[0035] The boom 22 includes a base boom 42, an intermediate boom 43, and a top boom 44. The base boom 42, the intermediate boom 43, and the top boom 44 are each shaped like a rectangular tube. The intermediate boom 43 is positioned inside the base boom 42, and the top boom 44 is positioned inside the intermediate boom 43. In other words, the base boom 42, the intermediate boom 43, and the top boom 44 are nested within each other, forming a so-called telescopic structure. The boom 22 extends and retracts as the intermediate boom 43 slides relative to the base boom 42 and the top boom 44 slides relative to the intermediate boom 43. The base boom 42, the intermediate boom 43, and the top boom 44 correspond to the "cylinder" described in the claims. The boom 22 may be composed of two cylinders or four or more cylinders.

[0036] The base boom 42 is located on the outermost side. Therefore, the outer surface of the base boom 42 is always exposed to the outside. The rectangular cylindrical base boom 42 has an upper surface 45 that becomes the upper surface when the boom 22 is in the horizontally laid position (see FIG. 3 ). A wide-angle camera 18, which will be described later, is provided on the upper surface 45.

[0037] The boom 22 is supported by the swivel base 21 at the base boom 42 so that it can be raised and lowered. In other words, the base boom 42 does not move (slide) when the boom 22 is extended or retracted. In other words, the position of the wide-angle camera 18 attached to the base boom 42 does not change when the boom 22 is extended or retracted.

[0038] The winch 23 is attached to the base end of the boom 22 or to the swivel base 21. The winch 23 has a drum 56 around which a wire rope 41 (hereinafter referred to as "wire 41") is wound, and a sheave 57 around which the wire 41 is reeled. When the winch 23 is driven, the wire 41 is wound onto the drum 56 or is unwound from the drum 56. Note that the winch 23, wire 41, sheave 57, drum 56, etc. are not shown in FIG. 3 .

[0039] The wire 41 is reeled around a sheave 57 provided at the base end of the boom 22, a sheave 58 provided at the tip of the boom 22, and a pulley device (not shown). The pulley device has a plurality of first sheaves (not shown) provided at the tip of the boom 22 and a plurality of second sheaves (not shown) provided on the hook block 32. Note that the hook block 32 is not shown in Figure 3.

[0040] The hook block 32 comprises a hook body 33, a plurality of the second sheaves rotatably held within the hook body 33, and a hook 34 provided on the underside of the hook body 33.

[0041] Two winches 23 may be provided on the crane apparatus 12. A hook block 32, which is a so-called main hook, is connected to a wire 41 of one winch 23, and a sub-hook (not shown) is connected to the wire 41 of the other winch 23.

[0042] As shown in FIG. 1 , the hydraulic actuator group 27 includes a swing motor 51 , a boom cylinder 52 , a telescopic cylinder 53 , a hydraulic motor 54 , and the jack 24 .

[0043] The swing motor 51 is a hydraulic motor that rotates via hydraulic oil supplied from the hydraulic supply unit 28, and rotates the swing base 21. The hoisting cylinder 52 is a hydraulic cylinder that extends and retracts via hydraulic oil supplied from the hydraulic supply unit 28, and raises and lowers the boom 22. The telescopic cylinder 53 is a hydraulic cylinder that extends and retracts via hydraulic oil supplied from the hydraulic supply unit 28, and extends and lowers the boom 22. The hydraulic motor 54 rotates via hydraulic oil supplied from the hydraulic supply unit 28, and rotates the drum 56 of the winch 23.

[0044] The hydraulic supply device 28 includes a hydraulic pump driven by the engine 15 mounted on the traveling vehicle 11, piping connecting the hydraulic pump to the swing motor 51 and other components of the hydraulic actuator group 27, and a hydraulic selector valve and other components provided on the piping. The hydraulic selector valve may be a so-called electromagnetic valve, which is driven by a drive signal input from the controller 70. Driving the electromagnetic valve drives the swing motor 51, the hoisting cylinder 52, the telescopic cylinder 53, the hydraulic motor 54, and the jack 24. In other words, by outputting drive signals, the controller 70 can swing, hoist, and extend the boom 22, reel in or pay out the wire 41, and extend or retract the jack 24.

[0045] The sensor group 26 includes a swing angle sensor 61 , a boom length sensor 62 , and a hoisting angle sensor 63 .

[0046] The swing angle sensor 61 outputs a detection value corresponding to the swing angle of the swivel base 21 from a swing reference position. The swing reference position is the position shown in FIG. 3 where the boom 22 protrudes forward. The swing angle sensor 61 is, for example, a rotary encoder provided on the swing shaft of the swivel base 21. The swing angle sensor 61 outputs, as a detection value, a number of pulse signals corresponding to the swing angle of the swivel base 21. In the following, the swing angle will be described as an angle measured counterclockwise from the swing reference position. In other words, the swing angle increases when the boom 22 swings left and decreases when the boom 22 swings right. The detection value output by the swing angle sensor 61 corresponds to the "second detection value" recited in the claims.

[0047] The boom length sensor 62 outputs a detection value corresponding to the length of the boom 22. The detection value output by the boom length sensor 62 may indicate the length from the base end to the tip end of the boom 22, or may indicate the extended length of the boom 22. The boom length sensor 62 may be a sensor that outputs a detection value that directly indicates the boom length, a sensor that outputs a detection value that indicates the extended length of the telescopic cylinder 53, or a sensor that outputs a detection value that indicates the operating time of the telescopic cylinder 53. The detection value output by the boom length sensor 62 corresponds to the "third detection value" recited in the claims.

[0048] The hoisting angle sensor 63 outputs a detection value corresponding to the hoisting angle of the boom 22. The hoisting angle sensor 63 is, for example, an inclination sensor or a horizontal sensor that outputs the angle relative to the horizontal plane. Alternatively, the hoisting angle sensor 63 may be a sensor that outputs a detection value indicating the extension length or operating time of the hoisting cylinder 52. The detection value output by the hoisting angle sensor 63 corresponds to the "first detection value" recited in the claims.

[0049] In the following description, the detection values ​​output by the swing angle sensor 61, the boom length sensor 62, and the hoisting angle sensor 63 are also referred to as "sensor detection values."

[0050] The swing angle sensor 61, boom length sensor 62, and derrick angle sensor 63 are connected to the controller 70 by cables. That is, the sensor detection values ​​output by the swing angle sensor 61 and the like are input to the controller 70. The sensor detection values ​​output by each sensor may also be input to the controller 70 via wireless communication.

[0051] Each sensor such as the turning angle sensor 61 performs detection continuously at predetermined time intervals (so-called sampling period / sampling cycle) and outputs a sensor detection value at the predetermined time intervals.

[0052] FIG. 4 is a diagram showing the control device 29 inside the cabin 13.

[0053] 4, the control device 29 is disposed in the cabin 13. The control device 29 includes an operation lever, a foot pedal, an operation button, and the like, which are operated by the operator.

[0054] The control device 29 is connected to a controller 70 (see FIG. 1 ) via a signal line (not shown). An operator operates the control device 29 to input instructions to the controller 70 and control the crane apparatus 12.

[0055] As shown in Fig. 4, the control monitoring device 80 is installed in the cabin 13. As shown in Fig. 1, the control monitoring device 80 includes a display 81, a transparent sheet-like touch sensor 82 overlaid on the display 81, and a speaker 83. In other words, the control monitoring device 80 is a so-called AML.

[0056] The operator inputs (selects) an image designation using the touch sensor 82. The image designation is information indicating one or more images from the rear monitoring image shown in Fig. 6(B), the boom left side monitoring image shown in Fig. 6(C), the boom right side monitoring image shown in Fig. 6(D), the boom tip monitoring image shown in Fig. 6(E), and the boom top side monitoring image shown in Fig. 6(F). The operator inputs designation of one or more images to be displayed on the display 84 from the above five images using the touch sensor 82. The input image designation is stored in the memory 72 (see Fig. 1) as second designation information.

[0057] As shown in FIG. 1, the safety device 17 includes a wide-angle camera 18, a display 84, and a controller 70.

[0058] The wide-angle camera 18 includes a condenser lens, an optical system, a plurality of image sensors, a control circuit, and a communication interface. The condenser lens is a so-called wide-angle lens or a fisheye lens, and the wide-angle camera 18 is a so-called wide-angle camera or a fisheye camera.

[0059] The optical system has, for example, multiple lenses, and guides light focused by a focusing lens to an imaging element. The multiple imaging elements are image sensors such as CCD or CMOS. The multiple imaging elements are arranged in a matrix (grid). The multiple imaging elements output image data according to the incident light. The image data consists of multiple pixel data and pixel position data. If the captured image is a color image, one pixel data consists of, for example, three color pixel data. The control circuit captures images using the wide-angle camera 18 and generates image data. The communication interface converts the image data into transmission data and outputs it. The control circuit is, for example, a driver IC, and the communication interface is, for example, a communication IC.

[0060] 2 and 3 , the wide-angle camera 18 is attached to the upper surface 45 of the base boom 42. The wide-angle camera 18 is disposed at a distance from the base end of the base boom 42. In other words, the wide-angle camera 18 is located at a position higher than the swivel base 21 when the boom 22 is raised (crane operation state). Therefore, the rear monitoring image (see FIG. 6(B) ) generated from the image captured by the wide-angle camera 18 captures a wider area behind the cabin 13. In other words, by attaching the wide-angle camera 18 to the upper surface 45 of the boom 22, it is possible to display a bird's-eye view image of a wider area behind the cabin 13 than when the wide-angle camera 18 is attached to the swivel base 21.

[0061] 2 and 3, the wide-angle camera 18 is positioned closer to the tip end than the center of the base boom 42 so that a rear monitoring image (see FIG. 6B) that captures a wider range can be generated. However, the wide-angle camera 18 may also be positioned at the center of the base boom 42, or closer to the base end than the center.

[0062] Wide-angle camera 18 may be a monochrome camera that outputs image data representing a monochrome image, or a color camera that outputs image data representing a color image. Furthermore, the image data output by wide-angle camera 18 may be data representing a still image or video data consisting of multiple frames.

[0063] 2 and 3 indicate the imaging range (angle of view) of wide-angle camera 18. As shown in Figures 2 and 3, the mounting position of wide-angle camera 18 and the angle of view of wide-angle camera 18 are determined so that the imaging range of wide-angle camera 18 includes the tip of boom 22, the left and right sides of boom 22, the area facing top surface 45 of boom 22, and the area behind cabin 13. In other words, the imaging range of wide-angle camera 18 includes all of the areas around the tip of boom 22, the area on the left side of boom 22, the area on the right side of boom 22, the area facing top surface 45 of boom 22, and the area behind cabin 13.

[0064] The angle of view of the wide-angle camera 18 is set to be 180 degrees or more. In the example shown in Figures 2 and 3, the angle of view of the wide-angle camera 18 is 220 degrees. Note that the angle of view of a wide-angle camera is generally up to 220 degrees. In other words, the angle of view of the wide-angle camera 18 is set in the range of 180 degrees or more and less than 220 degrees.

[0065] 1 , the wide-angle camera 18 and the controller 70 are connected by a cable 19. Captured image data output by the wide-angle camera 18 is input to the controller 70 via the cable 19. However, the captured image data may also be input to the controller 70 via wireless communication. In this case, a battery and a transmitting antenna are attached to the boom 22, and a receiving antenna connected to the controller 70 via a communication line is installed in the cabin 13. The wide-angle camera 18 captures images using power supplied from the battery and transmits the captured image data from the transmitting antenna. The transmitted captured image data is received by the receiving antenna and input to the controller 70.

[0066] The wide-angle camera 18 continuously captures images at predetermined time intervals and outputs captured image data at the predetermined time intervals, or continuously outputs video data consisting of a plurality of frames.

[0067] As shown in FIG. 4 , the display 84 is installed in the cabin 13. As shown in FIG. 1 , the display 84 is connected to the controller 70 by a signal line. The display 84 displays images represented by image data input from the controller 70. Specifically, the display 84 displays a rear monitoring image shown in FIG. 6(B), a boom left side monitoring image shown in FIG. 6(C), a boom right side monitoring image shown in FIG. 6(D), a boom tip monitoring image shown in FIG. 6(E), and a boom top side monitoring image shown in FIG. 6(F). The display 84 corresponds to the "display" recited in the claims. Note that the above five images may be displayed on the display 81 of the control monitor device 80. In that case, the display 81 corresponds to the "display" recited in the claims.

[0068] 1 , the controller 70 includes a CPU 71, which is a central processing unit, a memory 72, a power supply circuit 74, and a communication bus (not shown). The controller 70 is realized by, for example, an IC, a computer, resistors, diodes, coils, capacitors, etc. mounted on a control board. The control board is disposed, for example, in a control box disposed in the cabin 13.

[0069] The CPU 71, memory 72, hydraulic supply device 28, swing angle sensor 61 and the like of the sensor group 26, wide-angle camera 18, and control device 29 are connected to a communication bus (not shown). A control program 75 (described below) executed by the CPU 71 reads data and information from memory 72, stores the data and information in memory 72, and controls the operation of the hydraulic actuator group 27. The control program 75 also acquires sensor detection values ​​output by the sensor group 26, captured image data output by the wide-angle camera 18, and inputs made by the operator to the control monitor device 80 and the control device 29.

[0070] The memory 72 stores in advance an OS 77, which is an operating system, and a control program 75 executed by the CPU 71. The memory 72 also stores in advance first designation information, and stores the second designation information indicating an image designation input by the operator via the control monitor device 80. The first designation information corresponds to the "designation information" set forth in the claims. The second designation information corresponds to the "designation information" set forth in the claims. The first designation information and the second designation information correspond to the "designation information" set forth in the claims.

[0071] The first designation information, like the second designation information, is information indicating one or more of the five images. The developer of the crane apparatus 12 stores in the memory 72 the first designation information according to specifications determined by the model, type, destination, etc. of the crane apparatus 12.

[0072] The memory 72 also stores in advance a first correction function, a second correction function, a third correction function, a fourth correction function, a fifth correction function, a first conversion function, a second conversion function, and a third conversion function.

[0073] Fig. 6(A) is a schematic diagram of an image captured by the wide-angle camera 18. Fig. 6(B) is a schematic diagram showing a rear monitoring image, which is an image showing the area behind the cabin 13. Fig. 6(C) is a schematic diagram of a boom left side monitoring image showing the left side of the boom 22. Fig. 6(D) is a schematic diagram of a boom right side monitoring image showing the right side of the boom 22. Fig. 6(E) is a schematic diagram of a boom tip monitoring image showing the periphery of the tip of the boom 22. Fig. 6(F) is a schematic diagram of a boom top side monitoring image showing the area facing the top surface 45 of the boom 22.

[0074] The first correction function, the second correction function, the third correction function, the fourth correction function, the fifth correction function, the first conversion function, the second conversion function, and the third conversion function will be described with reference to FIG.

[0075] The first correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate corrected image data that shows the area behind the cabin 13. Specifically, the first correction function is a function that discards data of an area that is part of the captured image shown by the captured image data and that is other than the area behind the cabin 13, and corrects distortion. The first correction function discards data that shows an area other than the area around the fourth correction standard (see FIG. 6A ), that is, other than the area surrounded by the two-dot chain line in FIG. 6A , and corrects distortion.

[0076] The first correction function receives the captured image data and the hoisting angle detected by the hoisting angle sensor 63 as arguments, and outputs corrected image data as a return value. More specifically, the height of the wide-angle camera 18 changes depending on the hoisting angle of the boom 22. When the height of the wide-angle camera 18 changes, the range captured in the captured image represented by the captured image data changes. The first correction function determines the pixels in the area to be discarded based on the hoisting angle received. In other words, the range captured in the image represented by the generated corrected image data is a fixed range, regardless of the hoisting angle of the boom 22.

[0077] The first transformation function is a function that changes the viewpoint in the image represented by the corrected image data. The first transformation function is, for example, a function that performs projective transformation. The first transformation function receives the corrected image data and the boom hoist angle detected by the boom hoist angle sensor 63 as arguments, and outputs rear monitoring image data. The rear monitoring image data is an image showing the rear of the cabin 13 and is also a bird's-eye view image. More specifically, the position (viewpoint) of the wide-angle camera 18 changes depending on the boom hoist angle of the boom 22. The first transformation function changes the viewpoint based on the received boom hoist angle. In other words, the rear monitoring image represented by the generated rear monitoring image data is a bird's-eye view image from the same viewpoint regardless of the boom hoist angle of the boom 22.

[0078] The second correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom-tip monitoring image data that shows the periphery of the tip of the boom 22. Specifically, the second correction function is a function that discards data of a region that is part of the captured image indicated by the captured image data and that is not the region surrounding the tip of the boom 22 indicated by the first correction reference (see FIG. 6A), and corrects distortion. The second correction function corrects distortion by discarding data that indicates a region other than the periphery of the first correction reference (see FIG. 6A), that is, a region other than the region surrounded by the dotted line in FIG. 6A. The second correction function receives the captured image data as an argument and outputs the boom-tip monitoring image data as a return value.

[0079] The second transformation function is a function for enlarging an image. The second transformation function receives image data and an enlargement ratio as arguments, and outputs image data representing the enlarged image. The second transformation function is, for example, an affine transformation (affine matrix) that performs the enlargement.

[0080] The third correction function is a function for generating boom top-side monitoring image data by correcting the captured image data output by the wide-angle camera 18. The boom top-side monitoring image data is data representing a boom top-side monitoring image that shows the area facing the top surface 45 of the boom 22.

[0081] The third correction function is a function that discards data of a portion of the captured image represented by the captured image data other than the area facing the top surface 45 of the boom 22 and corrects distortion. Specifically, the third correction function discards data indicating an area other than the area determined by the third correction standard (see FIG. 6A ), i.e., an area other than the area surrounded by the dashed-dotted line in FIG. 6A , and corrects distortion. The third correction function receives the captured image data as an argument and outputs the boom top-side monitoring image data as a return value. The boom top-side monitoring image represented by the boom top-side monitoring image data (see FIG. 6F ) is composed of an image taken from the wide-angle camera 18 toward the base end of the boom 22 and an image taken from the wide-angle camera 18 toward the tip of the boom 22. The boom top-side monitoring image may be a single image synthesized from an image taken from the wide-angle camera 18 toward the base end of the boom 22 and an image taken from the wide-angle camera 18 toward the tip of the boom 22.

[0082] The fourth correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom left side monitoring image data that shows the area to the left of the boom 22. Specifically, the fourth correction function is a function that discards data of a portion of the captured image indicated by the captured image data other than the area to the left of the boom 22 indicated by the left second correction standard (see FIG. 6A ), i.e., other than the area to the left of the dotted line in FIG. 6A , and corrects distortion. The fourth correction function receives the captured image data as an argument and outputs boom left side monitoring image data as a return value. The left second correction standard corresponds to the "second correction standard" recited in the claims.

[0083] The fifth correction function is a function for correcting the captured image data output by the wide-angle camera 18 to generate boom right-side monitoring image data that shows the area to the right of the boom 22. Specifically, the fifth correction function is a function that discards data of a portion of the captured image indicated by the captured image data other than the area to the right of the boom 22 indicated by the right second correction reference (see FIG. 6A ), i.e., other than the area to the right of the dotted line in FIG. 6A , and corrects distortion. The fifth correction function receives the captured image data as an argument and outputs boom right-side monitoring image data as a return value. The right second correction reference corresponds to the "second correction reference" recited in the claims. The left second correction reference and the right second correction reference correspond to the "second correction reference" recited in the claims.

[0084] The third transformation function is a function that rotates an image represented by the image data. The third transformation function receives image data and a rotation angle as arguments, and outputs image data representing the rotated image as a return value. The third transformation function is, for example, an affine transformation (affine matrix) that performs rotation.

[0085] In Figure 6 (A), the area used to generate the monitoring image is shown in a rectangular shape or the like using dotted lines, dotted lines, dashed lines, and broken lines, but the area used to generate the monitoring image may be in a shape other than a rectangle.

[0086] The control program 75 shown in FIG. 1 is a program that controls the driving of the hydraulic actuator group 27 and the like based on operation signals input from the control device 29, controls the display on the display 81 of the control monitor device 80, accepts input from the operator through the control monitor device 80, and executes the monitoring image display process (see FIG. 5) described below.

[0087] The power supply circuit 74 converts the DC voltage supplied from the battery 16 mounted on the running object 11 into a DC voltage of a predetermined voltage value, such as 3.3 V or 5 V, and outputs the converted DC voltage. The power supply circuit 74 is, for example, a DC / DC converter (power supply IC) such as a switching regulator. The battery 16 is charged by the engine 15 of the running object 11. The DC voltage output by the power supply circuit 74 is supplied to the CPU 71, the control device 29, the sensor group 26, the wide-angle camera 18, etc. In FIG. 1 , power supply lines from the power supply circuit 74 to the sensor group 26, etc. are omitted from the illustration.

[0088] The following describes the processing executed by the controller 70 of the crane truck 10. The processing executed by the controller 70 is processing that the control program 75 causes the CPU 71 to execute.

[0089] First, a process will be described in which the controller 70 acquires second designation information, which is an image designation by the operator, through the control monitor device 80 and stores the acquired information in the memory 72. The controller 70 acquires first designation information previously stored in the memory 72. The controller 70 displays the type of image indicated by the first designation information on the display 81. The operator inputs, through the touch sensor 82, the type of image that the operator desires to display on the display 84 from among the displayed types of images. For example, if the first designation information indicates the above-mentioned five types of images, the operator's options are all of the above five types of images. However, of the above-mentioned five types of images, an image that is always displayed on the display 84 without any choice by the operator may be predetermined. In this case, information indicating the type of image that is always displayed on the display 84 is included in the first designation information. Furthermore, the image is excluded from the operator's options.

[0090] The controller 70 stores in the memory 72 second designation information indicating the type of image designated by the operator using the control monitor device 80 .

[0091] FIG. 5 is a flowchart of the monitoring image display process executed by the controller 70.

[0092] The monitoring image display process will be described with reference to Fig. 5. The controller 70 starts the monitoring image display process, for example, when the control device 29 is powered on, or when the operator makes a predetermined input to the control device 29 or the control monitor device 80.

[0093] The controller 70 acquires the first designation information and the second designation information stored in the memory 72 (S11). Note that if the operator can select all of the image types indicated by the first designation information, the controller 70 may acquire only the second designation information indicating the image type designated (selected) by the operator. Furthermore, if the first designation information includes information indicating the image type that is always displayed on the display 84, the controller 70 acquires both the first designation information and the second designation information. Furthermore, if the operator cannot designate (select) an image, the controller 70 acquires only the first designation information. Hereinafter, the first designation information and the second designation information will be collectively referred to as "designation information." The processing of step S11 corresponds to a "designation information acquisition process" set forth in the claims.

[0094] The controller 70 acquires the sensor detection values ​​output by each sensor of the sensor group 26 and stores them in the memory 72 (S12). The controller 70 repeatedly acquires the sensor detection values ​​at a predetermined sampling period and stores them in the memory 72. The controller 70 also acquires captured image data output by the wide-angle camera 18 (S13).

[0095] The controller 70 determines whether the designation information acquired in step S11 indicates the display of a "rear monitoring image" (S14). That is, in step S14, it is determined whether the driver has designated (selected) the display of a "rear monitoring image" or whether the "rear monitoring image" is an image that is always displayed.

[0096] If the controller 70 determines that the designation information indicates display of a "rear monitoring image" (S14: Yes), it generates rear monitoring image data based on the sensor detection value acquired in step S12, the captured image data acquired in step S13, and the first correction function and first conversion function stored in memory 72 (S15). More specifically, the controller 70 calls the first correction function, passes the captured image data and the hoisting angle θ detected by the hoisting angle sensor 63 as arguments to the first correction function, and receives the corrected image data as a return value. The controller 70 passes the received corrected image data and the hoisting angle θ as arguments to the first conversion function, and receives the rear monitoring image data as a return value. The image represented by the rear monitoring image data is an overhead image that captures a predetermined area around the rear of the cabin 13. The processing in step S15 corresponds to the "generation processing" and "fourth processing" recited in the claims. The process of acquiring the corrected image data based on the first correction function, the captured image data, and the hoisting angle θ corresponds to the "process of correcting the captured image data based on the fourth correction standard" described in the claims. The process of acquiring the rear monitoring image data based on the corrected image data, the hoisting angle θ, and the first conversion function corresponds to the "conversion process" described in the claims.

[0097] If the controller 70 determines that the designation information does not indicate the display of a "rear monitoring image" (S14: No), the controller 70 skips the process of step S15.

[0098] After executing the process of step S15, or when it is determined that the designation information does not indicate the display of a "rear monitoring image" (S14: No), the controller 70 determines whether or not the designation information acquired in step S11 indicates the display of a "boom-tip monitoring image" (S16). That is, in step S16, it is determined whether or not the operator has designated (selected) the display of a "boom-tip monitoring image", or whether or not the "boom-tip monitoring image" is an image that is always displayed.

[0099] If the controller 70 determines that the designation information acquired in step S11 indicates display of a "boom-tip monitoring image" (S16: Yes), it acquires a magnification factor based on the boom length indicated by the sensor detection value acquired in step S12 (S17). Specifically, the memory 72 stores in advance a table associating boom lengths with magnification factors. The controller 70 reads out the magnification factor associated with the boom length indicated by the sensor detection value from the table.

[0100] The controller 70 generates boom-tip monitor image data based on the captured image data acquired in step S13, the magnification factor acquired in step S17, and the second correction function and second conversion function stored in the memory 72 (S18). More specifically, the controller 70 calls the second correction function, passes the captured image data as an argument to the second correction function, and acquires the pre-magnification boom-tip monitor image data as a return value. Next, the controller 70 calls the second conversion function that performs the magnification, passes the pre-magnification boom-tip monitor image data and the magnification factor as arguments to the second conversion function, and acquires the boom-tip monitor image data showing the magnified boom-tip monitor image as a return value. In other words, the boom-tip monitor image showing the area around the tip of the boom 22 is enlarged by a magnification factor corresponding to the distance from the wide-angle camera 18 to the tip of the boom 22 and displayed on the display 84. The processing of step S18 corresponds to the "generation processing" and "first processing" recited in the claims. In step S18, the process of acquiring pre-magnification boom tip monitoring image data based on the second correction function and the captured image data corresponds to "processing of correcting the captured image data based on the first correction standard" as recited in the claims. In step S18, the process of acquiring boom tip monitoring image data based on the second conversion function, pre-magnification boom tip monitoring image data, and the magnification ratio corresponds to "fourth magnification process" as recited in the claims.

[0101] If the controller 70 determines that the designation information acquired in step S11 does not indicate the display of a "boom tip monitoring image" (S16: No), it skips the processes of steps S17 and S18.

[0102] After determining in step S16 that the designation information does not indicate the display of a "boom-tip monitoring image," or after executing the processing of step S18, the controller 70 determines whether or not the designation information acquired in step S11 indicates the display of a "boom upper surface monitoring image" (S19). That is, in step S19, it is determined whether or not the operator has designated (selected) the display of a "boom upper surface monitoring image," or whether or not the "boom upper surface monitoring image" is an image that is always displayed.

[0103] When the controller 70 determines that the designation information acquired in step S11 indicates display of a "boom top side monitoring image" (S19: Yes), it generates boom top side monitoring image data based on the captured image data acquired in step S13 and the third correction function stored in memory 72 (S20). More specifically, the controller 70 calls the third correction function, passes the captured image data to the third correction function as an argument, and acquires the boom top side monitoring image data as a return value. The processing of step S20 corresponds to the "generation processing" and "third processing" recited in the claims.

[0104] Next, the controller 70 determines whether the boom 22 is being raised or not based on the sensor detection value acquired in step S12 (S21). More specifically, the controller 70 determines whether the hoisting angle θ is increasing over time based on the multiple hoisting angles θ detected by the hoisting angle sensor 63 at a predetermined sampling period.

[0105] When the controller 70 determines that the boom 22 is being raised (S21: Yes), it executes a first enlargement process to enlarge the boom top-side monitoring image represented by the boom top-side monitoring image data generated in step S20 (S22). More specifically, the controller 70 calls a second conversion function for enlargement and reads a predetermined enlargement factor stored in advance from the memory 72. The controller 70 passes the boom top-side monitoring image data generated in step S20 and the predetermined enlargement factor as arguments to the second conversion function, and obtains boom top-side monitoring image data representing the enlarged boom top-side monitoring image as a return value. The process of step S22 corresponds to the "first enlargement process" recited in the claims.

[0106] If the controller 70 determines that the boom 22 is not being raised (S21: No), it skips the processing of step S22. That is, when the boom 22 is not being raised, a boom upper surface monitoring image of the specified size is displayed on the display 84, and when the boom 22 is being raised, a boom upper surface monitoring image of a size larger than the specified size is displayed on the display 84. It is while the boom 22 is being raised that there is a possibility that the top surface 45 of the boom 22 may come into contact with an obstacle. That is, the controller 70 enlarges and displays the boom upper surface monitoring image while the boom 22 is being raised, when there is a possibility that the top surface 45 of the boom 22 may come into contact with an obstacle.

[0107] If the controller 70 determines that the designation information acquired in step S11 does not indicate the display of the "boom upper surface monitoring image" (S19: No), it skips the processing of steps S20, S21, and S22.

[0108] After determining in step S19 that the designation information does not indicate the display of a "boom upper surface monitoring image," or after determining in step S21 that the boom 22 is not being raised, or after executing the processing of step S22, the controller 70 determines whether the designation information acquired in step S11 indicates the display of a "boom left side monitoring image" and a "boom right side monitoring image" (S23). That is, in step S23, it is determined whether the operator has designated (selected) the display of the "boom left side monitoring image" and the "boom right side monitoring image," or whether the "boom left side monitoring image" and the "boom right side monitoring image" are images that are always displayed.

[0109] If the controller 70 determines that the designation information acquired in step S11 indicates the display of a "boom left-side monitoring image" and a "boom right-side monitoring image" (S23: Yes), the controller 70 determines a first rotation angle and a second rotation angle based on the boom hoisting angle θ of the boom 22 indicated by the sensor detection value acquired in step S12 (S24). For example, the controller 70 determines the boom hoisting angle θ as the first rotation angle, and determines -θ as the second rotation angle. The first rotation angle is the angle by which the boom left-side monitoring image is rotated, and the second rotation angle is the angle by which the boom right-side monitoring image is rotated.

[0110] The controller 70 generates boom left-side side monitoring image data based on the captured image data acquired in step S13, the fourth correction function and the third conversion function stored in the memory 72, and the first rotation angle determined in step S24 (S25). The controller 70 also generates boom right-side side monitoring image data based on the captured image data acquired in step S13, the fifth correction function and the third conversion function stored in the memory 72, and the second rotation angle determined in step S24 (S25). Note that in the following description, the boom left-side side monitoring image data and the boom right-side side monitoring image data may be collectively referred to as boom side monitoring image data, and the boom left-side side monitoring image and the boom right-side side monitoring image may be collectively referred to as boom side monitoring images.

[0111] The left boom side monitoring image data corresponds to the "boom side monitoring image data" set forth in the claims. The right boom side monitoring image data corresponds to the "boom side monitoring image data" set forth in the claims. The left boom side monitoring image data and the right boom side monitoring image data correspond to the "boom side monitoring image data" set forth in the claims. The left boom side monitoring image corresponds to the "boom side monitoring image" set forth in the claims. The right boom side monitoring image corresponds to the "boom side monitoring image" set forth in the claims. The left boom side monitoring image and the right boom side monitoring image correspond to the "boom side monitoring image" set forth in the claims.

[0112] The processing of step S25 will be described in detail. The controller 70 calls the fourth correction function, passes the captured image data to the fourth correction function as an argument, and obtains the pre-rotation boom left side monitoring image data as a return value. Next, the controller 70 calls the third transformation function that rotates the image, passes the pre-rotation boom left side monitoring image data and the first rotation angle to the third transformation function as arguments, and obtains the boom left side monitoring image data showing the rotated boom left side monitoring image as a return value. The ground shown in the rotated boom left side monitoring image becomes approximately horizontal on the display 84. In other words, the image rotation by the third transformation function is performed to make the ground shown in the boom left side monitoring image appear horizontal on the display 84, eliminating any sense of discomfort felt by the operator. Similarly, the controller 70 calls the fifth correction function, passes the captured image data to the fifth correction function as an argument, and obtains the pre-rotation boom right side monitoring image data as a return value. Next, the controller 70 calls a third transformation function that rotates the image, passes the pre-rotation boom right-side monitoring image data and the second rotation angle as arguments to the third transformation function, and obtains boom right-side monitoring image data showing the rotated boom right-side monitoring image as a return value. The ground shown in the rotated boom right-side monitoring image appears approximately horizontal on the display 84.

[0113] The processing of step S25 corresponds to the "generation processing" and "second processing" recited in the claims. In step S25, the processing of acquiring pre-rotation boom left-side monitoring image data based on the captured image data and the fourth correction function corresponds to the "process of correcting the captured image data based on the second correction standard" recited in the claims. In step S25, the processing of acquiring pre-rotation boom right-side monitoring image data based on the captured image data and the fifth correction function corresponds to the "process of correcting the captured image data based on the second correction standard" recited in the claims. In step S25, the processing of acquiring pre-rotation boom left-side monitoring image data and pre-rotation boom right-side monitoring image data based on the captured image data, the fourth correction function, and the fifth correction function corresponds to the "process of correcting the captured image data based on the second correction standard" recited in the claims. In step S25, the processing of acquiring boom left-side monitoring image data based on pre-rotation boom left-side monitoring image data, the first rotation angle, and the third transformation function corresponds to the "rotation processing" recited in the claims. In step S25, the process of acquiring boom right-side monitoring image data based on the pre-rotation boom right-side monitoring image data, the second rotation angle, and the third transformation function corresponds to the "rotation process" recited in the claims. In step S25, the process of acquiring boom left-side monitoring image data and boom right-side monitoring image data based on the pre-rotation boom left-side monitoring image data, pre-rotation boom right-side monitoring image data, the first rotation angle, the second rotation angle, and the third transformation function corresponds to the "rotation process" recited in the claims.

[0114] Next, the controller 70 determines whether the boom 22 is swinging left based on the sensor detection value acquired in step S12 (S26). More specifically, the controller 70 determines whether the swing angle α detected by the swing angle sensor 61 at a predetermined sampling period is increasing over time. If the swing angle α is increasing, the controller 70 determines that the boom 22 is swinging left.

[0115] When the controller 70 determines that the boom 22 is rotating left (S26: Yes), it executes a second enlargement process (S27) to enlarge the boom left side monitoring image represented by the boom left side monitoring image data generated in step S25. More specifically, the controller 70 calls a second conversion function for enlargement and reads a predetermined enlargement factor stored in advance from the memory 72. The controller 70 passes the boom left side monitoring image data generated in step S25 and the predetermined enlargement factor as arguments to the second conversion function, and obtains boom left side monitoring image data representing the enlarged boom left side monitoring image as a return value. The process of step S27 corresponds to the "second enlargement process" recited in the claims.

[0116] If the controller 70 determines that the boom 22 is not turning left (S26: No), it skips the processing of step S27. That is, when the boom 22 is not turning left, a boom left side monitoring image of the specified size is displayed on the display 84, and when the boom 22 is turning left, a boom left side monitoring image of a size larger than the specified size is displayed on the display 84. It is during the left rotation of the boom 22 that there is a possibility that the left side of the boom 22 will come into contact with an obstacle. That is, the controller 70 enlarges and displays the boom left side monitoring image during the left rotation of the boom 22 when there is a possibility that the left side of the boom 22 will come into contact with an obstacle.

[0117] If the controller 70 determines that the boom 22 is not rotating left (S26: No), or after executing the process of step S27, the controller 70 determines whether the boom 22 is rotating right based on the sensor detection value acquired in step S12 (S28). More specifically, the controller 70 determines whether the rotation angle α detected by the rotation angle sensor 61 at a predetermined sampling period is decreasing over time. If the rotation angle α is decreasing, the controller 70 determines that the boom 22 is rotating right.

[0118] When the controller 70 determines that the boom 22 is turning to the right (S28: Yes), it executes a third enlargement process (S29) to enlarge the boom right-side monitoring image represented by the boom right-side monitoring image data generated in step S25. More specifically, the controller 70 calls a second conversion function for enlargement and reads a predetermined enlargement factor stored in advance from the memory 72. The controller 70 passes the boom right-side monitoring image data generated in step S25 and the predetermined enlargement factor as arguments to the second conversion function, and obtains boom right-side monitoring image data representing the enlarged boom right-side monitoring image as a return value. The process of step S29 corresponds to the "third enlargement process" recited in the claims.

[0119] If the controller 70 determines that the boom 22 is not turning to the right (S28: No), it skips the processing of step S29. That is, when the boom 22 is not turning to the right, a boom right side monitoring image of the specified size is displayed on the display 84, and when the boom 22 is turning to the right, a boom right side monitoring image of a size larger than the specified size is displayed on the display 84. It is during the right rotation of the boom 22 that there is a possibility that the right side of the boom 22 will come into contact with an obstacle. That is, the controller 70 enlarges and displays the boom right side monitoring image during the right rotation of the boom 22, when there is a possibility that the right side of the boom 22 will come into contact with an obstacle.

[0120] If the controller 70 determines that the designation information acquired in step S11 does not indicate the display of the "boom side monitoring image" (S23: No), it skips the processes from step S24 to S29.

[0121] After determining that the designation information does not indicate the display of a "boom side monitoring image" (S23: No), or after determining that a right turn is not being performed (S28: No), or after executing the processing of step S29, the controller 70 inputs the generated image data to the display 84 and causes the monitoring image to be displayed on the display 84 (S30). Specifically, the controller 70 inputs the generated monitoring image data to the display 84. The monitoring image displayed on the display 84 is an image determined to be displayed according to the specifications (first designation information) and an image designated (selected) by the operator. Furthermore, the monitoring image displayed on the display 84 is an image that has been appropriately enlarged in accordance with the extension, contraction, elevation, and rotation of the boom 22. The processing of step S30 corresponds to the "display processing" set forth in the claims.

[0122] Each monitoring image will now be described in detail. The wide-angle camera 18 is attached to the boom 22 and rotates integrally with the boom 22. Therefore, the rear monitoring image shown in FIG. 6(B) always captures the area behind the cabin 13 (see FIG. 2), not the area behind the crane truck 10. In other words, the rear monitoring image always captures the area behind the cabin 13, which is a blind spot for the operator. The rear monitoring image is also a bird's-eye view image that always looks down on the area behind the cabin 13 from the same viewpoint, regardless of whether the boom 22 is rising or falling. The rear monitoring image is also an image that always captures a fixed range, regardless of whether the boom 22 is rising or falling.

[0123] The boom left side monitoring image shown in Fig. 6(C) is an image viewed to the left from the boom 22, and the boom right side monitoring image shown in Fig. 6(D) is an image viewed to the right from the boom 22. The ground shown in the boom left side monitoring image and the ground shown in the boom right side monitoring image are approximately horizontal on the display 84, and obstacles standing vertically from the ground in the boom left side monitoring image and the boom right side monitoring image are approximately aligned in the up-and-down direction on the display 84.

[0124] 6(E) shows the area around the tip of the boom 22. The range shown in the boom tip monitoring image is, for example, a range in which an operator who notices an obstacle in the boom tip monitoring image can stop the crane apparatus 12 before the tip of the boom 22 comes into contact with the obstacle. This range is constant regardless of the extended length of the boom 22 due to the enlargement of the boom tip monitoring image (S17, S18).

[0125] 6(F) is made up of an image of the tip and upward view of the boom 22 taken from the wide-angle camera 18, and an image of the base and upward view of the boom 22 taken from the wide-angle camera 18. In other words, the boom top-side monitoring image is an image that shows the area facing the top surface 45 of the boom 22. The two images may be displayed side by side on the display 84, or may be displayed side by side on the display 84. Alternatively, the two images may be combined and displayed as a single image on the display 84.

[0126] The five monitoring images shown in FIGS. 6(B) to 6(F) may be displayed in any arrangement on the display 84.

[0127] Furthermore, the enlarged boom side monitoring image and boom top side monitoring image may be displayed separately from the boom side monitoring image and boom top side monitoring image before enlargement on the display 84. For example, when the boom 22 is not rotated, the boom side monitoring image before enlargement is displayed on the display 84, and when the boom 22 is rotated, the enlarged boom side monitoring image is displayed on the display 84 together with the boom side monitoring image before enlargement.

[0128] 5, after executing the process of step S30, the controller 70 determines whether or not to terminate the monitoring image display process (S31). The controller 70 determines to terminate the monitoring image display process based on, for example, the power supply to the control device 29 being turned off or based on the input of an end instruction via the control monitor device 80 (S31: Yes, End). If the controller 70 determines not to terminate the monitoring image display process (S31: No), it executes the processes from step S11 onwards again. That is, the controller 70 repeatedly executes the monitoring image display process. The cycle (sampling cycle) at which the controller 70 repeats the monitoring image display process is set to, for example, from several hundred milliseconds to several minutes.

[0129] [Effects of the embodiment]

[0130] Because the wide-angle camera 18 is attached to the top surface 45 of the boom 22, it can simultaneously capture images of the area around the tip of the boom 22, the left and right lateral areas of the boom 22, the area facing the top surface 45 of the boom 22, and the rear area around the base end of the boom 22. The controller 70 generates monitoring image data from the captured image data for the area indicated by the first designation information pre-stored in the memory 72 or the area designated (selected) by the operator, and displays the monitoring image represented by the generated monitoring image data on the display 84. Therefore, the safety device 17 can display a monitoring image of a blind spot or a monitoring area desired by the operator of the crane vehicle 10 on the display 84, regardless of whether the area is the area around the tip of the boom 22, a lateral area of ​​the boom 22, an area facing the top surface 45 of the boom 22, or the rear area around the base end of the boom 22 (the area rearward of the cabin 13). Therefore, the crane apparatus 12 can eliminate blind spots regardless of the type or specifications, or can eliminate blind spots according to the operator's request.

[0131] The safety device 17, which is made up of the wide-angle camera 18, the display 84, and the controller 70, can change the type of image to be displayed on the display 84 based on the first designation information stored in the memory 72. Therefore, the safety device 17 can be commonly used for various crane apparatuses 12. As a result, it is possible to standardize parts.

[0132] The controller 70 reads out second designation information indicating the operator's image designation from the memory 72, and causes the type of monitoring image designated by the second designation information to be displayed on the display 84. Therefore, the operator can display on the display 84 the monitoring image he or she desires.

[0133] The position and height of the wide-angle camera 18 relative to the swivel base 21 change as the boom 22 is raised or lowered. A viewpoint conversion process is performed to obtain a bird's-eye view image based on the boom 22 raising or lowering angle, which is a parameter that determines the position and height of the wide-angle camera 18. Therefore, the crane apparatus 12 can always display on the display 84 a rear monitoring image that provides a bird's-eye view of the area behind the cabin 13 from a fixed position, regardless of the boom 22 raising or lowering angle.

[0134] The boom side monitoring image tilts in accordance with the elevation of the boom 22. The controller 70 corrects the tilt of the boom side monitoring image by rotating the boom side monitoring image in accordance with the elevation angle of the boom 22. Therefore, the crane apparatus 12 allows the operator to easily recognize obstacles compared to a case in which the tilt of the boom side monitoring image is not corrected.

[0135] When the boom 22 is being raised, there is a risk of contact between the upper surface 45 of the boom 22 and an obstacle. When the boom 22 is being raised, the controller 70 enlarges and displays the boom upper surface monitoring image. Therefore, the crane apparatus 12 makes it easier for the operator to recognize obstacles compared to when the boom upper surface monitoring image is always displayed at a constant size on the display 84.

[0136] When the boom 22 is being rotated, there is a risk of contact between the side of the boom 22 and an obstacle. The controller 70 enlarges and displays the boom left side monitoring image when the boom 22 is being rotated left, and enlarges and displays the boom right side monitoring image when the boom 22 is being rotated right. Therefore, the crane apparatus 12 makes it easier for the operator to recognize obstacles compared to when the boom side monitoring image is always displayed at a constant size on the display 84.

[0137] Since the wide-angle camera 18 is mounted on the base boom 42, whose position does not change with the extension and contraction of the boom 22, wired connection between the wide-angle camera 18 and the controller 70 is easier than when the wide-angle camera 18 is mounted on the intermediate boom 43 or top boom 44, whose position changes with the extension and contraction of the boom 22.

[0138] If the wide-angle camera 18 is provided on the base boom 42, as the boom 22 extends, the tip of the boom 22 moves away from the wide-angle camera 18. The boom tip monitoring image is enlarged at a magnification corresponding to the extended length of the boom 22, so the crane apparatus 12 can make it easier for the operator to recognize obstacles approaching the tip of the boom 22.

[0139] [Modification]

[0140] In the above embodiment, an example has been described in which the angle of view of the wide-angle camera 18 is 220 degrees. However, the angle of view of the wide-angle camera 18 may be other angles between 180 degrees and 220 degrees, such as 180 degrees or 200 degrees.

[0141] In the above embodiment, an example has been described in which the crane apparatus 12 is a mobile type (crane vehicle 10). However, the crane apparatus 12 may be a fixed type such as a tower crane.

[0142] In the above-described embodiment, an example has been described in which the first designation information is pre-stored in the memory 72 and the second designation value input by the operator is stored in the memory 72. That is, an example has been described in which the monitoring image to be displayed is predetermined based on the format and specifications, and the operator can select the monitoring image to be displayed. However, the first designation information does not have to be pre-stored in the memory 72, or a function for the operator to input the second designation value does not have to be provided. That is, the operator may simply be able to select the monitoring image to be displayed, or the monitoring image to be displayed may be predetermined based on the format and specifications, and the operator may not be able to select the monitoring image.

[0143] In the above embodiment, an example has been described in which the boom left side monitoring image is enlarged when the boom 22 is being rotated left, and the boom right side monitoring image is enlarged when the boom 22 is being rotated right. However, when the boom 22 is not being rotated, the boom left side monitoring image and the boom right side monitoring image may not be displayed on the display 84, and when the boom 22 is being rotated left, the boom left side monitoring image may be displayed normally or enlarged, and when the boom 22 is being rotated right, the boom right side monitoring image may be displayed normally or enlarged.

[0144] In the above embodiment, an example has been described in which the boom top side monitoring image is enlarged when the boom 22 is raised. However, when the boom 22 is not raised, the boom top side monitoring image may not be displayed on the display 84, and when the boom 22 is raised, the boom top side monitoring image may be displayed in a normal manner or enlarged.

[0145] In the above-described embodiment, an example has been described in which the first correction function for correcting the captured image and the first conversion function for changing the viewpoint are separate functions. However, the first correction function and the first conversion function may be a single function. Similarly, the second correction function and the second conversion function may be a single function, the fourth correction function and the third conversion function may be a single function, or the fifth correction function and the third conversion function may be a single function.

[0146] In the above embodiment, an example of correcting and converting image data using a function has been described. However, as long as image data can be corrected and converted, a class that generates monitoring image data as an instance (object), an arithmetic expression, or the like may also be used.

[0147] DESCRIPTION OF SYMBOLS 10: Crane vehicle 11: Traveling body 12: Crane device 13: Cabin 17: Safety device 18: Wide-angle camera 19: Cable 21: Swing base 22: Boom 23: Winch 26: Sensor group 29: Control device 41: Wire rope 42: Base boom (cylindrical body) 43: Intermediate boom (cylindrical body) 44: Top boom (cylindrical body) 45: Upper surface 51: Swing motor 52: Drawer cylinder 53: Telescopic cylinder 54: Hydraulic motor 61: Swing angle sensor 62: Boom length sensor 63: Drawer angle sensor 70: Controller 71: CPU 72: Memory 75: Control program 80: Control monitor device 81: Display 82: Touch sensor 84: Display

Claims

1. A crane apparatus comprising: a rotating base; a cabin mounted on the rotating base and equipped with a control device; a boom that is rotatable on the rotating base and is extendable and retractable; and a safety device having a wide-angle camera attached to the top surface of the boom when it is in a lowered position, a controller, and a display, wherein the controller executes a generation process for generating surveillance image data based on captured image data generated by the wide-angle camera, and a display process for displaying a surveillance image represented by the surveillance image data on the display.

2. The crane apparatus according to claim 1, wherein the controller further executes a designated information acquisition process to acquire designated information, and the generation process is one or more of the following processes indicated by the designated information: a first process of correcting the captured image data based on a first correction standard corresponding to the tip of the boom shown in the captured image indicated by the captured image data to generate boom-tip monitoring image data; a second process of correcting the captured image data based on a predetermined second correction standard in an area shown in the captured image indicated by the captured image data that is to the side of the boom; a third process of correcting the captured image data based on a predetermined third correction standard in an area shown in the captured image indicated by the captured image data that faces the top of the boom, to generate boom top-side monitoring image data; and a fourth process of correcting the captured image data based on a fourth correction standard corresponding to the base end of the boom shown in the captured image indicated by the captured image data to generate rear monitoring image data.

3. A crane apparatus as described in claim 2, further comprising a boom hoisting angle sensor which outputs a first detection value indicating the boom hoisting angle, wherein the fourth process includes a process of correcting the captured image data based on the fourth correction standard, and a conversion process of converting a viewpoint based on the boom hoisting angle indicated by the first detection value, and wherein the rear monitoring image indicated by the rear monitoring image data is an overhead image showing the rear of the cabin.

4. A crane apparatus as described in claim 2, further comprising a boom hoisting angle sensor which outputs a first detection value indicating the boom hoisting angle, and wherein the second processing further includes: a process of correcting the captured image data based on the second correction standard; and a rotation process of rotating a boom side monitoring image, which is a monitoring image indicated by the boom side monitoring image data, at a rotation angle corresponding to the boom hoisting angle indicated by the first detection value.

5. A crane apparatus as described in claim 2, further comprising a boom hoisting angle sensor which outputs a first detection value indicating the boom hoisting angle, wherein the controller executes the third process based on the designation information indicating execution of the third process, and executes a first enlargement process to enlarge the boom top side monitoring image indicated by the boom top side monitoring image data generated in the third process before executing the display process based on the determination that the boom is being raised based on the change over time in the boom hoisting angle indicated by the first detection value.

6. A crane apparatus as set forth in claim 2, further comprising a rotation angle sensor which outputs a second detection value indicating a rotation angle of the rotating base, wherein the boom side monitoring image data includes boom left side monitoring image data indicating the left side of the boom and boom right side monitoring image data indicating the right side of the boom, and wherein the controller executes the second process based on the designation information indicating execution of the second process, executes a second enlargement process to enlarge the boom left side monitoring image indicated by the boom left side monitoring image data generated in the second process before executing the display process based on the determination of a left rotation based on the second detection value, and executes a third enlargement process to enlarge the boom right side monitoring image indicated by the boom right side monitoring image data generated in the second process before executing the display process based on the determination of a right rotation based on the second detection value.

7. The crane apparatus according to claim 1, wherein the boom has a plurality of cylindrical bodies including a base boom, and the wide-angle camera is provided on the base boom.

8. A crane apparatus as described in claim 2, further comprising a boom length sensor which outputs a third detection value indicating an extended length of the boom, the boom having a plurality of cylindrical bodies including a base boom, the wide-angle camera being provided on the base boom, and the first process includes: a process of correcting the captured image data based on the first correction standard; and a fourth enlargement process of enlarging the captured image data by a magnification factor corresponding to the third detection value.

9. The crane apparatus according to claim 1, wherein the angle of view of the wide-angle camera is 180 degrees or more.

Images