Calibration system of space recognition device for working machine and working machine

Abstract

To provide a calibration system of a space recognition devise for a working machine capable of improving efficiency of time required for the calibration among multiple space recognition devices.SOLUTION: A calibration system of a space recognition devise for a working machine according to one embodiment of the present disclosure comprises a working machine, multiple space recognition devices provided on the working machine and capable of acquiring the surrounding situation of the working machine, and a control device to perform the calibration among the multiple space recognition devices based on the objects existing in the surrounding area of the working machine. The control device is configured to indicate whether or not the objects exist in the position where the calibration can be made.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a calibration system for a spatial recognition device for a work machine, and to a work machine. [Background technology]

[0002] In recent years, there has been a trend to provide work machines with spatial recognition devices (e.g., LiDAR or imaging devices) to monitor the surroundings of the work machine.

[0003] When a spatial recognition device is provided on a work machine, the spatial recognition device needs to be calibrated. For the work machine described in Patent Document 1, a technique is proposed for performing calibration based on the shape of at least a part of the work machine that is included in the detection range. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2023 / 063219 Summary of the Invention [Problem to be solved by the invention]

[0005] Furthermore, when a work machine is provided with multiple spatial recognition devices, it is necessary to perform calibration between the multiple spatial recognition devices. Calibration between the multiple spatial recognition devices is performed based on objects that exist in overlapping ranges among the detection ranges of the multiple spatial recognition devices. Such calibration between the multiple spatial recognition devices can be time-consuming because it is difficult to adjust the position of the object that is the calibration target.

[0006] One aspect of the present invention is to improve the efficiency of the time required for calibration by teaching the position of an object for calibration among a plurality of spatial recognition devices. [Means for solving the problem]

[0007] A calibration system for a spatial recognition device for a work machine according to one embodiment of the present invention comprises a work machine, a plurality of spatial recognition devices provided on the work machine and capable of acquiring the surrounding conditions of the work machine, and a control device that performs calibration between the plurality of spatial recognition devices based on objects present in the vicinity of the work machine, and the control device is configured to instruct each of the plurality of spatial recognition devices whether or not the object is present in a position where calibration can be performed. Effect of the Invention

[0008] According to one aspect of the present invention, it is possible to improve the efficiency of the time required for calibration between a plurality of spatial recognition devices. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a side view showing a crawler crane according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating an example of the configuration of a crawler crane according to a first embodiment. [Diagram 3] 2 is a diagram showing a measurement range of a distance measuring device and an imaging range of an imaging device according to the first embodiment. FIG. [Figure 4] FIG. 2 is a block diagram showing a configuration example of a calibration system for a spatial recognition device provided in the crawler crane according to the first embodiment. [Diagram 5] 4 is a conceptual diagram showing an example of determination by a position determination unit according to the first embodiment. FIG. [Figure 6] FIG. 2 is a diagram showing a concept of calibrating the spatial recognition device of the crawler crane according to the first embodiment. [Figure 7] FIG. 2 is an explanatory diagram showing an example in which an operator is present at a position where calibration can be performed among a plurality of distance measuring devices according to the first embodiment. [Figure 8] 13A to 13C are diagrams showing transitions of screens displayed on a portable communication terminal according to a second embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a screen displayed on a display device by a display control unit according to a modified example of the second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a screen displayed on a display device by a display control unit according to a modified example of the second embodiment. [Figure 11] 13A to 13C are diagrams showing transitions of screens displayed on a portable communication terminal according to a third embodiment. [Figure 12] FIG. 13 is a conceptual diagram showing a method for generating a travel route by an output control unit according to a modified example of the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is an example, not a limitation of the invention, and all features and combinations described in the embodiment are not necessarily essential to the invention. In addition, the same or corresponding components in each drawing are denoted by the same or corresponding reference numerals, and the description may be omitted.

[0011] In the following embodiment of the present invention, an example will be described in which a crane shown below is used as an example of a work machine, but the work machine is not limited to the crane shown below.

[0012] (First embodiment) Hereinafter, an embodiment of the invention will be described with reference to the drawings.

[0013] First, with reference to FIG. 1, an overview of a crawler crane 100 as an example of a work machine will be described.

[0014] The crawler crane 100 comprises a lower running body 1, an upper rotating body 3 (an example of an upper rotating body 3) mounted on the lower running body 1 so as to be freely rotatable via a rotating mechanism 2, an attachment AT, a mast 5, a backstop 6, a hook HK, a counterweight 9, and a cabin 10.

[0015] The lower traveling body 1 includes, for example, a pair of left and right crawlers 1C. The lower traveling body 1 causes the crawler crane 100 to travel by hydraulically driving each of the crawlers 1C by a left traveling hydraulic motor 1ML and a right traveling hydraulic motor 1MR (see FIG. 2).

[0016] The upper rotating body 3 rotates relative to the lower traveling body 1 as the rotating mechanism 2 is hydraulically driven by a rotating hydraulic motor 2M (see FIG. 2).

[0017] The attachment AT (an example of a working device) includes a boom 4 and a main hoisting rope 7.

[0018] The boom 4 is movably attached to the center of the front part of the upper rotating body 3. A main hoisting rope 7 hangs down from the tip of the boom 4, and a hook HK is attached to the tip of the main hoisting rope 7. That is, the hook HK is attached to the tip of the boom 4 via the main hoisting rope 7.

[0019] The main hoisting rope 7 has its base end attached to a main hoisting winch 7a that is attached to the rear portion between the base end and tip of the boom 4, and its tip end attached to the hook HK. The main hoisting rope 7 is wound up and unwound by the main hoisting winch 7a that is hydraulically driven by a main hoisting hydraulic motor 7M (see FIG. 2), thereby allowing the hook HK to be raised and lowered.

[0020] The mast 5 is attached to the upper rotating body 3 slightly rearward of the base end of the boom 4 so as to be rotatable about a rotation axis parallel to the rotation axis of the boom 4. The tip of the mast 5 is connected to the tip of the boom 4 via a pendant rope 5a, and the boom 4 is raised and lowered via the mast 5 by winding in and out the boom hoist rope 5b using a boom hoist winch 5c that is hydraulically driven by a hoist hydraulic motor 5M (see Figure 2).

[0021] The backstop 6 has a base end attached to a portion of the upper rotating body 3 rearward of the base end of the boom 4 so as to be rotatable about a rotation axis parallel to the rotation axis of the boom 4, and a tip end attached to a rear portion between the base end and tip end of the boom 4 so as to be rotatable about a rotation axis parallel to the rotation axis of the boom 4. The backstop 6 expands and contracts in response to the raising and lowering operation of the boom 4, and has a function of supporting the boom 4 from the rear when the boom 4 is in a substantially upright state, for example.

[0022] The hook HK is attached to the end of the main hoisting rope 7 and is used to suspend a load.

[0023] The counterweight 9 is provided at the rear end of the upper rotating body 3 and has the function of balancing the weight of the boom 4 and the suspended load.

[0024] The cabin 10 is attached, for example, to the right front end portion of the upper rotating body 3. Inside the cabin 10, a cockpit and an operating device 26 (see FIG. 2) for operating various actuators are provided.

[0025] In this embodiment, an example will be described in which an imaging device S6 and a distance measuring device S7 are provided as a plurality of spatial recognition devices capable of acquiring the surrounding conditions of the crawler crane 100.

[0026] The imaging device S6 is attached, for example, to the upper rear end of the counterweight 9, and captures an image of the rear of the crawler crane 100. The imaging device S6 is, for example, a monocular wide-angle camera having a very wide angle of view. The imaging device S6 may also be a stereo camera or a distance imaging camera. The image captured by the imaging device S6 is input to the controller 30.

[0027] The imaging device S6 in this embodiment need only be installed in a position that allows monitoring of the periphery of the crawler crane 100, and the location at which the imaging device S6 is installed is not limited to the upper rear end of the counterweight 9, but may be installed, for example, on the upper left side, upper right side, or upper front part of the counterweight 9.

[0028] The distance measuring device S7 may be any device that is provided around the counterweight 9 and is capable of detecting the distance to an object present around the crawler crane 100. The distance measuring device S7 is, for example, a LiDAR (Light Detection and Ranging).

[0029] The distance measuring device S7, for example, irradiates infrared rays in a certain direction and receives reflected light from objects in that direction to obtain information about objects in the vicinity of the crawler crane 100, specifically, information about the received reflected light (hereinafter, "received light information"). The distance measuring device S7 is, for example, a scanning type LiDAR, and is a three-dimensional laser scanner that can scan the irradiation direction of an infrared laser in the up-down and left-right directions. The distance measuring device S7 may also be a so-called flash type LiDAR that irradiates infrared rays from a light emitting module over a wide three-dimensional range and captures the reflected light (infrared rays) with a three-dimensional distance image element.

[0030] Distance measuring devices S7 (distance measuring device S7B, distance measuring device S7L, and distance measuring device S7R) are attached to the lower rear end, rear left side, and rear right side of the counterweight 9, respectively, and perform detection at the rear, left side, and right side of the crawler crane 100.

[0031] Furthermore, the distance measuring device S7 may be provided on the upper surface of the counterweight 9 or near the front side of the upper rotating body 3 of the crawler crane 100 in order to detect an object present ahead.

[0032] Distance measuring device S7B detects a measurement range behind the counterweight 9, for example, a measurement range in the horizontal direction from the rear left to the rear right (i.e., the circumferential direction as seen from the crawler crane 100). Distance measuring device S7L captures, for example, a measurement range on the left side of the counterweight 9, for example, a measurement range in the horizontal direction from the front left to the rear left of the counterweight 9 (the circumferential direction as seen from the crawler crane 100). Distance measuring device S7R captures, for example, a measurement range on the right side of the counterweight 9, for example, a measurement range in the horizontal direction from the front right to the rear right of the counterweight 9 (the circumferential direction as seen from the crawler crane 100).

[0033] Additionally, each of the distance measuring devices S7 of the crawler crane 100 is attached to the counterweight 9 or the upper part of the upper rotating body 3 with its optical axis facing diagonally downward, and captures an image of a measurement range in the vertical direction including the ground near the crawler crane 100 to the far side of the crawler crane 100. In this manner, the distance measuring devices S7B, S7L, S7R are configured to be able to irradiate infrared rays to the above-mentioned measurement ranges.

[0034] The distance measuring devices S7B, S7L, and S7R have partially overlapping measurement ranges to avoid blind spots. For this reason, the distance measuring devices S7B, S7L, and S7R need to be calibrated with each other to prevent erroneous detection of objects in the overlapping measurement ranges (for example, to prevent the same object from being detected as a different object).

[0035] Similarly, the imaging range of the imaging device S6 overlaps with the measurement range of the distance measuring device S7, so calibration must be performed between the imaging device S6 and the distance measuring device S7.

[0036] In particular, the crawler crane 100 is assembled according to the site. The number of weights stacked on the counterweight 9 of the crawler crane 100 varies depending on the site. For this reason, the crawler crane 100 needs to calibrate the spatial recognition device (imaging device S6 and distance measuring device S7) every time it is assembled on site. For this reason, a method is required that enables an operator to easily perform calibration on site.

[0037] Therefore, in this embodiment, a method for the crawler crane 100 to easily calibrate between multiple spatial recognition devices (including the imaging device S6 and the distance measuring device S7) will be described.

[0038] In this embodiment, an example is described in which the spatial recognition device includes an imaging device S6 and a distance measuring device S7, but this embodiment is not limited to the example in which the imaging device S6 and the distance measuring device S7 are included, and a configuration in which multiple imaging devices S6 or multiple distance measuring devices S7 are provided is also possible. The spatial recognition device may be any device that can acquire the surrounding conditions of the crawler crane 100, and may be any device other than the imaging device S6 and the distance measuring device S7.

[0039] For example, the spatial recognition device may be provided with other object detection devices such as an ultrasonic sensor, a millimeter wave radar, a LIDAR, an infrared sensor, etc. When a millimeter wave radar, an ultrasonic sensor, a laser radar, or the like is used as the spatial recognition device, a number of signals (laser light, etc.) may be transmitted to an object, and the reflected signals may be received to detect the distance and direction of the object from the reflected signals.

[0040] <<Hydraulic drive system>> 2, the hydraulic drive system of the crawler crane 100 according to this embodiment includes hydraulic actuators HA that hydraulically drive each of the driven elements, such as the lower traveling body 1 (left and right crawlers 1C), the upper rotating body 3, and the attachment AT, as described above. The hydraulic drive system of the crawler crane 100 according to this embodiment also includes an engine 11, a regulator 13, a main pump 14, and a control valve unit 17.

[0041] The crawler crane 100 includes, as hydraulic actuators HA, traveling hydraulic motors 1ML, 1MR, a swing hydraulic motor 2M, a hoisting hydraulic motor 5M, and a main hoist hydraulic motor 7M.

[0042] The engine 11 is a prime mover and a main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 is mounted, for example, on the rear of the upper rotating body 3. The engine 11 rotates at a constant speed at a preset target speed under direct or indirect control by a controller 30 (described later), and drives the main pump 14 and the pilot pump 15.

[0043] The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30. For example, the regulator 13 adjusts the angle of the swash plate of the main pump 14 (hereinafter, the “tilt angle”) in response to a control command from the controller 30.

[0044] The main pump 14 supplies hydraulic oil to the control valve unit 17 through a high-pressure hydraulic line. The main pump 14 is mounted, for example, on the rear of the upper rotating body 3, similar to the engine 11. As described above, the main pump 14 is driven by the engine 11. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the controller 30, the tilt angle of the swash plate is adjusted by the regulator 13 to adjust the stroke length of the piston, thereby controlling the discharge flow rate (discharge pressure).

[0045] The control valve unit 17 is a hydraulic control device that controls the hydraulic actuators HA in response to the contents of an operator's operation on the operating device 26 or remote operation, or an operation command related to an automatic operation function output from the controller 30. The control valve unit 17 is mounted, for example, in the center of the upper rotating body 3. As described above, the control valve unit 17 is connected to the main pump 14 via a high-pressure hydraulic line, and selectively supplies hydraulic oil supplied from the main pump 14 to each hydraulic actuator in response to an operator's operation or an operation command output from the controller 30. Specifically, the control valve unit 17 includes a plurality of control valves (also referred to as "directional control valves") that control the flow rate and flow direction of the hydraulic oil supplied from the main pump 14 to each hydraulic actuator HA.

[0046] <<Operation system>> The operation system of the crawler crane 100 according to this embodiment includes a pilot pump 15, an operation device 26, and a proportional valve 31.

[0047] The pilot pump 15 supplies pilot pressure to various hydraulic devices via a pilot line 25. The pilot pump 15 is mounted on the rear of the upper rotating body 3, for example, similar to the engine 11. The pilot pump 15 is, for example, a fixed displacement hydraulic pump, and is driven by the engine 11 as described above.

[0048] The pilot pump 15 may be omitted. In this case, the relatively high pressure hydraulic oil discharged from the main pump 14 is reduced in pressure by a predetermined pressure reducing valve, and the resulting relatively low pressure hydraulic oil is supplied to various hydraulic devices as pilot pressure.

[0049] The operating device 26 is provided near the cockpit of the cabin 10, and is used by the operator to operate the various driven elements. In other words, the operating device 26 is used by the operator to operate the hydraulic actuators HA that drive the respective driven elements. The operating device 26 includes a pedal device and a lever device for operating the respective driven elements (hydraulic actuators HA).

[0050] For example, the operating device 26 is of an electrical type. Specifically, the operating device 26 outputs an electric signal (hereinafter, "operation signal") corresponding to the operation content, and the operation signal is input to the controller 30. The controller 30 then outputs a control command corresponding to the content of the operation signal, that is, a control signal corresponding to the operation content for the operating device 26, to the proportional valve 31. As a result, a pilot pressure corresponding to the operation content of the operating device 26 is input from the proportional valve 31 to the control valve unit 17, and the control valve unit 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26.

[0051] Furthermore, the control valves (directional control valves) that drive the respective hydraulic actuators built into the control valve unit 17 may be of the electromagnetic solenoid type. In this case, an operation signal output from the operating device 26 may be input directly to the control valve unit 17, i.e., to the electromagnetic solenoid type control valve.

[0052] A proportional valve 31 is provided for each driven element (hydraulic actuator HA) that is the object of operation of the operating device 26. The proportional valve 31 is disposed in a pipe that connects the pilot pump 15 and a pilot port of a control valve in the control valve unit 17, and is configured to be able to change the flow path area of ​​the pipe. In this embodiment, the proportional valve 31 operates in response to a control command output by the controller 30. Therefore, the controller 30 can supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the control valve in the control valve unit 17 via the proportional valve 31, regardless of the operation of the operating device 26 by the operator.

[0053] <<User interface system>> The user interface system of the crawler crane 100 according to this embodiment includes an operation device 26, an output device D1, and an input device D2.

[0054] The output device D1 outputs various types of information to a user (operator) of the crawler crane 100 inside the cabin 10.

[0055] For example, the output device D1 includes indoor lighting devices and display devices that are provided in a place that is easily visible to an operator seated in the cabin 10 and output various information in a visual manner. The lighting devices are, for example, warning lights, etc. The display devices are, for example, liquid crystal displays and organic EL (Electroluminescence) displays, etc.

[0056] Furthermore, for example, the output device D1 includes a sound output device that outputs various information by an auditory method, such as a buzzer or a speaker.

[0057] Moreover, for example, the output device D1 includes a device that outputs various information by a tactile method such as vibration of the cockpit.

[0058] The input device D2 is provided within a range close to the operator seated in the cabin 10, and receives various inputs from the operator.

[0059] For example, the input device D2 is an operation input device that accepts operation input. The operation input device may include a touch panel mounted on the display device, a touch pad installed around the display device, a button switch, a lever, a toggle, a knob switch provided on the operation device 26 (lever device), and the like.

[0060] Furthermore, for example, the input device D2 may be a voice input device that accepts voice input from an operator. The voice input device includes, for example, a microphone.

[0061] Furthermore, for example, the input device D2 may be a gesture input device that accepts a gesture input by an operator. The gesture input device includes, for example, an imaging device (an interior camera) installed in the cabin 10.

[0062] <<Communications>> The communication system of the crawler crane 100 in this embodiment includes a communication device T1 capable of communicating with an external device.

[0063] The communication device T1 is connected to the communication line NW and communicates with devices provided separately from the crawler crane 100. The devices provided separately from the crawler crane 100 may include devices outside the crawler crane 100 as well as portable terminal devices brought into the cabin 10 by the user of the crawler crane 100. The communication device T1 may include, for example, a mobile communication module conforming to standards such as 4G (4th Generation) and 5G (5th Generation). The communication device T1 may also include, for example, a satellite communication module. The communication device T1 may also include, for example, a Wi-Fi (registered trademark) communication module or a Bluetooth (registered trademark) communication module.

[0064] <<Control system>> The control system of the crawler crane 100 according to this embodiment includes a controller 30. The control system of the crawler crane 100 according to this embodiment includes an operation sensor 29 that detects the operation content of the operation device 26. Furthermore, the control system of the crawler crane 100 includes light sources L1 to L4 used when performing calibration. For example, the light sources L1 to L4 are attached at positions corresponding to each of the spatial recognition devices. Then, according to the control of the controller 30, the light sources L1 to L4 light up or blink in a predetermined color to indicate the status related to the spatial recognition device.

[0065] The controller 30 performs various controls related to the crawler crane 100. The functions of the controller 30 may be realized by any hardware or any combination of hardware and software. For example, the controller 30 is configured mainly with a computer including a central processing unit (CPU), a memory device such as a random access memory (RAM), a non-volatile auxiliary storage device such as a read only memory (ROM), and various interface devices for input and output. The controller 30 realizes various functions by, for example, loading a program installed in the auxiliary storage device into the memory device and executing the program on the CPU.

[0066] The controller 30 performs control related to the operation of a hydraulic actuator HA (driven element) of the crawler crane 100, for example, with the proportional valve 31 as a control object.

[0067] Specifically, the controller 30 may control the operation of the hydraulic actuator HA (driven element) of the crawler crane 100 based on the operation of the operation device 26, with the proportional valve 31 as the control object.

[0068] <<Overlapping range of spatial recognition devices>> 3 is a diagram showing the measurement range of the distance measuring device S7B according to this embodiment and the imaging range of the imaging device S6. In the example shown in Fig. 3, the distance measuring device S7B can detect objects included in a measurement range 1301. On the other hand, the imaging device S6 can detect objects included in an imaging range 1302.

[0069] Then, the controller 30 performs calibration between the distance measuring device S7B and the imaging device S6 based on objects present in the vicinity of the crawler crane 100.

[0070] Calibration according to the present embodiment refers to determining the positional relationship between the measurement range or image capture range of multiple spatial recognition devices. When calibration is performed, it is possible to suppress false detection in which the object is detected twice when the same object is detected by each of the multiple spatial recognition devices.

[0071] In this embodiment, a person is used as a calibration target. Usually, a marker with an "x" or the like is used as a calibration target. However, when the crawler crane 100 is assembled at a work site, it is necessary to bring the calibration target to the work site. If the calibration target is brought in every time work is performed at the work site, the burden on the worker increases. Therefore, in this embodiment, an example in which a person is used as a calibration target will be described. That is, since a worker is present at the work site, by using a person as a calibration target, it is not necessary to bring in a marker or the like, and therefore it is possible to improve work efficiency. However, this embodiment shows an example in which a person is used as a calibration target, and a conventional marker or the like may be used as a calibration target.

[0072] 3, when calibrating the distance measuring device S7B and the imaging device S6, there are cases where the person 1311 does not fall within the measurement range 1301 of the distance measuring device S7B and the imaging range 1302 of the imaging device S6. In this case, when the controller 30 calibrates the distance measuring device S7B and the imaging device S6, there is a possibility that the calibration will fail or that the calibration calculation will not converge, or other such problems will occur. In this case, the controller 30 needs to perform the calibration again after the person 1311 moves, which may reduce productivity.

[0073] Therefore, when the controller 30 of this embodiment performs calibration using one or more of the distance measuring device S7 and the imaging device S6, the controller 30 is configured to instruct each of the distance measuring device S7 and the imaging device S6 as to whether or not a person 1311 is present in a position where calibration can be performed.

[0074] <Block configuration of each device for calibration> Fig. 4 is a block diagram showing a configuration example of the calibration system SYS of the spatial recognition device provided in the crawler crane 100 according to the first embodiment. In the example shown in Fig. 4, the calibration system SYS of the spatial recognition device includes the crawler crane 100 and a mobile communication terminal 800. As shown in Fig. 4, the crawler crane 100 and the mobile communication terminal 800 are connected via a communication line NW.

[0075] The mobile communication terminal 800 is a communication terminal carried by a person (hereinafter also referred to as a worker) who is a calibration target. The controller 30 of the crawler crane 100 informs the person, via the mobile communication terminal 800, whether or not the person is in a position where calibration can be performed. Note that this embodiment is not limited to an example in which the person who is a calibration target carries the mobile communication terminal 800, and the person does not need to carry the mobile communication terminal 800. In this case, the controller 30 uses the light sources L1 to L4 to inform the worker whether or not the person is in a position where calibration can be performed.

[0076] The portable communication terminal 800 includes a display device 801, an input device 802, a communication device T2, and a controller 830.

[0077] The display device 801 is, for example, a liquid crystal panel provided in the mobile communication terminal 800, and displays various information images under the control of the controller 830.

[0078] The input device 802 is an interface for an operator to input information, and is, for example, a touch panel provided on the display device 801.

[0079] The communication device T2 is connected to the communication line NW and communicates with the crawler crane 100 and the like. The communication device T2 may include, for example, a mobile communication module conforming to standards such as 4G (4th Generation) and 5G (5th Generation). The communication device T2 may also include, for example, a satellite communication module. The communication device T2 may also include, for example, a Wi-Fi (registered trademark) communication module or a Bluetooth (registered trademark) communication module.

[0080] The controller 830 includes, for example, a CPU (Central Processing Unit) (not shown) and executes a program (not shown) stored in a storage medium (not shown) to perform various controls. The controller 830 according to this embodiment includes a display control unit 831 and a selection unit 832.

[0081] The display control unit 831 performs control for displaying various information on the display device 801. For example, the display control unit 831 displays a list of spatial recognition devices (e.g., a plurality of distance measuring devices S7 and an imaging device S6) provided on the crawler crane 100 in a selectable manner.

[0082] The selection unit 832 selects multiple spatial recognition devices to be calibrated. For example, the selection unit 832 may receive a selection of multiple spatial recognition devices from the operator via the input device 802 from a list of spatial recognition devices (e.g., multiple distance measuring devices S7 and imaging device S6) displayed on the display device 801. Alternatively, the selection unit 832 may automatically select multiple spatial recognition devices. Then, the selection unit 832 transmits information indicating the selected multiple spatial recognition devices to the crawler crane 100 via the communication device T2.

[0083] In this embodiment, an example will be described in which the selection unit 832 is provided in the mobile communication terminal 800. However, this embodiment is not limited to a method in which the selection unit 832 is provided in the mobile communication terminal 800, and the selection unit 832 may be provided in the controller 30 of the crawler crane 100.

[0084] The crawler crane 100 includes light sources L1 to L4, a communication device T1, a plurality of distance measuring devices S7, an imaging device S6, and a controller 30. The light sources L1 to L4, the communication device T1, the plurality of distance measuring devices S7, and the imaging device S6 are as described above, and therefore description thereof will be omitted.

[0085] The controller 30 includes, for example, a CPU (Central Processing Unit) (not shown) and performs various controls by executing a program (not shown) stored in a storage medium (not shown). The controller 30 according to this embodiment includes a first object detection unit 301, a second object detection unit 302, a target extraction unit 303, a position determination unit 304, an output control unit 305, a feature extraction unit 306, and a calibration unit 307.

[0086] The first object detection unit 301 detects an object present within the measurement range of the distance measurement device S7 based on the distance measurement results by the multiple distance measurement devices S7. Any method may be used to detect an object based on the distance measurement results, regardless of whether it is a well-known method. For example, the first object detection unit 301 may generate a distance image (in which colors are changed according to distance) based on distance information indicated by point cloud data, which is the distance measurement result, and detect an object from the visualized distance image.

[0087] The second object detection unit 302 detects an object present in the imaging range of the imaging device S6 based on image information captured by the imaging device S6. The method of detecting an object based on image information may be any method, regardless of whether it is a well-known method, and may, for example, specify the shape of the object by contour extraction and detect the object based on the specified shape of the object.

[0088] The target extraction section 303 extracts, from the object detected by the first object detection section 301, an area within the measurement range in which a worker who serves as a calibration target is present.

[0089] Furthermore, the target extraction section 303 extracts, from the object detected by the second object detection section 302, an area within the imaging range in which a worker who is to be a calibration target exists.

[0090] The position determination unit 304 determines, for each spatial recognition device to be calibrated, whether or not a calibration target exists in a position where calibration is possible, based on the area extracted by the target extraction unit 303.

[0091] When the position determination unit 304 according to this embodiment receives information indicating a calibration target from the mobile communication terminal 800, it selects a plurality of spatial recognition devices to be calibrated based on the received information. Then, for each of the selected plurality of spatial recognition devices, the position determination unit 304 determines whether or not the calibration target is present in a position where calibration is possible. In this embodiment, the spatial recognition device to be calibrated is determined according to the selection of the worker carrying the mobile communication terminal 800. Therefore, the spatial recognition devices can be calibrated in an order according to the worker's request, so that dissatisfaction among the worker can be suppressed.

[0092] Fig. 5 is a conceptual diagram showing an example of determination by the position determination unit 304 according to this embodiment. In the example shown in Fig. 5, calibration is performed between the first distance measuring device S7 and the second distance measuring device S7 or the imaging device S6.

[0093] 5, calibration is performed using each part of the person 1501. For this reason, the entire person 1501 needs to be included in the measurement range or image capture range.

[0094] 5 shows a distance image generated based on the measurement results by the first distance measuring device S7. The image 1511 shows the entire person 1501A. Therefore, the position determination unit 304 determines that the person 1501 is present in a position where calibration is possible based on the measurement results by the first distance measuring device S7.

[0095] 5 shows a distance image generated based on the measurement results by the second distance measuring device S7 or image information captured by the imaging device S6. In the image 1521, the person 1501B is cut off at the left edge. Therefore, the position determination unit 304 determines that the person 1501 is not present in a position where calibration is possible based on the measurement results by the second distance measuring device S7 or the imaging results by the imaging device S6.

[0096] 5 shows a distance image generated based on the measurement results by the second distance measuring device S7 or image information captured by the imaging device S6. In the image 1522, the person 1501C is cut off at the top. Therefore, the position determination unit 304 determines that the person 1501 is not in a position where calibration is possible based on the measurement results by the second distance measuring device S7 or the imaging results by the imaging device S6.

[0097] The output control unit 305 outputs the determination result by the position determination unit 304 for each of the multiple spatial recognition devices. In this embodiment, the determination result by the position determination unit 304 is represented by color. When the position determination unit 304 determines that the person 1501 is present in a position where calibration is possible for a specific spatial recognition device (ranging device S7 or imaging device S6), the output control unit 305 outputs blue for the specific spatial recognition device. When the position determination unit 304 determines that the person 1501 is not present in a position where calibration is possible for the specific spatial recognition device, the output control unit 305 outputs red for the specific spatial recognition device.

[0098] For example, the output control unit 305 causes the light sources L1 to L4, which are provided at positions corresponding to each of the multiple spatial recognition devices, to shine in a color (one example of a mode) that corresponds to whether or not the light sources are located at a position that can be calibrated by the position determination unit 304.

[0099] As another example, the output control unit 305 transmits screen information to the portable communication terminal 800 via the communication device T1. In the screen information, a color corresponding to the determination result is displayed for each spatial recognition device selected by the selection unit 832. In this manner, the output control unit 305 according to the present embodiment instructs the portable communication terminal 800 via the communication device T1 as to whether or not the person 1501 is present in a position where calibration is possible.

[0100] The output control unit 305 according to this embodiment outputs the determination result by the position determination unit 304 to the light sources L1 to L4 and the mobile communication terminal 800, thereby being able to recognize whether or not the person 1501, who is the calibration target, is in a position where calibration is possible. Therefore, when the person 1501 is not in a position where calibration is possible, the person 1501 can move to a position where calibration is possible. Therefore, it is possible to suppress failure of calibration by the controller 30, shorten the time until calibration is completed, and improve work efficiency.

[0101] Fig. 6 is a diagram showing a concept of calibrating the spatial recognition device of the crawler crane 100 according to this embodiment. As shown in Fig. 6, the crawler crane 100 is provided with an imaging device S6 and distance measuring devices S7B, S7R, and S7L. Furthermore, on the rear surface of the counterweight 9 of the crawler crane 100, a light source L1 corresponding to the imaging device S6, a light source L2 corresponding to the distance measuring device S7L, a light source L3 corresponding to the distance measuring device S7B, and a light source L4 corresponding to the distance measuring device S7R are provided.

[0102] In this example, the worker selects the imaging device S6 and the distance measuring device S7B as the calibration targets using the mobile communication terminal 800. As a result, the controller 30 performs a process for calibrating the imaging device S6 and the distance measuring device S7B.

[0103] The operator is not in a position where distance measuring device S7L and distance measuring device S7R can be calibrated. Therefore, the output control unit 305 causes the light source L2 corresponding to distance measuring device S7L, the light source L3 corresponding to distance measuring device S7B, and the light source L4 corresponding to distance measuring device S7R to shine in red.

[0104] When the worker selects the imaging device S6 and the distance measuring device S7B as the calibration targets using the mobile communication terminal 800, the output control unit 305 starts blinking the light source L1 corresponding to the imaging device S6 and the light source L3 corresponding to the distance measuring device S7B.

[0105] When the worker is at position 1601, the output control unit 305 causes the light source L1 corresponding to the image capture device S6 to blink blue, and the light source L3 corresponding to the distance measurement device S7B to blink red. Furthermore, the output control unit 305 transmits screen information to the mobile communication terminal 800 via the communication device T1. In the screen information, the area 1611 corresponding to the image capture device S6 blinks blue, and the area 1612 corresponding to the distance measurement device S7B blinks red. This allows the worker to recognize that the distance measurement device S7B cannot be calibrated at the current position.

[0106] Therefore, the worker moves from position 1601 to position 1602 .

[0107] When the worker is at position 1602, the output control unit 305 causes the light source L1 corresponding to the image capture device S6 to blink in blue, and causes the light source L3 corresponding to the distance measurement device S7B to blink in blue. Furthermore, the output control unit 305 transmits screen information to the mobile communication terminal 800 via the communication device T1. In the screen information, the area 1621 corresponding to the image capture device S6 blinks in blue, and the area 1622 corresponding to the distance measurement device S7B blinks in blue. This allows the worker to recognize that calibration can be performed between the image capture device S6 and the distance measurement device S7B at the current position. Therefore, calibration by the controller 30 is started.

[0108] Then, after the calibration by the controller 30 is completed, the output control unit 305 turns on the light source L1 and the light source L3 in blue. Furthermore, the screen information transmitted by the output control unit 305 also displays the areas 1621 and 1622 in blue. This allows the operator to recognize that the calibration between the image capture device S6 and the distance measurement device S7B has been completed.

[0109] 6, an example has been described in which the operator moves to a position where calibration can be performed in order to calibrate between the distance measuring device S7B and the image capturing device S6. However, this embodiment is not limited to the example of calibration between the distance measuring device S7B and the distance measuring device S7R.

[0110] Fig. 7 is an explanatory diagram showing an example where an operator is in a position where calibration is possible in order to perform calibration between the distance measuring device S7B and the distance measuring device S7R according to this embodiment. In the example shown in Fig. 7, the operator is in an overlapping area between the distance measuring range 1701 of the distance measuring device S7B and the distance measuring range 1702 of the distance measuring device S7R.

[0111] The operator can recognize that he / she is currently in a position where calibration is possible by referring to the portable communication terminal 800 or the light sources L3 and L4. Therefore, calibration by the controller 30 is performed.

[0112] Returning to FIG. 4, when the position determination unit 304 determines that the calibration target is present in a position where calibration is possible, the feature extraction unit 306 extracts characteristic portions and feature amounts of the calibration target from the area in which the calibration target extracted by the target extraction unit 303 is present, for each of the multiple spatial recognition devices to be calibrated.

[0113] The calibration unit 307 performs calibration between the multiple spatial recognition devices based on the characteristic parts and feature amounts of the calibration target extracted by the feature extraction unit 306. A specific calibration method is a well-known method, and a description thereof will be omitted.

[0114] In this embodiment, calibration is required so that the correspondence between the spatial recognition devices can be recognized. For example, when the distance measuring devices S7B, S7L, and S7R and the imaging device S6 are present as the spatial recognition devices, the calibration unit 307 performs the calibration between the distance measuring device S7B and the imaging device S6 and the calibration between the distance measuring device S7B and the distance measuring device S7R, as well as the calibration between the distance measuring device S7B and the distance measuring device S7L. This allows the controller 30 to recognize the correspondence between all the spatial recognition devices provided on the crawler crane 100.

[0115] This embodiment shows an example of the arrangement of the spatial recognition devices of the crawler crane 100, and is not limited to this arrangement. Even if the spatial recognition devices are arranged in another manner, the controller 30 only needs to perform calibration so that it can recognize the correspondence between all the spatial recognition devices provided on the crawler crane 100.

[0116] The controller 30 according to the present embodiment has been described as controlling the light sources L1 to L4 and providing output to the mobile communication terminal 800. However, the present embodiment is not limited to this control, and the controller 30 may perform only one of the control of the light sources L1 to L4 and the output to the mobile communication terminal 800.

[0117] In this embodiment, the embodiment shown below, and the modified examples, an example will be described in which the operator is instructed as to whether or not the operator is in a position where calibration can be performed. However, this embodiment, the embodiment shown below, and the modified examples are not limited to a method in which the operator is directly instructed as to whether or not the operator is in a position where calibration can be performed using any one or more of the light sources L1 to L4 and the mobile communication terminal 800. For example, the output device D1 in the cabin 10 of the crawler crane 100 may be instructed as to whether or not the operator is in a position where calibration can be performed. Then, the operator of the cabin 10 may instruct the operator to move to a position where calibration can be performed.

[0118] Second Embodiment In the above-described embodiment, an example was described in which the determination result by the position determination unit 304 was referred to by the worker who is the calibration target. However, the present invention is not limited to the method of teaching whether or not the worker is in a position where calibration is possible based on the determination result by the position determination unit 304 as in the above-described embodiment. Therefore, in the second embodiment, a method of teaching whether or not the worker is in a position where calibration is possible by having the worker refer to the detection result of the spatial recognition device will be described.

[0119] The controller 30 according to this embodiment does not necessarily have to include the position determination unit 304 as in the above-described embodiment. Instead, in this embodiment, the output control unit 305 outputs to the mobile communication terminal 800 one or more of a distance image showing the detection result of the first object detection unit 301 and a captured image showing the detection result of the second object detection unit 302.

[0120] The mobile communication terminal 800 displays an image (distance image or captured image) showing the detection result of the surrounding situation for each spatial recognition device to be calibrated, as an instruction as to whether the worker is in a position where calibration is possible. This allows the worker to recognize whether each spatial recognition device is in a position where calibration is possible. Next, the screen displayed on the mobile communication terminal 800 will be described.

[0121] Fig. 8 is a diagram showing the transition of screens displayed on portable communication terminal 800 according to this embodiment. Fig. 8(A) shows a menu screen 1801 displayed on portable communication terminal 800. Then, selection unit 832 of controller 830 accepts selection of "calibration" 1802 via input device 802.

[0122] Fig. 8(B) shows a selection screen of a sensor (spatial recognition device) displayed on the display device 801 by the display control unit 831. Fig. 8(B) shows check boxes 1811-1815 for selecting the sensor along with the sensor (spatial recognition device) to be calibrated. In the selection screen of Fig. 8(B), the check boxes 1811-1815 display sensors A, B, and C, which are distance measuring devices S7, and sensors D and E, which are imaging devices S6, as selectable. Note that the arrangement of the distance measuring devices S7 and imaging devices S6 shown in Fig. 8(B) shows another aspect of the first embodiment.

[0123] When the selection unit 832 receives a press of the OK button 1816 after receiving a selection of two of the sensors A to E shown in the check boxes 1811 to 1815, it recognizes that the selection of the spatial recognition device to be calibrated has been received.

[0124] Then, the selection unit 832 transmits information indicating the spatial recognition device whose selection has been accepted to the crawler crane 100 via the communication device T2. Thereafter, the communication device T2 receives image information indicating the detection result of the spatial recognition device whose selection has been accepted from the crawler crane 100. In the example shown in FIG. 8, it is assumed that the selection of sensor A and sensor D has been accepted.

[0125] Then, the display control unit 831 displays a display screen showing image information indicating the detection result of the spatial recognition device that has accepted the selection. As shown in Fig. 8C, the display screen shows an image (distance image) 1821 indicating the detection result of the sensor A (distance measuring device S7) that has accepted the selection, and an image (captured image) 1822 indicating the detection result of the sensor D (imaging device S6) that has accepted the selection.

[0126] 8C, the worker can recognize that the entire body is not included in the imaging range of the sensor D (imaging device S6). Therefore, the worker moves toward the imaging range of the sensor D (imaging device S6).

[0127] Even after the worker moves, the display control unit 831 continues to display the display screen showing image information indicating the detection result of the spatial recognition device that has accepted the selection. As shown in Fig. 8(D) , the display screen shows an image (distance image) 1831 indicating the detection result of the sensor A (distance measuring device S7) that has accepted the selection, and an image (captured image) 1832 indicating the detection result of the sensor D (imaging device S6) that has accepted the selection.

[0128] 8(D), the worker can recognize that the entire body is included in the imaging range of sensor A (distance measuring device S7) and the imaging range of sensor D (imaging device S6). Then, the worker presses the OK button 1833.

[0129] Then, after accepting pressing of OK button 1833, display control unit 831 displays a display screen showing a message for the worker. By referring to message 1841 included in the screen shown in FIG. 8(E), the worker can recognize that he or she will remain still in a predetermined posture for 15 seconds. The predetermined posture may be any posture that is determined in advance for performing calibration. After the worker assumes the predetermined posture, he or she presses OK button 1842.

[0130] When the selection unit 832 receives a press of the OK button 1842, it transmits an instruction to start calibration to the crawler crane 100 via the communication device T2.

[0131] After the calibration is completed, communication device T2 receives a notification from crawler crane 100 that the calibration is complete.

[0132] After that, the display control unit 831 displays the display screen shown in Fig. 8(F) By referring to the display screen, the operator can recognize that the calibration has been completed.

[0133] Thereafter, the operator may perform an operation for calibration among other spatial recognition devices, or may end the calibration. When the selection unit 832 receives a press of the move button 1851, the display control unit 831 displays the selection screen shown in Fig. 8(B). When the selection unit 832 receives a press of the end button 1852, the process ends.

[0134] The controller 30 of the crawler crane 100 and the controller 830 of the mobile communication terminal 800 according to this embodiment perform control so that the above-mentioned display screen can be displayed, allowing the operator to recognize whether or not the operator is in a position where calibration is possible. In addition, the operator can recognize the position where the operator is in relation to the distance measurement range or the image capture range by referring to the display screen. Since the operator can recognize the direction and approximate distance to move to the position where calibration is possible, the operator can quickly move to the position where calibration is possible. Therefore, the controller 30 of the crawler crane 100 and the controller 830 of the mobile communication terminal 800 according to this embodiment can prevent calibration failures, thereby improving the efficiency of the time required for calibration.

[0135] (Modification of the second embodiment) In the second embodiment, a display screen showing the detection result of the spatial recognition device is output, The method of instructing whether or not the device is in a position where calibration is possible has been described above. In the second embodiment, a case will be described in which the direction of movement is instructed in addition to the output of the display screen.

[0136] The position determination unit 304 of the controller 30 according to this modification determines whether or not a person who is a calibration target is present at a position where calibration is possible, based on the measurement result by the distance measuring device S7. Furthermore, when the position determination unit 304 determines that a person is not present at a position where calibration is possible, it specifies the direction in which the person moves to the position where calibration is possible.

[0137] Similarly, the position determination unit 304 determines whether or not a person who is a calibration target is present at a position where calibration is possible, based on the image capturing result by the image capturing device S6. Furthermore, when the position determination unit 304 determines that a person is not present at a position where calibration is possible, it specifies a direction in which the person moves to a position where calibration is possible.

[0138] Then, the output control unit 305 outputs, to the portable communication terminal 800, information indicating the moving direction identified by the position determination unit 304 together with at least one of the distance image and the captured image.

[0139] Then, the display control unit 831 of the portable communication terminal 800 displays a display screen based on at least one of the distance image and the captured image, and on the information indicating the moving direction.

[0140] Fig. 9 is a diagram showing an example of a screen displayed on the display device 801 by the display control unit 831 according to this modification. In the example shown in Fig. 9, it is assumed that sensors A and D are selected as calibration targets. Also, an example is shown in which the entire body of a worker who is the calibration target is not included in the imaging range of sensor D (imaging device S6) and is cut off at the left end.

[0141] In this case, the display control unit 831 displays a distance image 1901 showing the detection result of sensor A and an captured image 1902 showing the detection result of sensor D, and also superimposes an arrow mark 1903 indicating the received movement direction on the captured image 1902 showing the detection result of sensor D.

[0142] That is, by displaying the display screen shown in FIG. 9, the mobile communication terminal 800 can guide the worker to a position where calibration can be performed.

[0143] Furthermore, guidance to the operator according to this modified example is not limited to cases where the operator is out of view. For example, guidance may be given to the operator when calibration is difficult because the operator is far away from the crawler crane 100.

[0144] In other words, the determination by the position determination unit 304 of the controller 30 as to whether or not the operator is in a position where calibration is possible also includes a determination based on the distance from the crawler crane 100 to the operator. And the information indicating the movement direction output by the output control unit 305 also includes the direction of approaching the crawler crane 100.

[0145] Fig. 10 is a diagram showing an example of a screen displayed on the display device 801 by the display control unit 831 according to this modified example. In the example shown in Fig. 10, it is assumed that sensors A and D are selected as calibration targets. It is also assumed that the worker who is the calibration target is too far away from the crawler crane 100. For example, it is assumed that the worker is 7 m or more away from the spatial recognition device, even though the calibration possible range is within 5 m from the spatial recognition device.

[0146] In this case, as shown in the display screen of Figure 10 (A), the display control unit 831 displays a distance image 2001 indicating the detection result of sensor A and an captured image 2002 indicating the detection result of sensor D, and also superimposes arrow marks 2003 and 2004 indicating the direction of movement on each of the distance image 2001 and the captured image 2002.

[0147] 10(A), the worker can recognize that he or she needs to approach the crawler crane 100. In addition, this modified example is not limited to displaying an arrow, and a message may also be displayed.

[0148] 10(B) displays a distance image and a captured image, and an arrow indicating the direction of movement is superimposed on each of the distance image and the captured image, and also displays a message 2011. In the message 2011 shown in FIG. 10(B), an example has been described in which a character string indicating that the user is approaching is displayed. However, the displayed message is not limited to the character string indicating that the user is approaching, and any one or more of the distance to move and the direction to move may be displayed.

[0149] In this modification, an example has been described in which the instruction of the movement direction to the worker is displayed on the display device 801. However, in this modification, the instruction of the movement direction to the worker is not limited to being displayed on the display screen, and may be notified by voice, for example.

[0150] In addition, in this modified example, an example in which the OK button is always displayed on the display screen so that it can be pressed will be described, but the present invention is not limited to this display mode. For example, the display control unit 831 may hide or gray out the OK button until the worker moves to a position where calibration is possible.

[0151] In this modification, the direction of movement is indicated, so that the operator knows what to do until calibration is possible. Therefore, this modification can shorten the time until calibration can be started appropriately, and improve the efficiency of calibration time.

[0152] (Third embodiment) In the above-mentioned embodiment, the case where the calibration between the multiple spatial recognition devices is performed at one time has been described. However, it is preferable to perform the calibration between the multiple spatial recognition devices multiple times. Therefore, in the third embodiment, the case where the worker moves to multiple locations and performs the calibration at each of the moved locations will be described.

[0153] Fig. 11 is a diagram showing the transition of screens displayed on portable communication terminal 800 according to this embodiment. Fig. 11(A) shows a menu screen 2101 displayed on portable communication terminal 800. Then, selection unit 832 of controller 830 accepts selection of "calibration" 2102 via input device 802.

[0154] Fig. 11(B) shows a selection screen for sensors (spatial recognition devices) that the display control unit 831 displays on the display device 801. Fig. 11(B) shows check boxes 2111 to 2115 for selecting the sensors (spatial recognition devices) to be calibrated.

[0155] When the selection unit 832 receives a press of the OK button 2116 after receiving a selection of two of the sensors A to E shown in the check boxes 2111 to 2115, it recognizes that the selection of the spatial recognition device to be calibrated has been received.

[0156] Then, the selection unit 832 transmits information indicating the spatial recognition device that has accepted the selection to the crawler crane 100 via the communication device T2.

[0157] Then, the output control unit 305 of the crawler crane 100 generates a movement path for the worker to calibrate the multiple spatial recognition devices whose selections have been accepted.

[0158] The movement route is generated so as to be included in the measurement range or image capture range of each of the multiple spatial recognition devices. The movement route is a route in which multiple positions are set for the worker to move (be) in for calibration. In this example, the movement route is a random route generated so as to be included in the measurement range or image capture range of each of the multiple spatial recognition devices, but the movement route may be determined in advance.

[0159] Then, the output control unit 305 transmits the generated travel route to the mobile communication terminal 800 together with an image (distance image or captured image) indicating the detection result of each of the multiple spatial recognition devices whose selections have been accepted.

[0160] On the other hand, the display control unit 831 displays a display screen showing a message 2121 indicating that calibration will start. As shown in Fig. 11(C), the display screen displays the message indicating that calibration will start together with an OK button 2122. When the selection unit 832 accepts pressing of the OK button 2122, the display control unit 831 displays the screen shown in Fig. 11(D) based on the information received.

[0161] 11D shows an image (distance image) 2131 indicating the detection result of the sensor A (distance measuring device S7) that has been selected, and an image (captured image) 2132 indicating the detection result of the sensor D (imaging device S6) that has been selected. Furthermore, a movement path 2133 of the worker is superimposed on the image 2131.

[0162] The movement path 2133 is composed of positions where the worker stops to perform calibration and arrows indicating the movement between the positions. The worker can perform calibration multiple times by repeating stopping at the positions indicated by the movement path 2133 and moving.

[0163] The operator can recognize the movement path for calibration by referring to the display screen shown in Fig. 11(D). When the selection unit 832 accepts pressing of the OK button 2134, the display control unit 831 displays the screen shown in Fig. 11(E).

[0164] The display screen shown in Fig. 11(E) displays the procedure for performing calibration as a message 2141. The operator can recognize the procedure for calibration at the current position by referring to the display screen shown in Fig. 11(E). In other words, the operator can recognize that after pressing the OK button 2142, he or she should assume a predetermined posture within five seconds and remain still for 15 seconds.

[0165] When the selection unit 832 accepts pressing of the OK button 2142, the selection unit 832 transmits an instruction to start calibration to the crawler crane 100. As a result, the controller 30 of the crawler crane 100 starts calibration with the worker at the current position as the calibration target.

[0166] After the calibration is completed, the display control unit 831 displays the screen shown in Fig. 11(F). The display screen shown in Fig. 11(F) displays an image (distance image) 2151 showing the detection result of sensor A (distance measuring device S7) whose selection has been accepted, and an image (captured image) 2152 showing the detection result of sensor D (imaging device S6) whose selection has been accepted. Furthermore, an arrow mark 2153 for the worker to move to the next position is superimposed on the image 2151. Furthermore, the display control unit 831 displays a message 2154 to prompt the worker to move.

[0167] The display screen shown in FIG. 11(G) is displayed by the display control unit 831 after the worker moves to the next position. The display screen shown in FIG. 11(G) displays an image (distance image) 2161, an image (captured image) 2162, and an arrow mark 2163. The worker can recognize that he has moved to the next calibration position by referring to the arrow mark 2163 in the image (distance image) 2161 and an area 2164 in which the worker is displayed. The display screen shown in FIG. 11(G) does not display the message 2154 because the worker has moved to the next calibration position. Furthermore, there is also a method in which the display control unit 831 displays "Please stop" or the like when it is determined that the worker has reached the vicinity of the tip of the arrow mark 2163.

[0168] When the selection unit 832 receives a press of the OK button 2165, the display control unit 831 displays the screen shown in FIG.

[0169] The display screen shown in Fig. 11(H) displays the procedure for performing calibration as a message 2171. The operator can recognize the procedure for calibration at the current position by referring to the display screen shown in Fig. 11(H). In other words, the operator can recognize that after pressing the OK button 2172, he or she should assume a predetermined posture within five seconds and remain still for 15 seconds.

[0170] In this embodiment, the screen displays of FIG. 11(F) to FIG.

[0171] The controller 30 of the crawler crane 100 then calibrates between the multiple spatial recognition devices based on the worker present at each of the multiple positions indicated by the travel path 2133.

[0172] Then, when the calibration for each position indicated by the movement path 2133 is all completed, the display control unit 831 displays the display screen shown in Fig. 11(I). The operator can recognize that the calibration is completed by referring to the display screen shown in Fig. 11(I).

[0173] In this embodiment, the worker, who is the calibration target, moves to a plurality of positions, and the controller 30 performs calibration for each position, thereby achieving improvement in calibration accuracy.

[0174] (Modification of the fourth embodiment) In the above embodiment, an example has been described in which the output control unit 305 of the crawler crane 100 generates a random route. However, the present invention is not limited to generating a random route. Therefore, in a modified example, a method for generating a travel route will be described.

[0175] FIG. 12 is a conceptual diagram showing a method of generating a moving path by the output control unit 305 according to a modified example of the fourth embodiment. The area 2201 shown in FIG. 12 shows an area common to the range detectable by each of the selected spatial recognition devices. Calibration is possible by generating a moving path within the common area. Note that it is preferable that the positions included in the moving path are scattered in various places.

[0176] For this reason, the output control unit 305 divides the common area 2201 shown in Fig. 12 into a predetermined number of areas. In the example shown in Fig. 12, the area is divided into three vertically and four horizontally. Then, the output control unit 305 generates (sets) a movement path that includes the position where the worker stops (is present) and that spans three or more of the divided areas. In the example shown in Fig. 12, a movement path is generated so as to calibrate in each of the divided areas 2211, 2212, 2213, 2214, 2215, and 2216.

[0177] Other controls are the same as those in the fourth embodiment, and explanations thereof will be omitted. Note that this modified example shows one example, and the number of divisions of the range is not limited to the example shown in Fig. 12. The number of divisions in the vertical direction may be 2 or less and 4 or more, and the number of divisions in the horizontal direction may be 3 or less and 5 or more. Also, the number of areas spanned by the movement path may be 2 or less.

[0178] The method of generating the travel path is not limited to the method described in the modified example and the embodiment described above, and other methods may be used. For example, the travel path may be generated so as to avoid obstacles that exist in the common area.

[0179] The movement path generated by the output control unit 305 according to this modification has scattered positions where the operator stops to perform calibration. Therefore, the controller 30 can perform calibration at various positions included in the movement path, thereby improving the accuracy of calibration among multiple spatial recognition devices.

[0180] In the above-mentioned embodiment and modified examples, a case has been described in which a crawler crane 100 is used as the work machine. However, the embodiment and modified examples do not limit the work machine to the crawler crane 100, and the work machine may be a forklift or an overhead crane that can be moved according to the operation of an operator, or a fixed power crane. Furthermore, the work machine may be a road machine including an asphalt finisher or the like, or a construction machine including a shovel or the like.

[0181] <effect> The calibration system configured with the crawler crane 100 and the mobile communication terminal 800 according to the above-mentioned embodiment and modification can confirm whether the worker, who is the calibration target, is in a position where calibration is possible by using the light sources L1 to L4 or the information displayed on the mobile communication terminal 800. The controller 30 of the crawler crane 100 can perform calibration between multiple spatial recognition devices after confirming that the worker is in a position where light can be detected, thereby preventing the occurrence of calibration failures, etc. Therefore, the controller 30 can reduce the time required for calibration and improve the time efficiency required for calibration.

[0182] Although the embodiment of the calibration system for a spatial recognition device for a work machine and the work machine according to the present invention have been described above, the present invention is not limited to the above embodiment. Various changes, modifications, substitutions, additions, deletions, and combinations are possible within the scope of the claims. Naturally, these also fall within the technical scope of the present invention. [Explanation of symbols]

[0183] 100 Crawler Crane 1 Undercarriage 2 Swivel mechanism 3. Upper rotating body 4. Boom 5 Mast 5 6. Backstop 7 Main winding rope 7 9 Counterweight 10 Cabin T1 Communications Equipment S6 imaging device S7 range finder L1~L4 light source 30 Controller 301 First object detection unit 302 Second object detection unit 303 Target Extraction Unit 304 Position determination section 305 Output control section 306 Feature Extraction Unit 307 Proofreading Department 800 Mobile communication terminals T2 Communication Equipment 801 Display device 802 Input Device 830 Controller 831 Display control unit 832 Selection section

Claims

1. A work machine; A plurality of space recognition devices provided on the work machine and capable of acquiring a surrounding situation of the work machine; a control device that performs calibration between the plurality of spatial recognition devices based on an object present around the work machine, The control device is configured to instruct each of the plurality of spatial recognition devices whether or not the object is present at a calibrable position. A calibration system for spatial recognition devices for work machines.

2. The control device is configured to determine whether or not the object is present at a position capable of being calibrated for each of the plurality of spatial recognition devices, and to output a determination result. The calibration system of claim 1 .

3. The control device is configured to output information representing a direction in which the object moves when it is determined that the object is not present in a calibration-enabled position. The calibration system of claim 2 .

4. The work machine is provided with a light source at a position corresponding to each of the plurality of spatial recognition devices, The control device is configured to illuminate the light sources provided at positions corresponding to each of the plurality of spatial recognition devices based on whether or not the object is present at a calibrable position. The calibration system of claim 1 .

5. Further comprising a communication device capable of communicating with an external device; The control device is configured to inform the external device via the communication device whether or not the object is present at a calibrable position. The calibration system of claim 1 .

6. The external device further comprises: The external device is configured to display image information indicating the detection result of the surrounding situation for each of the spatial recognition devices as an indication of whether the object is present in a calibration position. The calibration system of claim 5.

7. The spatial recognition device is provided in a number of three or more. The control device receives information indicating a calibration target from the external device, and selects the spatial recognition device to be calibrated based on the information. The calibration system of claim 5.

8. the control device is configured to set a plurality of positions to which the object moves for calibration, and to output the set plurality of positions to the external device via the communication device; Calibrating the plurality of spatial recognition devices based on the object present at each of the plurality of positions; The calibration system of claim 5.

9. The control device divides a region common to the detectable ranges of each of the plurality of spatial recognition devices into a predetermined number of regions, and sets a position where the object exists for calibration in each of the divided ranges. The calibration system of claim 8.

10. A plurality of spatial recognition devices capable of acquiring surrounding conditions of the work machine; a control device that performs calibration between the plurality of spatial recognition devices based on an object present around the work machine, The control device is configured to instruct each of the plurality of spatial recognition devices whether or not the object is present at a calibrable position. Working machinery.

Images