Industrial machinery, information processing equipment

Abstract

To provide a technology capable of estimating a travel area of a work machine.SOLUTION: A crawler crane 100 in accordance with an embodiment of the present invention is provided with: a distance sensor 45 for acquiring data on objects around the crawler crane 100; a feature quantity extraction unit 303 for acquiring feature quantities regarding unique features at positions corresponding to the ground around the crawler crane 100 based on an output of the distance sensor 45; and a travel area estimation unit 304 for estimating a travelable area of a work machine or an area around the crawler crane 100 to allow it to travel based on the feature quantities extracted by the feature quantity extraction unit 303.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] This disclosure relates to working machines and the like.

Background Art

[0002] Techniques for recognizing continuous ground are known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, at a work site, there may be defined an area where a working machine such as a crane or a shovel can travel or is permitted to travel (hereinafter referred to as "travel area") and an area where travel is impossible or prohibited (hereinafter referred to as "non-travel area"). For example, at a ground work site, in the travel area, a rectangular iron plate may be laid on the ground surface, and in the non-travel area, the ground surface such as earth and sand may be exposed. Also, at a work site on an upper floor of a building under construction with two or more floors, in the travel area, an iron plate may be similarly laid, and in the non-travel area, a concrete floor surface may be exposed.

[0005] However, in Patent Document 1, although continuous ground can be recognized, it is impossible to recognize whether the ground is a travel area or not.

[0006] Therefore, in view of the above problems, an object is to provide a technique capable of estimating the travel area of a working machine.

Means for Solving the Problems

[0007] To achieve the above object, in one embodiment of the present disclosure, A sensor that acquires data about objects around the work machine, The system includes an estimation unit that estimates a first area on the ground around the work machine where the work machine can or is permitted to move, based on the output of the aforementioned sensor. 、 The sensor outputs a detection wave around the work machine and receives the reflected wave, thereby acquiring data regarding the distance to objects around the work machine and the intensity of the reflected wave. The estimation unit estimates the first area around the work machine based on the distance to objects around the work machine and the intensity of the reflected waves. ru, Work equipment will be provided.

[0008] In other embodiments of this disclosure, The system acquires the output of a sensor that obtains data on objects within a predetermined range, and estimates a first area within the predetermined range where the work machine can or is permitted to move. Equipped with an estimation unit , The sensor outputs a detection wave within the predetermined range and receives the reflected wave, thereby acquiring data regarding the distance to an object within the predetermined range and the intensity of the reflected wave. The estimation unit estimates the first region within the predetermined range based on the distance to the object within the predetermined range and the intensity of the reflected wave. An information processing device is provided. [Effects of the Invention]

[0009] According to the above-described embodiment, the travel area of ​​the work machine can be estimated. [Brief explanation of the drawing]

[0010] [Figure 1] This is a diagram showing an example of a work machine. [Figure 2] This figure shows an example of an operational support system. [Figure 3] Block diagram showing an example of the hardware configuration of a crawler crane. [Figure 4] This is a functional block diagram showing the first example of a functional configuration for estimating the travel range of a crawler crane. [Figure 5] This flowchart schematically shows the first example of the main routine processing related to the estimation of the driving area. [Figure 6] This flowchart schematically shows an example of a subroutine process related to the estimation of the driving area. [Figure 7]The figure shows an example of an image representing the traveling area around a crawler crane, which is displayed on a display device. [Figure 8] It is a functional block diagram showing a second example of the functional configuration regarding the estimation of the traveling area. [Figure 9] It is a flowchart schematically showing a second example of the main routine process regarding the estimation of the traveling area. [Figure 10] The figure shows another example of a working machine.

Embodiments for Carrying out the Invention

[0011] Hereinafter, embodiments will be described with reference to the drawings.

[0012] [Overview of the Working Machine] Referring to FIGS. 1 and 2, the overview of the working machine according to this embodiment will be described.

[0013] FIG. 1 is a figure showing an example of a working machine. Specifically, FIG. 1 is a side view showing an example of a crawler crane 100. FIG. 2 is a figure showing an example of an operation support system SYS.

[0014] Hereinafter, when explaining the direction in the crawler crane 100 or the direction seen from the crawler crane 100, the direction in which the boom 4 extends in the top view of the crawler crane 100 will be defined as "front".

[0015] The crawler crane 100 is an example of a working machine. The crawler crane 100 includes a lower traveling body 1, an upper slewing body 3 mounted on the lower traveling body 1 so as to be slewing freely via a slewing mechanism 2, a boom 4, a mast 5, a backstop 6, a main hoist rope 7, a hook HK, a counterweight 9, and a cab 10.

[0016] The lower traveling body 1 drives the crawler crane 100. The lower traveling body 1 includes, for example, a pair of left and right crawlers 1C. Each crawler 1C includes one left crawler 1CL and the other left crawler 1CR. Crawler 1CL is hydraulically driven by a travel hydraulic motor 1ML (see Figure 3). Similarly, crawler 1CR is hydraulically driven by a travel hydraulic motor 1MR (see Figure 3). This allows the lower traveling body 1 to move under its own power.

[0017] The upper rotating body 3 is mounted on the lower traveling body 1 so as to be able to rotate freely via the rotating mechanism 2. Specifically, the upper rotating body 3 rotates relative to the lower traveling body 1 when the rotating mechanism 2 is hydraulically driven by the rotating hydraulic motor 2M (see Figure 3).

[0018] The boom 4 is mounted to the center of the front of the upper slewing body 3 in a luffable manner. A main hoisting rope 7 hangs down from the tip of the boom 4, and a hook HK is attached to the end of the main hoisting rope 7. In other words, the hook HK is attached to the tip of the boom 4 via the main hoisting rope 7.

[0019] The mast 5 is mounted on the upper slewing body 3 slightly behind the base end of the boom 4, and is rotatable around a pivot axis parallel to the pivot axis of the boom 4. The tip of the mast 5 is connected to the tip of the boom 4 via a pendant rope 5a. The boom 4 is raised and lowered via the mast 5 by winding and unwinding the boom luffing rope 5b using a boom luffing winch 5c, which is hydraulically driven by a luffing hydraulic motor 5M (see Figure 3).

[0020] The backstop 6 is mounted so as to be rotatable around a pivot axis parallel to the pivot axis of the boom 4, with its base end located behind the base end of the boom 4 in the upper slewing body 3. The backstop 6 is also mounted so as to be rotatable around a pivot axis parallel to the pivot axis of the boom 4, with its tip located on the rear surface between the base end and tip of the boom 4. The backstop 6 extends and retracts in accordance with the luffing motion of the boom 4, and for example, it has the function of supporting the boom 4 from the rear when the boom 4 is in a nearly upright position.

[0021] The main hoisting rope 7 has its base end attached to the main hoisting winch 7a, which is mounted on the rear portion between the base and tip of the boom 4, and its tip is attached to the hook HK. The main hoisting rope 7 is wound up and unwound by the main hoisting winch 7a, which is hydraulically driven by the main hoisting hydraulic motor 7M (see Figure 3), thereby raising and lowering the hook HK.

[0022] Hook HK is attached to the end of the main winding rope 7 and is used to suspend the load.

[0023] The counterweight 9 is located at the rear end of the upper slewing body 3 and has the function of balancing the weight of the boom 4 and the suspended load.

[0024] The cabin 10 is attached, for example, to the front right side of the upper rotating body 3. Inside the cabin 10 are a cockpit and operating devices 26 (see Figure 3) for operating various actuators.

[0025] The crawler crane 100 operates actuators (e.g., hydraulic actuators) in response to the operation of an operator sitting in the cabin 10, driving driven elements such as the lower traveling body 1, the upper slewing body 3, the boom 4, and the main hoisting rope 7.

[0026] Furthermore, the crawler crane 100 may be configured to be operable by an operator in the cabin 10, or in addition to being operable by an operator in the cabin 10, and may also be configured to be remotely operated from outside the crawler crane 100. When the crawler crane 100 is remotely operated, the cabin 10 may be unoccupied. Also, when the crawler crane 100 is remotely operated, the cabin 10 may be omitted. The following explanation will proceed on the premise that operator operation includes at least one of operation of the control device 26 by an operator in the cabin 10 and remote operation by an external operator.

[0027] Remote operation includes, for example, a mode in which the crawler crane 100 is operated by input from a user (operator) regarding the actuators of the crawler crane 100, which is performed by a remote operation support device located outside the crawler crane 100. The remote operation support device is, for example, the support device 200 described later. In this case, the crawler crane 100 transmits, for example, image information of the area around the crawler crane 100 (hereinafter referred to as "surrounding image") based on the output of the imaging device 40 described later to the support device 200, and the image information may be displayed on a display device (hereinafter referred to as "remote operation display device") provided in the support device 200. Similarly, various information images (information screens) displayed on the output device 50 (display device) inside the cabin 10 of the crawler crane 100 may also be displayed on the remote operation display device of the support device 200. This allows an operator using the support device 200 to remotely operate the crawler crane 100 while checking the displayed content, such as the surrounding image and various information images showing the area around the crawler crane 100, which are displayed on the remote operation display device. The crawler crane 100 may then operate actuators in response to remote control signals received from the support device 200, which represent the content of the remote control operation, thereby driving driven elements such as the lower traveling body 1, the upper slewing body 3, the boom 4, and the main hoisting rope 7.

[0028] Furthermore, remote operation may include, for example, a mode in which the crawler crane 100 is operated by external voice input or gesture input from people (e.g., workers) in the vicinity of the crawler crane 100. Specifically, the crawler crane 100 recognizes voices spoken by surrounding workers or gestures made by workers through a voice input device (e.g., microphone) or imaging device mounted on the crawler crane 100 (itself). Then, the crawler crane 100 may operate actuators according to the content of the instructions corresponding to the recognized voices or gestures, and drive driven elements such as the lower traveling body 1, the upper slewing body 3, the boom 4, and the main hoisting rope 7.

[0029] Furthermore, the crawler crane 100 may operate its actuators automatically, without operator intervention. This enables the crawler crane 100 to automatically operate at least some of its driven elements, such as the lower traveling body 1, the upper slewing body 3, the boom 4, and the main hoisting rope 7, thus realizing a so-called "automatic driving function" or "machine control (MC) function."

[0030] The automatic driving function may include a function that automatically operates driven elements (actuators) other than the target driven element (actuator) in response to the operator's operation of the control device 26 or remote control. In other words, the automatic driving function may include a so-called "semi-automatic driving function" or "operation support type MC function". Furthermore, the automatic driving function may include a function that automatically operates at least some of the multiple driven elements (hydraulic actuators) on the premise that there is no operation of the operator's control device 26 or remote control. In other words, the automatic driving function may include a so-called "fully automatic driving function" or "fully automatic type MC function". When the fully automatic driving function of the crawler crane 100 is enabled, the cabin 10 may be unoccupied. Also, when the crawler crane 100 is operating with the fully automatic driving function, the cabin 10 may be omitted. Furthermore, the semi-automatic driving function and fully automatic driving function may include a mode in which the operation content of the driven elements (actuators) that are the target of automatic driving is automatically determined according to predetermined rules. Furthermore, the semi-autonomous driving function and the fully autonomous driving function may include a mode in which the crawler crane 100 autonomously makes various decisions, and the operation of the driven elements (hydraulic actuators) of the autonomous driving target is determined autonomously in accordance with the results of those decisions. In other words, the semi-autonomous driving function and the fully autonomous driving function may include so-called "autonomous driving functions."

[0031] Furthermore, the operation of the crawler crane 100 may be remotely monitored. In this case, a remote monitoring support device having similar functions to the remote operation support device may be provided. The remote monitoring support device is, for example, the support device 200 described later. This allows the monitor, who is the user of the support device 200, to monitor the status of the crawler crane 100's operation while checking the surrounding image displayed on the support device's display. Also, for example, if the monitor deems it necessary from a safety standpoint, they can intervene in the operation of the crawler crane 100 by making a predetermined input using the input device of the remote monitoring support device and bring it to an emergency stop.

[0032] Furthermore, as shown in Figure 2, the crawler crane 100 may be connected to the support device 200 via a communication line NW using a communication device 60 (see Figure 3), and together with the support device 200, it may be a component of the operational support system SYS.

[0033] The SYS operational support system uses the support device 200 to provide support for the operation of the crawler crane 100.

[0034] The support device 200 communicates with the crawler crane 100 to cooperate with it and provide support for the operation of the crawler crane 100.

[0035] The functions of the support device 200 are realized by any hardware or any combination of hardware and software. For example, the support device 200 is centered around a computer including a CPU (Central Processing Unit), memory device, auxiliary storage device, interface device, input device, and output device. The memory device is, for example, SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory). The auxiliary storage device is, for example, HDD (Hard Disk Drive), SSD (Solid State Disk), EEPROM (Electrically Erasable Programmable Read-Only Memory), or flash memory. The interface device includes an external interface for connecting to an external recording medium and a communication interface for communicating with the outside, such as the crawler crane 100. The input device includes, for example, a lever-type operation input device. The output device includes a display device and a sound output device. The display device is, for example, a liquid crystal display or an organic EL (Electroluminescence) display. The sound output device is, for example, a speaker.

[0036] The support device 200 is a server or management terminal device installed, for example, in a management office within the crawler crane 100's work site, or in a management center located in a different location from the crawler crane 100's work site, which manages the operating status of the crawler crane 100. The management terminal device may be a stationary terminal device such as a desktop PC (Personal Computer), or a portable terminal device (mobile terminal) such as a tablet, smartphone, or laptop PC. In the latter case, workers at the work site, supervisors overseeing the work, and managers managing the work site can carry the portable support device 200 and move around the work site. In the latter case, the operator can, for example, bring the portable support device 200 into the cabin of the crawler crane 100.

[0037] For example, the support device 200 acquires data regarding the operating status from the crawler crane 100. This allows the support device 200 to understand the operating status of the crawler crane 100 and monitor for any abnormalities. Furthermore, the support device 200 can display the data regarding the operating status of the crawler crane 100 through its own display device, allowing the user to confirm it.

[0038] Furthermore, the support device 200 may transmit various data, such as programs and reference data used in processing by the controller 30, to the crawler crane 100. This allows the crawler crane 100 to perform various operations related to its operation using the various data downloaded from the support device 200.

[0039] Furthermore, the support device 200 may assist in the remote operation and remote monitoring of the crawler crane 100, as described above.

[0040] [Hardware configuration of crawler crane] Next, with reference to Figures 1 and 2, as well as Figure 3, the hardware configuration of the crawler crane 100 will be described.

[0041] The crawler crane 100 includes various components such as a hydraulic drive system for hydraulically driving the driven elements, an operating system for operating the driven elements, a user interface system for exchanging information with the user, a communication system for communication with the outside, and a control system for various types of control.

[0042] <Hydraulic drive system> As shown in Figure 3, the hydraulic drive system of the crawler crane 100 according to this embodiment includes hydraulic actuators HA that hydraulically drive each of the driven elements, such as the lower traveling body 1 (left and right crawlers 1C), the upper slewing body 3, the boom 4, and the main hoisting rope 7, as described above. The hydraulic drive system of the crawler crane 100 according to this embodiment also includes an engine 11, a regulator 13, a main pump 14, and a control valve 17.

[0043] The hydraulic actuator HA includes travel hydraulic motors 1ML and 1MR, swing hydraulic motor 2M, luffing hydraulic motor 5M, and main hoisting hydraulic motor 7M, etc.

[0044] Furthermore, the crawler crane 100 may have some or all of its hydraulic actuator HA replaced with an electric actuator.

[0045] Engine 11 is the prime mover and the main power source in the hydraulic drive system. Engine 11 is, for example, a diesel engine that uses light oil as fuel. Engine 11 is mounted, for example, at the rear of the upper slewing body 3. Under direct or indirect control by the controller 30, which will be described later, Engine 11 rotates at a constant speed at a preset target speed and drives the main pump 14 and the pilot pump 15.

[0046] The regulator 13 controls (adjusts) the discharge rate of the main pump 14 under the control of the controller 30. For example, the regulator 13 adjusts the angle of the swash plate of the main pump 14 (hereinafter referred to as the "tilt angle") in response to a control command from the controller 30.

[0047] The main pump 14 supplies hydraulic fluid to the control valve 17 through a high-pressure hydraulic line. The main pump 14 is mounted, for example, at the rear of the upper slewing body 3, similar to the engine 11. The main pump 14 is driven by the engine 11, as described above. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the controller 30, the piston stroke length is adjusted by adjusting the tilt angle of the swash plate by the regulator 13, thereby controlling the discharge flow rate (discharge pressure).

[0048] The control valve 17 drives the hydraulic actuators HA in response to the operator's operation of the control device 26, the content of remote operation, or operation commands related to the automatic operation function output from the controller 30. The control valve 17 is mounted, for example, in the center of the upper rotating body 3. As described above, the control valve 17 is connected to the main pump 14 via a high-pressure hydraulic line and selectively supplies hydraulic fluid supplied from the main pump 14 to each hydraulic actuator in response to the operator's operation or operation commands output from the controller 30. Specifically, the control valve 17 includes a plurality of control valves (also called "direction control valves") that control the flow rate and direction of the hydraulic fluid supplied from the main pump 14 to each of the hydraulic actuators HA.

[0049] <Operation system> As shown in Figure 3, the operating system of the crawler crane 100 according to this embodiment includes a pilot pump 15, an operating device 26, a hydraulic control valve 31, a shuttle valve 32, and a hydraulic control valve 33.

[0050] The pilot pump 15 supplies pilot pressure to various hydraulic devices via the pilot line 25. The pilot pump 15 is mounted, for example, at the rear of the upper slewing body 3, similar to the engine 11. The pilot pump 15 is, for example, a fixed-displacement hydraulic pump and is driven by the engine 11 as described above.

[0051] The pilot pump 15 may be omitted. In this case, the relatively high-pressure hydraulic fluid discharged from the main pump 14 is reduced in pressure by a predetermined pressure reducing valve, and the resulting relatively low-pressure hydraulic fluid is supplied to various hydraulic devices as pilot pressure.

[0052] The control device 26 is located near the cockpit of the cabin 10 and is used by the operator to operate various driven elements. In other words, the control device 26 is used by the operator to operate the hydraulic actuators HA that drive each driven element. The control device 26 includes pedal devices and lever devices for operating each driven element (hydraulic actuator HA).

[0053] For example, as shown in Figure 3, the operating device 26 is hydraulically pilot operated. Specifically, the operating device 26 uses hydraulic fluid supplied from the pilot pump 15 through the pilot line 25 and the pilot line 25A branched therefrom, and outputs a pilot pressure corresponding to the operation to the secondary pilot line 27A. The pilot line 27A is connected to one inlet port of the shuttle valve 32 and, via the pilot line 27 connected to the outlet port of the shuttle valve 32, is connected to the control valve 17. As a result, the control valve 17 can receive a pilot pressure via the shuttle valve 32 corresponding to the operation of various driven elements (hydraulic actuators) in the operating device 26. Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation performed on the operating device 26 by an operator or the like.

[0054] Furthermore, the operating device 26 may be electrically operated. Specifically, the operating device 26 outputs an electrical signal (hereinafter referred to as "operating signal") corresponding to the operation content, and the operating signal is received by the controller 30. The controller 30 then outputs a control command corresponding to the content of the operating signal, that is, a control signal corresponding to the operation content of the operating device 26, to the hydraulic control valve 31. As a result, a pilot pressure corresponding to the operation content of the operating device 26 is input from the hydraulic control valve 31 to the control valve 17, and the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26.

[0055] Furthermore, if the operating device 26 is electrically operated, the operating pressure sensor 29, shuttle valve 32, and hydraulic control valve 33, which will be described later, are omitted.

[0056] Furthermore, the control valves (directional control valves) built into the control valve 17 that drive each hydraulic actuator may be of the electromagnetic solenoid type. In this case, the operating signal output from the operating device 26 may be directly input to the control valve 17, that is, to the electromagnetic solenoid type control valve.

[0057] Furthermore, as described above, some or all of the hydraulic actuator HA may be replaced with an electric actuator. In this case, the controller 30 may output control commands to the electric actuator or a driver that drives the electric actuator, etc., according to the operation content of the operating device 26 and the content of the remote operation specified by the remote operation signal.

[0058] Furthermore, if the crawler crane 100 is remotely controlled or operates using an automatic driving function, the control device 26 may be omitted.

[0059] A hydraulic control valve 31 is provided for each driven element (hydraulic actuator HA) that the operating device 26 is designed to operate, and for each driving direction of the driven element (hydraulic actuator HA) (for example, the upward and downward directions of the boom 4). In other words, two hydraulic control valves 31 are provided for each double-acting hydraulic actuator HA. The hydraulic control valve 31 is provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17, and may be configured to allow changes in its flow area (i.e., the cross-sectional area through which hydraulic fluid can flow). As a result, the hydraulic control valve 31 can use the hydraulic fluid from the pilot pump 15 supplied through the pilot line 25B to output a predetermined pilot pressure to the secondary pilot line 27B. Therefore, as shown in Figure 3, the hydraulic control valve 31 can indirectly apply a predetermined pilot pressure to the control valve 17 in accordance with the control signal from the controller 30 through the shuttle valve 32 between the pilot line 27B and the pilot line 27. Therefore, the controller 30 can supply pilot pressure from the hydraulic control valve 31 to the control valve 17 according to the operation of the operating device 26, thereby enabling the operation of the crawler crane 100 based on the operator's input.

[0060] Furthermore, the controller 30 may, for example, control the hydraulic control valve 31 to realize an automatic operation function. Specifically, the controller 30 outputs a control signal to the hydraulic control valve 31 corresponding to an operation command related to the automatic operation function, regardless of whether the operating device 26 is operated or not. As a result, the controller 30 causes the hydraulic control valve 31 to supply pilot pressure corresponding to the operation command related to the automatic operation function to the control valve 17, thereby realizing the operation of the crawler crane 100 based on the automatic operation function.

[0061] Furthermore, the controller 30 may, for example, control the hydraulic control valve 31 to enable remote operation of the crawler crane 100. Specifically, the controller 30 outputs a control signal to the hydraulic control valve 31 via the communication device 60 that corresponds to the content of the remote operation specified by the remote operation signal received from the support device 200. As a result, the controller 30 causes the hydraulic control valve 31 to supply pilot pressure corresponding to the content of the remote operation to the control valve 17, thereby enabling operation of the crawler crane 100 based on the operator's remote operation.

[0062] The shuttle valve 32 has two inlet ports and one outlet port, and outputs hydraulic fluid with the higher of the two pilot pressures input to the two inlet ports to the outlet port. A shuttle valve 32 is provided for each driven element (hydraulic actuator HA) that is operated by the operating device 26. One of the two inlet ports of the shuttle valve 32 is connected to the pilot line 27A on the secondary side of the operating device 26 (specifically, the lever device or pedal device included in the operating device 26), and the other is connected to the pilot line 27B on the secondary side of the hydraulic control valve 31. The outlet port of the shuttle valve 32 is connected to the pilot port of the corresponding control valve of the control valve 17 via the pilot line 27. The corresponding control valve is the control valve that drives the hydraulic actuator that is operated by the lever device or pedal device connected to one of the inlet ports of the shuttle valve 32. Therefore, each of these shuttle valves 32 can apply the higher of the pilot pressure of the pilot line 27A on the secondary side of the operating device 26 and the pilot pressure of the pilot line 27B on the secondary side of the hydraulic control valve 31 to the pilot port of the corresponding control valve. In other words, the controller 30 can control the corresponding control valve regardless of the operator's operation of the operating device 26 by outputting a pilot pressure from the hydraulic control valve 31 that is higher than the pilot pressure on the secondary side of the operating device 26. Thus, the controller 30 can control the operation of the driven elements (lower traveling body 1, upper slewing body 3, boom 4, and main hoisting rope 7) regardless of the operator's operation of the operating device 26, thereby realizing remote control and automatic operation functions.

[0063] The hydraulic control valve 33 is provided in the pilot line 27A connecting the operating device 26 and the shuttle valve 32. The hydraulic control valve 33 is configured, for example, to allow the flow path area to be changed. The hydraulic control valve 33 operates in response to a control signal input from the controller 30. This allows the controller 30 to forcibly reduce the pilot pressure output from the operating device 26 when the operating device 26 is operated by an operator. Therefore, even when the operating device 26 is being operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operating device 26. In addition, the controller 30 can reduce the pilot pressure output from the operating device 26, for example, even when the operating device 26 is being operated, to a level lower than the pilot pressure output from the hydraulic control valve 31. Therefore, by controlling the hydraulic control valve 31 and the hydraulic control valve 33, the controller 30 can reliably apply a desired pilot pressure to the pilot port of the control valve in the control valve 17, for example, regardless of the operation of the operating device 26. Therefore, the controller 30 can more effectively realize the automatic operation function and remote control function of the crawler crane 100 by controlling, for example, the hydraulic control valve 33 in addition to the hydraulic control valve 31.

[0064] <User Interface System> As shown in Figure 3, the user interface system of the crawler crane 100 according to this embodiment includes an operating device 26, an output device 50, and an input device 52.

[0065] The output device 50 outputs various information to the user (operator) of the crawler crane 100 inside the cabin 10.

[0066] For example, the output device 50 includes indoor lighting equipment and display devices 50A (see Figures 4 and 8) that are installed in a location easily visible to a seated operator inside the cabin 10 and output various information in a visual manner. The lighting equipment is, for example, a warning light (indicator lamp). The display device 50A is, for example, a liquid crystal display or an organic EL (electroluminescence) display. For example, the lighting equipment and display devices 50A are installed inside the cabin 10 and output various information in a visual manner to the operator inside the cabin 10. Alternatively, the lighting equipment and display devices 50A may be installed, for example, on the side of the upper rotating body 3 and output various information in a visual manner to workers around the crawler crane 100.

[0067] Furthermore, the output device 50 may include a sound output device 50B (see Figure 8) that outputs various information in an audible manner. The sound output device 50B may include, for example, a buzzer or a speaker. For example, the sound output device 50B may be installed inside and outside the cabin 10 and output various information in an audible manner to the operator inside the cabin 10 or to people (workers, etc.) around the crawler crane 100.

[0068] Furthermore, the output device 50 may also include a device that outputs various types of information through tactile means such as vibrations in the cockpit.

[0069] The input device 52 receives various inputs from the user of the crawler crane 100, and the signals corresponding to the received inputs are taken into the controller 30. The input device 52 is installed, for example, inside the cabin 10 and receives inputs from operators, etc., inside the cabin 10. Alternatively, the input device 52 may be installed, for example, on the side of the upper slewing body 3 and receive inputs from workers, etc., around the crawler crane 100.

[0070] For example, the input device 52 includes a mechanical input device that accepts mechanical input from a user. The mechanical input device may include a touch panel mounted on the display device 50A, a touchpad installed around the display device 50A, a button switch, a lever, a toggle, a knob switch provided on the operating device 26 (lever device), and the like.

[0071] Furthermore, the input device 52 may include a voice input device that accepts voice input from the user. The voice input device may include, for example, a microphone.

[0072] Furthermore, the input device 52 may include a gesture input device that receives gesture input from the user. The gesture input device may include, for example, an imaging device that captures images of the gestures performed by the user.

[0073] Furthermore, the input device 52 may include a biometric input device that accepts biometric input from the user. Biometric input may include, for example, the input of biometric information such as the user's fingerprints or iris scan.

[0074] <Communications System> As shown in Figure 3, the communication system of the crawler crane 100 according to this embodiment includes a communication device 60.

[0075] The communication device 60 connects to a communication line NW and communicates with a device (for example, a support device 200) that is provided separately from the crawler crane 100. The device provided separately from the crawler crane 100 may include not only devices located outside the crawler crane 100, but also portable terminal devices (for example, a support device 200) brought into the cabin 10 by the user of the crawler crane 100. The communication device 60 uses, for example, 4G (4 th Generation) and 5G (5 th The communication device 60 may include a mobile communication module that conforms to standards such as Generation. Furthermore, the communication device 60 may include, for example, a satellite communication module. Additionally, the communication device 60 may include, for example, a WiFi communication module or a Bluetooth® communication module.

[0076] <Control System> As shown in Figure 3, the control system of the crawler crane 100 according to this embodiment includes a controller 30. The control system of the crawler crane 100 according to this embodiment also includes an operating pressure sensor 29, an imaging device 40, a distance sensor 45, and sensors S1 to S4.

[0077] The controller 30 performs various controls related to the crawler crane 100.

[0078] The functions of the controller 30 may be realized by any hardware, or any combination of hardware and software. For example, as shown in Figure 3, the controller 30 includes an auxiliary storage device 30A, a memory device 30B, a CPU (Central Processing Unit) 30C, and an interface device 30D, all connected by bus B1.

[0079] The auxiliary storage device 30A is a non-volatile storage means that stores the installed program as well as necessary files and data. The auxiliary storage device 30A is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory) or flash memory.

[0080] The memory device 30B loads the program from the auxiliary storage device 30A into the CPU 30C's readable state, for example, when a program startup command is received. The memory device 30B is, for example, SRAM (Static Random Access Memory).

[0081] The CPU 30C, for example, executes a program loaded into the memory device 30B and implements various functions of the controller 30 according to the program's instructions.

[0082] The interface device 30D functions, for example, as a communication interface for connecting to a communication line inside the crawler crane 100. The interface device 30D may include multiple different types of communication interfaces to match the type of communication line to be connected.

[0083] Furthermore, the interface device 30D functions as an external interface for reading data from and writing data to a recording medium. The recording medium is, for example, a dedicated tool connected by a detachable cable to a connector installed inside the cabin 10. Alternatively, the recording medium may be a general-purpose recording medium such as an SD memory card or a USB (Universal Serial Bus) memory. Thus, the program that realizes the various functions of the controller 30 may be provided, for example, by a portable recording medium and installed in the auxiliary storage device 30A of the controller 30. Alternatively, the program may be downloaded from another computer outside the crawler crane 100 via the communication device 60 and installed in the auxiliary storage device 30A.

[0084] Furthermore, some of the functions of controller 30 may be implemented by other controllers (control devices). In other words, the functions of controller 30 may be implemented in a distributed manner by multiple controllers.

[0085] The operating pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot-operated operating device 26, that is, the pilot pressure corresponding to the operating state of each driven element (hydraulic actuator) in the operating device 26. The detection signal of the pilot pressure corresponding to the operating state of each driven element (hydraulic actuator HA) in the operating device 26, detected by the operating pressure sensor 29, is received by the controller 30.

[0086] The imaging device 40 is mounted on the upper part of the upper rotating body 3 and acquires images of the area around the crawler crane 100. The imaging device 40 may also acquire (generate) three-dimensional data (hereinafter simply referred to as "three-dimensional data of objects") representing the position and outline of objects around the crawler crane 100 within the imaging range (angle of view) based on the acquired images and distance data described later. The three-dimensional data of objects around the crawler crane 100 may include, for example, coordinate information data of point clouds representing the surface of objects or distance image data.

[0087] For example, as shown in Figure 1, the imaging device 40 includes a camera that images the front of the upper slewing body 3. The imaging device 40 may also include at least one of the following cameras: one that images the rear of the upper slewing body 3, one that images the left side of the upper slewing body 3, and one that images the right side of the upper slewing body 3. This allows the operator to view the images captured by the imaging device 40 and surrounding images such as processed images generated based on those images via the display device 50A or the remote control display device, and to check the condition of the upper slewing body 3 in at least one direction: front, left, right, and rear. Furthermore, for example, the operator can remotely operate the crawler crane 100 while checking the condition of the suspended load by viewing the images captured by the imaging device 40 and surrounding images such as processed images generated based on those images via the remote control display device.

[0088] The imaging device 40 is, for example, a monocular camera. Alternatively, the imaging device 40 may be capable of acquiring distance (depth) data in addition to two-dimensional images, such as a stereo camera or a TOF (Time of Flight) camera (hereinafter collectively referred to as a "3D camera").

[0089] The output data from the imaging device 40 (for example, image data and three-dimensional data of objects surrounding the crawler crane 100) is received by the controller 30 via a one-to-one communication line or an in-vehicle network. This allows the controller 30 to monitor objects surrounding the crawler crane 100 based on the output data from the imaging device 40. Furthermore, the controller 30 can understand the surrounding environment of the crawler crane 100 based on the output data from the imaging device 40. Additionally, the controller 30 can determine the attitude state of the crawler crane 100's body (upper rotating body 3) based on objects surrounding the crawler crane 100, using the output data from the imaging device 40 as a reference.

[0090] The distance sensor 45 is mounted on the upper part of the upper rotating body 3 and acquires data on the distance to the crawler crane 100 and its surroundings, with the crawler crane 100 or itself (distance sensor 45) as the reference. Specifically, the distance sensor 45 outputs a predetermined detection wave (also called an "irradiation wave") to the area around the crawler crane 100 and receives the reflected wave, thereby acquiring data on the distance to objects around the crawler crane 100 in a predetermined manner such as the TOF (Time of Flight) method. The predetermined detection wave is a laser (light) or millimeter wave, etc. The distance sensor may also acquire (generate) three-dimensional data (for example, point cloud coordinate information data) of objects around the crawler crane 100 within its sensing range based on the acquired data. The distance sensor 45 can acquire coordinate information of an object based on the output direction of the detection wave and the distance to the object that reflected the detection wave. The coordinates of each point in the point cloud data are represented, for example, on a moving coordinate system with a predetermined location on the crawler crane 100 as the reference. Furthermore, if it is possible to obtain information regarding the absolute position of the crawler crane 100 on a fixed coordinate system fixed to the ground using a positioning device or the like, the coordinates of each point in the point cloud data may be represented on the fixed coordinate system.

[0091] For example, as shown in Figure 2, the distance sensor 45 includes a distance sensor mounted on the front of the upper slewing body 3 (cabin 10) of the crawler crane 100, capable of acquiring data on the distance to an object in front of the crawler crane 100. The distance sensor 45 may also include a distance sensor mounted on the left side of the upper slewing body 3, capable of acquiring data on the distance to an object to the left of the crawler crane 100. The distance sensor 45 may also include a distance sensor mounted on the right side of the upper slewing body 3, capable of acquiring data on the distance to an object to the right of the crawler crane 100. The distance sensor 45 may also include a distance sensor mounted on the rear of the upper slewing body 3, capable of acquiring data on the distance to an object behind the crawler crane 100.

[0092] The distance sensor 45 is, for example, a LiDAR (Light Detection and Ranging) that outputs a laser (light) as a detection wave. Alternatively, the distance sensor may be, for example, a millimeter-wave radar that outputs millimeter waves as a detection wave.

[0093] Sensor S1 acquires detection data regarding the attitude angle of the boom 4 relative to a predetermined reference (for example, the horizontal plane or the state at either end of the boom 4's movable angle range). Sensor S1 may include, for example, a rotary encoder, an acceleration sensor, an angular velocity sensor, a six-axis sensor, an IMU (Inertial Measurement Unit), etc. The detection data from sensor S1 is input to the controller 30.

[0094] Sensor S2 acquires detection data regarding the tilt state of the crawler crane 100, including the lower traveling body 1 and the upper rotating body 3. Sensor S2 is mounted on the upper rotating body 3, for example, and acquires detection data regarding the tilt angles of the upper rotating body 3 in the longitudinal and lateral directions (hereinafter referred to as "longitudinal tilt angle" and "lateral tilt angle"). Sensor S2 may include, for example, an acceleration sensor (tilt sensor), an angular velocity sensor, a six-axis sensor, an IMU, etc. The detection data from Sensor S2 is input to the controller 30.

[0095] Sensor S3 acquires detection data regarding the rotation state of the upper rotating body 3. Sensor S3 also acquires detection data regarding the rotation angle of the upper rotating body 3 relative to a predetermined standard (for example, the state in which the forward direction of the lower traveling body 1 coincides with the forward direction of the upper rotating body 3). Sensor S3 includes, for example, a potentiometer, rotary encoder, resolver, etc. The detection data from sensor S3 is received by the controller 30.

[0096] Furthermore, if the components of sensor S2 (for example, a six-axis sensor or an IMU) can acquire detection data regarding the attitude state of the upper rotating body 3, including not only the tilt angle but also the rotation angle, then sensor S3 may be omitted.

[0097] Sensor S4 acquires detection data regarding the relative position of the suspended load with respect to boom 4. Sensor S4 is, for example, an image sensor (camera) or a distance sensor. The detection data from sensor S4 is input to controller 30.

[0098] Alternatively, a sensor capable of detecting the winding state of the main winding rope 7 may be provided instead of, or in addition to, sensor S4.

[0099] Furthermore, the crawler crane 100 may be equipped with a positioning device capable of determining its absolute position. The positioning device may be, for example, a GNSS (Global Navigation Satellite System) sensor. This can improve the accuracy of estimating the attitude state of the crawler crane 100.

[0100] The controller 30 can determine the position of the suspended load hanging from the hook HK based on data from sensors S1 to S4, etc.

[0101] Furthermore, depending on the specifications regarding the function of the crawler crane 100, sensors S1 to S4, etc., may be omitted.

[0102] [First example of a function related to estimating the driving area] Next, with reference to Figures 1 to 3, as well as Figures 4 to 7, we will describe a first example of a function related to estimating the area on the ground around the crawler crane 100 where travel is possible or permitted (travel area).

[0103] <Functional Configuration> Figure 4 is a functional block diagram showing a first example of the functional configuration for estimating the travel area of ​​the crawler crane 100.

[0104] As shown in Figure 4, the controller 30 includes, as functional units, a data acquisition unit 301, a preprocessing unit 302, a feature extraction unit 303, a driving area estimation unit 304, and a display processing unit 305.

[0105] The data acquisition unit 301 acquires the output data from the distance sensor 45. The output data from the distance sensor 45 is, for example, point cloud data representing the surface of objects around the crawler crane 100. The point cloud data includes, for example, the coordinate data of each point and data relating to the intensity of the reflected wave corresponding to each point (hereinafter, "reflection intensity"). The following explanation will proceed on the premise that the output data from the distance sensor 45 is point cloud data.

[0106] The preprocessing unit 302 applies known preprocessing to the output data (point cloud data) from the distance sensor 45. Preprocessing includes, for example, noise reduction and downsampling.

[0107] The feature extraction unit 303 extracts feature quantities related to features specific to the travel area around the crawler crane 100, based on the output data (point cloud data) of the distance sensor 45 that has been preprocessed by the preprocessing unit 302.

[0108] For example, in the area where the crawler crane 100 operates on the ground, rectangular steel plates (base plates) are laid out on the exposed ground surface where soil and sand are located. Similarly, in the area where the crawler crane 100 operates on the upper floors of a building under construction with two or more stories, rectangular steel plates (base plates) are laid out on the exposed concrete floor surface. Therefore, the feature extraction unit 303 extracts features related to the features unique to the steel plates. Specifically, the feature extraction unit 303 extracts (acquires) the vertical height value and the reflectance value of each point in the point cloud data as features. This is because the ground surface on which the steel plates are laid is approximately horizontal. Also, the vertical height of the steel plates is lower than in non-operational areas where movement is impossible due to obstacles such as people or traffic cones. Furthermore, the reflectivity of the steel plates can differ to a degree that allows them to be distinguished from areas where the crawler crane 100 cannot or is prohibited from traveling (non-traveling areas), such as areas where soil and sand on the ground surface are exposed. Specifically, when the distance sensor 45 is a LIDAR, the reflectivity is affected by the color of the object; for example, white objects have high reflectivity, and black objects have low reflectivity. Therefore, since the color of the steel plates and areas where soil and sand on the ground surface are exposed are usually completely different, the characteristics of the steel plates laid in the traveling area are reflected in their reflectivity. In addition, the reflectivity of the steel plates can differ to a degree that allows them to be distinguished from non-traveling areas where travel is impossible, such as areas where obstacles such as people or traffic cones are present.

[0109] The travel area estimation unit 304 estimates the travel area and non-travel area around the crawler crane 100 based on the feature quantities (height value and reflectance value) of each point data included in the point cloud data, for each position around the crawler crane 100.

[0110] Based on the estimation results of the travel area estimation unit 304, the display processing unit 305 displays an image on the display device 50A that superimposes on the surrounding image, representing the travel area and non-travel area within the range shown in the surrounding image (see, for example, Figure 8). The surrounding image may be the image captured by the imaging device 40 itself, as described above, or it may be a processed image based on the captured image (for example, an overhead view image). This allows the operator in the cabin 10 to operate the crawler crane 100 while checking the travel area and non-travel area around the crawler crane 100 through the display device 50A. Furthermore, when the crawler crane 100 is remotely operated or remotely monitored, the display processing unit 305 may transmit display image data, including the surrounding image and an image representing the travel area within the range shown in the surrounding image, to the support device 200 via the communication device 60. This allows the support device 200 to display the same image as the display device 50A on its own display device. Therefore, operators using the support device 200 can remotely control the movement of the crawler crane 100 while checking the travel and non-travel areas around the crawler crane 100 through the support device 200 (display device). In addition, monitors using the support device 200 can monitor the movement of the crawler crane 100 while checking the travel and non-travel areas around the crawler crane 100 through the support device 200 (display device).

[0111] <Processing> Figure 5 is a flowchart schematically showing the first example of the main routine processing for estimating the travel area around the crawler crane 100. Figure 6 is a flowchart of an example of subroutine processing for estimating the travel area around the crawler crane 100. Specifically, Figure 6 is a flowchart of the subroutine corresponding to the processing in step S108 of Figure 5 (travel area estimation processing). Figure 7 is a diagram showing an example of an image representing the travel area around the crawler crane 100, which is displayed on the display device 50A.

[0112] The flowchart in Figure 5 is executed repeatedly at predetermined intervals, for example, from the start of operation (key switch on) to the stop of operation (key switch on) of the crawler crane 100. Furthermore, the estimation function for the travel area may be switchable between enabled and disabled by a predetermined input from the input device 52. In this case, the flowchart in Figure 5 may be executed repeatedly at predetermined control intervals when the estimation function for the travel area is enabled, from the start of operation (key switch on) to the stop of operation (key switch on) of the crawler crane 100. The same applies to the flowchart in Figure 9, which will be described later.

[0113] As shown in Figure 5, in step S102, the data acquisition unit 301 acquires point cloud data from the distance sensor 45.

[0114] When the controller 30 completes the processing in step S102, it proceeds to step S104.

[0115] In step S104, the preprocessing unit 302 performs predetermined preprocessing on the point cloud data acquired in step S102.

[0116] When the controller 30 completes the processing in step S104, it proceeds to step S106.

[0117] In step S106, the feature extraction unit 303 extracts the height value and the reflection intensity value as feature quantities for each point data included in the point cloud data, based on the point cloud data that has been preprocessed in step S104.

[0118] When the controller 30 completes the processing in step S106, it proceeds to step S108.

[0119] In step S108, the travel area estimation unit 304 moves to the subroutine shown in Figure 8 and performs a process (travel area estimation process) to estimate the travel area around the crawler crane 100 based on the feature quantities acquired in step S106.

[0120] As shown in Figure 8, in step S202, the travel area estimation unit 304 sets a reference position on the ground around the crawler crane 100 for comparing feature quantities with the position to be determined. The reference position is represented on a moving coordinate system based on a predetermined point on the crawler crane 100. Alternatively, if information regarding the absolute position of the crawler crane 100 on a fixed coordinate system fixed to the ground can be obtained by a positioning device or the like, the reference position may be represented on the fixed coordinate system. The reference position is set to a position on the ground that has been previously recognized as a travel area. For example, the initial reference position is set from a point on the contact surface of the lower travel body 1 of the crawler crane 100. After the processing in step S212, if the determination result of the previous point data is a travel area, the reference position is set to the position of the point data that was the target of the previous determination. On the other hand, after the processing in step S212, if the determination result of the previous point data is a non-travel area, the reference position is maintained at the previous reference position.

[0121] When the controller 30 completes the processing in step S202, it proceeds to step S204.

[0122] In step S204, the travel area estimation unit 304 sets a location around the crawler crane 100 that is to be determined as either a travel area or a non-travel area. Specifically, the travel area estimation unit 304 sets (selects) a point data from among the point data included in the point cloud data that is to be determined as either a travel area or a non-travel area. The location (point data) to be determined is selected by a known method from the point data adjacent to the reference position in the point cloud data. However, if the previous determination result for the point data was a non-travel area, the location (point data) to be determined is set by a known method from the point data adjacent to the reference position and different from the location to be determined in the previous determination.

[0123] When the controller 30 completes the processing in step S204, it proceeds to step S206.

[0124] In step S206, the driving area estimation unit 304 determines whether the gradient of the feature quantities between the position (point data) to be determined and the reference position (point data) is below a certain threshold. Specifically, the driving area estimation unit 304 determines whether the gradient of the height value between the position (point data) to be determined and the reference position (point data) is below a certain threshold Th1, and whether the gradient of the reflection intensity value is below a certain threshold. Thresholds Th1 and Th2 are predetermined values ​​that take into account a margin over the maximum gradient expected when both the position to be determined and the reference position are within the driving area, based on experiments, computer simulations, etc. If the gradient of the feature quantities between the position (point data) to be determined and the reference position (point data) is below a certain threshold, the driving area estimation unit 304 proceeds to step S208; otherwise, it proceeds to step S210.

[0125] In step S208, the driving area estimation unit 304 determines that the location of the point data to be judged is within the driving area. This is because, as described above, the driving area is covered with steel plates (steel plates), and the change in reflectivity (gradient) is relatively small, as is the change in height (gradient).

[0126] When the controller 30 completes the processing in step S208, it proceeds to step S212.

[0127] In step S210, the travel area estimation unit 304 determines that the location of the point data to be judged is in a non-travel area. This is because in non-travel areas where soil and other materials are exposed, the value of the reflectivity differs significantly from that of the steel plates in adjacent travel areas, and the change in reflectivity (slope) between the location of the adjacent travel area is considered to be relatively large. Also, in non-travel areas where obstacles such as people or traffic cones exist, the vertical height is greater than that of the steel plates in adjacent travel areas, and the change in height (slope) between the location of the adjacent travel area is considered to be relatively large.

[0128] When the controller 30 completes the processing in step S210, it proceeds to step S212.

[0129] In step S212, the driving area estimation unit 304 determines whether the termination condition has been met. The termination condition is that it is not possible to set point data to be determined for the next determination. For example, if all the point cloud data is in the driving area, the driving area estimation unit 304 can determine whether all point data is in the driving area or not by appropriately selecting adjacent point data as data to be determined. The termination condition is met when the determination for all point data is completed. On the other hand, if the point cloud data includes a non-driving area, a group of point data in the non-driving area corresponding to the boundary line is formed by sequentially finding point data adjacent to the driving area that corresponds to the boundary between the driving area and the non-driving area. As a result, the point data in the non-driving area other than the point data in the driving area is not set as a point to be determined as an adjacent point to the point data in the driving area, and the termination condition is met. If the termination condition is met, the driving area estimation unit 304 proceeds to step S214.

[0130] In step S214, the driving area estimation unit 304 determines whether or not there are any undetermined positions (point data). If there are any undetermined positions (point data), the driving area estimation unit 304 proceeds to step S216; otherwise, it terminates the processing in this flowchart (the processing in step S108 in Figure 5).

[0131] In step S216, the driving area estimation unit 304 determines that the location of the undetermined point data is a non-driving area.

[0132] When the process in step S216 is completed, the controller 30 terminates the process in this flowchart (the process in step S108 in Figure 5).

[0133] Returning to Figure 5, once the controller 30 completes the processing in step S108, it proceeds to step S110.

[0134] In step S110, the display processing unit 305 displays an image representing the travel area around the crawler crane 100 on the display device 50A, superimposed on the surrounding image, based on the result of the travel area estimation process in step S108 (see Figure 7).

[0135] For example, as shown in Figure 7, the display device 50A displays an image 700 acquired by the imaging device 40, which includes the ground in front of the crawler crane 100. The image 700 shows an image area corresponding to the travel area where the steel plate IP in front of the crawler crane 100 is laid, including the ground contact position of the lower traveling body 1 of the crawler crane 100, and an image area corresponding to the non-travel area where the ground surface behind it (upper side in the figure) is exposed.

[0136] The display device 50A displays a line-shaped image 701 representing the travel area, superimposed on the captured image 700. Image 701 represents the boundary line between the image area corresponding to the travel area and the image area corresponding to the non-travel area in the captured image 700. This makes it easy for the user (operator) to understand the outer edge of the travel area, i.e., the boundary line between it and the non-travel area. Therefore, it is possible to prevent situations in which the crawler crane 100 approaches the non-travel area or actually enters the non-travel area.

[0137] Furthermore, the display device 50A displays an image 702 corresponding to the non-traveling area superimposed on the captured image 700. Image 702 grays out the image area in the captured image 700 that corresponds to the non-traveling area. This makes it easy for the user (operator) to understand that the image area corresponding to image 702 is a non-traveling area. Therefore, it is possible to prevent situations in which the crawler crane 100 approaches the non-traveling area or actually enters the non-traveling area.

[0138] Furthermore, the display device 50A displays an image 703 superimposed on the captured image 700, representing the travel trajectory of the lower traveling body 1 (crawler 1CL, 1CR) assuming that the lower traveling body 1 travels in the current state of the crawler crane 100. This makes it easy to understand the relationship between the travel trajectory, travel area, and non-travel area when the lower traveling body 1 travels in the current state of the crawler crane 100. As a result, situations where the crawler crane 100 approaches a non-travel area or actually enters a non-travel area can be more appropriately prevented.

[0139] Furthermore, a screen similar to that in Figure 7 may also be displayed on the support device 200 (display device) used by the operator performing the remote control.

[0140] Returning to Figure 5, once the processing in step S110 is complete, the controller 30 terminates the processing in this flowchart.

[0141] In this way, the controller 30 can estimate the travel area and non-travel area on the ground around the crawler crane 100 based on the output of the distance sensor 45. Furthermore, the controller 30 can visually display the estimated travel area and non-travel area through the display device 50A. This helps to prevent situations where the crawler crane 100 approaches or enters a non-travel area.

[0142] [Second example of a function related to estimating the driving area] In addition to Figures 1 to 3, a second example of the function for estimating the area on the ground around the crawler crane 100 where travel is possible or permitted (travel area) will be described with reference to Figures 8 and 9.

[0143] The following explanation will focus on the differences from the first example described above, and explanations of content that is the same as or corresponds to the first example may be simplified or omitted.

[0144] <Functional Configuration> FIG. 8 is a functional block diagram showing a second example of the functional configuration related to the estimation of the traveling area of the crawler crane 100.

[0145] As shown in FIG. 8, the controller 30 includes, as functional units, a data acquisition unit 301, a preprocessing unit 302, a feature extraction unit 303, a traveling area estimation unit 304, a display processing unit 305, and an operation support control unit 306.

[0146] The operation support control unit 306 performs control related to the support of the traveling operation of the crawler crane 100 by the operator based on the estimation result of the traveling area estimation unit 304, that is, the estimation result of the traveling area and the non-traveling area on the ground around the crawler crane 100.

[0147] For example, when the distance between the crawler crane 100 and the non-traveling area becomes equal to or less than the threshold Th3, the operation support control unit 306 notifies the user (operator) to that effect through the display device 50A or the sound output device 50B. Thereby, the operator in the cab 10 can recognize that the crawler crane 100 is approaching or has entered the non-traveling area too closely. Further, when the distance between the crawler crane 100 and the non-traveling area becomes equal to or less than the threshold Th3, the operation support control unit 306 may transmit a signal indicating that fact to the support device 200 through the communication device 60. Thereby, the operator using the support device 200 can recognize that the crawler crane is approaching or has entered the non-traveling area too closely through the support device 200 (display device or sound output device).

[0148] Also, the operation support control unit 306 may change the notification mode as the distance between the crawler crane 100 and the non-traveling area becomes closer. For example, when the distance between the crawler crane 100 and the non-traveling area is equal to or less than the threshold Th3 and greater than the threshold Th4 (<Th3), the operation support control unit 306 increases the sound pressure, pitch, etc. of the notification sound from the sound output device 50B when the distance is equal to or less than the threshold Th4.

[0149] Further, when the distance between the crawler crane 100 and the non - traveling area becomes equal to or less than the threshold value Th5, the operation support control unit 306 may control the hydraulic control valve 33 or the like to restrict the traveling operation of the lower traveling body 1 (crawler 1C) of the crawler crane 100. The restriction function of the traveling operation of the lower traveling body 1 includes, for example, a deceleration function that reduces the traveling speed of the lower traveling body 1 in response to an operator's operation input. Further, the restriction function of the traveling operation of the lower traveling body 1 may include a stop function that stops the lower traveling body 1 and maintains the stopped state regardless of the operator's operation input. Thereby, when the crawler crane 100 approaches the non - traveling area too closely, the traveling operation of the crawler crane 100 can be decelerated or stopped, and as a result, the occurrence of a situation where the crawler crane 100 intrudes into the non - traveling area can be suppressed.

[0150] Also, the operation support control unit 306 may change the mode of restricting the operation of the lower traveling body 1 as the distance between the crawler crane 100 and the non - traveling area becomes closer. For example, the operation support control unit 306 increases the degree of restriction when the distance between the crawler crane 100 and the non - traveling area is equal to or less than the threshold value Th6 (< Th5) compared to when the distance is equal to or less than the threshold value Th5 and greater than the threshold value Th6. Increasing the degree of restriction includes increasing the deceleration rate of the deceleration function and shifting from the deceleration function to the stop function.

[0151] <Process> FIG. 9 is a flowchart schematically showing a second example of the main routine process regarding the estimation of the traveling area around the crawler crane 100.

[0152] Note that since the processing of the sub - routine regarding the estimation of the traveling area around the crawler crane 100 is the same as that of the first example described above, FIG. 6 is incorporated and the illustration is omitted.

[0153] As shown in FIG. 9, the processing of steps S302 to S310 is the same as the processing of steps S102 to S110 in FIG. 5, and thus the description is omitted.

[0154] When the controller 30 completes the processing in step S310, it proceeds to step S312.

[0155] In step S312, the operation support control unit 306 determines whether the distance between the crawler crane 100 and the non-traveling area is below a certain threshold. For example, in the case of a notification function, the operation support control unit 306 determines whether the distance between the crawler crane 100 and the non-traveling area is below a threshold Th3. In the case of a function to limit the travel movement of the lower traveling body 1, the operation support control unit 306 may determine whether the distance between the crawler crane 100 and the non-traveling area is below a threshold Th5. If the distance between the crawler crane 100 and the non-traveling area is below a certain threshold, the operation support control unit 306 proceeds to step S314; otherwise, it terminates this flowchart.

[0156] In step S314, the operation support control unit 306 performs control processing (travel operation support control processing) related to supporting the travel operation of the crawler crane 100. Specifically, the operation support control unit 306 may activate a notification function and a function to limit the travel movement of the lower traveling body 1.

[0157] When the processing in step S314 is completed, the controller 30 terminates the processing of this flowchart.

[0158] Thus, in this example, the controller 30 can provide support for the operation of the lower vehicle 1 based on the estimation results of the driving area and the non-driving area.

[0159] [Other embodiments] Next, other embodiments will be described with reference to Figure 10.

[0160] Figure 10 shows another example of a work machine. Specifically, Figure 10 is a side view showing an example of a shovel 300.

[0161] The embodiments described above may be combined with other individual embodiments as appropriate, or modified or altered as needed.

[0162] For example, in the above embodiment, the functions related to estimating the travel area around the crawler crane 100, that is, some or all of the functions of the controller 30 related to estimating the travel area around the crawler crane 100, may be transferred to the support device 200. For example, the functions of the preprocessing unit 302, the feature extraction unit 303, and the travel area estimation unit 304 may be transferred to the support device 200. In addition to the functions of the preprocessing unit 302, the feature extraction unit 303, and the travel area estimation unit 304, the functions of the display processing unit 305 may also be transferred to the support device 200. In this case, the controller 30 transmits the point cloud data acquired from the distance sensor 45 to the support device 200 via the communication device 60. Alternatively, in this case, instead of the distance sensor 45, the output data of an external distance sensor (for example, a distance sensor fixed at the work site or a distance sensor mounted on a drone flying over the work site) may be used to estimate the travel area within a predetermined range of the work site. This allows, for example, the support device 200 to generate a map of the travel area within the work site and distribute it to the crawler crane 100 in advance.

[0163] Furthermore, in the embodiments and their modifications described above, the travel area of ​​other work machines different from the crawler crane 100 may be estimated instead of, or in addition to, the crawler crane 100. For example, as shown in Figure 10, the travel area around the shovel 300 at the work site may be estimated. This is because, in urban work sites or work sites where the shovel 300 is used for crane work, steel plates may also be laid in the travel area of ​​the shovel 300.

[0164] Furthermore, in the embodiments and their modifications described above, the running area may be estimated to be formed of materials other than steel plates or materials other than steel. For example, in a work site in a mountainous area, the running area may consist only of the ground compacted by a road roller or the like. In this case, differences in flatness between the running area and the non-running area may be used as a characteristic feature.

[0165] Furthermore, in the embodiments and modifications thereof described above, the driving area may be estimated by using known image processing techniques for the unique characteristics of the driving area, based on image data (for example, image data from the imaging device 40) instead of, or in addition to, the output data of the distance sensor.

[0166] Furthermore, in the embodiments and variations thereof described above, the driving area may be estimated using a trained model based on distance sensors and image data. In this case, the trained model is, for example, a model trained using a training dataset consisting of a combination of point cloud data or image data and point data or image region data that corresponds to the driving area within the point cloud data or image data.

[0167] [Effect] Next, the operation of the work machine and information processing device according to this embodiment will be described.

[0168] In this embodiment, the work machine includes a sensor that acquires data about objects around the work machine, and an estimation unit that estimates a first area (travel area) on the ground around the work machine in which the work machine can travel or is permitted to travel, based on the output of the sensor. The work machine is, for example, a crawler crane 100 or an excavator 300. The sensor is, for example, a distance sensor 45 or an imaging device 40. The estimation unit is, for example, a travel area estimation unit 304.

[0169] Furthermore, in this embodiment, the information processing device acquires the output of a sensor that acquires data on objects within a predetermined range, and estimates a first area (travel area) within a predetermined range in which the work machine can travel or is permitted to travel. The information processing device is, for example, a controller 30 or a support device 200. The work machine is, for example, a crawler crane 100 or a shovel 300. The sensor is, for example, a distance sensor 45 or an imaging device 40.

[0170] This allows for the estimation of the travel area of ​​the work site where the work machine is performing its tasks. Therefore, for example, it can prevent situations where an autonomous work machine deviates from its designated travel area. Furthermore, it can provide information about the travel area to users who remotely operate the work machine, for example, in situations where it is relatively difficult to grasp the surrounding environment of the work machine. Therefore, it can prevent situations where a remotely operated work machine deviates from its designated travel area.

[0171] Furthermore, in this embodiment, the first region (travel region) may have unique characteristics that can be extracted from the sensor output. The estimation unit may then estimate the first region around the work machine, taking these unique characteristics into consideration, based on the sensor output.

[0172] This allows for the accurate estimation of the driving area from the sensor output by considering the unique characteristics of the driving area.

[0173] Furthermore, in this embodiment, the work machine may include an extraction unit that acquires feature quantities relating to unique characteristics for each location corresponding to the ground around the work machine based on the sensor output. The extraction unit is, for example, a feature quantity extraction unit 303. The estimation unit may then estimate a first region around the work machine based on the feature quantities.

[0174] This allows for a more accurate estimation of the driving area using feature data.

[0175] Furthermore, in this embodiment, the sensor may obtain data regarding the distance to objects around the work machine and the intensity of the reflected wave (reflection intensity) by outputting a detection wave around the work machine and receiving the reflected wave. The detection wave is, for example, a laser (light) or millimeter wave. The extraction unit may then extract the reflection intensity value as a characteristic quantity for each position corresponding to the ground around the work machine based on the sensor output.

[0176] This allows us to estimate a travel area (for example, a travel area where steel plates are laid) that has unique characteristics related to reflectance, using reflectance as a feature.

[0177] Furthermore, in this embodiment, the estimation unit may estimate the first area around the work machine based on the change in intensity value associated with the movement of a position corresponding to the ground around the work machine.

[0178] This allows for a more accurate estimation of the travel area by utilizing the fact that within a travel area with unique characteristics regarding reflectivity, the change in reflectivity due to positional movement is relatively small.

[0179] Furthermore, in this embodiment, the extraction unit may extract intensity values ​​and height values ​​as feature quantities for each position corresponding to the ground around the work machine, based on the output of the sensor.

[0180] This allows for a more accurate estimation of flat driving areas by using vertical height values ​​as features. Furthermore, using vertical height values ​​as features helps to prevent situations where areas that are actually impassable due to obstacles are mistakenly estimated as part of the driving area.

[0181] Furthermore, in this embodiment, the estimation unit may estimate the first area around the work machine based on the change in height value associated with the movement of a position corresponding to the ground around the work machine.

[0182] This allows for a more accurate estimation of the driving area by utilizing the fact that the change in height associated with movement within the driving area is relatively small.

[0183] Furthermore, in this embodiment, the first region may have a higher reflectivity than the second region (non-traveling region) on the ground around the work machine where the work machine cannot or is prohibited from traveling.

[0184] This allows us to estimate the travel area using reflectivity as a feature.

[0185] Furthermore, in this embodiment, a metal plate is laid in the first region.

[0186] This allows us to estimate the travel area using reflectivity as a feature.

[0187] Furthermore, in this embodiment, the work machine may include an imaging device for imaging the area around the work machine and a display device. The imaging device is, for example, imaging device 40. The display device is, for example, display device 50A. Specifically, the display device may display a first image representing the area around the work machine based on the image captured by the imaging device, and may also display a second image representing a first region superimposed on the first image. The first image is, for example, the captured image 700. The second image is, for example, image 701.

[0188] This allows the user (operator) to recognize the travel area around the work machine. Therefore, it is possible to appropriately prevent situations in which the work machine deviates from its designated travel area.

[0189] In this embodiment, the display device may also superimpose a third image onto the first image, which represents the trajectory of the vehicle assuming the work machine is running in its current state. The third image is, for example, image 703.

[0190] This allows the user (operator) of the cabin 10 to recognize the relationship between the travel path of the work machine and the outer edge of the travel area, i.e., the boundary with the non-travel area. Therefore, it is possible to more effectively prevent situations in which the work machine deviates from the travel area.

[0191] Furthermore, in this embodiment, a notification device may be provided to notify the operator if there is a possibility of deviating from the first area.

[0192] This makes it possible to more effectively prevent situations in which the work machine deviates from its designated travel area.

[0193] Although embodiments have been described in detail above, this disclosure is not limited to these specific embodiments, and various modifications and changes are possible within the scope of the gist described in the claims. [Explanation of symbols]

[0194] 1. Lower running body 1C, 1CL, 1CR Crawler 1ML, 1MR Hydraulic Motor for Travel 2. Swivel mechanism 2M Swivel Hydraulic Motor 3. Upper rotating body 4 Boom 5 Mast 5a Pendant Rope 5b Boom luffing rope 5c Boom luffing winch 5M luffing hydraulic motor 6 Backstop 7 Main winding rope 7a Main hoist winch 7M Main Winding Hydraulic Motor 9 Counterweight 10 cabins 11 Engine 13 Regulator 14 Main pump 15 Pilot pump 17 Control valve 26 Operating device 29 Operating pressure sensor 30 controllers 31 Hydraulic control valve 32 Shuttle valve 33 Hydraulic control valve 40 Imaging device 45 Distance Sensor 50 Output device 50A display device 50B Sound Output Device 52 Input devices 60 Communication equipment 100 Crawler Cranes 200 Support equipment 300 Shovel 301 Data Acquisition Unit 302 Pre-processing section 303 Feature Extraction Unit 304 Driving Area Estimation Unit 305 Display Processing Unit 306 Operation Support Control Unit 700 captured images 701 images 702 images 703 images HA Hydraulic Actuator HK Hook IP Iron Plate Network communication lines S1~S4 Sensors SYS Operation Support System

Claims

1. A sensor that acquires data about objects around the work machine, The system includes an estimation unit that estimates a first area on the ground around the work machine where the work machine can or is permitted to move, based on the output of the aforementioned sensor. The sensor outputs a detection wave around the work machine and receives the reflected wave, thereby acquiring data regarding the distance to objects around the work machine and the intensity of the reflected wave. The estimation unit estimates the first region around the work machine based on the distance to objects around the work machine and the intensity of the reflected waves. A type of machinery used for industrial work.

2. The first region has unique characteristics that can be extracted from the output of the sensor, The estimation unit estimates the first region around the work machine based on the output of the sensor and taking into consideration the unique characteristics. The work machine according to claim 1.

3. The system includes an extraction unit that acquires feature quantities related to the unique characteristics of each location corresponding to the ground around the work machine, based on the output of the sensor. The estimation unit estimates the first region around the work machine based on the feature quantities. The working machine according to claim 2.

4. The extraction unit extracts the intensity values ​​of the feature quantities for each position corresponding to the ground around the work machine, based on the output of the sensor. The work machine according to claim 3.

5. The estimation unit estimates the first area around the work machine based on the change in the intensity value accompanying the movement of a position corresponding to the ground around the work machine. The work machine according to claim 4.

6. The extraction unit extracts the intensity value and height value as feature quantities for each position corresponding to the ground around the work machine, based on the output of the sensor. The working machine according to claim 4 or 5.

7. The estimation unit estimates the first area around the work machine based on the change in the height value accompanying the movement of a position corresponding to the ground around the work machine. The work machine according to claim 6.

8. The first area has a higher intensity than the second area on the ground around the work machine where the work machine cannot or cannot move. A working machine according to any one of claims 4 to 7.

9. A metal plate is laid in the first region. The working machine according to claim 8.

10. An imaging device for imaging the area around the aforementioned work machine, The system includes a display device that displays a first image representing the surroundings of a work machine based on the image captured by the imaging device, and a second image representing the first region superimposed on the first image. A working machine according to any one of claims 1 to 9.

11. The display device superimposes a third image onto the first image, representing the trajectory of the vehicle assuming the work machine is running in its current state. The work machine according to claim 10.

12. The system includes a notification device that notifies the operator if there is a possibility of deviating from the first area. A working machine according to any one of claims 1 to 11.

13. The system includes an estimation unit that acquires the output of a sensor that acquires data on objects within a predetermined range, and estimates a first area within the predetermined range in which the work machine can or is permitted to move, based on the output of the sensor. The sensor outputs a detection wave within the predetermined range and receives the reflected wave, thereby acquiring data regarding the distance to an object within the predetermined range and the intensity of the reflected wave. The estimation unit estimates the first region within the predetermined range based on the distance to the object within the predetermined range and the intensity of the reflected wave. Information processing device.

Images