Control method of work machine, control program for work machine, control system for work machine, and work machine

Abstract

To provide a control method of a work machine, a control program for the work machine, a control system for the work machine, and the work machine allowing an operator to feel no troublesome.SOLUTION: A control method of a work machine includes: executing restraint processing for restraining motion of the work machine on the basis of a detected result by a detection section for detecting an object to be detected in a monitoring area around the work machine; switching valid and invalid of a function relating to the restraint processing; and displaying a display screen Dp1 including restrained state information I1 indicating whether the function relating to the restraint processing is valid or invalid and a picked-up image of the monitoring area on a display device. The restraint processing includes sound output processing for outputting announcing sound and limiting processing for limiting the motion of the work machine. The control method displays first restrained state information I11 of the restrained state information I1 relating to the sound output processing and second restrained state information I12 of the restrained state information I1 relating to the limiting processing outside of the picked-up imaged on the display screen Dp1.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a work machine control method, a work machine control program, a work machine control system, and a work machine that are used in a work machine having a function of detecting an object to be detected in a surrounding monitoring area. [Background technology]

[0002] As a related art, a work machine (excavator) is known in which a rotating section (upper rotating body) is rotatably mounted on a traveling section (lower traveling body) (see, for example, Patent Document 1). The work machine according to the related art has an object detection function that uses a camera attached to the rotating section to detect a predetermined object within a predetermined detection range set around the work machine. When the work machine detects an object (for example, a person), it activates a buzzer installed in the cabin to notify the operator that an object has been detected. Then, in this work machine, the operating status of the object detection function (detectable state / undetectable state) is displayed on a display device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2018 / 008542A1 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned related art, as long as the object detection function is in a detectable state, the buzzer will be activated when an object is detected, which may be annoying to the operator.

[0005] An object of the present invention is to provide a work machine control method, a work machine control program, a work machine control system, and a work machine that are less likely to be bothersome to the operator. [Means for solving the problem]

[0006] A control method for a work machine according to one embodiment of the present invention includes: executing a check process to check the operation of the work machine based on the detection results of a detection unit that detects a detection target in a monitoring area around the work machine; switching a function related to the check process between enabled and disabled; and displaying on a display device a display screen including check state information indicating whether the function related to the check process is enabled or disabled and an image of the monitoring area. The check process includes a sound output process that outputs an alarm sound, and a restriction process that restricts the operation of the work machine. The control method displays first check state information, which is the check state information related to the sound output process, and second check state information, which is the check state information related to the restriction process, outside the image on the display screen.

[0007] A work machine control program according to one aspect of the present invention is a program for causing one or more processors to execute the work machine control method.

[0008] A work machine control system according to one aspect of the present invention comprises a restraint processing unit, a switching processing unit, and a display processing unit. The restraint processing unit executes restraint processing to restrain the operation of the work machine based on the detection results of a detection unit that detects a detection target in a monitoring area around the work machine. The switching processing unit switches between enabling and disabling the restraint processing unit. The display processing unit displays on a display device a display screen including restraint status information indicating whether the restraint processing unit is enabled or disabled and a captured image of the monitoring area. The restraint processing includes a sound output process that outputs an alarm sound, and a restriction process that restricts the operation of the work machine. The display processing unit displays first restraint status information, which is the restraint status information related to the sound output processing, and second restraint status information, which is the restraint status information related to the restriction processing, outside the captured image on the display screen.

[0009] A work machine according to one aspect of the present invention includes the work machine control system and a machine body on which the display device is mounted. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a work machine control method, a work machine control program, a work machine control system, and a work machine that are less likely to be bothersome to the operator. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic perspective view showing the overall configuration of a work machine according to a first embodiment. [Figure 2] FIG. 2 is a schematic diagram showing a hydraulic circuit and the like of the work machine according to the first embodiment. [Figure 3] FIG. 3 is a schematic plan view of the work machine according to the first embodiment, seen from above, that schematically shows a monitoring area set around the work machine, etc. [Figure 4] FIG. 4 is a schematic external view of a display device on which a display screen is displayed by the work machine control system according to the first embodiment. [Figure 5] FIG. 5 is a diagram showing an example of a display screen displayed by the work machine control system according to the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a display mode of restraint state information on a display screen displayed by the work machine control system according to the first embodiment. [Figure 7] FIG. 7 is a diagram showing a display example of the second area of the display screen displayed by the work machine control system according to the first embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a display screen displayed by the work machine control system according to the first embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of the operation of the work machine control system according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a display mode of the restraint state information on a display screen displayed by a work machine control system according to a modified example of the first embodiment. [Figure 11] FIG. 11 is a diagram showing a display example of the second area of the display screen displayed by the work machine control system according to the second embodiment. [Figure 12] FIG. 12 is a diagram showing a display example of the second area of the display screen displayed by the work machine control system according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following description will discuss preferred embodiments of the present invention with reference to the accompanying drawings. The preferred embodiments are merely examples of the present invention and are not intended to limit the technical scope of the present invention.

[0013] (Embodiment 1) [1] Overall structure As shown in Figure 1, the work machine 3 according to this embodiment is equipped with a traveling section 31, a swivel section 32, and a working section 33 on a machine body 30. Furthermore, as shown in Figure 2, the work machine 3 is further equipped with a work machine control system 1 (hereinafter simply referred to as the "control system 1"). In addition, the machine body 30 is further equipped with a display device 2, an operating device, etc.

[0014] In this disclosure, the term "work machine" refers to various types of work machinery, and examples include work vehicles such as backhoes (including hydraulic excavators, mini excavators, etc.), wheel loaders, and carriers. The work machine 3 is equipped with a working unit 33 configured to be able to perform at least one task, including lifting. The work machine 3 is not limited to a "vehicle," but may be, for example, a work vessel, a work air vehicle such as a drone or multicopter, or the like. Furthermore, the work machine 3 is not limited to a construction machine (construction equipment), but may be, for example, an agricultural machine (farm equipment) such as a rice transplanter, tractor, or combine harvester. In this embodiment, unless otherwise specified, the work machine 3 is a backhoe with a lifting function (crane function) that can perform tasks such as excavation, leveling, trench digging, and loading in addition to lifting.

[0015] Furthermore, in this embodiment, for ease of explanation, the vertical direction when the work machine 3 is in a usable state is defined as the up-down direction D1. Furthermore, when the swivel unit 32 is in a non-swivel state, the front-to-rear direction D2 and the left-to-right direction D3 are defined based on the direction as seen by the user (operator) aboard the work machine 3 (the driving unit 321 thereof). In other words, all directions used in this embodiment are directions defined based on the machine body 30 of the work machine 3, with the direction in which the machine body 30 moves when the work machine 3 moves forward being "forward" and the direction in which the machine body 30 moves when the work machine 3 moves backward being "rear". Similarly, the direction in which the front end of the machine body 30 moves when the work machine 3 turns right being "rightward", and the direction in which the front end of the machine body 30 moves when the work machine 3 turns left being "leftward". However, these directions are not intended to limit the direction in which the work machine 3 is used (the direction in use).

[0016] The work machine 3 is equipped with an engine that serves as a power source. In the work machine 3, for example, the engine drives a hydraulic pump 41 (see FIG. 2), and hydraulic oil is supplied from the hydraulic pump 41 to hydraulic actuators (including hydraulic motors 43 and hydraulic cylinders 44, etc.) in various parts of the machine body 30, thereby driving the machine body 30. Furthermore, the work machine 3 is controlled, for example, by a user (operator) aboard the driving section 321 of the machine body 30 operating an operating lever or the like of an operating device.

[0017] In this embodiment, as described above, it is assumed that the work machine 3 is a riding-type backhoe, and therefore the working unit 33 is driven in accordance with the operation of a user (operator) riding in the driving unit 321 to perform work such as excavation work. The driving unit 321 on which the user rides is provided on the swivel unit 32.

[0018] The traveling unit 31 has a traveling function and is configured to be able to travel (including turn) on the ground. The traveling unit 31 has, for example, a pair of left and right crawlers 311 and a blade 312. The traveling unit 31 further has a traveling hydraulic motor 43 (hydraulic actuator) for driving the crawlers 311.

[0019] The swivel unit 32 is located above the travel unit 31 and is configured to be swivelable relative to the travel unit 31 around a rotation axis that is aligned in the vertical direction. The swivel unit 32 has a hydraulic motor (hydraulic actuator) for rotation and the like. In addition to a driving unit 321, the swivel unit 32 is equipped with an engine, a hydraulic pump 41 and the like. Furthermore, a boom bracket 322 to which the working unit 33 is attached is provided at the front end of the swivel unit 32.

[0020] The working unit 33 is configured to be able to perform work including lifting work. The working unit 33 is supported by a boom bracket 322 of the swivel unit 32 and performs work. The working unit 33 has a bucket 331, a boom 332, an arm 333, etc. The working unit 33 also has hydraulic actuators (including hydraulic cylinders 44, hydraulic motors, etc.) for driving each part.

[0021] The bucket 331 is a type of attachment (work implement) that is attached to the body 30 of the work machine 3, and is any tool selected from multiple types of attachments depending on the type of work to be done. As an example, the bucket 331 is removably attached to the body 30 and is replaced depending on the type of work to be done. In addition to the bucket 331, attachments for the work machine 3 include, for example, various tools such as breakers, augers, crushers, forks, fork claws, steel frame cutters, asphalt cutters, brush cutters, rippers, mulchers, tiltrotators, and tampers. The working unit 33 performs work by driving the bucket 331 with power from a drive device.

[0022] The boom 332 is rotatably supported by the boom bracket 322 of the swivel section 32. Specifically, the boom 332 is supported by the boom bracket 322 so as to be rotatable around a rotation axis that extends in the horizontal direction. The boom 332 has a shape that extends upward from a base end supported by the boom bracket 322. The arm 333 is connected to the tip of the boom 332. The arm 333 is supported relative to the boom 332 so as to be rotatable around a rotation axis that extends in the horizontal direction. A bucket 331 is attached to the tip of the arm 333.

[0023] The working unit 33 operates by receiving power from an engine as a power source. Specifically, the engine drives a hydraulic pump 41, and hydraulic oil is supplied from the hydraulic pump 41 to hydraulic actuators (hydraulic cylinder 44, etc.) of the working unit 33, thereby operating each part of the working unit 33 (bucket 331, boom 332, and arm 333).

[0024] Particularly in this embodiment, the working unit 33 has an articulated structure in which the boom 332 and the arm 333 are configured to be independently rotatable. In other words, by rotating each of the boom 332 and the arm 333 around a horizontal rotation axis, the articulated working unit 33 including the boom 332 and the arm 333 can be extended or folded as a whole.

[0025] Like the working unit 33, the traveling unit 31 and the swivel unit 32 each receive power from an engine as a power source and operate. That is, the swivel unit 32 and the traveling unit 31 operate when hydraulic oil is supplied from the hydraulic pump 41 to the hydraulic motor 43 of the traveling unit 31 and the hydraulic motor of the swivel unit 32, etc.

[0026] As described above, the engine functions as a power source that supplies power to each part. Here, the engine is mounted on the slewing part 32 together with the hydraulic pump 41 and the like. In this embodiment, as an example, the engine is a diesel engine. The engine is driven by fuel (here, diesel) supplied from a fuel tank.

[0027] Here, the machine body 30 is equipped with various sensors (including cameras), such as cameras that capture images of the periphery of the machine body 30, for detecting a detection object Ob1 (see FIG. 3) in a monitoring area A1 (see FIG. 3) around the work machine 3. In this embodiment, as an example, as shown in FIG. 3, multiple cameras (three in this case) including a left camera 341, a right camera 342, and a rear camera 343 are mounted on the swivel section 32 of the machine body 30. The left camera 341, the right camera 342, and the rear camera 343 are connected to the control system 1, and images captured by each camera are output to the control system 1. FIG. 3 is a plan view of the work machine 3 viewed from above, and schematically shows the monitoring area A1 set around the work machine 3, the detection object Ob1, and the machine body 30 of the work machine 3 (including the left camera 341, right camera 342, and rear camera 343).

[0028] The left camera 341, the right camera 342, and the rear camera 343 are installed facing left, right, and rear with respect to the driving section 321 so as to capture images of a monitoring area A1 that is to the left, right, and rear as seen from the operator in the driving section 321 of the swivel section 32. That is, the monitoring area A1 includes a plurality of (three in this case) small areas A11, A12, and A13 as shown in FIG. 3 , and the left camera 341 captures an image of the small area A11 (left area) that is to the left as seen from the operator in the driving section 321. Similarly, the right camera 342 captures an image of the small area A12 (right area) that is to the right as seen from the operator in the driving section 321, and the rear camera 343 captures an image of the small area A13 (rear area) that is rear as seen from the operator in the driving section 321. This makes it possible to cover the sides (left and right) and rear, which tend to be blind spots for the operator, with the left camera 341, right camera 342 and rear camera 343.

[0029] Fig. 2 schematically shows the hydraulic circuit and electrical circuit (electrical connection relationships) of the work machine 3 according to this embodiment. In Fig. 2, solid lines indicate high-pressure oil passages (for hydraulic oil), dotted lines indicate low-pressure oil passages (for pilot oil), and dashed arrows indicate electrical signal paths.

[0030] As shown in FIG. 2, the work machine 3 is equipped with a hydraulic pump 41, a hydraulic motor 43 (not shown in FIG. 2), and a hydraulic cylinder 44, as well as a pilot pump 42, a remote control valve 45, a control valve 46, and a directional control valve (control valve) 47.

[0031] Hydraulic oil from a hydraulic pump 41 driven by the engine is supplied to a hydraulic motor 43 of the traveling unit 31, a hydraulic motor of the swivel unit 32, a hydraulic cylinder 44 of the working unit 33, etc. This drives hydraulic actuators such as the hydraulic motor 43 and the hydraulic cylinder 44.

[0032] Hydraulic actuators such as the hydraulic motor 43 and the hydraulic cylinder 44 are provided with a pilot-type directional control valve 47 that can switch the direction and flow rate of hydraulic oil from the hydraulic pump 41. The directional control valve 47 is driven by the supply of pilot oil, which serves as an input command, from the pilot pump 42.

[0033] Here, for example, a remote control valve 45 is provided in a supply path for pilot oil to a directional control valve 47 corresponding to the hydraulic cylinder 44 of the working unit 33. The remote control valve 45 outputs a work operation command for the working unit 33 in response to the operation of an operating lever. The work operation command instructs the extension operation, retraction operation, etc. of the working unit 33. In addition, an electromagnetic control valve 46 (solenoid valve) is inserted between the remote control valve 45 and the pilot pump 42. The control valve 46 is connected to a power source 351 via a cutoff relay 352 and a cutoff switch 353, and operates in response to the current supplied from the power source 351.

[0034] Similarly, a remote control valve is provided in a pilot oil supply path to a directional control valve corresponding to the hydraulic motor 43 of the traveling unit 31. This remote control valve outputs a travel operation command for the traveling unit 31 in response to operation of an operating lever. The travel operation command instructs the traveling operation (forward or reverse, etc.) of the traveling unit 31. Furthermore, a remote control valve is provided in a pilot oil supply path to a directional control valve corresponding to the hydraulic motor of the slewing unit 32. This remote control valve outputs a swing operation command for the slewing unit 32 in response to operation of an operating lever. The swing operation command instructs the swing operation (left swing, right swing, etc.) of the slewing unit 32. Electromagnetic control valves 46 (solenoid valves) are also inserted between these remote control valves and the pilot pump 42. The control valve 46 is connected to a power source 351 via a cutoff relay 352 and a cutoff switch 353, and operates in response to the current supplied from the power source 351.

[0035] When the control valve 46 is energized, that is, when current is being supplied, it opens the flow path of pilot oil from the pilot pump 42 to the remote control valve 45, and when it is de-energized, that is, when the supply current is cut off, it cuts off the flow path of pilot oil. Therefore, when the supply current to the control valve 46 is cut off, the hydraulic actuator corresponding to the remote control valve 45 becomes inoperable, and the output of the hydraulic actuator is forcibly stopped regardless of the operation of the operating lever.

[0036] Here, the cutoff relay 352 is connected to the control system 1 and is switched on / off in response to a control signal (electrical signal) from the control system 1. The cutoff switch 353 is switched on / off in response to the operation of a cutoff lever, and is turned on, for example, when the cutoff lever is operated downward. Therefore, when both the cutoff relay 352 and the cutoff switch 353 are on, the control valve 46 is energized and the flow path of pilot oil from the pilot pump 42 to the remote control valve 45 is opened, so that the hydraulic actuator is driven in response to the operation of the operation lever. On the other hand, when at least one of the cutoff relay 352 and the cutoff switch 353 is off, the control valve 46 is deenergized and the flow path of pilot oil is blocked, so that the hydraulic actuator cannot be driven.

[0037] For example, when at least one of the cutoff relay 352 and the cutoff switch 353 connected to the control valve 46 inserted between the pilot pump 42 and the remote control valve corresponding to the hydraulic motor of the swivel unit 32 is in an off state, the hydraulic motor of the swivel unit 32 cannot be driven. In this state, the output of the hydraulic actuator (hydraulic motor of the swivel unit 32) is forcibly stopped regardless of the operation of the control lever, and therefore the swivel operation of the swivel unit 32 is prohibited.

[0038] The control system 1 is primarily configured as a computer system having one or more processors such as a CPU (Central Processing Unit) and one or more memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and executes various processes (information processing). In this embodiment, the control system 1 is an integrated controller that controls the entire work machine 3, and is composed of, for example, an electronic control unit (ECU). However, the control system 1 may also be provided separately from the integrated controller. The control system 1 will be explained in more detail in the section "[2] Configuration of the Control System."

[0039] The display device 2 is disposed in the driving section 321 of the machine body 30, and is a user interface for receiving operation inputs from a user (operator) and outputting various information to the user. The display device 2 receives various operations from the user, for example, by outputting electrical signals in response to the user's operations. This allows the user (operator) to view the display screen Dp1 (see FIG. 4) displayed on the display device 2, and also allows the user (operator) to operate the display device 2 as necessary.

[0040] 2, the display device 2 includes a control unit 21, an operation unit 22, and a display unit 23. The display device 2 is configured to be able to communicate with the control system 1, and is able to send and receive data to and from the control system 1. In this embodiment, as an example, the display device 2 is a dedicated device used in the work machine 3.

[0041] The control unit 21 controls the display device 2 in accordance with data from the control system 1. Specifically, the control unit 21 outputs an electrical signal in response to a user operation received by the operation unit 22, and displays a display screen Dp1 generated by the control system 1 on the display unit 23.

[0042] The operation unit 22 is a user interface for accepting operation inputs by a user (operator) to a display screen Dp1 displayed on the display unit 23. The operation unit 22 accepts various operations by the user U1 (see FIG. 4), for example, by outputting an electrical signal in response to an operation by the user U1. In the present embodiment, as an example, the operation unit 22 includes a plurality of (six in this case) mechanical push button switches 221-226, as shown in FIG. 4. These plurality of push button switches 221-226 are arranged close to the display area (below in the example of FIG. 4) so as to follow the periphery of the display area of the display unit 23. These plurality of push button switches 221-226 are associated with items displayed on the display screen Dp1, which will be described later, and by operating any of the plurality of push button switches 221-226, any item on the display screen Dp1 is operated (selected).

[0043] Furthermore, the operation unit 22 may include a touch panel, an operation dial, etc. In this case as well, an operation on the operation unit 22 causes an operation (selection) of any item on the display screen Dp1.

[0044] The display unit 23 is a user interface for presenting information to the user U1 (operator), such as a liquid crystal display or an organic EL display that displays various types of information. The display unit 23 presents various types of information to the user by displaying them. In this embodiment, as an example, the display unit 23 is a full-color liquid crystal display with a backlight, and has a "horizontally elongated" display area that is long in the horizontal direction, as shown in FIG. 4.

[0045] The display device 2 presents various pieces of information on the display screen Dp1 to the user U1 (operator) who operates the work machine 3. In other words, the user U1 who operates the work machine 3 can visually obtain various pieces of information related to the work machine 3 by looking at the display screen Dp1 displayed on the display device 2. As an example, by displaying information about the operating state of the work machine 3, such as the coolant temperature and hydraulic oil temperature, the user U1 can check the information about the operating state of the work machine 3 that is necessary for operating the work machine 3 on the display device 2. The display device 2 can also display, on the display screen Dp1, images of the surroundings of the work machine 3 (images of the monitoring area A1) captured by the left camera 341, right camera 342, and rear camera 343. This allows the user U1 (operator) to check, for example, the conditions to the sides and rear of the work machine 3, which are likely to be blind spots from the driving unit 321, on the display screen Dp1 displayed on the display device 2 when operating the work machine 3.

[0046] Furthermore, the work machine 3 is equipped with a sound output unit 36 (see FIG. 2 ) that outputs sounds (including voice) to the user U1 (operator). The sound output unit 36 includes a buzzer, a speaker, or the like, and outputs sound in response to an electrical signal. The sound output unit 36 is connected to the control system 1, and outputs sounds such as beeps or voice in response to a sound control signal from the control system 1. In this embodiment, the sound output unit 36 is provided in the driving unit 321 of the machine body 30, similar to the display device 2. The sound output unit 36 may be provided integrally with the display device 2.

[0047] In addition to the above-described configuration, the machine 30 further includes an operating lever, a cut-off lever, a communication terminal, a fuel tank, a battery, etc. Furthermore, the machine 30 is equipped with sensors for monitoring the operating state of the machine 30, such as a coolant temperature sensor, a hydraulic oil temperature sensor, a tachometer for measuring the engine RPMs, and an hour meter for measuring the operating time. The machine 30 also includes sensors for detecting the state of the gate lock lever, starter key switch, etc.

[0048] [2] Control system configuration Next, the configuration of the control system 1 according to this embodiment will be described with reference to Figure 2. The control system 1 controls the display device 2 to cause the display device 2 to display the display screen Dp1. In this embodiment, the display device 2 is mounted on the body 30 of the work machine 3, as described above. The control system 1 is a component of the work machine 3, and together with the body 30 and the like, forms the work machine 3. In other words, the work machine 3 according to this embodiment comprises at least the control system 1 and the body 30 (including the traveling unit 31, the swivel unit 32, and the working unit 33) on which the display device 2 is mounted.

[0049] In the present disclosure, the term "screen" such as the display screen Dp1 refers to an image (picture) displayed on the display device 2, and includes icons, figures, photographs, text, and video. That is, the control system 1 is capable of causing the display device 2 to display the display screen Dp1, which includes images and the like that represent information relating to the operating state of the work machine 3, such as the cooling water temperature and hydraulic oil temperature. Here, when the display screen Dp1 includes video and the like, the display screen Dp1 does not include a fixed image, but includes an image that changes every moment.

[0050] As shown in Fig. 2, the control system 1 includes a display processing unit 11, a restraint processing unit 12, a switching processing unit 13, an image acquisition unit 14, a detection unit 15, and an abnormality determination unit 16. In this embodiment, as an example, the control system 1 is mainly configured as a computer system having one or more processors, and these multiple functional units (display processing unit 11, etc.) are realized by the one or more processors executing a work machine control program. These multiple functional units included in the control system 1 may be provided separately in multiple housings, or may be provided in a single housing.

[0051] The control system 1 is configured to be able to communicate with devices provided in various parts of the aircraft 30. That is, at least the display device 2, the sound output unit 36, the cutoff relay 352, the left camera 341, the right camera 342, and the rear camera 343 are connected to the control system 1. This enables the control system 1 to control the display device 2 and the sound output unit 36, etc., control the cutoff relay 352 to control the control valve 46, and acquire images captured by the left camera 341, the right camera 342, and the rear camera 343, etc. Here, the control system 1 may exchange various types of information (data) directly with each device or indirectly via a relay or the like.

[0052] The image acquisition unit 14 periodically or irregularly acquires the outputs of the left camera 341, right camera 342, and rear camera 343 from the left camera 341, right camera 342, and rear camera 343. In other words, the image acquisition unit 14 acquires image data of the monitoring area A1 (each of the small areas A11, A12, A13) around the work machine 3. The data acquired by the image acquisition unit 14 is stored in, for example, a memory or the like.

[0053] The detection unit 15 detects a detection object Ob1 in the monitoring area A1 around the work machine 3. In other words, the detection unit 15 determines the presence or absence (existence) of the detection object Ob1 in the monitoring area A1, and outputs a detection result indicating whether or not the detection object Ob1 is present in the monitoring area A1. In this embodiment, as an example, the detection object Ob1 is a "person." In other words, if the work machine 3 moves, or a "person" around the work machine 3 moves, and as a result a "person" enters the monitoring area A1 around the work machine 3, the detection unit 15 detects the "person" as the detection object Ob1. If multiple detection objects Ob1 are present in the monitoring area A1, the detection unit 15 may also detect the number of detection objects Ob1 (number of people).

[0054] In this embodiment, the detection unit 15 detects a detection object Ob1 in the monitoring area A1 based on the outputs (image data) of the left camera 341, the right camera 342, and the rear camera 343. Specifically, the detection unit 15 extracts feature amounts in the image by performing image processing on the image data acquired by the image acquisition unit 14, and determines whether or not the detection object Ob1 (a "person" in this embodiment) is captured in the image based on the feature amounts. Here, if the detection object Ob1 is captured in the image, the detection unit 15 determines whether the detection object Ob1 is captured in the image captured by the left camera 341, the right camera 342, or the rear camera 343. In other words, the detection unit 15 detects the object to be detected Ob1 by distinguishing whether the object to be detected Ob1 is located in the small area A11 captured by the left camera 341, the small area A12 captured by the right camera 342, or the small area A13 captured by the rear camera 343.

[0055] The abnormality determination unit 16 determines whether or not there is an abnormality regarding the detection by the detection unit 15. In other words, if there is an abnormality (failure, etc.) in any of the detection unit 15, the left camera 341, the right camera 342, the rear camera 343, or the lighting device that illuminates the monitoring area A1, the detection unit 15 cannot normally detect the detection target Ob1. Therefore, the abnormality determination unit 16 determines that a state in which the detection unit 15 cannot normally detect the detection target Ob1 is a state in which an abnormality regarding the detection by the detection unit 15 exists.

[0056] The restraint processing unit 12 executes restraint processing to restrain the operation of the work machine 3 based on the detection results of the detection unit 15. In this embodiment, the restraint processing unit 12 executes restraint processing when the detection results of the detection unit 15 indicate the presence of a detection object Ob1 (here, a person) in the monitoring area A1. In this disclosure, "restraint processing" refers to processing that acts in some way to restrain the operation of the work machine 3. As an example, the restraint processing includes processing that indirectly restrains the operation of the work machine 3 by warning the user U1 (operator) who operates the work machine 3 using sound or light (including display). Furthermore, the restraint processing includes processing that directly restrains the operation of the work machine 3 by controlling the traveling unit 31, swivel unit 32, working unit 33, etc. of the work machine 3.

[0057] In this embodiment, the restraint processing unit 12 includes a sound output processing unit 121 and a restriction processing unit 122.

[0058] When a detection object Ob1 is present in the monitoring area A1, the sound output processing unit 121 controls the sound output unit 36 to output an alarm sound from the sound output unit 36. That is, in this embodiment, the restraint processing includes sound output processing that outputs an alarm sound. The alarm sound may be a simple beep, or a voice message such as "please be careful." Furthermore, the alarm sound may change depending on the detection result of the detection unit 15 (such as the distance from the machine body 30 to the detection object Ob1). This allows the user U1 (operator) operating the work machine 3 to be warned by the alarm sound, thereby indirectly restraining the operation of the work machine 3, thereby increasing the degree of freedom in operating the work machine 3. In other words, by operating the work machine 3 while paying attention to the detection object Ob1, the user U1 can continue operating the work machine 3 while avoiding contact with the detection object Ob1.

[0059] When a detection object Ob1 is present in the monitoring area A1, the restriction processing unit 122 controls the cutoff relay 352 to turn off the cutoff relay 352. As a result, the control valve 46, which is connected to the power source 351 via the cutoff relay 352, is de-energized, and the output of the hydraulic actuator corresponding to the control valve 46 is forcibly stopped. That is, in this embodiment, the restraint processing includes a restriction processing that restricts the operation of the work machine 3. The "restriction processing" referred to in this disclosure means a processing that acts in some way to restrict the operation of the work machine 3. As an example, the restriction processing includes a processing that prohibits the traveling operation of the traveling unit 31 (making the traveling operation impossible), a processing that prohibits the swing operation of the swing unit 32 (making the swing operation impossible), and a processing that prohibits the operation of the working unit 33 (making the work impossible). This makes it possible to forcibly restrict the operation of the work machine 3 without the operation of the user U1 (operator). That is, it is possible to avoid contact between the machine body 30 and the detection object Ob1 due to the operation of the work machine 3.

[0060] Here, the restriction processing executed by the restriction processing unit 122 includes at least processing to restrict the swing operation of the swing unit 32. Specifically, the restriction processing unit 122 is configured to be able to control a cutoff relay 352 connected to a control valve 46 corresponding to the hydraulic motor of the swing unit 32, and turns off the cutoff relay 352 when a detection object Ob1 is present in the monitoring area A1. As a result, when a detection object Ob1 is present in the monitoring area A1, the hydraulic motor of the swing unit 32 becomes inoperable, the swing unit 32 is brought to an emergency stop if the swing unit 32 is in the middle of swinging, and the swing operation of the swing unit 32 is prohibited if the swing unit 32 is not in the middle of swinging. That is, in this embodiment, the work machine 3 includes a traveling unit 31 and a swing unit 32 that is capable of swinging relative to the traveling unit 31. The restriction processing at least restricts the swing operation of the swing unit 32. This makes it possible to avoid contact between the machine body 30 and the detection object Ob1 due to the rotation of the swivel unit 32 when the detection object Ob1 is present in the monitoring area A1 that is a blind spot for the user U1 (operator).

[0061] The switching processing unit 13 switches between enabling and disabling the check processing unit 12. In other words, the switching processing unit 13 switches between enabling and disabling the functions related to check processing. In short, the check processing unit 12 is not always enabled, but can be switched between enabled and disabled. If the check processing unit 12 is enabled, check processing is performed by the check processing unit 12 when a detection object Ob1 is present in the monitoring area A1. On the other hand, if the check processing unit 12 is disabled, check processing is not performed by the check processing unit 12 even if a detection object Ob1 is present in the monitoring area A1.

[0062] In this embodiment, as an example, the checking processing-related function (check processing unit 12) is switched between enabled and disabled by user U1 (operator) operating the display device 2. That is, when user U1 operates the operation unit 22 of the display device 2 to enable the checking processing-related function, the switching processing unit 13 receives this operation and enables the checking processing-related function. On the other hand, when user U1 operates the operation unit 22 of the display device 2 to disable the checking processing-related function, the switching processing unit 13 receives this operation and disables the checking processing-related function.

[0063] Furthermore, in this embodiment, the restraint processing performed by the restraint processing unit 12 includes sound output processing executed by the sound output processing unit 121 and restriction processing executed by the restriction processing unit 122. In this way, the restraint processing includes a plurality of specific processes (sound output processing, restriction processing, etc.) for restraining the operation of the work machine 3. Here, the restraint processing can be switched between enabled and disabled individually for each specific process. In other words, the switching processing unit 13 can switch between enabled and disabled individually for the sound output processing unit 121 and the restriction processing unit 122 in the restraint processing unit 12. As an example, it is possible to enable the sound output processing unit 121 and disable the restriction processing unit 122, or to disable the sound output processing unit 121 and enable the restriction processing unit 122. This makes it possible to enable only the specific processes that are necessary depending on the situation, improving the flexibility of the restraint processing.

[0064] The display processing unit 11 has a function of displaying at least the display screen Dp1 on the display device 2. Specifically, the display processing unit 11 generates the display screen Dp1 based on data acquired by the image acquisition unit 14, etc., and controls the display device 2 to display the display screen Dp1 on the display unit 23 of the display device 2. Furthermore, the display processing unit 11 operates in response to operations received by the operation unit 22 of the display device 2. The display processing unit 11 displays, for example, captured images Im11, Im12, and Im13 (see FIG. 5 ) captured by the left camera 341, right camera 342, and rear camera 343 on the display screen Dp1. In other words, the display processing unit 11 causes the image acquisition unit 14 to display images of the monitoring area A1 (each of the small areas A11, A12, and A13) around the work machine 3 on the display device 2.

[0065] Here, the display processing unit 11 is capable of causing the display device 2 to display a display screen Dp1 including check status information I1 (see FIG. 5) that indicates whether the check processing related functions are enabled or disabled. In other words, the switching processing unit 13 switches between enabling and disabling the check processing related functions, and the current status (enabled / disabled) of the check processing related functions is displayed on the display screen Dp1 as check status information I1. In this way, the display processing unit 11 causes the display device 2 to display the display screen Dp1. The display screen Dp1 includes check status information I1 that indicates whether the check processing unit 12 is enabled or disabled. This allows the user U1 (operator) to visually confirm whether the check processing related functions are enabled or disabled, and can operate the work machine 3 knowing whether the check processing related functions are enabled or disabled.

[0066] Incidentally, the detection unit 15 is not an essential component of the control system 1. For example, the control system 1 may be configured to acquire the detection result of an external detection unit, and the check processing unit 12 may execute check processing based on the detection result.

[0067] [3] Control method for work machine An example of a control method for the work machine 3 (hereinafter simply referred to as "control method") that is executed mainly by the control system 1 will be described below with reference to FIGS.

[0068] The control method according to this embodiment is executed by a control system 1 whose main component is a computer system, and in other words, is embodied in a work machine control program (hereinafter simply referred to as a "control program"). In other words, the control program according to this embodiment is a computer program for causing one or more processors to execute each process related to the control method. Such a control program may be executed, for example, by the control system 1 and the display device 2 working together.

[0069] Here, the control system 1 executes the various processes described below related to the control method when a specific, preset start operation is performed to execute the control program. The start operation is, for example, an operation to start the engine of the work machine 3. On the other hand, the control system 1 ends the various processes described below related to the control method when a specific, preset end operation is performed. The end operation is, for example, an operation to stop the engine of the work machine 3.

[0070] [3.1] Display screen Here, we will first explain the configuration of the display screen Dp1 displayed on the display unit 23 of the display device 2 by the control method according to this embodiment. In the drawings showing the display screen Dp1 displayed on the display unit 23 of the display device 2, such as Fig. 5, the dashed dotted lines, lead lines, and reference symbols representing areas are added merely for the purpose of explanation and are not actually displayed on the display device 2.

[0071] 5 is a home screen that is first displayed by the control method. The home screen is a basic display screen Dp1 that is first displayed on the display device 2 while the work machine 3 is in operation. The display screen Dp1 can transition from the home screen to various display screens Dp1, including a menu screen, a crane screen, a mode screen, a PTO screen, and the like, in accordance with operations on the operation unit 22.

[0072] 5, the display screen Dp1 includes a first region R1, a second region R2, a third region R3, a fourth region R4, a fifth region R5, a sixth region R6, a seventh region R7, an eighth region R8, a ninth region R9, and a tenth region R10. In the control method according to this embodiment, as an example, the captured images Im11, Im12, and Im13 of the monitoring area A1 (each of the small areas A11, A12, and A13), the checking status information I1, etc. are displayed in the second region R2, which occupies the majority of the display screen Dp1. The region in the second region R2 where the checking status information I1 is displayed is designated as the eleventh region R11.

[0073] Specifically, the display screen Dp1 is divided into four regions vertically (up and down). Each of the top three regions is further divided horizontally (left and right). This divides the display screen Dp1 into a total of ten regions. The second-highest region is, from left to right, the first region R1, the second region R2, and the third region R3. The bottom region is the fourth region R4. The third-highest region is, from left to right, the fifth region R5, the sixth region R6, and the seventh region R7, and the top region is, from left to right, the eighth region R8, the ninth region R9, and the tenth region R10. In terms of vertical size, of the four regions divided vertically, the second-highest region (the first region R1, the second region R2, and the third region R3) is the largest. In terms of horizontal size, the middle region (second region R2, sixth region R6, ninth region R9) is the largest among the three regions divided horizontally.

[0074] However, the arrangement and size of each of these regions are merely examples and can be changed as appropriate. Furthermore, it is not essential that each region be clearly divided by a boundary line. For example, in the example of Figure 5, the second region R2 and the third region R3 are clearly divided by a boundary line, while there is no boundary line between the first region R1 and the second region R2. Of course, the first region R1 and the second region R2 may also be clearly divided by a boundary line.

[0075] The first area R1 is a rectangular area that is long in the vertical direction. In the first area R1, for example, remaining amount information G1 relating to the remaining amount of fuel (e.g., diesel) for the engine is displayed. The display processing unit 11 generates the remaining amount information G1 on the display screen Dp1 based on the output (sensor signal) of the remaining amount sensor, etc.

[0076] The second area R2 is a rectangular area that is elongated in the horizontal direction. In the second area R2, captured images Im11, Im12, and Im13 of the monitoring area A1, as well as the restraint state information I1, etc. are displayed. The captured image Im11 is an image of the small area A11 to the left of the driving section 321 captured by the left camera 341, and the captured image Im12 is an image of the small area A12 to the right of the driving section 321 captured by the right camera 342. The captured image Im13 is an image of the small area A13 behind the driving section 321 captured by the rear camera 343. The display processing unit 11 displays the captured images Im11, Im12, and Im13 acquired by the image acquisition unit 14 in real time. In this embodiment, these captured images Im11, Im12, and Im13 are displayed in the upper left of the second area R2, the captured image Im12 in the upper right of the second area R2, and the captured image Im13 in the lower center of the second area R2 so as to correspond to the positional relationship when the aircraft 30 is viewed from above. Furthermore, an icon Im10 that resembles the aircraft 30 as seen from the information side is displayed in the center of the second area R2. The icon Im10 schematically represents the positional relationship between the imaging ranges (small areas A11, A12, and A13) of the left camera 341, right camera 342, and rear camera 343 as seen from the aircraft 30.

[0077] The check status information I1 is displayed in the eleventh area R11 set to the lower right of the second area R2. As described above, the check status information I1 is information that indicates whether the check processing function is enabled or disabled, that is, the status (enabled / disabled) of the check processing function selected by the switching processing unit 13. In this embodiment, the check status information I1 includes an image (icon). The check status information I1 indicates the status (enabled / disabled) of the check processing function, for example, by the display mode such as the display color or size of the image. The display processing unit 11 determines the status (enabled / disabled) of the check processing function based on the operating state of the switching processing unit 13. The display processing unit 11 then determines the display mode of the image in the check status information I1 depending on the status (enabled / disabled) of the check processing function.

[0078] Here, the check processing includes multiple specific processes (sound output processing, restriction processing, etc.), and each specific process can be individually switched between enabled and disabled, and check status information I1 is displayed separately for each specific process. In the example of Figure 5, check status information I1 includes first check status information I11 and second check status information I12, each corresponding to a specific process. First check status information I11 indicates whether a function related to sound output processing is enabled / disabled, and second check status information I12 indicates whether a function related to restriction processing is enabled / disabled. First check status information I11 and second check status information I12 indicate whether the corresponding process corresponds to sound output processing or restriction processing, depending on the design (picture) of each individual icon.

[0079] The third area R3 is a vertically long rectangular area. In the third area R3, an icon (icon) Im1 corresponding to the operating status of each part of the work machine 3 is displayed. Multiple icons Im1 can be displayed in the third area R3, and the design (picture) of each icon Im1 indicates which status it represents, for example, the battery, seat belt, coolant temperature, hydraulic oil temperature, etc. Here, each icon Im1 indicates its operating status by its display mode, such as its display color or size. The display processing unit 11 determines the status of each part of the work machine 3 using the outputs of various sensors (including the coolant temperature sensor 34 and the hydraulic oil temperature sensor 35) that detect the operating status of each part of the work machine 3. Then, if an abnormal value is detected in any part, the display processing unit 11 displays a warning by, for example, changing the display mode, such as the display color, of the icon Im1 for that part.

[0080] The fourth area R4 is a strip-shaped area extending across the entire width of the display screen Dp1. The fourth area R4 displays various items for operating the display screen Dp1. As an example, in FIG. 5, the fourth area R4 displays six items, "Menu," "Crane," "Mode," "Camera," "PTO," and "Switch," arranged in this order from left to right. These six items are associated with six pushbutton switches 221 to 226 of the operation unit 22 located directly below them. For example, the "Menu" item is associated with pushbutton switch 221, and the "Crane" item is associated with pushbutton switch 222. Therefore, for example, when the pushbutton switch 224 corresponding to the "Camera" item is operated by the user U1 (see FIG. 4), the "Camera" item is operated (selected).

[0081] Furthermore, in this embodiment, one of the items is highlighted in the fourth region R4 so as to correspond to the operation of the operation dial (or cursor key) of the operation unit 22, etc. In the example of FIG. 5, the "Menu" item is highlighted, and the highlighted item changes by operating the operation dial (or cursor key), etc. The user U1 can select the desired item by operating the enter button while the desired item is highlighted. Therefore, for example, when the enter button is operated while the highlight is moved to the "Camera" item, the "Camera" item is operated (selected). Furthermore, if the operation unit 22 includes a touch panel, the user U1 can select the desired item by touching the desired item on the display screen Dp1.

[0082] The fifth area R5 displays warning images (icons) that indicate that abnormal values have been detected by various sensors (including the cooling water temperature sensor and hydraulic oil temperature sensor). The sixth area R6, for example, displays information related to the working unit 33 currently in operation on the work machine 3. The seventh area R7, for example, displays information related to the operating state of the work machine 3, such as the engine RPM. The eighth area R8, for example, displays the current time. The ninth area R9, for example, displays information indicating the item to which the currently displayed display screen Dp1 belongs. The tenth area R10, for example, displays information related to the operating time (hour meter) of the work machine 3.

[0083] [3.2] Details Next, the control method according to this embodiment will be described in detail.

[0084] In this embodiment, as described above, the checking processing function (check processing unit 12) is switched between enabled and disabled in response to an operation by user U1. Therefore, the checking processing unit 12 is not always enabled, and user U1 can arbitrarily switch between enabled and disabled states of the checking processing unit 12.

[0085] Moreover, the enabled / disabled status of the check processing unit 12 is displayed as check status information I1 on the display screen Dp1. In other words, the check status information I1 displayed in the 11th area R11 of the display screen Dp1 indicates the status (enabled / disabled) of the function related to the check processing currently selected by the switching processing unit 13. Therefore, the user U1 can confirm the enabled / disabled status of the function related to the check processing from the check status information I1 on the display screen Dp1.

[0086] This configuration has the advantage that it is less bothersome to the operator (user U1) than a configuration in which, for example, a buzzer is activated when an object is detected as long as the object detection function is in a detectable state. That is, according to the control method of this embodiment, the function related to the check processing (check processing unit 12) can be switched between enabled and disabled. Therefore, in situations where check processing (sound output processing, restriction processing, etc.) is not necessary, the function can be disabled, thereby preventing the operator (user U1) from feeling bothered. Furthermore, the enabled / disabled status of the function related to the check function is displayed on the display screen Dp1 as check status information I1, allowing the operator (user U1) to easily confirm the current enabled / disabled status of the function related to the check function. Therefore, it is easy to avoid a situation in which, for example, the operator (user U1) mistakenly operates the work machine 3 under the assumption that the function related to the check function is enabled, even though it is actually disabled.

[0087] 6 shows an example of the display mode of the checking status information I1. In this embodiment, the checking status information I1 includes first checking status information I11 and second checking status information I12, and therefore the display mode is switched between enabled and disabled for each of the first checking status information I11 and the second checking status information I12 individually. As a result, there are four display modes for the checking status information I1: "Pattern 1" in which both the sound output processing and the restriction processing are enabled, "Pattern 2" in which only the sound output processing is enabled, "Pattern 3" in which only the restriction processing is enabled, and "Pattern 4" in which both the sound output processing and the restriction processing are disabled.

[0088] As an example, in this embodiment, if the check processing function is enabled, the check status information I1 is displayed as active, and if the check processing function is disabled, the check status information I1 is displayed as inactive. In other words, check status information I1 displayed as active indicates that the check processing function is enabled, and check status information I1 displayed as inactive indicates that the check processing function is disabled. Check status information I1 displayed as active is displayed in a high-contrast display color, such as green, while check status information I1 displayed as inactive is displayed in a low-contrast display color, such as gray. Therefore, if the sound output processing unit 121 of the check processing unit 12 is enabled and the restriction processing unit 122 is disabled, for example, as in "Pattern 2" shown in FIG. 6, the first check status information I11 is displayed as active in green, and the second check status information I12 is displayed as inactive by being grayed out.

[0089] Thus, in this embodiment, the restraint status information I1 indicates whether each specific process is valid or invalid. Therefore, when the restraint process includes multiple specific processes (sound output process, restriction process, etc.) for restraining the operation of the work machine 3, the operator (user U1) can individually check whether each of these multiple specific processes is valid or invalid.

[0090] Furthermore, in this embodiment, when a function related to check processing is disabled, the fact that the function related to check processing is disabled is displayed as check status information I1. In other words, as described above, when a function related to check processing is disabled, check status information I1 is displayed as inactive, thereby indicating that the function related to check processing is disabled. In this way, check status information I1 is displayed regardless of whether the function related to check processing is enabled or disabled, making it easy for the operator (user U1) to understand whether the function related to check processing is enabled or disabled.

[0091] In this embodiment, as shown in FIG. 7, the display screen Dp1 includes detection result information I2 and I3 that represent the detection results of the detection unit 15. FIG. 7 shows only the second region R2 of the display screen Dp1, and regions other than the second region R2 are not shown. The detection result information I2 is a band-shaped (frame-shaped) image that highlights captured images Im11, Im12, and Im13 that include the detection object Ob1. The detection result information I3 is an image that indicates the direction in which the detection object Ob1 exists as viewed from the driving unit 321. The example in FIG. 7 assumes that the detection object Ob1 (here, a "person") exists in a small area A11 to the left of the driving unit 321, which is captured by the left camera 341. Therefore, of the captured images Im11, Im12, and Im13, the captured image Im11 is highlighted by the detection result information I2, and detection result information I3 is displayed below the captured image Im11, indicating that the detected object Ob1 is present to the left of the driver's section 321.

[0092] The display manner of the detection result information I2 and I3 is preferably changed depending on the position of the detection object Ob1 in the monitoring area A1. For example, the display manner, such as the display color, size, or display pattern (including a blinking pattern) of the detection result information I2 and I3, is changed depending on the position of the detection object Ob1 in the monitoring area A1. For example, the closer the detection object Ob1 is to the aircraft 30, the more the display manner of the detection result information I2 and I3 is changed to a more conspicuous display color. As an example, when the detection object Ob1 approaches the aircraft 30, the display color of the detection result information I2 and I3 changes from yellow to red.

[0093] In this way, the display screen Dp1 not only displays the captured images Im11, Im12, and Im13 of the monitoring area A1, but also displays the detection result of the detection target Ob1 in the monitoring area A1 as detection result information I2 and I3. Therefore, the operator (user U1) can easily confirm the presence or absence (existence or non-existence) of the detection target Ob1 in the monitoring area A1 by looking at the display screen Dp1.

[0094] In short, in the control method according to this embodiment, the display screen Dp1 includes captured images Im11, Im12, and Im13 of the monitoring area A1. This allows the operator (user U1) to check the conditions of the sides and rear of the work machine 3, which are likely to be blind spots from the driving unit 321, on the display screen Dp1 displayed on the display device 2. Therefore, compared to a configuration in which only the detection result information I2 and I3 is displayed, when a detection object Ob1 is present in the monitoring area A1, the condition of the detection object Ob1 can be more easily grasped in detail on the display screen Dp1.

[0095] Furthermore, the captured images Im11, Im12, and Im13 include images of a plurality of small areas A11, A12, and A13 included in the monitoring area A1. Then, on the display screen Dp1, an image of a small area among the plurality of small areas A11, A12, and A13 in which the detection object Ob1 is present is highlighted. That is, in the detection result information I2, among the captured images Im11, Im12, and Im13, the captured image including the detection object Ob1 is highlighted. This makes it easier for the operator (user U1) to know in which of the plurality of small areas A11, A12, and A13 the detection object Ob1 is present when the detection object Ob1 is present in the monitoring area A1.

[0096] Here, the detection result information I2 and I3 are displayed not only when the check processing function is enabled, but also when the check processing function is disabled. In other words, even when the check processing unit 12 is disabled and both the first check status information I11 and the second check status information I12 are grayed out and displayed as inactive ("Pattern 4" in FIG. 6), the detection result information I2 and I3 are displayed if a detection target Ob1 is present in the monitoring area A1. As described above, in the control method according to this embodiment, the display screen Dp1 includes the detection result information I2 and I3 representing the detection results of the detection unit 15, and the detection result information I2 and I3 are displayed regardless of whether the check processing function is enabled or disabled. Therefore, even when the check processing function is disabled, the operator (user U1) can check the detection results of the detection unit 15 on the display screen Dp1.

[0097] Furthermore, in this embodiment, when an abnormality occurs in the detection by the detection unit 15, the display mode of the check status information I1 is set to an error mode. Here, the "error mode" refers to the display mode of the check status information I1 when an abnormality occurs in the detection by the detection unit 15, and is distinguished from a case where there is no abnormality in the detection by the detection unit 15, for example, by not displaying the check status information I1, or by changing the display color, size, or display pattern (including a blinking pattern) of the check status information I1. In other words, when the abnormality determination unit 16 determines that there is an abnormality in the detection by the detection unit 15, as shown in FIG. 8, the check status information I1 in the eleventh region R11 is not displayed, thereby changing the display mode to an error mode and notifying the user that there is an abnormality in the detection by the detection unit 15. Furthermore, in the example of FIG. 8, presentation information Im2 indicating that an abnormality has been detected, such as "Error Occurring," is also displayed in the ninth region R9.

[0098] According to this configuration, the check status information I1 not only indicates whether the check processing functions are enabled or disabled, but also whether there is an abnormality in the detection by the detection unit 15. Therefore, the operator (user U1) can check on the display screen Dp1 whether there is an abnormality in the detection by the detection unit 15, and can take necessary measures when an abnormality in the detection by the detection unit 15 occurs.

[0099] [3.3] Overall processing Next, the overall flow of processing related to the control method will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of processing related to the control method.

[0100] 9, the display processing unit 11 of the control system 1 first displays a display screen Dp1 on the display device 2 (S1). Here, the control system 1 determines whether the function related to the check processing (check processing unit 12) currently selected by the switching processing unit 13 is enabled or disabled.

[0101] Specifically, the control system 1 first determines whether the function related to sound output processing in the check processing is enabled (S2). If the function related to sound output processing is enabled (S2: Yes), the control system 1 enables the sound output processing unit 121 (S3). Then, the display processing unit 11 of the control system 1 actively displays the first check status information I11 of the check status information I1 on the display screen Dp1 (S4). On the other hand, if the function related to sound output processing is disabled (S2: No), the control system 1 disables the sound output processing unit 121 (S5). Then, the display processing unit 11 of the control system 1 inactively displays the first check status information I11 of the check status information I1 on the display screen Dp1 (S6).

[0102] The control system 1 also determines whether the function related to the restriction processing in the check processing is enabled (S7). If the function related to the restriction processing is set to enabled (S7: Yes), the control system 1 enables the restriction processing unit 122 (S8). Then, the display processing unit 11 of the control system 1 actively displays the second check status information I12 of the check status information I1 on the display screen Dp1 (S9). On the other hand, if the function related to the restriction processing is set to disabled (S7: No), the control system 1 disables the restriction processing unit 122 (S10). Then, the display processing unit 11 of the control system 1 inactively displays the second check status information I12 of the check status information I1 on the display screen Dp1 (S11).

[0103] The control system 1 repeatedly executes the processing of steps S1 to S11. As a result, check status information I1 indicating the status (enabled / disabled) of the function related to the check processing is displayed on the display screen Dp1. However, the flowchart shown in Fig. 9 is only an example, and processing may be added or omitted as appropriate, and the order of processing may be changed as appropriate.

[0104] [4] Variation Below, we will list some modified examples of embodiment 1. The modified examples explained below can be applied in appropriate combinations.

[0105] The control system 1 in the present disclosure includes a computer system. The computer system is primarily composed of one or more processors and one or more memories as hardware. The functions of the control system 1 in the present disclosure are realized by the processor executing a program recorded in the memory of the computer system. The program may be pre-recorded in the memory of the computer system, provided via a telecommunications line, or provided by being recorded on a non-transitory recording medium such as a memory card, optical disk, or hard disk drive that is readable by the computer system. In addition, some or all of the functional units included in the control system 1 may be configured with electronic circuits.

[0106] Furthermore, it is not essential for the control system 1 that at least some of the functions of the control system 1 are concentrated in one housing, and the components of the control system 1 may be distributed across multiple housings. Conversely, in the first embodiment, functions distributed across multiple devices (for example, the control system 1 and the display device 2) may be concentrated in one housing. Furthermore, at least some of the functions of the control system 1 may be realized by the cloud (cloud computing) or the like.

[0107] Furthermore, the power source of the work machine 3 is not limited to a diesel engine, and may be, for example, an engine other than a diesel engine, or may be a motor (electric motor), or a hybrid power source including an engine and a motor (electric motor).

[0108] The display device 2 is not limited to a dedicated device, but may be a general-purpose terminal such as a laptop computer, a tablet terminal, a smartphone, etc. Furthermore, the display unit 23 is not limited to a device that directly displays a display screen, such as a liquid crystal display or an organic EL display, but may be configured to display a display screen by projection, such as a projector.

[0109] Furthermore, modes other than push button switches, touch panels, and operation dials may be adopted as the mode of inputting information to the operation unit 22. For example, the operation unit 22 may adopt modes such as a keyboard, a pointing device such as a mouse, voice input, gesture input, or input of an operation signal from another terminal.

[0110] Furthermore, the error mode, which is the display mode of the check status information I1 when an abnormality occurs in the detection of the detection unit 15, is not limited to "non-display" as shown in Figure 8. For example, when there is an abnormality in the detection of the detection unit 15, the display color of the check status information I1 may be changed to red, etc., to set the display mode of the check status information I1 to the error mode. In this case, regardless of whether the function related to the check processing is enabled or disabled, the check status information I1 is displayed in the error mode.

[0111] Furthermore, the restriction processing executed by the restriction processing unit 122 may be processing that restricts the operation of the work machine 3, and is not limited to processing that prohibits (makes inoperable) the operation (turning operation, etc.) of the work machine 3. The restriction processing may be, for example, processing that reduces the speed of the operation (turning operation, etc.) of the work machine 3, processing that narrows the operating range (turning angle, etc.) of the work machine 3, processing that restricts the allowable area for operation of the work machine 3, or the like.

[0112] As a specific configuration for slowing down the operation speed of the work machine 3, for example, instead of an on-off valve that can switch between opening and closing the flow path, an (electromagnetic) proportional control valve may be used as the control valve 46 inserted in the pilot oil flow path between the remote control valve 45 and the pilot pump 42. The control valve 46, which is a proportional control valve, is connected to the control system 1 and operates in response to an electrical signal (supply current) from the control system 1. As an example, the control valve 46 is an inverse proportional valve that reduces the output of the corresponding hydraulic actuator as the supply current increases. As a result, the restriction processing unit 122 can reduce the output of the corresponding hydraulic actuator by increasing the current supplied to the control valve 46, thereby slowing down the operation speed of the work machine 3. As an alternative method, for example, a variable displacement pump may be used as the pilot pump 42. In this case, the restriction processing unit 122 can reduce the output of the corresponding hydraulic actuator by controlling the variable displacement pump to reduce the flow rate of pilot oil, thereby slowing down the operation speed of the work machine 3.

[0113] Furthermore, the sensors for detecting the detection object Ob1 in the monitoring area A1 around the work machine 3 are not limited to the left camera 341, right camera 342, and rear camera 343, and may include one, two, four or more cameras (image sensors). Furthermore, the detection object Ob1 in the monitoring area A1 may be detected by a camera that can capture images in all directions as seen from the work machine 3, such as an omnidirectional camera (360-degree camera). The sensors for detecting the detection object Ob1 in the monitoring area A1 may include sensors such as a human presence sensor, sonar sensor, radar, or LiDAR (Light Detection and Ranging) in addition to or instead of a camera. Here, the sensor that detects the detection object Ob1 in the monitoring area A1 may be a three-dimensional sensor that measures the distance to the detection object Ob1 using a Time Of Flight (TOF) method, which measures the distance to a ranging point based on the round-trip time it takes for light or sound to reach and return from a ranging point.

[0114] In addition, the detection object Ob1 may include, in addition to or instead of a "person," a moving object such as a vehicle (including other work machinery), a structure such as a wall or pillar, a plant, an animal, a step, a ditch, or other obstacle.

[0115] Furthermore, it is not essential that the check processing include multiple specific processes (such as sound output processing and restriction processing) for checking the operation of the work machine 3. Furthermore, even if the check processing includes multiple specific processes, it is not essential that each specific process can be switched between enabled and disabled individually, and multiple specific processes may be switched between enabled and disabled collectively. It is also not essential that the check status information I1 indicate whether each specific process is enabled or disabled. Furthermore, when a function related to the check processing is disabled, it is not essential to display as check status information I1 that the function related to the check processing is disabled, and when a function related to the check processing is disabled, for example, the check status information I1 may be hidden.

[0116] Furthermore, it is not essential that the display screen Dp1 include captured images Im11, Im12, and Im13 of the monitoring area A1. Furthermore, it is not essential that the captured images Im11, Im12, and Im13 include images of multiple small areas A11, A12, and A13 included in the monitoring area A1, and they may include only an image of a single small area. Furthermore, the restraint processing does not have to include at least one of a sound output processing that outputs an alarm sound and a restriction processing that restricts the operation of the work machine 3. Furthermore, the restriction processing does not have to include a processing that restricts the rotational movement of the swivel unit 32, and may, for example, restrict the traveling movement of the traveling unit 31. Furthermore, it is not essential that the detection result information I2 and I3 be displayed regardless of whether the function related to the restraint processing is enabled or disabled. For example, the detection result information I2 and I3 may be displayed only when the function related to the restraint processing is enabled. Furthermore, it is not essential that the display mode of the restraint status information I1 be set to an error mode when an abnormality occurs in the detection of the detection unit 15.

[0117] Furthermore, the inactive display of the check status information I1, which indicates that the check processing function is disabled, is not limited to a display mode such as graying out. For example, as shown in FIG. 10 , when the check processing function is disabled, the check status information I1 may be displayed as inactive by superimposing a mark M1, such as a cross, indicating invalidity. Specifically, if the sound output processing unit 121 of the check processing unit 12 is enabled and the restriction processing unit 122 is disabled, for example, as in "Pattern 2" shown in FIG. 10 , the first check status information I11 is displayed as active, and the second check status information I12 is displayed as inactive by superimposing the mark M1. In "Pattern 4" in which both the sound output processing and the restriction processing are disabled, as shown in FIG. 10 , a single mark M1 may be superimposed on the first check status information I11 and the second check status information I12, or a separate mark M1 may be superimposed on each of the first check status information I11 and the second check status information I12. Furthermore, the mark M1 indicating invalidity is not limited to a cross, but may be any shape such as a diagonal line, a strikethrough, a rectangle, or text such as "invalid." Furthermore, instead of being limited to superimposing the mark M1 on the checking status information I1, the checking status information I1 may be displayed as inactive by a display mode in which the mark M1 is displayed in association with the checking status information I1 near the checking status information I1.

[0118] (Embodiment 2) As shown in Figures 11 and 12, the work machine 3 according to this embodiment differs from the work machine 3 according to the first embodiment in the display content of the second region R2 of the display screen Dp1. Hereinafter, the same components as those in the first embodiment will be assigned the same reference numerals and explanations thereof will be omitted as appropriate. Figures 11 and 12 show only the second region R2 of the display screen Dp1, and regions other than the second region R2 are not shown.

[0119] 11, two screens are displayed in the second area R2, one of which is the captured image Im12 captured by the right camera 342 and the other is the captured image Im13 captured by the rear camera 343, out of the captured images Im11, Im12, and Im13 (see FIG. 7) of the monitoring area A1. In this example, the captured images Im12 and Im13 are arranged side by side in the left-right direction in the second area R2, and an icon Im10 is displayed below these captured images Im12 and Im13. The icon Im10 schematically represents the positional relationship between the imaging ranges (small areas A12, A13) of the right camera 342 and the rear camera 343 as seen from the aircraft 30.

[0120] In the example of Fig. 12, an overhead image Im100 of the monitoring area A1 is displayed in the second region R2. The overhead image Im100 is a type of captured image generated by combining, through coordinate transformation, images Im11, Im12, and Im13 captured by the left camera 341, right camera 342, and rear camera 343. In this overhead image Im100 as well, the detection result information I2 is a band-shaped (frame-shaped) image that highlights a small area in which the detection target object Ob1 exists. The overhead image Im100 may be an image in which captured images are combined with animation.

[0121] The dual-screen display (see FIG. 11) and the overhead image Im100 (see FIG. 12) as exemplified in this embodiment may be arbitrarily switchable by operating the operation unit 22 of the display device 2. In this case, it is preferable that the user U1 can select any display format, including the triple-screen display (see FIG. 7) described in the first embodiment.

[0122] The configuration according to the second embodiment can be adopted in appropriate combination with the various configurations (including modified examples) described in the first embodiment.

[0123] <Notes on the invention> A method for controlling a work machine according to one embodiment of the present invention includes executing a restraint process to restrain the operation of the work machine based on the detection results of a detection unit that detects a detection object in a monitoring area surrounding the work machine, switching between enabling and disabling a function related to the restraint process, and displaying on a display device a display screen including restraint status information indicating whether the function related to the restraint process is enabled or disabled.

[0124] A work machine control program according to one aspect of the present invention is a program for causing one or more processors to execute the work machine control method.

[0125] A work machine control system according to one aspect of the present invention includes a check processing unit, a switching processing unit, and a display processing unit. The check processing unit executes check processing to check the operation of the work machine based on the detection results of a detection unit that detects a detection target in a monitoring area around the work machine. The switching processing unit switches between enabling and disabling the check processing unit. The display processing unit displays on a display device a display screen including check status information indicating whether the check processing unit is enabled or disabled.

[0126] A work machine according to one aspect of the present invention includes the work machine control system and a machine body on which the display device is mounted. [Explanation of symbols]

[0127] 1. Work machine control system 2 Display device 3. Work machinery 11 Display processing section 12 Check and balance processing section 13 Switching processing section 15 Detection unit 30 aircraft 31 Running part 32 Swivel section A1 Surveillance Area A11, A12, A13 small area Dp1 display screen I1 Check status information Im11, Im12, Im13 captured images Im100 overhead image (captured image) Ob1 Detected object

Claims

1. Executing a restraining process to restrain the operation of the work machine based on the detection results of a detection unit that detects a detection target in a monitoring area around the work machine; Switching between enabling and disabling a function related to the check processing; and displaying on a display device a display screen including check status information indicating whether a function related to the check processing is enabled or disabled and an image of the monitoring area, The restraint processing includes a sound output processing for outputting an alarm sound, and a restriction processing for restricting the operation of the work machine, The first checking state information, which is the checking state information related to the sound output processing, and the second checking state information, which is the checking state information related to the restriction processing, are displayed outside the captured image on the display screen. A method for controlling a work machine.

2. The first checking state information and the second checking state information are displayed next to the captured image. A method for controlling a work machine according to claim 1.

3. the display screen further includes a status display image that indicates a status of the work machine, The first checking state information and the second checking state information are displayed between the captured image and the state display image. A control method for a work machine according to claim 1 or 2.

4. the captured image includes images of a plurality of small areas included in the monitoring area, On the display screen, an image of a small area among the plurality of small areas in which the detection object exists is highlighted regardless of whether the first checking state information and the second checking state information are displayed. A control method for a work machine according to any one of claims 1 to 3.

5. The first check status information and the second check status information are displayed side by side in the left-right direction. A control method for a work machine according to any one of claims 1 to 4.

6. A control method for a work machine according to any one of claims 1 to 5, A control program for a work machine for execution by one or more processors.

7. a restraint processing unit that executes restraint processing to restrain the operation of the work machine based on the detection results of a detection unit that detects a detection target in a monitoring area around the work machine; a switching processing unit that switches between enabled and disabled states of the check processing unit; a display processing unit that displays on a display device a display screen including check status information indicating whether the check processing unit is enabled or disabled and a captured image of the monitoring area; The restraint processing includes a sound output processing for outputting an alarm sound, and a restriction processing for restricting the operation of the work machine, The display processing unit displays first checking state information, which is the checking state information related to the sound output processing, and second checking state information, which is the checking state information related to the restriction processing, outside the captured image on the display screen. Control systems for work machines.

8. A work machine control system according to claim 7; and a body on which the display device is mounted. Work machinery.

Images