Work machine monitoring system and work machine monitoring method

Abstract

A work machine monitoring system according to the present invention comprises: a measurement data acquisition unit that acquires measurement data of the ground on which a work machine travels; a measurement data adjustment unit that adjusts at least a part of the measurement data on the basis of the distance from the work machine; and a rock determination unit that determines the presence or absence of a rock on the ground in the adjusted measurement data.

Description

Work machine monitoring system and work machine monitoring method

[0001] The present disclosure relates to a monitoring system for a work machine and a monitoring method for a work machine.

[0002] A work machine operates at a work site. There is a possibility that boulders may be present on the ground at the work site. If the work machine runs over a boulder, it may cause problems with the work machine's travel. Patent Document 1 discloses a technology that outputs a warning to the driver's cab of a wheel loader when there is a rock that requires attention and that may damage the tires.

[0003] Japanese Patent Application Laid-Open No. 2021-080790

[0004] If there are boulders on the ground, a warning is output, allowing the operator of the work machine to recognize the presence of the boulders. For example, there is a demand for technology that allows the conditions for determining whether or not there are boulders on the ground to be adjusted in accordance with the needs of the operator.

[0005] The present disclosure aims to adjust the conditions for determining whether or not there are boulders on the ground.

[0006] According to the present disclosure, there is provided a work machine monitoring system comprising a measurement data acquisition unit that acquires measurement data of the ground on which the work machine is traveling, a measurement data adjustment unit that adjusts at least a portion of the measurement data based on the distance from the work machine, and a boulder determination unit that determines the presence or absence of boulders on the ground in the adjusted measurement data.

[0007] According to the present disclosure, it is possible to adjust the conditions for determining whether or not boulders are present on the ground.

[0008] FIG. 1 is a side view showing a work machine according to an embodiment. FIG. 2 is a top view showing a work machine according to an embodiment. FIG. 3 is a front view showing a work machine according to an embodiment. FIG. 4 is a view showing a driver's cab of a driver's cab according to an embodiment. FIG. 5 is a front view showing an enlarged portion of the work machine according to an embodiment. FIG. 6 is a view showing a drive system of a work machine according to an embodiment. FIG. 7 is a view explaining the operation of a work implement according to an embodiment. FIG. 8 is a view explaining the operation of a work machine according to an embodiment. FIG. 9 is a side view showing a work machine performing loading work according to an embodiment. FIG. 10 is a top view showing a work machine performing loading work according to an embodiment. FIG. 11 is a view showing an example of image data captured by an imaging device on the left side according to an embodiment. FIG. 12 is a functional block diagram showing a monitoring system according to an embodiment. FIG. 13 is a view explaining warning criteria according to an embodiment. FIG. 14 is a view explaining adjustment of measurement data according to an embodiment. FIG. 15 is a view explaining adjustment of the number of pixels according to an embodiment. FIG. 16 is a view explaining determination of the presence or absence of boulders according to an embodiment. FIG. 17 is a view showing an example of an output device according to an embodiment. FIG. 18 is a flowchart showing a ground surface monitoring method according to an embodiment. FIG. 19 is a block diagram showing a computer system according to an embodiment.

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiments. The components of the embodiments described below can be combined as appropriate. In addition, some components may not be used.

[0010] [Working Machine] Fig. 1 is a side view showing a working machine 1 according to an embodiment. Fig. 2 is a top view showing the working machine 1 according to an embodiment. Fig. 3 is a front view showing the working machine 1 according to an embodiment.

[0011] The work machine 1 operates at a work site. In this embodiment, the work machine 1 is a wheel loader, which is a type of articulated work machine. The work machine 1 performs work at the work site. The work performed by the work machine 1 includes an excavation operation in which an excavation target is excavated, and a loading operation in which the material excavated by the excavation operation is loaded onto a loading target.

[0012] As shown in Figures 1, 2, and 3, the work machine 1 includes a vehicle body 2, a cab 3, a traveling device 4, a front fender 7, a support member 8, a housing 9, a work implement 10, an angle sensor 90, and a load sensor 93.

[0013] The vehicle body 2 includes a front vehicle body 2F and a rear vehicle body 2R. The front vehicle body 2F and the rear vehicle body 2R are connected via a joint mechanism 2A.

[0014] The cab 3 is supported by the vehicle body 2. A driver's cab is provided in the cab 3. A driver's seat is provided in the cab. The work machine 1 is operated by a driver who sits in the cab 3.

[0015] The traveling device 4 supports the vehicle body 2 and travels on the ground 200 at the work site. The traveling device 4 has wheels 5 and tires 6. The tires 6 are attached to the wheels 5. The wheels 5 include front wheels 5F supported on the front vehicle body 2F and rear wheels 5R supported on the rear vehicle body 2R. The tires 6 include front tires 6F attached to the front wheels 5F and rear tires 6R attached to the rear wheels 5R.

[0016] The front wheel 5F and the front tire 6F are rotatable about a rotation axis FX, and the rear wheel 5R and the rear tire 6R are rotatable about a rotation axis RX.

[0017] In the embodiment, the direction parallel to the rotation axis FX of the front wheel 5F will be referred to as the vehicle width direction, as appropriate. The direction perpendicular to the contact surface of the front tire 6F that contacts the ground 200 will be referred to as the up-down direction, as appropriate. The direction perpendicular to both the vehicle width direction and the up-down direction will be referred to as the fore-aft direction, as appropriate. When the body 2 of the work machine 1 travels in a straight line, the rotation axis FX and the rotation axis RX are parallel to each other.

[0018] In addition, in the embodiment, a position or direction closer to the center CL of the work machine 1 in the vehicle width direction of the work machine 1 is appropriately referred to as the inside in the vehicle width direction, and a position or direction farther from the center CL is appropriately referred to as the outside in the vehicle width direction.

[0019] In the vehicle width direction, one side of the center CL of the work machine 1 is the left, and the opposite side of the left is the right. In the longitudinal direction, the position or direction closer to the work implement 10 using the driver's seat of the cab 3 as the reference is the front, and the opposite side of the front is the rear. In the vertical direction, the position or direction closer to the contact surface of the front tire 6F is the bottom, and the opposite side of the bottom is the top.

[0020] The rear vehicle body 2R is disposed rearward of the front vehicle body 2F. The front vehicle body 2F is bent to the left and right relative to the rear vehicle body 2R. The rear wheels 5R are disposed rearward of the front wheels 5F. The rear tires 6R are disposed rearward of the front tires 6F. The front wheels 5F and front tires 6F are disposed to the left and right of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The rear wheels 5R and rear tires 6R are disposed to the left and right of the center CL of the work machine 1 in the vehicle width direction of the work machine 1.

[0021] The tires 6 are rotating members that rotate while in contact with the ground surface 200. The tires 6 rotate while in contact with the ground surface 200, causing the work machine 1 to travel on the ground surface 200. The front tires 6F are front rotating members that rotate while in contact with the ground surface 200. The rear tires 6R are positioned rearward of the front tires 6F, and are rear rotating members that rotate while in contact with the ground surface 200.

[0022] The front fenders 7 prevent earth and sand scattered from the ground surface 200 from hitting the vehicle body 2 and the cab 3 when the work machine 1 is traveling. A portion of the front fender 7 is disposed above the front tires 6F. A portion of the front fender 7 is disposed rearward of the front tires 6F. The front fenders 7 are disposed on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The front fenders 7 include a front fender 7L disposed to the left of the center CL and a front fender 7R disposed to the right of the center CL. The left front fender 7L is attached to the left side of the front vehicle body 2F. The right front fender 7R is attached to the right side of the front vehicle body 2F.

[0023] The support member 8 is a rod-shaped member. The lower end of the support member 8 is fixed to the front vehicle body 2F. The support member 8 is inclined upward, forward, and outward in the vehicle width direction. The support members 8 are arranged on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The support members 8 include a support member 8L arranged to the left of the center CL and a support member 8R arranged to the right of the center CL. The lower end of the left support member 8L is fixed to the left section of the front vehicle body 2F. The left support member 8L is inclined upward, forward, and to the left. The lower end of the right support member 8R is fixed to the right section of the front vehicle body 2F. The right support member 8R is inclined upward, forward, and to the right.

[0024] The housing 9 is fixed to the upper end of the support member 8. The housing 9 is supported on the front vehicle body 2F via the support member 8. In the longitudinal direction, the housing 9 is disposed between the joint mechanism 2A and the front end of the front vehicle body 2F. In the vertical direction, the housing 9 is disposed between the upper end of the cab 3 and the upper end of the front vehicle body 2F. The housings 9 are disposed on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The housings 9 include a housing 9L disposed to the left of the center CL and a housing 9R disposed to the right of the center CL. The left housing 9L is fixed to the upper end of the left support member 8L. The right housing 9R is fixed to the upper end of the right support member 8R. The left housing 9L is disposed above the left front fender 7L. The right housing 9R is disposed above the right front fender 7R.

[0025] The work implement 10 is capable of excavating an excavation target. The work implement 10 operates during excavation work and loading work. The work implement 10 is coupled to a front vehicle body 2F. At least a portion of the work implement 10 is disposed forward of the front wheels 5F. The work implement 10 has a boom 11, a bucket 12, a bell crank 15, and a bucket link 16.

[0026] The boom 11 is rotatably connected to the front vehicle body 2F. The boom 11 connects the front vehicle body 2F and the bucket 12. The boom 11 operates by a driving force generated by a boom cylinder 13. The boom cylinder 13 operates the boom 11. The boom cylinder 13 is a hydraulic cylinder. One end of the boom cylinder 13 is connected to the front vehicle body 2F. The other end of the boom cylinder 13 is connected to the boom 11. The boom cylinder 13 is arranged on both the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1.

[0027] The bucket 12 excavates an excavation target. The bucket 12 is rotatably connected to the tip of the boom 11. The bucket 12 is positioned forward of the front wheels 5F. The bucket 12 is operated by a driving force generated by a bucket cylinder 14. The bucket cylinder 14 operates the bucket 12. The bucket cylinder 14 is a hydraulic cylinder. A central portion of a bell crank 15 is rotatably connected to the boom 11. One end of the bucket cylinder 14 is connected to the front vehicle body 2F. The other end of the bucket cylinder 14 is connected to one end of the bell crank 15. The other end of the bell crank 15 is connected to the bucket 12 via a bucket link 16. One bucket cylinder 14 is arranged in the central portion in the vehicle width direction.

[0028] The bucket 12 has a bucket body 17 , bucket teeth 18 , and inter-tooth protectors 19 .

[0029] The bucket body 17 holds the excavated material. The bucket body 17 includes a bottom plate portion 17A, an upper plate portion 17B, a left plate portion 17C, and a right plate portion 17D. The bottom plate portion 17A, the upper plate portion 17B, the left plate portion 17C, and the right plate portion 17D form a holding space in which the excavated material is accommodated. The tip ends of the bottom plate portion 17A and the upper plate portion 17B each extend in the vehicle width direction. The tip ends of the left plate portion 17C and the right plate portion 17D each extend in the up-down direction or the front-to-rear direction. The tip ends of the bottom plate portion 17A, the upper plate portion 17B, the left plate portion 17C, and the right plate portion 17D define an opening 12M in the bucket 12. The excavated material can enter the holding space of the bucket 12 through the opening 12M.

[0030] 2 and 3 , both end portions 12E of the bucket body 17 in the vehicle width direction are disposed outward in the vehicle width direction from the tires 6. The distance in the vehicle width direction between the left end portion 12E and the right end portion 12E is greater than the distance in the vehicle width direction between the left side surface of the left tire 6 and the right side surface of the right tire 6.

[0031] The bucket teeth 18 form the cutting edges of the bucket 12. The bucket teeth 18 are replaceably attached to the bucket body 17. The bucket teeth 18 are attached to the tip of the bottom plate portion 17A. A plurality of bucket teeth 18 are attached to the bucket body 17. The plurality of bucket teeth 18 are arranged at intervals in the vehicle width direction. In this embodiment, eight bucket teeth 18 are arranged at intervals in the vehicle width direction.

[0032] The inter-tooth protectors 19 protect the tip end of the bottom plate portion 17A. The inter-tooth protectors 19 are replaceably attached to the bucket body 17. The inter-tooth protectors 19 are arranged between a pair of adjacent bucket teeth 18. The inter-tooth protectors 19 are attached to the tip end of the bottom plate portion 17A. Multiple inter-tooth protectors 19 are attached to the bucket body 17. The multiple inter-tooth protectors 19 are arranged at intervals in the vehicle width direction. In the embodiment, seven inter-tooth protectors 19 are arranged at intervals in the vehicle width direction.

[0033] The angle sensor 90 is an attitude sensor that detects the attitude of the work implement 10. The attitude of the work implement 10 includes the angle of the work implement 10. The angle sensor 90 is provided on the work implement 10. The angle sensor 90 includes a boom angle sensor 91 that detects the angle of the boom 11 and a bucket angle sensor 92 that detects the angle of the bucket 12. The boom angle sensor 91 detects the angle of the boom 11 with respect to a reference axis of a vehicle body coordinate system defined in the front vehicle body 2F, for example. The bucket angle sensor 92 detects the angle of the bucket 12 with respect to the boom 11. An example of the angle sensor 90 is a potentiometer. Note that the attitude sensor that detects the attitude of the work implement 10 may be a stroke sensor that detects the stroke of the hydraulic cylinders (boom cylinder 13, bucket cylinder 14). The attitude sensor that detects the attitude of the work implement 10 may be an IMU (Inertial Measurement Unit) that detects acceleration and angular velocity.

[0034] The load sensor 93 detects the load applied to the work implement 10. The load sensor 93 can detect whether or not an excavated object is held in the bucket 12. The load sensor 93 can detect the weight of the excavated object held in the bucket 12. An example of the load sensor 93 is a pressure sensor that detects the pressure of hydraulic oil in the bottom chamber of the boom cylinder 13.

[0035] [Driver's Cabin] Fig. 4 is a diagram showing the driver's cab of the driver's seat 3 according to the embodiment. Fig. 4 is a diagram showing a schematic view seen from a driver seated in the driver's seat in the driver's cab.

[0036] The driver's cab is equipped with a driving operation device 20, an operation panel 21, a monitor device 22, a rearview monitor device 23, an output device 24, and an input device 25. The driving operation device 20 is operated by the driver. The driving operation device 20 includes an accelerator pedal 20A and a pair of brake pedals 20B. Although not shown, the driving operation device 20 also includes a steering lever, a shift lever, a forward / reverse lever, and an operation lever. Operating the accelerator pedal 20A increases the traveling speed of the work machine 1. Operating the brake pedal 20B decreases the traveling speed of the work machine 1 or stops the work machine 1 from traveling. Operating the steering lever causes the work machine 1 to turn. Operating the shift lever changes the speed stage of the work machine 1. Operating the forward / reverse lever switches the traveling direction of the work machine 1 between forward and reverse. Operating the operation lever causes the work implement 10 to operate. The monitor device 22 displays data relating to a plurality of functions of the work machine 1. The rearview monitor device 23 displays an image of the rear of the body 2 of the work machine 1 captured by a camera provided at the rear of the work machine 1 (not shown).

[0037] The output device 24 provides output data to the driver, including warnings. The output device 24 includes a display device 24A and a sound device 24B.

[0038] The display device 24A provides display data to the driver as output data. Examples of the display device 24A include a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OELD).

[0039] The sound generating device 24B provides the driver with voice data as output data. Examples of the sound generating device 24B include a buzzer and a speaker.

[0040] The input device 25 generates input data when operated by the driver. Examples of the input device 25 include a button or a keyboard. The input device 25 may include, for example, a touch panel.

[0041] A windshield is provided in front of the driver's cab. The windshield is attached to a pillar 27 of the driver's cab. In the embodiment, the output device 24 is supported by the pillar 27. The output device 24 may be a tablet terminal or the like. The monitor device 22, the rearview monitor device 23, and the output device 24 may be integrated in part or in whole.

[0042] [Measuring Device] Fig. 5 is an enlarged front view of a portion of the work machine 1 according to the embodiment. As shown in Figs. 3 and 5, the work machine 1 has a headlight 28, a turn signal lamp 29, and a measuring device 32.

[0043] The headlights 28 emit illumination light forward, illuminating an illumination range ahead of the front vehicle body 2F. The headlights 28 are held in the housing 9. The headlights 28 are arranged on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The headlights 28 include a headlight 28L arranged on the left side of the center CL and a headlight 28R arranged on the right side of the center CL. The left headlight 28L is held in the left housing 9L. The right headlight 28R is held in the right housing 9R.

[0044] The turn signal lamps 29 indicate the turning direction of the work machine 1 by lighting up or flashing. The turn signal lamps 29 are held in the housing 9. The turn signal lamps 29 are arranged on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The turn signal lamps 29 include a turn signal lamp 29L arranged on the left side of the center CL, and a turn signal lamp 29R arranged on the right side of the center CL. The left turn signal lamp 29L is held in the left housing 9L. The right turn signal lamp 29R is held in the right housing 9R. The turn signal lamps 29 are arranged outward of the headlights 28 in the vehicle width direction of the work machine 1.

[0045] The measuring device 32 measures a predetermined target. The target measured by the measuring device 32 includes the ground surface 200 on which the traveling device 4 travels. The measuring device 32 measures the ground surface 200 on which the work machine 1 travels. The measuring device 32 acquires measurement data of the ground surface 200. In the embodiment, the measuring device 32 is an imaging device that images the ground surface 200. The measurement data of the ground surface 200 includes image data of the ground surface 200. In the following description, the measuring device 32 will be referred to as the imaging device 32 as appropriate.

[0046] The imaging device 32 includes an optical system and an image sensor onto which light passing through the optical system is incident. Examples of the image sensor include a CCD (Couple Charged Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0047] The imaging device 32 is disposed in the housing 9. In the vehicle width direction of the work machine 1, the imaging device 32 is disposed further outward than the headlights 28.

[0048] The imaging devices 32 are arranged on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The imaging devices 32 are arranged in each of the left housing 9 and the right housing 9. The imaging devices 32 include an imaging device 32L arranged on the left side of the center CL, and an imaging device 32R arranged on the right side of the center CL.

[0049] 6 is a diagram showing a drive system 40 of the work machine 1 according to the embodiment. The drive system 40 has an engine 41, a fuel injection device 42, a power take-off (PTO) 43, a transmission 44, a front axle 45F, a rear axle 45R, a hydraulic pump 46, a control valve 47, and a drive controller 48.

[0050] The engine 41 is, for example, a diesel engine. The fuel injection device 42 injects fuel into the cylinders of the engine 41. The driving force of the engine 41 is adjusted by adjusting the amount of fuel injected from the fuel injection device 42 into the engine 41.

[0051] The power take-off 43 distributes the driving force of the engine 41 to the transmission 44 and the hydraulic pump 46. The driving force of the engine 41 is transmitted to both the transmission 44 and the hydraulic pump 46 via the power take-off 43.

[0052] The transmission 44 has an input shaft to which the driving force of the engine 41 is input, and an output shaft that changes the speed of the driving force input to the input shaft and outputs it. The input shaft of the transmission 44 is connected to the power take-off 43. The output shafts of the transmission 44 are connected to the front axle 45F and the rear axle 45R, respectively. The driving force of the engine 41 is transmitted to the front axle 45F and the rear axle 45R, respectively, via the power take-off 43 and the transmission 44.

[0053] The transmission 44 switches the traveling direction of the work machine 1 between a forward direction and a reverse direction. The transmission 44 has a forward gear 44F and a reverse gear 44R. When the forward gear 44F is engaged, the traveling direction of the work machine 1 is determined to be a forward direction. When the reverse gear 44R is engaged, the traveling direction of the work machine 1 is determined to be a reverse direction.

[0054] The front axle 45F transmits the driving force transmitted from the transmission 44 to the front wheels 5F. The front wheels 5F rotate based on the driving force transmitted from the front axle 45F.

[0055] The rear axle 45R transmits the driving force transmitted from the transmission 44 to the rear wheels 5R. The rear wheels 5R rotate based on the driving force transmitted from the rear axle 45R.

[0056] Engaging the forward gear 44F puts the work machine 1 in a state where it can move forward. With the forward gear 44F engaged, the accelerator pedal 20A is operated and the driving force of the engine 41 is transmitted to each of the front axle 45F and rear axle 45R via the transmission 44, causing the work machine 1 to move forward. Note that even with the forward gear 44F engaged, the work machine 1 will not move forward if, for example, the brake pedal 20B is operated.

[0057] Engaging the reverse gear 44R puts the work machine 1 in a state where it can move in reverse. With the reverse gear 44R engaged, the accelerator pedal 20A is operated and the driving force of the engine 41 is transmitted to each of the front axle 45F and the rear axle 45R via the transmission 44, causing the work machine 1 to move in reverse. Note that even with the reverse gear 44R engaged, the work machine 1 will not move in reverse if, for example, the brake pedal 20B is operated.

[0058] The hydraulic pump 46 discharges hydraulic oil. The hydraulic pump 46 is a variable displacement hydraulic pump. The hydraulic pump 46 is driven based on the driving force of the engine 41. The hydraulic oil discharged from the hydraulic pump 46 is supplied to at least one of the boom cylinder 13 and the bucket cylinder 14 via a control valve 47.

[0059] The control valve 47 controls the flow rate and direction of hydraulic oil supplied to each of the boom cylinder 13 and the bucket cylinder 14. The work machine 10 is operated by hydraulic oil supplied from the hydraulic pump 46 via the control valve 47.

[0060] The drive controller 48 controls the work machine 1 based on the operation signals output from the driving operation device 20. The drive controller 48 includes a computer system.

[0061] 7 is a diagram illustrating the operation of the work machine 10 according to the embodiment. In the embodiment, the work machine 10 is a front-loading type work machine in which the opening 12M of the bucket 12 faces forward during excavation work.

[0062] The boom cylinder 13 extends and retracts, causing the boom 11 to move up and down.

[0063] The raising operation of the boom 11 refers to the operation of the boom 11 rotating so that the tip of the boom 11 moves away from the ground 200. The boom cylinder 13 extends, causing the boom 11 to perform the raising operation.

[0064] The lowering operation of the boom 11 refers to the operation of the boom 11 rotating so that the tip of the boom 11 approaches the ground 200. The boom cylinder 13 contracts, causing the boom 11 to perform the lowering operation.

[0065] The bucket cylinder 14 extends and retracts, causing the bucket 12 to tilt or dump.

[0066] The tilting of the bucket 12 refers to the rotation of the bucket 12 so that the tip of the bucket 12 moves away from the ground 200 with the opening 12M of the bucket 12 facing upward. As the bucket cylinder 14 extends, the bell crank 15 rotates so that the upper end of the bell crank 15 moves forward and the lower end of the bell crank 15 moves rearward. When the lower end of the bell crank 15 moves rearward, the bucket 12 is pulled rearward by the bucket link 16, causing a tilting motion. As the bucket 12 tilts, the excavation target is scooped up by the bucket 12 and the excavated material is held in the bucket 12.

[0067] The dumping operation of the bucket 12 refers to the operation of the bucket 12 rotating so that the tip of the bucket 12 approaches the ground surface 200 with the opening 12M of the bucket 12 facing downward. When the bucket cylinder 14 contracts, the bell crank 15 rotates so that the upper end of the bell crank 15 moves rearward and the lower end of the bell crank 15 moves forward. When the lower end of the bell crank 15 moves forward, the bucket 12 is pushed forward by the bucket link 16, performing a dumping operation. When the bucket 12 performs a dumping operation, the excavated material held in the bucket 12 is ejected from the bucket 12.

[0068] [Operation of the Work Machine] Figure 8 is a diagram illustrating the operation of the work machine 1 according to this embodiment. The work machine 1 performs excavation work and loading work at a work site.

[0069] Excavation work is work to excavate an excavation target. Examples of the excavation target include natural ground or a rocky mountain. Natural ground is a mountain made up of earth and sand placed on the ground surface 200. A rocky mountain is a mountain made up of rocks or stones placed on the ground surface 200. In the embodiment, the excavation target is natural ground 210. The excavated material is a portion of the natural ground 210 excavated and held by the bucket 12.

[0070] The loading operation is an operation of loading the excavated material excavated by the excavation operation onto a loading target. Examples of the loading target include a transport vehicle, a hopper, or a belt conveyor. In the embodiment, the loading target is a dump body 230 of a dump truck 220, which is a type of transport vehicle.

[0071] The work machine 1 is operated by the driver to perform cyclic work, which involves repeating a series of work modes. The cyclic work is made up of a plurality of interrelated work modes. The cyclic work includes travel of the traveling device 4 and operation of the work implement 10.

[0072] In this embodiment, the cycle operation is made up of six work modes. In this embodiment, the series of work modes of the work machine 1 includes an empty forward mode, an excavation mode, a loaded reverse mode, a loaded forward mode, a loading mode, and an empty reverse mode.

[0073] The sequence of the series of operation modes is fixed. After the empty forward mode is performed, the excavation mode is performed. After the excavation mode is performed, the loaded reverse mode is performed. After the loaded reverse mode is performed, the loaded forward mode is performed. After the loaded forward mode is performed, the loading mode is performed. After the loading mode is performed, the empty reverse mode is performed.

[0074] The empty forward movement mode is a work mode in which the work machine 1 moves forward to approach the excavation target with no excavation material held in the bucket 12. In the empty forward movement mode, the work machine 1 moves forward to approach the natural ground 210 as shown by arrow M1 in Figure 8 with no excavation material held in the bucket 12.

[0075] The excavation mode is a work mode in which an excavation target is excavated with the bucket 12 of the work machine 10. In the excavation mode, excavation work is performed. With at least a portion of the bucket 12 inserted into the natural ground 210, the bucket 12 performs a tilting operation, thereby excavating the natural ground 210 with the bucket 12 and holding the excavated material in the bucket 12.

[0076] The loaded reverse mode is a work mode in which the work machine 1 moves backward so as to move away from the excavation target with an excavated object held in the bucket 12. In the loaded reverse mode, the work machine 1 moves backward so as to move away from the natural ground 210 with an excavated object held in the bucket 12, as shown by arrow M2 in Figure 8.

[0077] The load forward mode is a work mode in which the work machine 1 moves forward to approach the loading target with an excavated work held in the bucket 12. In the load forward mode, the work machine 1 moves forward while swinging toward the dump truck 220, as shown by arrow M3 in Figure 8, with an excavated work held in the bucket 12. When the work machine 1 is moving forward toward the dump truck 220, the boom 11 performs a raising operation so that the bucket 12 is positioned above the dump body 230.

[0078] The loading mode is a work mode in which the excavated material held in the bucket 12 of the work machine 10 is loaded onto a loading target. In the loading mode, loading work is carried out. After the bucket 12 is positioned above the dump body 230, the bucket 12 performs a dumping operation, whereby the excavated material held in the bucket 12 is discharged from the bucket 12 and loaded into the dump body 230.

[0079] The empty reverse mode is a work mode in which the work machine 1 moves backward so as to move away from the loading target with no excavated material held in the bucket 12. With no excavated material held in the bucket 12, the work machine 1 moves backward while swinging so as to move away from the dump truck 220, as shown by arrow M4 in Figure 8.

[0080] The work machine 1 repeats a cycle of operations including an empty forward mode, an excavation mode, a loaded reverse mode, a loaded forward mode, a loading mode, and an empty reverse mode until the excavated material is loaded into the dump body 230 at the target load amount.

[0081] [Measurement range] Fig. 9 is a side view showing a work machine 1 that performs loading work according to this embodiment. Fig. 10 is a top view showing a work machine 1 that performs loading work according to this embodiment.

[0082] 9 and 10 , when loading excavated material into the dump body 230 of the dump truck 220, the work machine 1 performs a raising operation of the boom 11 so that the bucket 12 is positioned above the dump body 230. After the boom 11 has performed the raising operation and the bucket 12 has been positioned above the dump body 230, the work machine 1 performs a dumping operation of the bucket 12. With the boom 11 performing the raising operation, the bucket 12 performs a dumping operation so that the excavated material held in the bucket 12 is discharged into the dump body 230 of the dump truck 220. Due to the dumping operation of the bucket 12, the excavated material held in the bucket 12 is discharged from the bucket 12 and loaded into the dump body 230.

[0083] The measurement range Rb of the imaging device 32 includes the imaging range of the imaging device 32. The measurement range Rb is determined based on the angle of view β that indicates the angle of view of the optical system of the imaging device 32. The imaging device 32 images at least the ground surface 200 in the traveling direction of the tires 6. In the embodiment, the imaging device 32 images the ground surface 200 in front of the front tires 6F when the work machine 1 moves forward. The imaging device 32 is fixed to the housing 9 so that at least the ground surface 200 in front of the front tires 6F is located within the measurement range Rb when the work machine 1 moves forward. In the embodiment, the imaging device 32 is fixed to the housing 9 so that the front end of the front tires 6F, the ground surface 200 in front of the front tires 6F, and the ground surface 200 on the outer side of the front tires 6F in the vehicle width direction are all located within the measurement range Rb. The ground surface 200 in front of the front tires 6F includes the ground surface 200 between the front end of the front tires 6F and the rear end of the bucket 12 in the front-to-rear direction.

[0084] 10, the imaging device 32L on the left side of the center CL captures an image of the ground surface 200 in front of the front tire 6F on the left side of the center CL. The imaging device 32R on the right side of the center CL captures an image of the ground surface 200 in front of the front tire 6F on the right side of the center CL.

[0085] 10 , the imaging device 32L on the left side of the center CL is fixed to the left housing 9 so that the front end of the left front tire 6F, the ground surface 200 in front of the left front tire 6F, and the ground surface 200 to the left of the left front tire 6F are positioned within the measurement range Rb. The imaging device 32R on the right side of the center CL is fixed to the right housing 9 so that the front end of the right front tire 6F, the ground surface 200 in front of the right front tire 6F, and the ground surface 200 to the right of the right front tire 6F are positioned within the measurement range Rb.

[0086] 11 is a diagram showing an example of image data captured by the left imaging device 32L according to the embodiment. As shown in FIG. 11 , the imaging device 32L captures images of the ground 200 in front of the front tire 6F and the ground 200 to the left of the front tire 6F from above. This allows the imaging device 32L to capture an image of the ground 200 around the front tire 6F. For example, if a boulder 240 is present on the ground 200 around the front tire 6F, the imaging device 32L can capture an image of the boulder 240 on the ground 200.

[0087] [Monitoring System] Fig. 12 is a functional block diagram showing a monitoring system 30 according to this embodiment. The work machine 1 has a monitoring system 30. The monitoring system 30 monitors the ground surface 200 on which the work machine 1 travels.

[0088] The monitoring system 30 includes an imaging device 32, a driving operation device 20, an angle sensor 90, a load sensor 93, an output device 24, an input device 25, and a monitoring controller 33. The imaging device 32 includes an imaging device 32L arranged to the left of the center CL and an imaging device 32R arranged to the right of the center CL.

[0089] The imaging device 32, the driving operation device 20, the angle sensor 90, the load sensor 93, the output device 24, and the input device 25 are each connected to the monitoring controller 33. Image data captured by the imaging device 32 is input to the monitoring controller 33. Operation data of the driving operation device 20 is input to the monitoring controller 33. Detection data of the angle sensor 90 is input to the monitoring controller 33. Detection data of the load sensor 93 is input to the monitoring controller 33. Input data generated by the input device 25 is input to the monitoring controller 33.

[0090] In the embodiment, the imaging device 32 acquires measurement data at a predetermined sampling rate (SPS: samples per second). The measurement data includes image data. The sampling rate includes a frame rate (FPS: frames per second). The imaging device 32 captures the image data at the predetermined frame rate. A moving image is captured by the imaging device 32 capturing the image data at the predetermined frame rate.

[0091] The monitoring controller 33 recognizes the presence or absence of boulders 240 on the ground 200 on which the work machine 1 is traveling, based on image data captured by the imaging device 32. When a boulder 240 on the ground 200 is recognized, the monitoring controller 33 causes the output device 24 to output a warning indicating the presence of a boulder 240 on the ground 200. The boulders 240 on the ground 200 include the boulders 240 on the ground 200 in the traveling direction of the front tires 6F. The boulders 240 on the ground 200 in the traveling direction of the front tires 6F include the boulders 240 on the ground 200 in front of the front tires 6F when the work machine 1 moves forward.

[0092] The monitoring controller 33 recognizes the presence or absence of the boulder 240 during the cycle work described with reference to Fig. 8. Note that the monitoring controller 33 may also recognize the presence or absence of the boulder 240 during work other than the cycle work.

[0093] The monitoring controller 33 has a measurement data acquisition unit 331, a vehicle body data acquisition unit 332, a state determination unit 333, a boulder detection unit 334, a warning standard setting unit 335, a measurement data adjustment unit 336, a boulder determination unit 337, a warning control unit 338, a detection control unit 339, and a coefficient memory unit 340.

[0094] The measurement data acquisition unit 331 acquires measurement data from the imaging device 32. The measurement data acquisition unit 331 acquires measurement data of the ground surface 200 measured by the imaging device 32 from the imaging device 32 at a predetermined sampling rate. The measurement data of the ground surface 200 includes image data of the ground surface 200 imaged by the imaging device 32. The sampling rate includes a frame rate. The measurement data acquisition unit 331 acquires image data of the ground surface 200 from the imaging device 32 at a predetermined frame rate. The measurement data acquisition unit 331 acquires image data of the ground surface 200 in front of the front tire 6F.

[0095] The vehicle body data acquisition unit 332 acquires vehicle body data indicating the working state of the work machine 1. The working state of the work machine 1 includes the state of the work implement 10 and the state of the travelling device 4. The state of the work implement 10 includes the attitude of the work implement 10 and the load on the work implement 10. The state of the travelling device 4 includes forward and reverse movement of the travelling device 4. The vehicle body data includes attitude data indicating the attitude of the work implement 10, load data indicating the load on the work implement 10, and forward / reverse movement data indicating the forward or reverse movement of the travelling device 4.

[0096] The attitude data of the work implement 10 includes the angle of the work implement 10. The angle of the work implement 10 is detected by the angle sensor 90. The attitude data of the work implement 10 includes detection data of the angle sensor 90. The vehicle body data acquisition unit 332 acquires the detection data of the angle sensor 90 as the attitude data of the work implement 10.

[0097] The load data of the work implement 10 includes the pressure of the hydraulic oil in the bottom chamber of the boom cylinder 13. The pressure of the hydraulic oil in the bottom chamber of the boom cylinder 13 is detected by a load sensor 93. The load data of the work implement 10 includes the detection data of the load sensor 93. The vehicle body data acquisition unit 332 acquires the detection data of the load sensor 93 as the load data of the work implement 10.

[0098] The travel device 4 is switched between forward and reverse movement by a forward / reverse lever 20C of the driving operation device 20. The forward / reverse movement data of the travel device 4 includes operation data of the forward / reverse movement lever 20C. The vehicle body data acquisition unit 332 acquires the operation data of the forward / reverse movement lever 20C as the forward / reverse movement data of the travel device 4.

[0099] For example, if a rotation sensor that detects the rotation of the wheels 5 is provided, the rotation sensor can detect whether the traveling device 4 is moving forward or backward by detecting the rotation direction of the wheels 5. The vehicle body data acquisition unit 332 may acquire detection data of the rotation sensor as forward / reverse movement data of the traveling device 4.

[0100] The vehicle body data acquisition unit 332 may acquire status data indicating the status of the drive system 40 as forward / reverse travel data for the traveling device 4. The drive controller 48 inputs status data indicating the status of the drive system 40 to the monitoring controller 33. The vehicle body data acquisition unit 332 determines the status of the drive system 40 based on the status data from the drive controller 48. The status of the drive system 40 includes the operating state of the transmission 44. The transmission 44 switches the traveling direction of the work machine 1 between a forward direction and a reverse direction. The transmission 44 includes a forward gear 44F that operates to move the work machine 1 forward, and a reverse gear 44R that operates to move the work machine 1 backward. The vehicle body data acquisition unit 332 may acquire status data indicating the status of the transmission 44 as forward / reverse travel data for the traveling device 4.

[0101] The state determination unit 333 determines the working state of the work machine 1 based on the vehicle body data acquired by the vehicle body data acquisition unit 332. The state determination unit 333 can determine whether the boom 11 is performing a raising operation or a lowering operation based on the attitude data of the work implement 10 acquired by the vehicle body data acquisition unit 332. The state determination unit 333 can determine whether the bucket 12 is performing a tilting operation or a dumping operation based on the attitude data of the work implement 10 acquired by the vehicle body data acquisition unit 332. The state determination unit 333 can determine whether the work implement 10 is excavating an excavation target based on the load data of the work implement 10 acquired by the vehicle body data acquisition unit 332. The state determination unit 333 can determine whether the work machine 10 is moving forward or backward based on forward / reverse travel data of the traveling device 4 acquired by the vehicle body data acquisition unit 332.

[0102] The working state of the work machine 1 includes the working mode of the work machine 1. The working mode of the work machine 1 indicates the working state of the work machine 1. The state determination unit 333 can determine whether the working mode of the work machine 1 is empty forward mode, excavation mode, loaded reverse mode, loaded forward mode, loading mode, or empty reverse mode, based on the posture data of the work implement 10, the load data of the work implement 10, and the forward / reverse data of the traveling device 4.

[0103] The unladen forward mode indicates a working state in which the work machine 1 moves forward and the work implement 10 does not excavate the natural ground 210. The excavation mode indicates a working state in which the work machine 1 moves forward and the work implement 10 excavates the natural ground 210. The loaded reverse mode indicates a working state in which the work machine 1 moves backward. The loaded forward mode indicates a working state in which the work machine 1 moves forward and the work implement 10 does not excavate the natural ground 210. The loading mode indicates a working state in which the work machine 1 moves forward and the work implement 10 does not excavate the natural ground 210. The unladen reverse mode indicates a working state in which the work machine 1 moves backward.

[0104] Based on the posture data of the work machine 10, the load data of the work machine 10, and the forward and backward movement data of the traveling device 4, the state determination unit 333 can determine whether the working state of the work machine 1 is one of a working state in which the work machine 1 moves forward and the work machine 10 is not digging (first state), a working state in which the work machine 1 moves forward and the work machine 10 is digging (second state), or a working state in which the work machine 1 moves backward (third state).

[0105] The boulder detection unit 334 detects the boulder 240 from the image data of the ground 200 captured by the imaging device 32. The boulder detection unit 334 detects the boulder 240 from the image data of the ground 200 acquired by the measurement data acquisition unit 331 based on a predetermined algorithm. The algorithm for detecting the boulder 240 from the image data of the ground 200 includes an artificial intelligence (AI) algorithm. The AI ​​algorithm detects the boulder 240 from the image data of the ground 200 captured by the imaging device 32 based on a learning model generated by learning the feature amounts of the boulder 240. The learning model is a learning model that receives the feature amounts of an object as input and outputs the boulder 240.

[0106] The warning criteria setting unit 335 sets the warning criteria when a warning is output from the output device 24 .

[0107] 13 is a diagram illustrating warning criteria according to an embodiment. In the embodiment, the warning criteria include a detection area 34 set in at least a part of the image data of the ground 200, a first warning area 35 set inside the detection area 34, and a second warning area 36 set inside the first warning area 35.

[0108] The detection area 34 is rectangular in shape. A boulder 240 present inside the detection area 34 is detected. The boulder detector 334 detects the boulder 240 present in the detection area 34.

[0109] The first warning area 35 is smaller than the detection area 34. The first warning area 35 is rectangular. The first warning area 35 is set inside the detection area 34. At least a portion of the first warning area 35 is set on the ground surface 200 in front of the front tires 6F. The front end of the first warning area 35 is defined rearward (toward the work machine 1) of the front end of the detection area 34.

[0110] The second warning area 36 is smaller than the first warning area 35. The second warning area 36 is rectangular. The second warning area 36 is set inside the first warning area 35. At least a portion of the second warning area 36 is set on the ground surface 200 in front of the front tires 6F. The front end of the second warning area 36 is defined rearward (toward the work machine 1) than the front end of the first warning area 35.

[0111] The measurement data adjustment unit 336 adjusts at least a portion of the measurement data of the ground surface 200 based on the distance from the work machine 1. When the boulder 240 is detected by the boulder detection unit 334, the measurement data adjustment unit 336 can calculate the distance from the work machine 1 to the boulder 240. The measurement data adjustment unit 336 adjusts at least a portion of the measurement data of the ground surface 200 based on the distance from the work machine 1 to the boulder 240.

[0112] 14 is a diagram illustrating adjustment of measurement data according to an embodiment. As shown in FIG. 14 , the measurement data adjustment unit 336 divides the detection area 34 into a plurality of divided areas 50. In the example shown in FIG. 14 , the detection area 34 is divided into nine divided areas 50 in a matrix. The divided areas 50 include a first divided area 51, a second divided area 52, a third divided area 53, a fourth divided area 54, a fifth divided area 55, a sixth divided area 56, a seventh divided area 57, an eighth divided area 58, and a ninth divided area 59.

[0113] The distance between the work machine 1 and each of the multiple defined areas 50 is different. The defined areas 50 (first defined area 51, second defined area 52, third defined area 53) that are defined at the top of the detection area 34 have a longer distance from the work machine 1, and the defined areas 50 (seventh defined area 57, eighth defined area 58, ninth defined area 59) that are defined at the bottom of the detection area 34 have a shorter distance from the work machine 1. In the example shown in Figure 14, the distance between the work machine 1 and the first defined area 51 is the longest. The distance between the work machine 1 and the ninth defined area 59 is the shortest.

[0114] When a boulder 240 is detected in the detection area 34, the measurement data adjustment unit 336 adjusts a portion of the detection area 34 that includes the detected boulder 240 based on the distance between the work machine 1 and the boulder 240. When the distance between the work machine 1 and the boulder 240 is a first distance, the measurement data adjustment unit 336 makes a first adjustment to the portion of the measurement data that includes the boulder 240, and when the distance between the work machine 1 and the boulder 240 is a second distance, the measurement data adjustment unit 336 makes a second adjustment to the portion of the measurement data that includes the boulder 240.

[0115] In this embodiment, a coefficient related to the distance between the work machine 1 and the boulder 240 is set in advance. The coefficient storage unit 340 stores the coefficient that is set in advance based on the distance between the work machine 1 and the boulder 240. The measurement data adjustment unit 336 adjusts a portion of the detection area 34 based on the coefficient stored in the coefficient storage unit 340.

[0116] The coefficients may be input to the coefficient storage unit 340 via the input device 25. The operator or a maintenance person may input the coefficients by operating the input device 25. The coefficients input via the input device 25 may be stored in the coefficient storage unit 340.

[0117] As shown in FIG. 14 , in an embodiment, a coefficient is set for each of a plurality of defined areas 50. The coefficient may be a real number greater than 1, may be 1, or may be a positive real number less than 1. In the example shown in FIG. 14 , the farther the defined area 50 is from the work machine 1, the larger the coefficient value, and the shorter the defined area 50 is from the work machine 1, the smaller the coefficient value. In the example shown in FIG. 14 , the coefficient set for the first defined area 51 is 1.4. The coefficient set for the second defined area 52 is 1.3. The coefficient set for the third defined area 53 is 1.3. The coefficient set for the fourth defined area 54 is 1.2. The coefficient set for the fifth defined area 55 is 1.0. The coefficient set for the sixth defined area 56 is 1.0. The coefficient set for the seventh defined area 57 is 1.0. The coefficient set for the eighth defined area 58 is 1.0. The coefficient set for the ninth defined area 59 is 1.0.

[0118] The coefficients shown in Figure 14 are merely an example. The coefficient value may be smaller for a divided area 50 that is farther away from the work machine 1, and larger for a divided area 50 that is closer to the work machine 1.

[0119] In an embodiment, the measurement data adjustment unit 336 adjusts the number of pixels of a portion of the detection area 34 defined in the image data of the ground surface 200 based on the coefficient.

[0120] FIG. 15 is a diagram illustrating adjustment of the pixel count according to an embodiment. As shown in FIG. 15 , image data is composed of a plurality of pixels 60. In the embodiment, the number of pixels 60 in the image data is appropriately referred to as the pixel count. The pixel count may be the number of horizontal pixels (10 in the case shown in FIG. 15 ) or the number of vertical pixels (10 in the case shown in FIG. 15 ) in the sectional area 50 in which the boulder 240 is detected, or the larger of the number of horizontal and vertical pixels. In the following examples, the number of pixels 60 in the image data will be described as the pixel count. When the boulder 240 is detected in a certain sectional area 50 by the boulder detection unit 334, the measurement data adjustment unit 336 multiplies the number of pixels in the sectional area 50 in which the boulder 240 is detected by a coefficient. The number of pixels in the sectional area 50 is adjusted by multiplying the number of pixels in the sectional area 50 by the coefficient.

[0121] If the coefficient is a real number greater than 1, the number of pixels in the partitioned area 50 after adjustment increases compared to the number of pixels in the partitioned area 50 before adjustment. If the coefficient is 1, the number of pixels in the partitioned area 50 after adjustment remains the same as the number of pixels in the partitioned area 50 before adjustment. If the coefficient is a positive real number less than 1, the number of pixels in the partitioned area 50 after adjustment decreases compared to the number of pixels in the partitioned area 50 before adjustment.

[0122] 15 , if the number of pixels in one divided area 50 is 100 and the coefficient set for that divided area 50 is 1.4, the measurement data adjustment unit 336 multiplies the number of pixels, 100, by the coefficient 1.4. As a result, the number of pixels in the divided area 50 increases to 140. If the number of pixels in one divided area 50 is 100 and the coefficient set for that divided area 50 is 0.9, the number of pixels in the divided area 50 decreases to 90.

[0123] Increasing the number of pixels in the defined area 50 includes increasing the number of pixels in the image of the boulder 240. Increasing the number of pixels in the image of the boulder 240 includes increasing the amount of image data of the boulder 240. Decreasing the number of pixels in the defined area 50 includes decreasing the number of pixels in the image of the boulder 240. Decreasing the number of pixels in the image of the boulder 240 includes decreasing the amount of image data of the boulder 240.

[0124] The boulder determination unit 337 determines whether or not there is a boulder 240 on the ground 200 in the image data after it has been adjusted by the measurement data adjustment unit 336. The boulder determination unit 337 determines whether or not there is a boulder 240 on the ground 200 based on predetermined determination criteria. If the boulder 240 detected by the boulder detection unit 334 satisfies the determination criteria, the boulder determination unit 337 determines that there is a boulder 240. If the boulder 240 is detected by the boulder detection unit 334 but does not satisfy the determination criteria, the boulder determination unit 337 determines that there is no boulder 240.

[0125] FIG. 16 is a diagram illustrating the determination of the presence or absence of a boulder 240 according to the embodiment. The boulder determination unit 337 compares the number of pixels in the image of the boulder 240 after the pixel number has been adjusted by the measurement data adjustment unit 336 with a predetermined threshold Sh related to the number of pixels. As shown in FIG. 16 , the boulder determination unit 337 determines that a boulder 240 is present when the number of pixels in the image of the boulder 240 exceeds the threshold Sh, and determines that a boulder 240 is not present when the number of pixels in the image of the boulder 240 is equal to or less than the threshold Sh. Even if the actual size of the boulder 240 is the same, the boulder 240 may be determined to be present or absent depending on the value of the coefficient.

[0126] That is, even if the rolling stone detection unit 334 detects the rolling stone 240, if the rolling stone 240 does not satisfy the determination criterion (if the number of pixels of the rolling stone 240 is equal to or less than the threshold value Sh), the rolling stone determination unit 337 determines that the rolling stone 240 is not present. If the rolling stone detection unit 334 detects the rolling stone 240 and the rolling stone 240 satisfies the determination criterion (if the number of pixels of the rolling stone 240 exceeds the threshold value Sh), the rolling stone determination unit 337 determines that the rolling stone 240 is present.

[0127] The warning control unit 338 causes the output device 24 to output a warning based on the presence or absence of a rolling stone 240. The warning control unit 338 causes the output device 24 to output a warning based on the determination result of the rolling stone determination unit 337.

[0128] 17 is a diagram illustrating an example of the output device 24 according to the embodiment. The output device 24 includes a display device 24A that displays an image of the ground 200. The warning control unit 338 causes the output device 24 to output a warning based on the determination by the boulder determination unit 337 of the presence or absence of a boulder 240 and the relationship between the boulder 240 and the warning criteria.

[0129] In an embodiment, outputting the warning includes displaying a warning image 80 that highlights the boulder 240 included in the image of the ground 200. In an embodiment, the warning image 80 includes a frame image that is displayed to surround the boulder 240.

[0130] When the boulder 240 is inside the detection area 34 but outside the first warning area 35, the warning control unit 338 surrounds the boulder 240 with the first frame image 81. When the boulder 240 is inside the first warning area 35 but outside the second warning area 36, ​​the warning control unit 338 surrounds the boulder 240 with the second frame image 82. When the boulder 240 is inside the second warning area 36, ​​the warning control unit 338 surrounds the boulder 240 with the third frame image 83. The display forms of the first frame image 81, the second frame image 82, and the third frame image 83 are different. For example, the first frame image 81 is displayed in green, the second frame image 82 is displayed in yellow, and the third frame image 83 is displayed in red. By checking the warning image 80, the driver can intuitively recognize the distance between the work machine 1 and the boulder 240.

[0131] The warning control unit 338 may cause the display device 24A to display a frame image indicating the outer edge of the detection area 34, a frame image indicating the outer edge of the first warning area 35, and a frame image indicating the outer edge of the second warning area 36. The frame image indicating the outer edge of the detection area 34 may be displayed in green, the frame image indicating the outer edge of the first warning area 35 may be displayed in yellow, and the frame image indicating the outer edge of the second warning area 36 may be displayed in red.

[0132] In this embodiment, when a boulder 240 is present in the second warning area 36, ​​the warning control unit 338 causes the sound generating device 24B to emit a warning sound indicating that a boulder 240 is present on the ground 200 in front of the front tire 6F.

[0133] When the state determination unit 333 determines that the working state of the work machine 1 is in the first state, the detection control unit 339 causes the warning control unit 338 to execute a first process related to the boulders 240 on the ground surface 200. When the state determination unit 333 determines that the working state of the work machine 1 is in the second state, the detection control unit 339 causes the warning control unit 338 to execute a second process related to the boulders 240 on the ground surface 200. When the state determination unit 333 determines that the working state of the work machine 1 is in the third state, the detection control unit 339 causes the warning control unit 338 to execute a third process related to the boulders 240 on the ground surface 200. The first process and the second process are different. The first process and the third process may be the same or different.

[0134] In this embodiment, the first state is a working state in which the work machine 1 moves forward and the work implement 10 does not excavate. The second state is a working state in which the work machine 1 moves forward and the work implement 10 excavates. The third state is a working state in which the work machine 1 moves backward.

[0135] In an embodiment, the first state is at least one of an empty forward mode, a loaded forward mode, and a loading mode, the second state is a dig mode, and the third state is at least one of a loaded reverse mode and an empty reverse mode.

[0136] The first process is a process for causing the warning control unit 338 to output a warning sound when the boulder determination unit 337 determines that a boulder 240 is present. In other words, when the work machine 1 is operating in at least one of the work modes of the empty forward mode, the loaded forward mode, and the loading mode, if it is determined that a boulder 240 is present in the second warning area 36, ​​a warning sound is output from the sound generation device 24B.

[0137] The second process is a process that does not cause the warning control unit 338 to output a warning sound even if the boulder determination unit 337 determines that there is a boulder 240. In other words, when the work machine 1 is working in excavation mode, even if it is determined that there is a boulder 240 in the second warning area 36, ​​no warning sound is output from the sound generation device 24B.

[0138] The third process is a process that does not cause the warning control unit 338 to output a warning sound. In other words, when the work machine 1 is working in at least one of the work modes of the loaded reverse mode and the unladen reverse mode, no warning sound is output from the sound generation device 24B.

[0139] When the work machine 1 is operating in at least one of the work modes of the unladen forward mode, the loaded forward mode, and the loading mode, if it is determined that there is a boulder 240 in the second warning area 36, ​​a warning sound is output from the sound generating device 24B, allowing the driver to recognize the presence of the boulder 240.

[0140] When the work machine 1 is operating in excavation mode, the bucket 12 penetrates into the natural ground 210. When the bucket 12 has penetrated into the natural ground 210, a large number of boulders 240 are present around the front tires 6F. Furthermore, when the bucket 12 has penetrated into the natural ground 210, the driver is fully aware of the presence of the boulders 240. Furthermore, the state in which the bucket 12 has penetrated into the natural ground 210 is maintained for several seconds. If a warning sound continues to be output from the sound generation device 24B while the bucket 12 is penetrated into the natural ground 210, the driver is likely to find it annoying. Furthermore, when the bucket 12 has penetrated into the natural ground 210, the driver is fully aware of the presence of the boulders 240, and therefore the process of outputting a warning sound is likely to be unnecessary.

[0141] Therefore, in this embodiment, when the state determination unit 333 determines that the work machine 1 is working in excavation mode, the detection control unit 339 outputs a warning sound stop command to the warning control unit 338 so that a warning sound is not output while the excavation mode is being executed. While work is being executed in excavation mode, even if it is determined that a boulder 240 is present, no warning sound is output from the sound generation device 24B, which reduces the annoyance felt by the driver.

[0142] On the other hand, when the state determination unit 333 determines that the work machine 1 is operating in at least one of the work modes of the unladen forward mode, the loaded forward mode, and the loading mode, the detection control unit 339 outputs a warning sound permission command to the warning control unit 338 so that a warning sound is output when it is determined that a boulder 240 is present.

[0143] [Monitoring Method] FIG. 18 is a flowchart showing a method for monitoring the ground surface 200 according to the embodiment.

[0144] The measurement data acquisition unit 331 acquires image data of the ground surface 200 from the imaging device 32 (step S1).

[0145] The rolling stone detection unit 334 detects the rolling stone 240 from the image data acquired in step S1. The rolling stone detection unit 334 detects the rolling stone 240 from the image data based on a predetermined algorithm (step S2).

[0146] The measurement data adjustment unit 336 adjusts the number of pixels in the image of the boulder 240 based on the distance between the work machine 1 and the boulder 240. The measurement data adjustment unit 336 multiplies the number of pixels in the image of the boulder 240 by a coefficient that is set in advance based on the distance between the work machine 1 and the boulder 240 (step S3).

[0147] The boulder determination unit 337 determines whether or not the boulder 240 after the pixel number has been adjusted is in the second warning area 36. The boulder determination unit 337 determines whether or not the boulder 240 is in the second warning area 36 based on a predetermined determination criterion. The boulder determination unit 337 determines whether or not the boulder 240 is in the second warning area 36 based on the image of the boulder 240 after the pixel number has been adjusted and a predetermined threshold Sh related to the pixel number (step S4).

[0148] The vehicle body data acquisition unit 332 acquires vehicle body data indicating the state of the work machine 1. In this embodiment, the vehicle body data acquisition unit 332 acquires, as the vehicle body data, detection data from the angle sensor 90, detection data from the load sensor 93, and operation data of the forward / reverse lever 20C (step S5).

[0149] The state determination section 333 determines whether the work mode of the work machine 1 is the excavation mode based on the vehicle body data acquired in step S5 (step S6).

[0150] If it is determined in step S4 that a boulder 240 is present (step S4: Yes) and if it is determined in step S6 that the excavation mode is active (step S6: Yes), the detection control unit 339 does not output a warning sound (step S7).

[0151] If it is determined in step S4 that a boulder 240 is present (step S4: Yes) and it is determined in step S6 that the excavation mode is not being used (step S6: No), the state determination unit 333 determines whether the work mode of the work machine 1 is a forward mode (step S8). The forward mode is at least one of an empty forward mode, a loaded forward mode, and an excavation mode.

[0152] If it is determined in step S8 that the work mode of the work machine 1 is the forward mode (step S8: Yes), the warning control section 338 causes the sound generating device 24B to output a warning sound (step S9).

[0153] If it is determined in step S8 that the vehicle is not in the forward travel mode (step S8: No), the warning control unit 338 does not output a warning sound (step S10).

[0154] After any one of the processes in step S7, step S9, and step S10 has been performed, the boulder detection unit 334 determines whether or not to end the monitoring process (step S11).

[0155] In step S11, if it is determined that the monitoring process is to be continued (step S11: No), the process returns to steps S1 and S5.

[0156] In step S11, if it is determined that the monitoring process is to be ended (step S11: Yes), the monitoring process ends.

[0157] [Computer System] FIG. 19 is a block diagram showing a computer system 1000 according to an embodiment. The monitoring controller 33 and the drive controller 48 each include a computer system 1000. The computer system 1000 includes a processor 1001 such as a central processing unit (CPU), a main memory 1002 including nonvolatile memory such as read-only memory (ROM) and volatile memory such as random access memory (RAM), a storage 1003, and an interface 1004 including an input / output circuit. The functions of the monitoring controller 33 and the drive controller 48 are stored in the storage 1003 as computer programs. The processor 1001 reads the computer programs from the storage 1003, loads them into the main memory 1002, and executes the above-described processing in accordance with the programs. The computer programs may be distributed to the computer system 1000 via a network.

[0158] In accordance with the above-described embodiment, the computer program or computer system 1000 can acquire measurement data of the ground surface 200 on which the work machine 1 is traveling, adjust at least a portion of the measurement data based on the distance from the work machine 1, and determine the presence or absence of boulders 240 on the ground surface 200 in the adjusted measurement data.

[0159] [Effects] As described above, the monitoring system 30 for the work machine 1 comprises a measurement data acquisition unit 331 that acquires measurement data of the ground surface 200 on which the work machine 1 is traveling, a measurement data adjustment unit 336 that adjusts at least a portion of the measurement data based on the distance from the work machine 1, and a boulder determination unit 337 that determines whether or not there are boulders 240 on the ground surface 200 in the adjusted measurement data.

[0160] According to the embodiment, at least a portion of the image data of the ground surface 200 is adjusted based on the distance from the work machine 1 to the boulder 240, and the presence or absence of the boulder 240 is determined based on the adjusted measurement data. By adjusting at least a portion of the image data based on the distance from the work machine 1 to the boulder 240, the determination conditions for determining the presence or absence of the boulder 240 are adjusted. In the embodiment, a coefficient set based on the distance from the work machine 1 to the boulder 240 is multiplied by the number of pixels in the partitioned area 50 in which the boulder 240 is detected. If the number of pixels of the boulder 240 after multiplication by the coefficient is equal to or less than a threshold value Sh, it is determined that the boulder 240 is not present, and a warning sound is not output. If the number of pixels of the boulder 240 after multiplication by the coefficient exceeds the threshold value Sh, it is determined that the boulder 240 is present, and a warning sound is output from the sound generation device 24B. In other words, even if the actual size of the boulder 240 is the same, a warning sound may or may not be output depending on the value of the coefficient.

[0161] In the image data captured by the imaging device 32, boulders 240 that are located far from the work machine 1 appear small. In the image data captured by the imaging device 32, boulders 240 that are located close to the work machine 1 appear large. In other words, in the image data, the number of pixels of boulders 240 that are located far from the work machine 1 is small. In the image data, the number of pixels of boulders 240 that are located close to the work machine 1 is large. If the number of pixels of the boulders 240 is not adjusted, a warning sound is output for boulders 240 that are located close to the work machine 1, but no warning sound is output for boulders 240 that are located far from the work machine 1.

[0162] In the embodiment, for example, if the driver requests that a warning sound be output even for boulders 240 that are located far from the work machine 1, the coefficient for the sectioned area 50 that is far from the work machine 1 is set to a large value. For example, if the driver does not request that a warning sound be output even for boulders 240 that are located close to the work machine 1, the coefficient for the sectioned area 50 that is close to the work machine 1 is set to a small value. By adjusting the number of pixels of at least a portion of the measurement data based on the distance from the work machine 1, the determination conditions for determining the presence or absence of boulders 240 on the ground 200 are adjusted to suit the driver's request. In the embodiment, by adjusting the coefficient based on the distance from the work machine 1, the determination conditions for determining the presence or absence of boulders 240 on the ground 200 are adjusted to suit the driver's request.

[0163] [Other Embodiments] In the above-described embodiment, the first process may be a process in which the rolling stone detection unit 334 detects the rolling stone 240, and the second process may be a process in which the rolling stone detection unit 334 does not detect the rolling stone 240. Furthermore, the third process may be a process in which the rolling stone detection unit 334 does not detect the rolling stone 240.

[0164] That is, while work is being performed in the excavation mode, the detection control unit 339 outputs a detection stop command to the boulder detection unit 334 so that detection of the boulder 240 is not performed. While work is being performed in the excavation mode, detection of the boulder 240 is not performed, and therefore no warning sound is output from the sounding device 24B regardless of whether a boulder 240 is present or not. Furthermore, since the detection process by the boulder detection unit 334 is stopped, the calculation load on the monitoring controller 33 is reduced. While work is being performed in at least one of the work modes of the empty forward mode, the loaded forward mode, and the loading mode, the detection control unit 339 outputs a detection execution command to the boulder detection unit 334 so that the boulder 240 is detected.

[0165] In the above-described embodiment, the first process may be a process in which the rolling stone determination unit 337 determines whether or not there is a rolling stone 240, and the second process may be a process in which the rolling stone determination unit 337 does not determine whether or not there is a rolling stone 240. Furthermore, the third process may be a process in which the rolling stone determination unit 337 does not determine whether or not there is a rolling stone 240.

[0166] That is, while work is being performed in the excavation mode, the detection control unit 339 outputs a determination stop command to the boulder determination unit 337 so that the presence or absence of a boulder 240 is not determined. While work is being performed in the excavation mode, the presence or absence of a boulder 240 is not determined, and therefore, a warning sound is not output from the sounding device 24B regardless of the presence or absence of a boulder 240. Furthermore, since the determination process by the boulder determination unit 337 is stopped, the calculation load on the monitoring controller 33 is reduced. While work is being performed in at least one of the work modes of the empty forward mode, the loaded forward mode, and the loading mode, the detection control unit 339 outputs a determination execution command to the boulder determination unit 337 so that the presence or absence of a boulder 240 is determined.

[0167] In the above-described embodiment, the state determination unit 333 determines the working state of the work machine 1 based on detection data from the load sensor 93. The state determination unit 333 may determine the working state of the work machine 1 based on measurement data acquired by the measurement data acquisition unit 331. The measurement data includes an image of the ground 200. The state determination unit 333 may determine the working state of the work machine 1 based on an image of boulders 240 included in the image of the ground 200. As described above, in the excavation mode, the bucket 12 penetrates into the natural ground 210, and a large number of boulders 240 are present around the front tires 6F. If the number of boulders 240 detected by the boulder detection unit 334 is equal to or greater than a predetermined first specified value, the state determination unit 333 can determine that the working state of the work machine 1 is in the excavation mode. Furthermore, when the bucket 12 is penetrated into the natural ground 210, the natural ground 210 prevents the work machine 1 from moving forward. In the excavation mode, the boulders 240 imaged by the imaging device 32 do not change. That is, in the excavation mode, the imaging device 32 continues to image the same boulders 240. In the excavation mode, the boulders detection unit 334 continues to detect the same boulders 240. If the number of boulders 240 that the boulders detection unit 334 continues to detect is equal to or greater than a predetermined second specified value, the state determination unit 333 can determine that the working state of the work machine 1 is in the excavation mode.

[0168] In the above-described embodiment, the driving operation device 20 and the output device 24 are arranged in the driver's cab of the work machine 1. The driving operation device 20 and the output device 24 may also be arranged outside the work machine 1. The work machine 1 may also be remotely operated.

[0169] In the above-described embodiment, the measuring device 32 is an imaging device. The measuring device 32 may be a laser device. The measuring device 32 may be a radar device. A laser device irradiates a measurement object with laser light to acquire measurement data of the measurement object. A radar device irradiates a measurement object with sound waves to acquire measurement data of the measurement object.

[0170] In the above-described embodiment, the work machine 1 is a wheel loader. However, the work machine 1 may be another work machine such as a bulldozer or a hydraulic excavator. The bulldozer and the hydraulic excavator each have a work implement and tracks that rotate while in contact with the ground 200. The tracks are rotating members that rotate while in contact with the ground 200. The work machine travels as the tracks rotate.

[0171] DESCRIPTION OF SYMBOLS 1...working machine, 2...body, 2A...joint mechanism, 2F...front body, 2R...rear body, 3...driver's cab, 4...traveling gear, 5...wheels, 5F...front wheels, 5R...rear wheels, 6...tires, 6F...front tires, 6R...rear tires, 7...front fender, 7L...front fender, 7R...front fender, 8...support member, 8L...support member, 8R...support member, 9...housing, 9L...housing, 9R...housing, 10...working machine, 11...boom, 12...bucket, 12E...end, 12M...opening, 13...boom cylinder, 14...bucket cylinder, 15...bell crank, 16...bucket Link, 17... bucket body, 17A... bottom plate portion, 17B... upper plate portion, 17C... left plate portion, 17D... right plate portion, 18... bucket teeth, 19... tooth inter-protector, 20... driving operation device, 20A... accelerator pedal, 20B... brake pedal, 20C... forward / reverse lever, 21... operation panel, 22... monitor device, 23... rear view monitor device, 24... output device, 24A... display device, 24B... sound device, 25... input device, 27... pillar, 28... headlight, 28L... headlight, 28R... headlight, 29... turn signal lamp, 29L... turn signal lamp, 29R... turn signal lamp, 3 0...Monitoring system, 32...Imaging device (measuring device), 32L...Imaging device, 32R...Imaging device, 33...Monitoring controller, 34...Detection area, 35...First warning area, 36...Second warning area, 40...Drive system, 41...Engine, 42...Fuel injection device, 43...Power take-off, 44...Transmission, 44F...Forward gear, 44R...Reverse gear, 45F...Front axle, 45R...Rear axle, 46...Hydraulic pump, 47...Control valve, 48...Drive controller, 50...Division area, 51...First division area, 52...Second division area, 53...Third division area, 5 4...fourth sectioned area, 55...fifth sectioned area, 56...sixth sectioned area, 57...seventh sectioned area, 58...eighth sectioned area, 59...ninth sectioned area, 60...pixels, 80...warning image, 81...first frame image, 82...second frame image, 83...third frame image, 90...angle sensor, 91...boom angle sensor, 92...bucket angle sensor, 93...load sensor, 200...ground, 210...natural ground, 220...dump truck, 230...dump body, 240...rolling stone, 331...measurement data acquisition unit, 332...vehicle body data acquisition unit, 333...state determination unit, 334...rolling stone detection unit,335... warning standard setting unit, 336... measurement data adjustment unit, 337... boulder determination unit, 338... warning control unit, 339... detection control unit, 340... coefficient storage unit, 1000... computer system, 1001... processor, 1002... main memory, 1003... storage, 1004... interface, CL... center, FX... rotation axis, Rb... measurement range, RX... rotation axis, β... angle of view.

Claims

1. A work machine monitoring system comprising: a measurement data acquisition unit that acquires measurement data of the ground surface on which a work machine travels; a measurement data adjustment unit that adjusts at least a portion of the measurement data based on the distance from the work machine; and a boulder determination unit that determines the presence or absence of boulders on the ground surface based on the adjusted measurement data.

2. A monitoring system for a work machine as described in claim 1, further comprising a boulder detection unit that detects a boulder from the measurement data, and wherein the measurement data adjustment unit makes a first adjustment to a portion of the measurement data including the boulder when the distance between the work machine and the boulder is a first distance, and makes a second adjustment to a portion of the measurement data including the boulder when the distance between the work machine and the boulder is a second distance.

3. A monitoring system for a work machine as described in claim 2, further comprising a coefficient memory unit which stores a coefficient set based on the distance between the work machine and the boulder, and the measurement data adjustment unit adjusts a portion of the measurement data based on the coefficient.

4. A monitoring system for a work machine according to claim 3, wherein the measurement data includes an image of the ground, and the measurement data adjustment unit adjusts the number of pixels of a portion of the measurement data based on the coefficient.

5. A monitoring system for a work machine as described in claim 4, wherein the boulder determination unit determines that a boulder is present when the number of pixels exceeds a predetermined threshold value, and determines that a boulder is not present when the number of pixels is equal to or less than the threshold value.

6. A monitoring system for a work machine as set forth in claim 1, further comprising a warning control unit which causes an output device to output a warning based on the presence or absence of said boulders.

7. A monitoring system for a work machine as described in claim 6, wherein the measurement data includes an image of the ground, the output device includes a display device that displays the image of the ground, and outputting the warning includes displaying a warning image that highlights boulders included in the image of the ground.

8. The monitoring system for a work machine according to claim 1, wherein the work machine has a rotating member, and the work machine travels on the ground by rotating the rotating member in contact with the ground.

9. A method for monitoring a work machine, comprising: acquiring measurement data of a ground surface on which a work machine is traveling; adjusting at least a portion of the measurement data based on a distance from the work machine; and determining the presence or absence of boulders on the ground surface in the adjusted measurement data.

Images