Monitor system of work machine, and monitor method of work machine

Abstract

To provide a work machine capable of detecting rolling stones on the ground while reducing unnecessary processing.SOLUTION: The monitor system of a work machine includes: a measurement data acquisition unit that acquires measurement data of the ground on which the work machine runs; a status determination unit that determines the working status of the work machine; and a detection control unit that is configured so as to, when the working status is determined as a first status, execute a series of first processing relevant to the rolling stones on the ground, and when the working status is determined as a second status, execute a series of second processing different from the first processing.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present disclosure relates to a work machine monitoring system and a work machine monitoring method. [Background technology]

[0002] A work machine operates at a work site. There may be boulders on the ground at the work site. If the work machine runs over a boulder, it may cause a hindrance to the running of the work machine. Patent Document 1 discloses a technology that outputs a warning to the driver's cab of a wheel loader when there is a rock that requires attention and may damage tires. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-080790 Summary of the Invention [Problem to be solved by the invention]

[0004] By outputting the warning, the operator of the work machine can recognize the presence of boulders. On the other hand, for example, if the operator is fully aware of the presence of boulders, the process of outputting the warning is likely to be unnecessary.

[0005] The present disclosure aims to reduce unnecessary processing in a work machine capable of detecting boulders on the ground. [Means for solving the problem]

[0006] According to the present disclosure, there is provided a monitoring system for a work machine, comprising a measurement data acquisition unit that acquires measurement data of the ground on which the work machine travels, a state determination unit that determines the working state of the work machine, and a detection control unit that, if the working state is determined to be a first state, executes a first process related to boulders on the ground, and, if the working state is determined to be a second state, executes a second process different from the first process. Effect of the Invention

[0007] According to the present disclosure, unnecessary processing can be reduced in a work machine capable of detecting boulders on the ground. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a side view showing a work machine according to an embodiment. [Diagram 2] FIG. 2 is a top view showing the work machine according to the embodiment. [Diagram 3] FIG. 3 is a front view showing the work machine according to the embodiment. [Figure 4] FIG. 4 is a diagram showing a driver's cab of a driver's cab according to the embodiment. [Diagram 5] FIG. 5 is an enlarged front view of a portion of the work machine according to the embodiment. [Figure 6] FIG. 6 is a diagram showing a drive system of a work machine according to an embodiment. [Figure 7] FIG. 7 is a diagram illustrating the operation of the work machine according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating the operation of the work machine according to the embodiment. [Figure 9] FIG. 9 is a side view showing a work machine for carrying out loading work according to this embodiment. [Figure 10] FIG. 10 is a top view showing a work machine performing loading work according to this embodiment. [Figure 11] FIG. 11 is a diagram showing an example of image data captured by the left imaging device according to the embodiment. [Figure 12]FIG. 12 is a functional block diagram showing a monitoring system according to an embodiment. [Figure 13] FIG. 13 is a diagram illustrating warning criteria according to the embodiment. [Figure 14] FIG. 14 is a diagram for explaining adjustment of measurement data according to the embodiment. [Figure 15] FIG. 15 is a diagram illustrating adjustment of the number of pixels according to the embodiment. [Figure 16] FIG. 16 is a diagram for explaining the determination of the presence or absence of a rolling stone according to the embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of an output device according to an embodiment. [Figure 18] FIG. 18 is a flowchart showing a ground monitoring method according to the embodiment. [Figure 19] FIG. 19 is a block diagram illustrating a computer system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment according to the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiment. The components of the embodiments described below can be appropriately combined. In addition, some components may not be used.

[0010] [Work machinery] Fig. 1 is a side view showing the work machine 1 according to the embodiment. Fig. 2 is a top view showing the work machine 1 according to the embodiment. Fig. 3 is a front view showing the work machine 1 according to the embodiment.

[0011] The work machine 1 operates at a work site. In the embodiment, the work machine 1 is a wheel loader, which is a type of articulated work machine. The work machine 1 performs work at the work site. The work performed by the work machine 1 includes an excavation work for excavating an excavation target, and a loading work for loading the material excavated by the excavation work onto a loading target.

[0012] As shown in Figures 1, 2, and 3, the work machine 1 includes a vehicle body 2, a cab 3, a traveling gear 4, a front fender 7, a support member 8, a housing 9, a work implement 10, an angle sensor 90, and a load sensor 93.

[0013] The vehicle body 2 includes a front vehicle body 2F and a rear vehicle body 2R. The front vehicle body 2F and the rear vehicle body 2R are connected via a joint mechanism 2A.

[0014] The cab 3 is supported by the vehicle body 2. A cab is provided in the cab 3. A driver's seat is provided in the cab. The work machine 1 is operated by a driver who sits in the cab 3.

[0015] The traveling device 4 supports the vehicle body 2 and travels on the ground 200 at the work site. The traveling device 4 has wheels 5 and tires 6. The tires 6 are attached to the wheels 5. The wheels 5 include front wheels 5F supported by the front vehicle body 2F and rear wheels 5R supported by the rear vehicle body 2R. The tires 6 include front tires 6F attached to the front wheels 5F and rear tires 6R attached to the rear wheels 5R.

[0016] The front wheel 5F and the front tire 6F are rotatable about a rotation axis FX, and the rear wheel 5R and the rear tire 6R are rotatable about a rotation axis RX.

[0017] In the embodiment, the direction parallel to the rotation axis FX of the front wheel 5F is appropriately referred to as the vehicle width direction. The direction perpendicular to the ground surface of the front tire 6F that contacts the ground 200 is appropriately referred to as the up-down direction. The direction perpendicular to both the vehicle width direction and the up-down direction is appropriately referred to as the fore-aft direction. When the body 2 of the work machine 1 travels in a straight line, the rotation axis FX and the rotation axis RX are parallel to each other.

[0018] In addition, in the embodiment, a position or direction closer to the center CL of the work machine 1 in the vehicle width direction of the work machine 1 is appropriately referred to as the inner side in the vehicle width direction, and a position or direction farther from the center CL is appropriately referred to as the outer side in the vehicle width direction.

[0019] In the vehicle width direction, one side of the center CL of the work machine 1 is the left, and the opposite side of the left is the right. In the front-to-rear direction, the position or direction closer to the work machine 10 using the driver's seat of the cab 3 as the reference is the front, and the opposite side of the front is the rear. In the up-down direction, the position or direction closer to the ground contact surface of the front tires 6F is the bottom, and the opposite side of the bottom is the top.

[0020] The rear body 2R is disposed rearward of the front body 2F. The front body 2F is bent to the left and right with respect to the rear body 2R. The rear wheels 5R are disposed rearward of the front wheels 5F. The rear tires 6R are disposed rearward of the front tires 6F. The front wheels 5F and the front tires 6F are disposed to the left and right of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The rear wheels 5R and the rear tires 6R are disposed to the left and right of the center CL of the work machine 1 in the vehicle width direction of the work machine 1.

[0021] The tires 6 are rotating members that rotate while in contact with the ground 200. The tires 6 rotate while in contact with the ground 200, causing the work machine 1 to travel on the ground 200. The front tires 6F are front rotating members that rotate while in contact with the ground 200. The rear tires 6R are disposed rearward of the front tires 6F, and are rear rotating members that rotate while in contact with the ground 200.

[0022] The front fender 7 prevents soil scattered from the ground 200 from hitting the vehicle body 2 and the cab 3 while the work machine 1 is traveling. A portion of the front fender 7 is disposed above the front tires 6F. A portion of the front fender 7 is disposed rearward of the front tires 6F. The front fenders 7 are disposed on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The front fenders 7 include a front fender 7L disposed left of the center CL and a front fender 7R disposed right of the center CL. The left front fender 7L is attached to the left part of the front vehicle body 2F. The right front fender 7R is attached to the right part of the front vehicle body 2F.

[0023] The support member 8 is a rod-shaped member. The lower end of the support member 8 is fixed to the front body 2F. The support member 8 is inclined upward, forward, and outward in the vehicle width direction. The support member 8 is disposed on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The support member 8 includes a support member 8L disposed to the left of the center CL and a support member 8R disposed to the right of the center CL. The lower end of the left support member 8L is fixed to the left part of the front body 2F. The left support member 8L is inclined upward, forward, and to the left. The lower end of the right support member 8R is fixed to the right part of the front body 2F. The right support member 8R is inclined upward, forward, and to the right.

[0024] The housing 9 is fixed to the upper end of the support member 8. The housing 9 is supported by the front vehicle body 2F via the support member 8. In the front-rear direction, the housing 9 is disposed between the joint mechanism 2A and the front end of the front vehicle body 2F. In the up-down direction, the housing 9 is disposed between the upper end of the cab 3 and the upper end of the front vehicle body 2F. The housing 9 is disposed on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The housing 9 includes a housing 9L disposed to the left of the center CL and a housing 9R disposed to the right of the center CL. The left housing 9L is fixed to the upper end of the left support member 8L. The right housing 9R is fixed to the upper end of the right support member 8R. The left housing 9L is disposed above the left front fender 7L. The right housing 9R is disposed above the right front fender 7R.

[0025] The work machine 10 is capable of excavating an excavation target. The work machine 10 operates in excavation work and loading work. The work machine 10 is coupled to a front vehicle body 2F. At least a portion of the work machine 10 is disposed forward of the front wheels 5F. The work machine 10 has a boom 11, a bucket 12, a bell crank 15, and a bucket link 16.

[0026] The boom 11 is rotatably connected to the front vehicle body 2F. The boom 11 connects the front vehicle body 2F and the bucket 12. The boom 11 operates by a driving force generated by a boom cylinder 13. The boom cylinder 13 operates the boom 11. The boom cylinder 13 is a hydraulic cylinder. One end of the boom cylinder 13 is connected to the front vehicle body 2F. The other end of the boom cylinder 13 is connected to the boom 11. The boom cylinder 13 is disposed on both the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1.

[0027] The bucket 12 excavates an excavation target. The bucket 12 is rotatably connected to the tip of the boom 11. The bucket 12 is disposed forward of the front wheels 5F. The bucket 12 is operated by a driving force generated by a bucket cylinder 14. The bucket cylinder 14 operates the bucket 12. The bucket cylinder 14 is a hydraulic cylinder. A central portion of a bell crank 15 is rotatably connected to the boom 11. One end of the bucket cylinder 14 is connected to the front vehicle body 2F. The other end of the bucket cylinder 14 is connected to one end of the bell crank 15. The other end of the bell crank 15 is connected to the bucket 12 via a bucket link 16. One bucket cylinder 14 is disposed in the central portion in the vehicle width direction.

[0028] The bucket 12 has a bucket body 17 , bucket teeth 18 , and an inter-tooth protector 19 .

[0029] The bucket body 17 holds the excavated material. The bucket body 17 includes a bottom plate portion 17A, an upper plate portion 17B, a left plate portion 17C, and a right plate portion 17D. The bottom plate portion 17A, the upper plate portion 17B, the left plate portion 17C, and the right plate portion 17D form a holding space in which the excavated material is accommodated. The tip portion of the bottom plate portion 17A and the tip portion of the upper plate portion 17B extend in the vehicle width direction. The tip portion of the left plate portion 17C and the tip portion of the right plate portion 17D extend in the up-down direction or the front-rear direction. The tip portion of the bottom plate portion 17A, the tip portion of the upper plate portion 17B, the tip portion of the left plate portion 17C, and the tip portion of the right plate portion 17D define an opening portion 12M of the bucket 12. The excavated material can enter the holding space of the bucket 12 through the opening portion 12M.

[0030] 2 and 3, both end portions 12E of the bucket body 17 in the vehicle width direction are disposed outboard in the vehicle width direction of the tires 6. The distance in the vehicle width direction between the left end portion 12E and the right end portion 12E is greater than the distance in the vehicle width direction between the left side surface of the left tire 6 and the right side surface of the right tire 6.

[0031] The bucket teeth 18 form the cutting edges of the bucket 12. The bucket teeth 18 are replaceably attached to the bucket body 17. The bucket teeth 18 are attached to the tip of the bottom plate portion 17A. A plurality of bucket teeth 18 are attached to the bucket body 17. The plurality of bucket teeth 18 are arranged at intervals in the vehicle width direction. In the embodiment, eight bucket teeth 18 are arranged at intervals in the vehicle width direction.

[0032] The inter-tooth protector 19 protects the tip of the bottom plate portion 17A. The inter-tooth protector 19 is replaceably attached to the bucket body 17. The inter-tooth protector 19 is disposed between a pair of adjacent bucket teeth 18. The inter-tooth protector 19 is attached to the tip of the bottom plate portion 17A. A plurality of inter-tooth protectors 19 are attached to the bucket body 17. The multiple inter-tooth protectors 19 are arranged at intervals in the vehicle width direction. In the embodiment, seven inter-tooth protectors 19 are arranged at intervals in the vehicle width direction.

[0033] The angle sensor 90 is a posture sensor that detects the posture of the work implement 10. The posture of the work implement 10 includes the angle of the work implement 10. The angle sensor 90 is provided on the work implement 10. The angle sensor 90 includes a boom angle sensor 91 that detects the angle of the boom 11 and a bucket angle sensor 92 that detects the angle of the bucket 12. The boom angle sensor 91 detects the angle of the boom 11 with respect to a reference axis of a vehicle body coordinate system defined in the front vehicle body 2F, for example. The bucket angle sensor 92 detects the angle of the bucket 12 with respect to the boom 11. An example of the angle sensor 90 is a potentiometer. The posture sensor that detects the posture of the work implement 10 may be a stroke sensor that detects the stroke of hydraulic cylinders (boom cylinder 13, bucket cylinder 14). The posture sensor that detects the posture of the work implement 10 may be an IMU (Inertial Measurement Unit) that detects acceleration and angular velocity.

[0034] The load sensor 93 detects the load applied to the work machine 10. The load sensor 93 can detect whether or not an excavated object is held in the bucket 12. The load sensor 93 can detect the weight of the excavated object held in the bucket 12. An example of the load sensor 93 is a pressure sensor that detects the pressure of hydraulic oil in the bottom chamber of the boom cylinder 13.

[0035] [Driver's cab] Fig. 4 is a diagram showing a driver's cab of the driver's seat 3 according to the embodiment. Fig. 4 is a diagram showing a schematic view seen by a driver seated in the driver's seat in the driver's cab.

[0036] In the cab, a driving operation device 20, an operation panel 21, a monitor device 22, a rear view monitor device 23, an output device 24, and an input device 25 are arranged. The driving operation device 20 is operated by the driver. The driving operation device 20 includes an accelerator pedal 20A and a pair of brake pedals 20B. Although not shown, the driving operation device 20 also includes a steering lever, a shift lever, a forward / reverse lever, and an operation lever. Operation of the accelerator pedal 20A increases the traveling speed of the work machine 1. Operation of the brake pedal 20B decreases the traveling speed of the work machine 1 or stops the traveling of the work machine 1. Operation of the steering lever causes the work machine 1 to turn. Operation of the shift lever changes the speed stage of the work machine 1. Operation of the forward / reverse lever switches the traveling direction of the work machine 1 between the forward direction and the reverse direction. Operation of the operation lever causes the work equipment 10 to operate. The monitor device 22 displays data related to a plurality of functions of the work machine 1. The rearview monitor device 23 displays an image of the rear of the body 2 of the work machine 1 captured by a camera provided at the rear of the work machine 1 (not shown).

[0037] The output device 24 provides output data to the driver. The output data includes a warning. The output device 24 includes a display device 24A and a sound device 24B.

[0038] The display device 24A provides display data to the driver as output data. Examples of the display device 24A include a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OELD).

[0039] The sound output device 24B provides the driver with voice data as output data. Examples of the sound output device 24B include a buzzer and a speaker.

[0040] The input device 25 generates input data by being operated by the driver. Examples of the input device 25 include a button or a keyboard. The input device 25 may include, for example, a touch panel.

[0041] A windshield is provided in front of the cab. The windshield is attached to a pillar 27 of the cab. In the embodiment, the output device 24 is supported by the pillar 27. The output device 24 may be a tablet terminal or the like. The monitor device 22, the rearview monitor device 23, and the output device 24 may be integrated in part or in whole.

[0042] [Measuring equipment] 5 is an enlarged front view of a portion of the work machine 1 according to the embodiment. As shown in FIGS.

[0043] The headlights 28 emit illumination light forward to illuminate an illumination range in front of the front vehicle body 2F. The headlights 28 are held in the housing 9. The headlights 28 are arranged on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The headlights 28 include a headlight 28L arranged on the left side of the center CL, and a headlight 28R arranged on the right side of the center CL. The left headlight 28L is held in the left housing 9L. The right headlight 28R is held in the right housing 9R.

[0044] The turn signal lamps 29 indicate the turning direction of the work machine 1 by lighting up or flashing. The turn signal lamps 29 are held in the housing 9. The turn signal lamps 29 are arranged on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The turn signal lamps 29 include a turn signal lamp 29L arranged on the left side of the center CL, and a turn signal lamp 29R arranged on the right side of the center CL. The left turn signal lamp 29L is held in the left housing 9L. The right turn signal lamp 29R is held in the right housing 9R. The turn signal lamps 29 are arranged outboard of the headlights 28 in the vehicle width direction of the work machine 1.

[0045] The measuring device 32 measures a specific target. The target measured by the measuring device 32 includes the ground surface 200 on which the traveling device 4 travels. The measuring device 32 measures the ground surface 200 on which the work machine 1 travels. The measuring device 32 acquires measurement data of the ground surface 200. In the embodiment, the measuring device 32 is an imaging device that images the ground surface 200. The measurement data of the ground surface 200 includes image data of the ground surface 200. In the following description, the measuring device 32 will be referred to as the imaging device 32 as appropriate.

[0046] The imaging device 32 includes an optical system and an image sensor into which light that has passed through the optical system is incident. Examples of the image sensor include a CCD (Couple Charged Device) image sensor and a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0047] The imaging device 32 is disposed in the housing 9. The imaging device 32 is disposed outside the headlights 28 in the vehicle width direction of the work machine 1.

[0048] The imaging devices 32 are arranged on the left and right sides of the center CL of the work machine 1 in the vehicle width direction of the work machine 1. The imaging devices 32 are arranged in each of the left and right housings 9. The imaging devices 32 include imaging device 32L arranged on the left side of the center CL and imaging device 32R arranged on the right side of the center CL.

[0049] [Drive system] 6 is a diagram showing a drive system 40 of the work machine 1 according to the embodiment. The drive system 40 has an engine 41, a fuel injector 42, a power take off (PTO) 43, a transmission 44, a front axle 45F, a rear axle 45R, a hydraulic pump 46, a control valve 47, and a drive controller 48.

[0050] The engine 41 is, for example, a diesel engine. The fuel injector 42 injects fuel into cylinders of the engine 41. The driving force of the engine 41 is adjusted by adjusting the amount of fuel injected from the fuel injector 42 to the engine 41.

[0051] The power take-off 43 distributes the driving force of the engine 41 to the transmission 44 and the hydraulic pump 46. The driving force of the engine 41 is transmitted to both the transmission 44 and the hydraulic pump 46 via the power take-off 43.

[0052] The transmission 44 has an input shaft to which the driving force of the engine 41 is input, and an output shaft that changes the speed of the driving force input to the input shaft and outputs it. The input shaft of the transmission 44 is connected to the power take-off 43. The output shafts of the transmission 44 are connected to each of the front axle 45F and the rear axle 45R. The driving force of the engine 41 is transmitted to each of the front axle 45F and the rear axle 45R via the power take-off 43 and the transmission 44.

[0053] The transmission 44 switches the traveling direction of the work machine 1 between a forward direction and a reverse direction. The transmission 44 has a forward gear 44F and a reverse gear 44R. When the forward gear 44F is engaged, the traveling direction of the work machine 1 is determined to be the forward direction. When the reverse gear 44R is engaged, the traveling direction of the work machine 1 is determined to be the reverse direction.

[0054] The front axle 45F transmits the driving force transmitted from the transmission 44 to the front wheels 5F. The front wheels 5F rotate based on the driving force transmitted from the front axle 45F.

[0055] The rear axle 45R transmits the driving force transmitted from the transmission 44 to the rear wheels 5R. The rear wheels 5R rotate based on the driving force transmitted from the rear axle 45R.

[0056] By engaging the forward gear 44F, the work machine 1 is placed in a state in which it can move forward. With the forward gear 44F engaged, the accelerator pedal 20A is operated and the driving force of the engine 41 is transmitted to each of the front axle 45F and the rear axle 45R via the transmission 44, causing the work machine 1 to move forward. Note that even with the forward gear 44F engaged, the work machine 1 will not move forward if, for example, the brake pedal 20B is operated.

[0057] Engaging the reverse gear 44R puts the work machine 1 in a state in which it can move in reverse. With the reverse gear 44R engaged, the accelerator pedal 20A is operated and the driving force of the engine 41 is transmitted to each of the front axle 45F and the rear axle 45R via the transmission 44, causing the work machine 1 to move in reverse. Note that even with the reverse gear 44R engaged, the work machine 1 will not move in reverse if, for example, the brake pedal 20B is operated.

[0058] The hydraulic pump 46 discharges hydraulic oil. The hydraulic pump 46 is a variable displacement hydraulic pump. The hydraulic pump 46 is driven based on the driving force of the engine 41. The hydraulic oil discharged from the hydraulic pump 46 is supplied to at least one of the boom cylinder 13 and the bucket cylinder 14 via a control valve 47.

[0059] The control valve 47 controls the flow rate and direction of the hydraulic oil supplied to each of the boom cylinder 13 and the bucket cylinder 14. The work machine 10 is operated by the hydraulic oil supplied from the hydraulic pump 46 via the control valve 47.

[0060] The drive controller 48 controls the work machine 1 based on operation signals output from the driving operation device 20. The drive controller 48 includes a computer system.

[0061] [Working machine operation] 7 is a diagram illustrating the operation of the work machine 10 according to the embodiment. In the embodiment, the work machine 10 is a front-loading type work machine in which the opening 12M of the bucket 12 faces forward during excavation work.

[0062] The boom cylinder 13 extends and retracts, thereby causing the boom 11 to move up and down.

[0063] The raising operation of the boom 11 refers to the operation of the boom 11 rotating so that the tip of the boom 11 moves away from the ground 200. The boom cylinder 13 extends, thereby causing the boom 11 to perform the raising operation.

[0064] The lowering operation of the boom 11 refers to the operation of the boom 11 rotating so that the tip of the boom 11 approaches the ground 200. The boom cylinder 13 contracts, thereby causing the boom 11 to perform the lowering operation.

[0065] The bucket cylinder 14 extends and contracts, causing the bucket 12 to perform a tilt motion or a dump motion.

[0066] The tilting motion of bucket 12 refers to the motion of bucket 12 rotating so that the tip of bucket 12 moves away from ground surface 200 with opening 12M of bucket 12 facing upward. As bucket cylinder 14 extends, bell crank 15 rotates so that the upper end of bell crank 15 moves forward and the lower end of bell crank 15 moves rearward. When the lower end of bell crank 15 moves rearward, bucket 12 is pulled rearward by bucket link 16, performing a tilting motion. As bucket 12 tilts, the excavation target is scooped up by bucket 12 and the excavated material is held in bucket 12.

[0067] The dumping operation of the bucket 12 refers to an operation in which the bucket 12 rotates so that the tip of the bucket 12 approaches the ground 200 with the opening 12M of the bucket 12 facing downward. When the bucket cylinder 14 contracts, the bell crank 15 rotates so that the upper end of the bell crank 15 moves rearward and the lower end of the bell crank 15 moves forward. When the lower end of the bell crank 15 moves forward, the bucket 12 is pushed forward by the bucket link 16, performing a dumping operation. When the bucket 12 performs a dumping operation, the excavated material held in the bucket 12 is discharged from the bucket 12.

[0068] [Work machine operation] 8 is a diagram illustrating the operation of the work machine 1 according to the embodiment. The work machine 1 carries out excavation work and loading work at a work site.

[0069] The excavation work is a work of excavating an excavation target. Examples of the excavation target include natural ground or a rocky mountain. The natural ground is a mountain composed of earth and sand placed on the ground 200. The rocky mountain is a mountain composed of rocks or stones placed on the ground 200. In the embodiment, the excavation target is the natural ground 210. The excavated material is a part of the natural ground 210 excavated and held by the bucket 12.

[0070] The loading operation is an operation of loading the excavated material excavated by the excavation operation onto a loading target. The loading target is, for example, a transport vehicle, a hopper, or a belt conveyor. In the embodiment, the loading target is a dump body 230 of a dump truck 220, which is a type of transport vehicle.

[0071] The work machine 1 is operated by a driver to perform a cyclical work in which a series of work modes are repeated. The cyclical work is made up of a plurality of interrelated work modes. The cyclical work includes travel of the traveling gear 4 and operation of the work implement 10.

[0072] In the embodiment, the cycle work is configured by six work modes. In the embodiment, the series of work modes of the work machine 1 includes an empty forward mode, an excavation mode, a loaded reverse mode, a loaded forward mode, a loading mode, and an empty reverse mode.

[0073] The sequence of the series of operation modes is fixed. After the empty forward mode is performed, the excavation mode is performed. After the excavation mode is performed, the loaded reverse mode is performed. After the loaded reverse mode is performed, the loaded forward mode is performed. After the loaded forward mode is performed, the loading mode is performed. After the loading mode is performed, the empty reverse mode is performed.

[0074] The empty forward movement mode is a work mode in which the work machine 1 moves forward to approach the excavation target with no excavated material held in the bucket 12. In the empty forward movement mode, the work machine 1 moves forward to approach the natural ground 210 as shown by the arrow M1 in FIG. 8 with no excavated material held in the bucket 12.

[0075] The excavation mode is a work mode in which an excavation target is excavated with the bucket 12 of the work machine 10. In the excavation mode, excavation work is performed. With at least a portion of the bucket 12 inserted into the natural ground 210, the bucket 12 performs a tilting operation, whereby the natural ground 210 is excavated by the bucket 12 and the excavated material is held by the bucket 12.

[0076] The loaded reverse mode is a work mode in which the work machine 1 moves backward so as to move away from the excavation target with an excavated work held in the bucket 12. In the loaded reverse mode, the work machine 1 moves backward so as to move away from the natural ground 210 with an excavated work held in the bucket 12, as shown by the arrow M2 in FIG.

[0077] The load forward mode is a work mode in which the work machine 1 moves forward to approach the loading target with an excavated work held in the bucket 12. In the load forward mode, the work machine 1 moves forward while swinging toward the dump truck 220, as shown by arrow M3 in Fig. 8, with an excavated work held in the bucket 12. With the work machine 1 moving forward toward the dump truck 220, the boom 11 performs a raising operation so that the bucket 12 is positioned above the dump body 230.

[0078] The loading mode is a work mode in which the excavated material held in the bucket 12 of the work machine 10 is loaded onto a loading target. In the loading mode, a loading operation is performed. After the bucket 12 is positioned above the dump body 230, the bucket 12 performs a dumping operation, whereby the excavated material held in the bucket 12 is discharged from the bucket 12 and loaded into the dump body 230.

[0079] The empty reverse mode is a work mode in which the work machine 1 moves backward so as to move away from the loading target with no excavated material held in the bucket 12. With no excavated material held in the bucket 12, the work machine 1 moves backward while turning so as to move away from the dump truck 220, as shown by arrow M4 in Figure 8.

[0080] The work machine 1 repeats a cycle operation including an empty forward mode, an excavation mode, a loaded reverse mode, a loaded forward mode, a loading mode, and an empty reverse mode until the excavated material is loaded into the dump body 230 at the target load amount.

[0081] [Measurement range] Fig. 9 is a side view showing the work machine 1 performing loading work according to the embodiment. Fig. 10 is a top view showing the work machine 1 performing loading work according to the embodiment.

[0082] 9 and 10 , when loading excavated material into the dump body 230 of the dump truck 220, the work machine 1 performs a raising operation of the boom 11 so that the bucket 12 is disposed above the dump body 230. After the boom 11 performs the raising operation and the bucket 12 is disposed above the dump body 230, the work machine 1 performs a dumping operation of the bucket 12. With the boom 11 performing the raising operation, the bucket 12 performs a dumping operation so that the excavated material held in the bucket 12 is discharged into the dump body 230 of the dump truck 220. Due to the dumping operation of the bucket 12, the excavated material held in the bucket 12 is discharged from the bucket 12 and loaded into the dump body 230.

[0083] The measurement range Rb of the imaging device 32 includes the imaging range of the imaging device 32. The measurement range Rb is determined based on the angle of view β indicating the angle of view of the optical system of the imaging device 32. The imaging device 32 images at least the ground 200 in the traveling direction of the tire 6. In the embodiment, the imaging device 32 images the ground 200 in front of the front tire 6F when the work machine 1 moves forward. The imaging device 32 is fixed to the housing 9 so that at least the ground 200 in front of the front tire 6F is located in the measurement range Rb when the work machine 1 moves forward. In the embodiment, the imaging device 32 is fixed to the housing 9 so that the front end of the front tire 6F, the ground 200 in front of the front tire 6F, and the ground 200 on the outer side of the front tire 6F in the vehicle width direction are each located in the measurement range Rb. The ground 200 in front of the front tire 6F includes the ground 200 between the front end of the front tire 6F and the rear end of the bucket 12 in the front-rear direction.

[0084] 10, the imaging device 32L on the left side of the center CL captures an image of the ground 200 in front of the front tire 6F on the left side of the center CL. The imaging device 32R on the right side of the center CL captures an image of the ground 200 in front of the front tire 6F on the right side of the center CL.

[0085] 10, the imaging device 32L on the left side of the center CL is fixed to the left housing 9 so that the front end of the left front tire 6F, the ground surface 200 in front of the left front tire 6F, and the ground surface 200 to the left of the left front tire 6F are located in the measurement range Rb. The imaging device 32R on the right side of the center CL is fixed to the right housing 9 so that the front end of the right front tire 6F, the ground surface 200 in front of the right front tire 6F, and the ground surface 200 to the right of the right front tire 6F are located in the measurement range Rb.

[0086] Fig. 11 is a diagram showing an example of image data captured by the left imaging device 32L according to the embodiment. As shown in Fig. 11, the imaging device 32L captures images of the ground 200 in front of the front tire 6F and the ground 200 to the left of the front tire 6F from above. This allows the imaging device 32L to capture an image of the ground 200 around the front tire 6F. For example, when a boulder 240 is present on the ground 200 around the front tire 6F, the imaging device 32L can capture an image of the boulder 240 on the ground 200.

[0087] [Surveillance system] 12 is a functional block diagram showing a monitoring system 30 according to an embodiment. The work machine 1 has the monitoring system 30. The monitoring system 30 monitors the ground surface 200 on which the work machine 1 travels.

[0088] The monitoring system 30 has an imaging device 32, a driving operation device 20, an angle sensor 90, a load sensor 93, an output device 24, an input device 25, and a monitoring controller 33. The imaging device 32 includes an imaging device 32L arranged on the left side of a center CL, and an imaging device 32R arranged on the right side of the center CL.

[0089] The imaging device 32, the driving operation device 20, the angle sensor 90, the load sensor 93, the output device 24, and the input device 25 are each connected to the monitoring controller 33. Image data captured by the imaging device 32 is input to the monitoring controller 33. Operation data of the driving operation device 20 is input to the monitoring controller 33. Detection data of the angle sensor 90 is input to the monitoring controller 33. Detection data of the load sensor 93 is input to the monitoring controller 33. Input data generated by the input device 25 is input to the monitoring controller 33.

[0090] In the embodiment, the imaging device 32 acquires measurement data at a predetermined sampling rate (SPS: samples per second). The measurement data includes image data. The sampling rate includes a frame rate (FPS: frames per second). The imaging device 32 captures the image data at the predetermined frame rate. A moving image is shot by the imaging device 32 capturing the image data at the predetermined frame rate.

[0091] The monitoring controller 33 recognizes the presence or absence of boulders 240 on the ground 200 on which the work machine 1 travels, based on image data captured by the imaging device 32. When the monitoring controller 33 recognizes the boulders 240 on the ground 200, it causes the output device 24 to output a warning indicating the presence of the boulders 240 on the ground 200. The boulders 240 on the ground 200 include the boulders 240 on the ground 200 in the traveling direction of the front tires 6F. The boulders 240 on the ground 200 in the traveling direction of the front tires 6F include the boulders 240 on the ground 200 in front of the front tires 6F when the work machine 1 moves forward.

[0092] The monitoring controller 33 recognizes the presence or absence of the rolling stone 240 in the cycle work described with reference to Fig. 8. Note that the monitoring controller 33 may recognize the presence or absence of the rolling stone 240 in a work other than the cycle work.

[0093] The monitoring controller 33 has a measurement data acquisition unit 331, a vehicle body data acquisition unit 332, a condition discrimination unit 333, a boulder detection unit 334, a warning standard setting unit 335, a measurement data adjustment unit 336, a boulder determination unit 337, a warning control unit 338, a detection control unit 339, and a coefficient memory unit 340.

[0094] The measurement data acquisition unit 331 acquires measurement data of the imaging device 32. The measurement data acquisition unit 331 acquires measurement data of the ground 200 measured by the imaging device 32 from the imaging device 32 at a predetermined sampling rate. The measurement data of the ground 200 includes image data of the ground 200 imaged by the imaging device 32. The sampling rate includes a frame rate. The measurement data acquisition unit 331 acquires image data of the ground 200 from the imaging device 32 at a predetermined frame rate. The measurement data acquisition unit 331 acquires image data of the ground 200 in front of the front tire 6F.

[0095] The vehicle body data acquisition unit 332 acquires vehicle body data indicating the working state of the work machine 1. The working state of the work machine 1 includes the state of the work implement 10 and the state of the traveling gear 4. The state of the work implement 10 includes the posture of the work implement 10 and the load applied to the work implement 10. The state of the traveling gear 4 includes forward and reverse movement of the traveling gear 4. The vehicle body data includes posture data indicating the posture of the work implement 10, load data indicating the load applied to the work implement 10, and forward / reverse data indicating the forward or reverse movement of the traveling gear 4.

[0096] The attitude data of the work machine 10 includes the angle of the work machine 10. The angle of the work machine 10 is detected by an angle sensor 90. The attitude data of the work machine 10 includes detection data of the angle sensor 90. The vehicle body data acquisition unit 332 acquires the detection data of the angle sensor 90 as the attitude data of the work machine 10.

[0097] The load data of the work machine 10 includes the pressure of the hydraulic oil in the bottom chamber of the boom cylinder 13. The pressure of the hydraulic oil in the bottom chamber of the boom cylinder 13 is detected by a load sensor 93. The load data of the work machine 10 includes the detection data of the load sensor 93. The vehicle body data acquisition unit 332 acquires the detection data of the load sensor 93 as the load data of the work machine 10.

[0098] The travel device 4 is switched between forward and reverse movement by a forward / reverse lever 20C of the driving operation device 20. Forward / reverse movement data of the travel device 4 includes operation data of the forward / reverse lever 20C. The vehicle body data acquisition unit 332 acquires the operation data of the forward / reverse lever 20C as the forward / reverse movement data of the travel device 4.

[0099] For example, in the case where a rotation sensor that detects the rotation of the wheels 5 is provided, the rotation sensor can detect the forward or backward movement of the traveling device 4 by detecting the rotation direction of the wheels 5. The vehicle body data acquisition unit 332 may acquire detection data of the rotation sensor as forward / reverse movement data of the traveling device 4.

[0100] The vehicle body data acquisition unit 332 may acquire status data indicating the status of the drive system 40 as forward / reverse data of the traveling device 4. The drive controller 48 inputs status data indicating the status of the drive system 40 to the monitoring controller 33. The vehicle body data acquisition unit 332 determines the status of the drive system 40 based on the status data from the drive controller 48. The status of the drive system 40 includes the operating state of the transmission 44. The transmission 44 switches the traveling direction of the work machine 1 between a forward direction and a reverse direction. The transmission 44 includes a forward gear 44F that operates to move the work machine 1 forward, and a reverse gear 44R that operates to move the work machine 1 backward. The vehicle body data acquisition unit 332 may acquire status data indicating the status of the transmission 44 as forward / reverse data of the traveling device 4.

[0101] The state discrimination unit 333 discriminates the working state of the work machine 1 based on the vehicle body data acquired by the vehicle body data acquisition unit 332. The state discrimination unit 333 can discriminate whether the boom 11 is performing a raising operation or a lowering operation based on the attitude data of the work machine 10 acquired by the vehicle body data acquisition unit 332. The state discrimination unit 333 can discriminate whether the bucket 12 is performing a tilt operation or a dump operation based on the attitude data of the work machine 10 acquired by the vehicle body data acquisition unit 332. The state discrimination unit 333 can discriminate whether the work machine 10 is excavating an excavation target based on the load data of the work machine 10 acquired by the vehicle body data acquisition unit 332. The state discrimination unit 333 can discriminate whether the work machine 10 is moving forward or backward based on the forward / reverse data of the traveling device 4 acquired by the vehicle body data acquisition unit 332.

[0102] The working state of the work machine 1 includes the working mode of the work machine 1. The working mode of the work machine 1 indicates the working state of the work machine 1. The state determination unit 333 can determine whether the working mode of the work machine 1 is any of the empty forward mode, excavation mode, loaded reverse mode, loaded forward mode, loading mode, and empty reverse mode, based on the posture data of the work machine 10, the load data of the work machine 10, and the forward and reverse data of the traveling gear 4.

[0103] The unladen forward mode indicates a working state in which the work machine 1 moves forward and the work machine 10 does not excavate the ground 210. The excavation mode indicates a working state in which the work machine 1 moves forward and the work machine 10 excavates the ground 210. The loaded reverse mode indicates a working state in which the work machine 1 moves backward. The loaded forward mode indicates a working state in which the work machine 1 moves forward and the work machine 10 does not excavate the ground 210. The loading mode indicates a working state in which the work machine 1 moves forward and the work machine 10 does not excavate the ground 210. The unladen reverse mode indicates a working state in which the work machine 1 moves backward.

[0104] Based on the posture data of the work machine 10, the load data of the work machine 10, and the forward and reverse data of the traveling gear 4, the state determination unit 333 can determine whether the working state of the work machine 1 is in a working state where the work machine 1 moves forward and the work machine 10 is not digging (first state), a working state where the work machine 1 moves forward and the work machine 10 is digging (second state), or a working state where the work machine 1 moves backward (third state).

[0105] The boulder detection unit 334 detects the boulder 240 from the image data of the ground 200 captured by the imaging device 32. The boulder detection unit 334 detects the boulder 240 from the image data of the ground 200 acquired by the measurement data acquisition unit 331 based on a predetermined algorithm. The algorithm for detecting the boulder 240 from the image data of the ground 200 includes an artificial intelligence (AI) algorithm. The AI ​​algorithm detects the boulder 240 from the image data of the ground 200 captured by the imaging device 32 based on a learning model generated by learning the feature amount of the boulder 240. The learning model is a learning model that uses the feature amount of an object as input and outputs the boulder 240.

[0106] The warning criterion setting unit 335 sets the warning criterion when a warning is output from the output device 24 .

[0107] 13 is a diagram illustrating warning criteria according to an embodiment. In the embodiment, the warning criteria include a detection area 34 set in at least a part of image data of the ground 200, a first warning area 35 set inside the detection area 34, and a second warning area 36 set inside the first warning area 35.

[0108] The detection area 34 is rectangular. A boulder 240 is detected inside the detection area 34. The boulder detector 334 detects the boulder 240 in the detection area 34.

[0109] The first warning area 35 is smaller than the detection area 34. The first warning area 35 is rectangular. The first warning area 35 is set inside the detection area 34. At least a portion of the first warning area 35 is set on the ground surface 200 in front of the front tires 6F. The front end of the first warning area 35 is defined rearward (toward the work machine 1) of the front end of the detection area 34.

[0110] The second warning area 36 is smaller than the first warning area 35. The second warning area 36 is rectangular. The second warning area 36 is set inside the first warning area 35. At least a portion of the second warning area 36 is set on the ground surface 200 in front of the front tires 6F. The front end of the second warning area 36 is defined rearward (toward the work machine 1) than the front end of the first warning area 35.

[0111] The measurement data adjustment unit 336 adjusts at least a portion of the measurement data of the ground surface 200 based on the distance from the work machine 1. When the boulder 240 is detected by the boulder detection unit 334, the measurement data adjustment unit 336 can calculate the distance from the work machine 1 to the boulder 240. The measurement data adjustment unit 336 adjusts at least a portion of the measurement data of the ground surface 200 based on the distance from the work machine 1 to the boulder 240.

[0112] Fig. 14 is a diagram for explaining adjustment of measurement data according to the embodiment. As shown in Fig. 14, the measurement data adjustment unit 336 divides the detection area 34 into a plurality of divided areas 50. In the example shown in Fig. 14, the detection area 34 is divided into nine divided areas 50 in a matrix shape. The divided areas 50 include a first divided area 51, a second divided area 52, a third divided area 53, a fourth divided area 54, a fifth divided area 55, a sixth divided area 56, a seventh divided area 57, an eighth divided area 58, and a ninth divided area 59.

[0113] The distance between the work machine 1 and each of the multiple designated areas 50 is different. The closer to the upper part of the detection area 34 the designated area 50 (first designated area 51, second designated area 52, third designated area 53) is, the longer the distance from the work machine 1, and the closer to the lower part of the detection area 34 the designated area 50 (seventh designated area 57, eighth designated area 58, ninth designated area 59) is, the shorter the distance from the work machine 1. In the example shown in Fig. 14, the distance between the work machine 1 and the first designated area 51 is the longest. The distance between the work machine 1 and the ninth designated area 59 is the shortest.

[0114] When a boulder 240 is detected in the detection area 34, the measurement data adjustment unit 336 adjusts a portion of the detection area 34 including the detected boulder 240 based on the distance between the work machine 1 and the boulder 240. When the distance between the work machine 1 and the boulder 240 is a first distance, the measurement data adjustment unit 336 makes a first adjustment to the portion of the measurement data including the boulder 240, and when the distance between the work machine 1 and the boulder 240 is a second distance, the measurement data adjustment unit 336 makes a second adjustment to the portion of the measurement data including the boulder 240.

[0115] In this embodiment, a coefficient related to the distance between the work machine 1 and the boulder 240 is set in advance. The coefficient storage unit 340 stores the coefficient that is set in advance based on the distance between the work machine 1 and the boulder 240. The measurement data adjustment unit 336 adjusts a part of the detection area 34 based on the coefficient stored in the coefficient storage unit 340.

[0116] The coefficients may be input to the coefficient storage unit 340 via the input device 25. An operator or a maintenance person may input the coefficients by operating the input device 25. The coefficients input by the input device 25 may be stored in the coefficient storage unit 340.

[0117] As shown in FIG. 14, in the embodiment, a coefficient is set for each of the multiple divided areas 50. The coefficient may be a real number greater than 1, may be 1, or may be a positive real number less than 1. In the example shown in FIG. 14, the coefficient value is greater for divided areas 50 that are farther away from the work machine 1, and the coefficient value is smaller for divided areas 50 that are closer to the work machine 1. In the example shown in FIG. 14, the coefficient set for the first divided area 51 is 1.4. The coefficient set for the second divided area 52 is 1.3. The coefficient set for the third divided area 53 is 1.3. The coefficient set for the fourth divided area 54 is 1.2. The coefficient set for the fifth divided area 55 is 1.0. The coefficient set for the sixth divided area 56 is 1.0. The coefficient set for the seventh divided area 57 is 1.0. The coefficient set for the eighth divided area 58 is 1.0. The coefficient set for the ninth divided area 59 is 1.0.

[0118] 14 is an example. The coefficient value may be smaller for a divided area 50 that is farther away from the work machine 1, and the coefficient value may be larger for a divided area 50 that is nearer to the work machine 1.

[0119] In an embodiment, the measurement data adjustment unit 336 adjusts the number of pixels of a portion of the detection area 34 defined in the image data of the ground surface 200 based on the coefficient.

[0120] FIG. 15 is a diagram for explaining the adjustment of the number of pixels according to the embodiment. As shown in FIG. 15, the image data is composed of a plurality of pixels 60. In the embodiment, the number of pixels 60 in the image data is appropriately referred to as the number of pixels. The number of pixels may be the number of pixels in the horizontal direction (10 in the case shown in FIG. 15) of the divided area 50 in which the boulder 240 is detected, or the number of pixels in the vertical direction (10 in the case shown in FIG. 15), or the larger number of pixels in the horizontal direction and the vertical direction. In the following examples, the number of pixels 60 in the image data will be explained as the number of pixels. When the boulder 240 is detected by the boulder detection unit 334 in a certain divided area 50, the measurement data adjustment unit 336 multiplies the number of pixels in the divided area 50 in which the boulder 240 is detected by a coefficient. The number of pixels in the divided area 50 is adjusted by multiplying the number of pixels in the divided area 50 by the coefficient.

[0121] When the coefficient is a real number greater than 1, the number of pixels in the partitioned area 50 after the adjustment is increased compared to the number of pixels in the partitioned area 50 before the adjustment. When the coefficient is 1, the number of pixels in the partitioned area 50 after the adjustment is the same as the number of pixels in the partitioned area 50 before the adjustment. When the coefficient is a positive real number less than 1, the number of pixels in the partitioned area 50 after the adjustment is decreased compared to the number of pixels in the partitioned area 50 before the adjustment.

[0122] 15, if one divided area 50 has 100 pixels and the coefficient set for that divided area 50 is 1.4, the measurement data adjustment unit 336 multiplies the number of pixels, 100, by the coefficient 1.4. As a result, the number of pixels in the divided area 50 increases to 140. If one divided area 50 has 100 pixels and the coefficient set for that divided area 50 is 0.9, the number of pixels in the divided area 50 decreases to 90.

[0123] An increase in the number of pixels in the partitioned area 50 includes an increase in the number of pixels in the image of the rolling stone 240. An increase in the number of pixels in the image of the rolling stone 240 includes an increase in the amount of image data of the rolling stone 240. A decrease in the number of pixels in the partitioned area 50 includes a decrease in the number of pixels in the image of the rolling stone 240. A decrease in the number of pixels in the image of the rolling stone 240 includes a decrease in the amount of image data of the rolling stone 240.

[0124] The boulder determining unit 337 determines the presence or absence of a boulder 240 on the ground 200 in the image data after it has been adjusted by the measurement data adjusting unit 336. The boulder determining unit 337 determines the presence or absence of a boulder 240 on the ground 200 based on a predetermined determination criterion. If the boulder 240 detected by the boulder detecting unit 334 satisfies the determination criterion, the boulder determining unit 337 determines that the boulder 240 is present. If the boulder 240 is detected by the boulder detecting unit 334 but does not satisfy the determination criterion, the boulder determining unit 337 determines that the boulder 240 is not present.

[0125] FIG. 16 is a diagram for explaining the determination of the presence or absence of the boulder 240 according to the embodiment. The boulder determination unit 337 compares the number of pixels of the image of the boulder 240 after the number of pixels is adjusted by the measurement data adjustment unit 336 with a threshold Sh related to the number of pixels that is determined in advance. As shown in FIG. 16, the boulder determination unit 337 determines that the boulder 240 is present when the number of pixels of the image of the boulder 240 exceeds the threshold Sh, and determines that the boulder 240 is not present when the number of pixels of the image of the boulder 240 is equal to or less than the threshold Sh. Even if the actual size of the boulder 240 is the same, the boulder 240 may be determined to be present or not present depending on the value of the coefficient.

[0126] That is, even if the rolling stone detection unit 334 detects the rolling stone 240, if the rolling stone 240 does not satisfy the judgment criterion (if the number of pixels of the rolling stone 240 is equal to or less than the threshold value Sh), the rolling stone determination unit 337 determines that the rolling stone 240 is not present. If the rolling stone detection unit 334 detects the rolling stone 240 and the rolling stone 240 satisfies the judgment criterion (if the number of pixels of the rolling stone 240 exceeds the threshold value Sh), the rolling stone determination unit 337 determines that the rolling stone 240 is present.

[0127] The warning control unit 338 causes the output device 24 to output a warning based on the presence or absence of the rolling stone 240. The warning control unit 338 causes the output device 24 to output a warning based on the determination result of the rolling stone determination unit 337.

[0128] 17 is a diagram showing an example of the output device 24 according to the embodiment. The output device 24 includes a display device 24A that displays an image of the ground 200. A warning control unit 338 causes the output device 24 to output a warning based on the determination of the presence or absence of a boulder 240 by the boulder determination unit 337 and the relationship between the boulder 240 and the warning criterion.

[0129] In an embodiment, outputting the warning includes displaying a warning image 80 that highlights the boulder 240 included in the image of the ground 200. In an embodiment, the warning image 80 includes a frame image that is displayed to surround the boulder 240.

[0130] When the boulder 240 is inside the detection area 34 and outside the first warning area 35, the warning control unit 338 surrounds the boulder 240 with the first frame image 81. When the boulder 240 is inside the first warning area 35 and outside the second warning area 36, ​​the warning control unit 338 surrounds the boulder 240 with the second frame image 82. When the boulder 240 is inside the second warning area 36, ​​the warning control unit 338 surrounds the boulder 240 with the third frame image 83. The display form of the first frame image 81, the display form of the second frame image 82, and the display form of the third frame image 83 are different. For example, the first frame image 81 is displayed in green, the second frame image 82 is displayed in yellow, and the third frame image 83 is displayed in red. By checking the warning image 80, the driver can intuitively recognize the distance between the work machine 1 and the boulder 240.

[0131] The warning control unit 338 may cause the display device 24A to display a frame image indicating the outer edge of the detection area 34, a frame image indicating the outer edge of the first warning area 35, and a frame image indicating the outer edge of the second warning area 36. The frame image indicating the outer edge of the detection area 34 may be displayed in green, the frame image indicating the outer edge of the first warning area 35 may be displayed in yellow, and the frame image indicating the outer edge of the second warning area 36 may be displayed in red.

[0132] In the embodiment, when the rolling stone 240 is in the second warning area 36, ​​the warning control unit 338 causes the sound generating device 24B to generate a warning sound indicating that the rolling stone 240 is on the ground 200 in front of the front tire 6F.

[0133] When the state discrimination unit 333 discriminates that the working state of the work machine 1 is the first state, the detection control unit 339 causes the warning control unit 338 to execute a first process related to the boulders 240 on the ground surface 200. When the state discrimination unit 333 discriminates that the working state of the work machine 1 is the second state, the detection control unit 339 causes the warning control unit 338 to execute a second process related to the boulders 240 on the ground surface 200. When the state discrimination unit 333 discriminates that the working state of the work machine 1 is the third state, the detection control unit 339 causes the warning control unit 338 to execute a third process related to the boulders 240 on the ground surface 200. The first process and the second process are different. The first process and the third process may be the same or different.

[0134] In the embodiment, the first state is a working state in which the work machine 1 moves forward and the work implement 10 does not excavate. The second state is a working state in which the work machine 1 moves forward and the work implement 10 excavates. The third state is a working state in which the work machine 1 moves backward.

[0135] In an embodiment, the first state is at least one of an empty forward mode, a loaded forward mode, and a loading mode, the second state is a digging mode, and the third state is at least one of a loaded reverse mode and an empty reverse mode.

[0136] The first process is a process for causing the warning control unit 338 to output a warning sound when the rolling stone determination unit 337 determines that a rolling stone 240 is present. That is, when the work machine 1 is working in at least one of the work modes of the empty forward movement mode, the loaded forward movement mode, and the loading mode, if it is determined that a rolling stone 240 is present in the second warning area 36, ​​a warning sound is output from the sound generating device 24B.

[0137] The second process is a process that does not cause the warning control unit 338 to output a warning sound even if the boulder determination unit 337 determines that there is a boulder 240. In other words, when the work machine 1 is working in the excavation mode, even if it is determined that there is a boulder 240 in the second warning area 36, ​​no warning sound is output from the sound generation device 24B.

[0138] The third process is a process that does not cause the warning control unit 338 to output a warning sound. That is, when the work machine 1 is working in at least one of the work modes of the loaded reverse mode and the unloaded reverse mode, a warning sound is not output from the sound generation device 24B.

[0139] When the work machine 1 is operating in at least one of the work modes of the unladen forward mode, the loaded forward mode, and the loading mode, if it is determined that a boulder 240 is present in the second warning area 36, ​​a warning sound is output from the sound generating device 24B, thereby enabling the driver to recognize the presence of the boulder 240.

[0140] When the work machine 1 is working in the excavation mode, the bucket 12 penetrates into the natural ground 210. When the bucket 12 penetrates into the natural ground 210, many boulders 240 are present around the front tire 6F. Furthermore, when the bucket 12 penetrates into the natural ground 210, the driver is fully aware of the presence of the boulders 240. Furthermore, the state in which the bucket 12 penetrates into the natural ground 210 is maintained for several seconds. If the warning sound continues to be output from the sound generating device 24B during the period in which the bucket 12 penetrates into the natural ground 210, the driver is likely to feel annoyed. Furthermore, when the bucket 12 penetrates into the natural ground 210, the driver is fully aware of the presence of the boulders 240, and therefore the process of outputting the warning sound is likely to be unnecessary.

[0141] Therefore, in the embodiment, when the state determination unit 333 determines that the work machine 1 is working in the excavation mode, the detection control unit 339 outputs an alarm sound stop command to the alarm control unit 338 so that an alarm sound is not output while the excavation mode is being executed. Even if it is determined that there is a boulder 240 while work is being executed in the excavation mode, an alarm sound is not output from the sound generation device 24B, so that the driver is prevented from feeling annoyed.

[0142] On the other hand, when the state determination unit 333 determines that the work machine 1 is operating in at least one of the work modes of the empty forward mode, the loaded forward mode, and the loading mode, the detection control unit 339 outputs a warning sound permission command to the warning control unit 338 so that a warning sound is output when it is determined that a rolling stone 240 is present.

[0143] [Monitoring method] FIG. 18 is a flowchart showing a method for monitoring the ground surface 200 according to the embodiment.

[0144] The measurement data acquisition unit 331 acquires image data of the ground surface 200 from the imaging device 32 (step S1).

[0145] The rolling stone detection unit 334 detects the rolling stone 240 from the image data acquired in step S1. The rolling stone detection unit 334 detects the rolling stone 240 from the image data based on a predetermined algorithm (step S2).

[0146] The measurement data adjustment unit 336 adjusts the number of pixels in the image of the boulder 240 based on the distance between the work machine 1 and the boulder 240. The measurement data adjustment unit 336 multiplies the number of pixels in the image of the boulder 240 by a coefficient that is set in advance based on the distance between the work machine 1 and the boulder 240 (step S3).

[0147] The boulder determination unit 337 determines whether or not the boulder 240 after the pixel number has been adjusted is in the second warning area 36. The boulder determination unit 337 determines whether or not the boulder 240 is in the second warning area 36 based on a predetermined determination criterion. The boulder determination unit 337 determines whether or not the boulder 240 is in the second warning area 36 based on the image of the boulder 240 after the pixel number has been adjusted and a predetermined threshold value Sh related to the pixel number (step S4).

[0148] The vehicle body data acquisition unit 332 acquires vehicle body data indicating the state of the work machine 1. In an embodiment, the vehicle body data acquisition unit 332 acquires, as the vehicle body data, detection data of the angle sensor 90, detection data of the load sensor 93, and operation data of the forward / reverse lever 20C (step S5).

[0149] The state determination section 333 determines whether or not the work mode of the work machine 1 is the excavation mode based on the vehicle body data acquired in step S5 (step S6).

[0150] If it is determined in step S4 that a boulder 240 is present (step S4: Yes) and if it is determined in step S6 that the excavation mode is active (step S6: Yes), the detection control unit 339 does not output an alarm sound (step S7).

[0151] If it is determined in step S4 that there is a boulder 240 (step S4: Yes) and it is determined in step S6 that the excavation mode is not being set (step S6: No), the state determination unit 333 determines whether the work mode of the work machine 1 is a forward mode or not (step S8). The forward mode is at least one of an unladen forward mode, a loaded forward mode, and an excavation mode.

[0152] In step S8, if it is determined that the work mode of the work machine 1 is the forward mode (step S8: Yes), the warning control section 338 causes the sound generating device 24B to output a warning sound (step S9).

[0153] If it is determined in step S8 that the mode is not the forward travel mode (step S8: No), the warning control unit 338 does not output a warning sound (step S10).

[0154] After any one of the processes in steps S7, S9, and S10 has been performed, the rolling stone detection unit 334 determines whether or not to end the monitoring process (step S11).

[0155] In step S11, if it is determined that the monitoring process is to be continued (step S11: No), the process returns to steps S1 and S5.

[0156] In step S11, if it is determined that the monitoring process is to be ended (step S11: Yes), the monitoring process ends.

[0157] [Computer System] FIG. 19 is a block diagram showing a computer system 1000 according to an embodiment. Each of the above-mentioned monitoring controller 33 and the drive controller 48 includes a computer system 1000. The computer system 1000 has a processor 1001 such as a CPU (Central Processing Unit), a main memory 1002 including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory), a storage 1003, and an interface 1004 including an input / output circuit. The functions of the above-mentioned monitoring controller 33 and the drive controller 48 are stored in the storage 1003 as computer programs. The processor 1001 reads the computer program from the storage 1003, expands it in the main memory 1002, and executes the above-mentioned processing according to the program. The computer program may be distributed to the computer system 1000 via a network.

[0158] According to the above-described embodiment, the computer program or computer system 1000 can acquire measurement data of the ground surface 200 on which the work machine 1 is traveling, determine the working state of the work machine 1, and, if the working state is determined to be a first state, execute a first process related to boulders 240 on the ground surface 200, and, if the working state is determined to be a second state, execute a second process different from the first process.

[0159] [effect] As described above, the monitoring system 30 for the work machine 1 comprises a measurement data acquisition unit 331 that acquires measurement data of the ground surface 200 on which the work machine 1 travels, a state discrimination unit 333 that discriminates the working state of the work machine 1, and a detection control unit 339 that executes a first process related to the boulders 240 on the ground surface 200 when the working state is determined to be a first state, and executes a second process different from the first process when the working state is determined to be a second state.

[0160] According to the embodiment, when the working state of the work machine 1 is in the first state, a process of outputting a warning sound is executed as a first process. By outputting the warning sound, the driver of the work machine 1 can recognize the presence of the boulder 240. When the working state of the work machine 1 is in the second state, a process of not outputting a warning sound is executed as a second process. When the working state of the work machine 1 is in the second state, the process of outputting a warning sound is likely to be unnecessary. According to the embodiment, when the working state of the work machine 1 is in the second state, the process of outputting a warning sound is stopped, so that unnecessary processes are reduced in the work machine 1 that can detect the boulder 240 on the ground surface 200.

[0161] [Another embodiment] In the above-described embodiment, the first process may be a process for causing the rolling stone detection unit 334 to detect the rolling stone 240, and the second process may be a process for not causing the rolling stone detection unit 334 to detect the rolling stone 240. Also, the third process may be a process for not causing the rolling stone detection unit 334 to detect the rolling stone 240.

[0162] That is, during a period when work is being performed in the excavation mode, the detection control unit 339 outputs a detection stop command to the boulder detection unit 334 so that detection of the boulder 240 is not performed. During a period when work is being performed in the excavation mode, detection of the boulder 240 is not performed, so that a warning sound is not output from the sounding device 24B regardless of the presence or absence of the boulder 240. In addition, since the detection process by the boulder detection unit 334 is stopped, the calculation load of the monitoring controller 33 is reduced. During a period when work is being performed in at least one of the work modes of the empty forward movement mode, the loaded forward movement mode, and the loading mode, the detection control unit 339 outputs a detection execution command to the boulder detection unit 334 so that the boulder 240 is detected.

[0163] In the above embodiment, the first process may be a process in which the boulder determining unit 337 determines whether or not there is a boulder 240, and the second process may be a process in which the boulder determining unit 337 does not determine whether or not there is a boulder 240. Also, the third process may be a process in which the boulder determining unit 337 does not determine whether or not there is a boulder 240.

[0164] That is, during a period when work is being performed in the excavation mode, the detection control unit 339 outputs a determination stop command to the boulder determining unit 337 so as not to determine the presence or absence of the boulder 240. During a period when work is being performed in the excavation mode, the presence or absence of the boulder 240 is not determined, and therefore, regardless of the presence or absence of the boulder 240, a warning sound is not output from the sounding device 24B. Furthermore, since the determination process by the boulder determining unit 337 is stopped, the calculation load of the monitoring controller 33 is reduced. During a period when work is being performed in at least one of the work modes of the empty forward mode, the loaded forward mode, and the loading mode, the detection control unit 339 outputs a determination execution command to the boulder determining unit 337 so as to determine the presence or absence of the boulder 240.

[0165] In the above embodiment, the state discrimination unit 333 discriminates the working state of the work machine 1 based on the detection data of the load sensor 93. The state discrimination unit 333 may discriminate the working state of the work machine 1 based on the measurement data acquired by the measurement data acquisition unit 331. The measurement data includes an image of the ground 200. The state discrimination unit 333 may discriminate the working state of the work machine 1 based on an image of boulders 240 included in the image of the ground 200. As described above, in the excavation mode, the bucket 12 penetrates into the natural ground 210, and a large number of boulders 240 are present around the front tire 6F. When the number of boulders 240 detected by the boulder detection unit 334 is equal to or greater than a first specified value that is set in advance, the state discrimination unit 333 can discriminate that the working state of the work machine 1 is the excavation mode. Furthermore, in a state in which the bucket 12 penetrates into the natural ground 210, the forward movement of the work machine 1 is blocked by the natural ground 210. In the excavation mode, the boulders 240 imaged by the imaging device 32 do not change. That is, in the excavation mode, the imaging device 32 continues to image the same boulders 240. In the excavation mode, the boulders detection unit 334 continues to detect the same boulders 240. When the number of boulders 240 that the boulders detection unit 334 continues to detect is equal to or greater than a predetermined second specified value, the state determination unit 333 can determine that the working state of the work machine 1 is the excavation mode.

[0166] In the above-described embodiment, the driving operation device 20 and the output device 24 are arranged in the driver's cab of the work machine 1. The driving operation device 20 and the output device 24 may be arranged outside the work machine 1. The work machine 1 may be remotely operated.

[0167] In the above-described embodiment, the measuring device 32 is an imaging device. The measuring device 32 may be a laser device. The measuring device 32 may be a radar device. The laser device irradiates a measurement object with laser light and acquires measurement data of the measurement object. The radar device irradiates a measurement object with sound waves and acquires measurement data of the measurement object.

[0168] In the above-described embodiment, the work machine 1 is a wheel loader. The work machine 1 may be another work machine, such as a bulldozer or a hydraulic excavator. The bulldozer and the hydraulic excavator each have a work implement and a track that rotates while in contact with the ground 200. The track is a rotating member that rotates while in contact with the ground 200. The work machine travels as the track rotates. [Explanation of symbols]

[0169] Reference Signs List 1...working machine, 2...body, 2A...joint mechanism, 2F...front body, 2R...rear body, 3...driver's cab, 4...running gear, 5...wheels, 5F...front wheels, 5R...rear wheels, 6...tires, 6F...front tires, 6R...rear tires, 7...front fender, 7L...front fender, 7R...front fender, 8...support member, 8L...support member, 8R...support member, 9...housing, 9L...housing, 9R...housing, 10...working machine, 11...boom, 12...bucket, 12E...end, 12M...opening, 13...boom cylinder, 14...bucket cylinder, 15...bell crank, 16...bucket Link, 17... bucket body, 17A... bottom plate portion, 17B... upper plate portion, 17C... left plate portion, 17D... right plate portion, 18... bucket teeth, 19... tooth gap protector, 20... driving operation device, 20A... accelerator pedal, 20B... brake pedal, 20C... forward / reverse lever, 21... operation panel, 22... monitor device, 23... rear view monitor device, 24... output device, 24A... display device, 24B... sound device, 25... input device, 27... pillar, 28... headlight, 28L... headlight, 28R... headlight, 29... turn signal lamp, 29L... turn signal lamp, 29R... turn signal lamp, 3 0...Monitoring system, 32...Imaging device (measuring device), 32L...Imaging device, 32R...Imaging device, 33...Monitoring controller, 34...Detection area, 35...First warning area, 36...Second warning area, 40...Drive system, 41...Engine, 42...Fuel injector, 43...Power take-off, 44...Transmission, 44F...Forward gear, 44R...Reverse gear, 45F...Front axle, 45R...Rear axle, 46...Hydraulic pump, 47...Control valve, 48...Drive controller, 50...Division area, 51...First division area, 52...Second division area, 53...Third division area, 5 4...4th section area, 55...5th section area, 56...6th section area, 57...7th section area, 58...8th section area, 59...9th section area, 60...pixel, 80...warning image, 81...1st frame image, 82...2nd frame image, 83...3rd frame image, 90...angle sensor, 91...boom angle sensor, 92...bucket angle sensor, 93...load sensor, 200...ground, 210...natural ground, 220...dump truck, 230...dump body, 240...rolling stone, 331...measurement data acquisition unit, 332...vehicle body data acquisition unit, 333...state discrimination unit, 334...rolling stone detection unit,335: warning standard setting unit, 336: measurement data adjustment unit, 337: boulder determination unit, 338: warning control unit, 339: detection control unit, 340: coefficient storage unit, 1000: computer system, 1001: processor, 1002: main memory, 1003: storage, 1004: interface, CL: center, FX: rotation axis, Rb: measurement range, RX: rotation axis, β: angle of view.

Claims

1. a measurement data acquisition unit that acquires measurement data of the ground on which the work machine travels; a state determination unit that determines a working state of the work machine; a detection control unit that executes a first process related to boulders on the ground when the working state is determined to be a first state, and executes a second process different from the first process when the working state is determined to be a second state. Work machine monitoring system.

2. The work machine has a work implement capable of excavating an excavation target, The first state is a state in which the work machine moves forward and does not excavate, The second state is a state in which the work machine moves forward and the work implement excavates.

2. A monitoring system for a work machine according to claim 1.

3. a boulder detection unit that detects boulders from the measurement data based on a predetermined algorithm; The first process is a process of detecting a boulder, The second process is a process of not detecting a boulder.

3. A monitoring system for a work machine according to claim 2.

4. a boulder determination unit that determines whether or not there is a boulder on the ground based on a predetermined determination criterion; The first process is a process for determining the presence or absence of a boulder, The second process is a process that does not determine the presence or absence of a boulder.

3. A monitoring system for a work machine according to claim 2.

5. a warning control unit that outputs a warning sound from an output device; The first process is a process of outputting the warning sound when it is determined that a rolling stone is present, The second process is a process of not outputting the warning sound even if it is determined that a rolling stone is present.

3. A monitoring system for a work machine according to claim 2.

6. The state determination unit determines the working state based on a load applied to the working machine.

3. A monitoring system for a work machine according to claim 2.

7. The work machine has a hydraulic cylinder that operates the work implement, The load applied to the working machine includes the pressure of the hydraulic oil in the hydraulic cylinder. A monitoring system for a work machine according to claim 5.

8. the measurement data includes an image of the ground; The state determination unit determines the work state based on an image of boulders included in the image of the ground.

3. A monitoring system for a work machine according to claim 2.

9. When the work state is determined to be the third state, the detection control unit executes a third process related to boulders on the ground, the third state is a state in which the work machine moves backward, The third process is a process that does not determine the presence or absence of the boulder.

3. A monitoring system for a work machine according to claim 2.

10. The work machine has a rotating member, The rotating member rotates while in contact with the ground, thereby causing the work machine to travel on the ground.

2. A monitoring system for a work machine according to claim 1.

11. Obtaining measurement data of a ground surface on which a work machine travels; determining a working state of the work machine; When the working state is determined to be a first state, executing a first process related to boulders on the ground; and executing a second process different from the first process when the working state is determined to be a second state. A method for monitoring a work machine.

Images