Object Detection System for a Working Machine

Abstract

To provide an object detection system for a work machine capable of inexpensively detecting an object in a blind spot.SOLUTION: An object detection system for a work machine that detects an object existing around a work machine (10) having an upper revolving body (12) mounted on a lower traveling body (11) via a rotating mechanism comprises: a camera (20CB) provided on an upper part of the upper revolving body, including the ground around the work machine as an imaging area (21CB), and imaging the imaging area to generate a captured image; a light (25CB) for emitting light toward the ground (22CB) included in the imaging area; and a controller (110) for detecting objects around the work machine based on the captured image, wherein an image feature quantity of the captured image is analyzed to detect the object based on whether a shadow (92CB1) of the object (91CB1) existing on an optical path from the light to the ground included in the imaging area appears in the captured image.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an object detection system for a working machine.

Background Art

[0002] In working machines such as hydraulic excavators, detection system technologies that limit the vehicle body operation when detecting workers or objects around the working machine are known.

[0003] For example, Patent Document 1 describes, "An object detection system that detects an object existing around a construction machine including an upper swing body mounted on a lower traveling body via a swing mechanism has an object detection unit that detects an object based on the output of a scanning distance measuring device attached to the upper swing body, and the light emitted by the scanning distance measuring device passes through the gap between the lower traveling body and the upper swing body. (Summary excerpt)"

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the detection system, relatively expensive detection devices such as infrared sensors and laser scanners are used. In large working machines such as hydraulic excavators, a plurality of these detection devices are arranged to minimize dead spots as much as possible. Therefore, it costs a lot to install a large number of devices to supplement dead spots.

[0006] In addition, in order to prevent damage and dirt during operation, these devices are often installed above the vehicle body. Therefore, dead spots are likely to occur near the working machine or directly below the vehicle body. Therefore, it is desired to supplement dead spots inexpensively and efficiently.

[0007] Particularly, when the detection range in the vertical direction is limited to a plane passing between the upper revolving body and the lower traveling body, the lower traveling body that rotates relative to the upper revolving body can be excluded from the detection target. However, there is a problem that when there are workers or other obstacles at a position lower than the lower surface of the upper revolving body, they cannot be detected.

[0008] The present invention has been made in view of the above circumstances, and an object thereof is to provide an object detection system for a working machine that can detect an object in a blind spot at a lower cost.

Means for Solving the Problems

[0009] In order to solve the above problems, the present invention is an object detection system for a working machine that detects an object existing around the working machine including an upper revolving body mounted on a lower traveling body via a slewing mechanism. The object detection system includes an imaging device provided on the upper part of the upper revolving body, including the ground around the working machine as an imaging area, and generating an imaging image by imaging the imaging area; a light that emits light toward the ground included in the imaging area of the imaging device; and a controller connected to the imaging device and detecting an object around the working machine based on the imaging image acquired from the imaging device. The controller determines whether a shadow of the object existing on the optical path from the light to the ground included in the imaging area is imaged in the imaging image based on the image feature amount of the imaging image, and detects the object based on the determination result.

Effects of the Invention

[0010] According to the present invention, it is possible to provide an object detection system for a working machine that can detect an object in a blind spot at a lower cost. Problems, configurations, and effects other than those described above will be clarified by the description of the following embodiments.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In all the drawings for explaining the embodiments, members having the same function are denoted by the same or related reference numerals, and repeated explanations thereof are omitted.

[0013] With reference to FIGS. 1 and 2, the appearance of the hydraulic excavator 10 will be described. FIG. 1 is a top view showing the configuration of the hydraulic excavator 10. FIG. 2 is a left - side view of the hydraulic excavator 10.

[0014] FIG. 1 shows a hydraulic excavator 10 equipped with an object detection system 1. The hydraulic excavator 10 includes left crawlers 11L, right crawlers 11R (corresponding to the lower traveling body), and an upper slewing body 12 that is rotatably supported on the lower traveling body via a slewing mechanism, a cab 13 attached to the upper slewing body 12, and a front working machine attached to the upper slewing body 12.

[0015] The front working machine includes a boom 14 that is supported so as to be vertically swingable with respect to the upper swing body 12, an arm 15 (see FIG. 2) that is supported so as to be vertically swingable on the boom 14, and a bucket 16 that is supported so as to be swingable on the arm 15.

[0016] In addition, in order to perform forward movement, backward movement using the lower traveling body, turning movement of the upper swing body 12, and swinging movements of the boom 14, arm 15, and bucket 16, respectively, an actuator 106 (see FIG. 3) such as a hydraulic motor or a hydraulic cylinder is provided. The hydraulic motor is provided for turning movements and the like, and the hydraulic cylinder is provided for swinging movements of the boom 14 and the like. Here, a hydraulic type is assumed, but it is not limited thereto, and for example, an electric type such as an electric motor or a linear actuator may be used.

[0017] On the upper side of the left rear of the upper swing body 12, a left rear camera 20LB (corresponding to the imaging device of the present invention) including the ground 22LB (see FIG. 2) at the left rear of the hydraulic excavator 10 as an angular range is arranged. The imaging area of the left rear camera 20LB is indicated by reference numeral 21LB.

[0018] In the upper swing body 12, below the position where the left rear camera 20LB is attached, a left rear light 25LB (see FIG. 2) that emits light toward the ground 22LB included in the imaging area 21LB is provided. The left rear light 25LB emits light having a wavelength ranging from visible light to near-infrared light.

[0019] At the rear center of the upper part of the upper swing body 12, a rear center camera 20CB (corresponding to the imaging device of the present invention) including the ground 22CB (see FIG. 5) at the rear center of the hydraulic excavator 10 as an angular range is arranged. The imaging area of the rear center camera 20CB is indicated by reference numeral 21CB.

[0020] In the upper swing body 12, below the position where the rear center camera 20CB is attached, a rear center light 25CB (see FIG. 5) that emits light toward the ground 22CB included in the imaging area 21CB is provided.

[0021] At the upper right rear of the upper swing body 12, a right rear camera 20RB (corresponding to the imaging device of the present invention) is arranged, including the ground at the right rear of the hydraulic excavator 10 as an included angle. The imaging area of the right rear camera 20RB is indicated by reference numeral 21RB.

[0022] In the upper swing body 12, below the position where the right rear camera 20RB is attached, a right rear light 25RB (see Fig. 5(d)) that emits light toward the ground included in the imaging area 21RB is provided.

[0023] On the upper right side of the upper swing body 12, a right camera 20R is arranged, including the ground on the right side of the hydraulic excavator 10 as an included angle. The imaging area of the right camera 20R is indicated by reference numeral 21R.

[0024] In the upper swing body 12, below the position where the right camera 20R is attached, a right light 25R (see Fig. 5(d)) that emits light toward the ground included in the imaging area 21R is provided.

[0025] The left rear camera 20LB, the center rear camera 20CB, the right rear camera 20RB, and the right camera 20R may have the same configuration and are named differently according to their arrangement positions. Therefore, when there is no need to specifically explain the position where they are arranged, they may simply be referred to as cameras. The same applies to the imaging area, the ground, and the lights.

[0026] The cab 13 is provided with an operation lever 18 including a right lever and a left lever shown in Fig. 2. By operating the right lever forward and backward, the boom 14 is lowered and raised, and by operating it left and right, the bucket 16 is lowered and raised. By operating the left lever forward and backward, the arm 15 is extended and retracted, and by operating it left and right, an execution instruction for the left or right rotation of the upper swing body 12 is input.

[0027] The cab 13 is also provided with a controller 110 that houses a processor 111 for controlling the hydraulic excavator 10, and a notification device 60 for notifying the result of object detection. The notification device 60 is, for example, a display, a buzzer, or a warning light.

[0028] FIG. 3 is a block diagram showing the hardware configuration of the hydraulic excavator 10.

[0029] The controller 110 includes a processor 111 such as a CPU (Central Processing Unit), a RAM (Random Access Memory) 112, a ROM (Read Only Memory) 113, a storage 114, an input I / F 115, and an output I / F 116, and these are connected to each other via a bus 117.

[0030] The storage 114 may be any device that can store data non-volatile, and may be of any type such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0031] Connected to the input I / F 115 are an operation lever 18, a left rear camera 20LB, a center rear camera 20CB, a right rear camera 20RB, and a right camera 20R.

[0032] Connected to the output I / F 116 are a left rear light 25LB, a center rear light 25CB, a right rear light 25RB, a right light 25R, and a notification device 60.

[0033] The hydraulic excavator 10 also includes a pilot pump 102, a main pump 103, an electromagnetic proportional valve 104, a control valve 105, an actuator 106, an engine 107 serving as a power source, and an ECU (Engine Control Unit) 108. For convenience of explanation, it is described as if there is one electromagnetic proportional valve 104, one control valve 105, and one actuator 106 each, but actually, a plurality of systems may be provided.

[0034] An operation signal from a potentiometer that converts and outputs an electric signal corresponding to the operation amount of the operation lever 18 drives the electromagnetic proportional valve 104 via the controller 110, and drives the control valve 105 by the pilot pressure output from this electromagnetic proportional valve 104 to operate the actuator 106. The pilot pump 102 discharges pressure oil to the electromagnetic proportional valve 104, and the main pump 103 discharges pressure oil to the control valve 105.

[0035] The output I / F 116 is connected to the electromagnetic proportional valve 104. When the result of object detection is to stop the swing, forward, or backward movement of the hydraulic excavator 10, a close signal is output to the electromagnetic proportional valve 104. Furthermore, the output I / F 116 is also connected to the ECU 108. An engine speed control signal (including a signal to reduce the speed or a stop signal) is output from the controller 110 to the engine 107 via the ECU 108, and as a result, a decrease or stop in the discharge amount of the main pump 103 is achieved. When performing a swing, forward, or backward movement, a signal to increase the speed is output as the engine speed control signal.

[0036] FIG. 4 is a block diagram showing the functional configuration of the controller 110.

[0037] The processor 111 executes the functions of each part of the operation signal determination unit 111a, the light control unit 111b, the camera control unit 111c, the object detection unit 111d, and the operation control unit 111e. Each of these parts is realized by the processor 111 loading and executing an object detection program in the RAM 112.

[0038] The storage 114 stores object determination threshold data used for determining the size of the shadow.

[0039] <First Embodiment> The first embodiment relates to an object detection system in which one camera is combined with one light. FIG. 5 is an explanatory diagram of the detection range in the object detection system 1 of the working machine according to the first embodiment. In FIG. 5, the positional relationship between the imaging region 21CB of the rear center camera 20CB and the rear center light 25CB is taken as an example for explanation.

[0040] As shown in FIG. 5(a), in the rear center camera 20CB, a dead angle region 29CB occurs between the lower part of the upper swing body 12 and between the left crawler 11L and the right crawler 11R. In FIG. 5(a), for the sake of illustration, the dead angle region 29CB is shown in a right side view with the illustration of the right crawler 11R omitted. The same applies to FIGS. 5(b) and 5(c).

[0041] Since the operator 91CB2 is included in the imaging area 21CB, the operator 91CB2 is reflected in the captured image captured by the rear center camera 20CB. Therefore, the operator 91CB2 is detected by the object detection system 1.

[0042] However, as shown in Fig. 5(b), when the operator 91CB1 is present in the blind spot area 29CB in a low posture, the operator 91CB1 is not included in the imaging area 21CB, so the operator 91CB1 is not reflected in the captured image captured by the rear center camera 20CB.

[0043] Therefore, the rear center light 25CB is attached to the lower part of the upper swing body 12 at the position and orientation where the shadow 92CB1 of the operator 91CB1 is projected onto the ground 22CB. That is, the rear center light 25CB is attached below the upper swing body 12 so as to irradiate light at an angle at which the shadow of an object existing in the blind spot area 29CB from the outside of the angle of view of the rear center camera 20CB to the hydraulic excavator 10, more preferably, the space generated under the upper swing body 12 reaches the ground 22CB behind the hydraulic excavator 10.

[0044] As shown in Fig. 5(a), when there is nothing in the blind spot area 29CB, since there is nothing to block the optical path 26CB from the rear center light 25CB toward the ground 22CB, no shadow of an object is reflected on the ground 22CB behind the hydraulic excavator 10, and a captured image with little change in the image feature amount can be obtained.

[0045] On the other hand, as shown in Fig. 5(b), when the operator 91CB1 is in the blind spot area 29CB, the operator 91CB1 blocks the optical path 26CB, so the shadow 92CB1 of the operator 91CB1 is projected onto the ground 22CB behind the hydraulic excavator 10. Therefore, a captured image with a large change in the image feature amount compared to Fig. 5(a) can be obtained.

[0046] Furthermore, Fig. 5(c) shows a state where the worker 91CB3 is crawling under the vehicle body of the upper swing body 12. In this case, at the mounting position of the center rear light 25CB shown in Fig. 5(b), since there is no worker on the optical path 26CB, no shadow is generated. Therefore, the mounting position of the center rear light 25CB is arranged at the lower end of the upper swing body 12 and in front of the rearmost part of the upper swing body 12. The mounting angle of the center rear light 25CB is set such that the light emitted from the center rear light 25CB reaches the ground 22CB behind the hydraulic excavator 10. As a result, the shadow 92CB3 of the worker 91CB3 existing between the center rear light 25CB and the ground 22CB behind the hydraulic excavator 10 is projected onto the ground 22CB behind the hydraulic excavator 10, and an imaging image with a large change in image feature amount compared to Fig. 5(a) can be obtained.

[0047] Thus, when a shadow is reflected in the imaging image of the ground 22CB behind the hydraulic excavator 10, the image feature amounts such as the brightness, chroma, and lightness of the imaging image change between the area where the shadow is imaged and the surrounding area. Based on this change in the image feature amount, it is determined whether a shadow is reflected, and based on the determination result, an object existing outside the imaging area 21CB of the center rear camera 20CB is detected.

[0048] Fig. 5(d) shows the mounting positions of the left rear light 25LB, the center rear light 25CB, the right rear light 25RB, and the right light 25R.

[0049] Each of the left rear light 25LB, the center rear light 25CB, the right rear light 25RB, and the right light 25R is arranged at the lower part of the upper swing body 12. And they are mounted in such a way that the shadow 92CB1 of the worker 91CB1 who crawls under the vehicle body of the hydraulic excavator 10 and the shadow 92R of the worker 91R who is located close to the right side surface of the hydraulic excavator 10 can be captured by the center rear camera 20CB and the right camera 20R.

[0050] In Fig. 5, the positional relationship between the center rear camera 20CB and the center rear light 25CB is mainly taken as an example for explanation, but the same applies to the right rear camera 20RB, the left rear camera 20LB, and the right camera 20R.

[0051] FIG. 6 is a flowchart showing the processing flow of the object detection system 1 according to the first embodiment.

[0052] When the engine 107 of the hydraulic excavator 10 is started, the main power supply of the controller 110 is turned on, and the operation signal determination unit 111a waits for the acquisition of an operation signal from the operation lever 18. If the operation signal is not acquired (S01: No), it continues to wait.

[0053] The operation signal determination unit 111a determines the operation indicated by the acquired operation signal. When it determines that an operation signal for any of right turn, left turn, forward, or backward has been acquired (S01: Yes), it outputs the operation content indicated by the operation signal to the light control unit 111b.

[0054] The light control unit 111b determines which lights to turn on in which order according to the operation content (S02).

[0055] For example, when about to make a right turn, in the order of the right light 25R, the right rear light 25RB, the center rear light 25CB, and the left rear light 25LB; when making a left turn, in the reverse order; when moving forward, from the rear to the front in the order of the right rear light 25RB and the right light 25R; when moving backward, from the front to the rear in the order of the right light 25R and the right rear light 25RB, the lights are turned on to announce the operation of the hydraulic excavator 10.

[0056] The light control unit 111b outputs a lighting signal to the first light to be turned on according to the lighting order, and the light is turned on (S03).

[0057] The light control unit 111b outputs information indicating the lit light to the camera control unit 111c. The camera control unit 111c captures an image with the camera corresponding to the lit light (S04), and outputs the captured image to the controller 110. After imaging, the light is turned off.

[0058] If the light control unit 111b has not lit all the lights determined in S02 (S05: No), it outputs a lighting signal for the next light (S06). Then, it returns to S04.

[0059] When the light control unit 111b lights all the lights determined in S02 (S05: Yes), the object detection unit 111d executes subject detection of the captured image and image analysis based on the image feature amount (S07).

[0060] Specifically, the object detection unit 111d acquires the captured images of each camera, performs subject detection on the captured images, performs object recognition processing on the detected subjects, and determines whether objects such as people, reflective materials, and cars are captured within the detection range in the captured images. When it is determined that an object is captured (S08: Yes), it outputs notification information indicating that an object has been detected to the notification device 60. Further, the object detection unit 111d outputs information indicating that an object has been detected to the operation control unit 111e, and the operation control unit 111e outputs a close signal to the electromagnetic proportional valve 104 (S09).

[0061] When the object detection unit 111d determines that no object is detected (S08: No), it determines whether a shadow is captured based on the image feature amount of the captured image (S10). As an example of image analysis, if there is no shadow, the brightness of the captured image is relatively bright, and if a shadow is reflected, the brightness of the captured image decreases. Therefore, this image feature amount is used to capture the shadow and estimate the presence of an object in the blind spot area.

[0062] When the object detection unit 111d determines that a shadow is reflected (S10: Yes), it compares the size of the shadow with an object determination threshold value for determining whether operation stop is necessary (S11). When it is determined that the size of the shadow is equal to or greater than the object determination threshold value (S11: Yes), it outputs notification information indicating that an object has been detected to the notification device 60. Further, the object detection unit 111d outputs information indicating that an object has been detected to the operation control unit 111e, and the operation control unit 111e outputs a close signal to the electromagnetic proportional valve 104 (S09).

[0063] The "object determination threshold value" is a threshold value determined based on the size of the shadow of an object of a size to be detected. For example, if a pylon is not detected as an object and an object larger than the pylon, such as a worker working in a crouched and low posture, is to be detected, the length of the shadow of the worker working in a crouched and low posture is measured in advance and stored in the storage 114 as the object determination threshold value. Then, in step S11, the length of the shadow shown in the captured image is compared with the object determination threshold value. If the length of the shadow shown in the captured image is equal to or greater than the object determination threshold value, it is determined that there is an object to be detected.

[0064] When the object detection unit 111d determines that no shadow is shown (S10: No), or when it determines that the size of the shadow is less than the object determination threshold value (S11: No), the operation control unit 111e outputs an open signal to the electromagnetic proportional valve 104, and the hydraulic excavator 10 starts an operation according to the operation signal (S12).

[0065] When continuing the swing, forward, or backward operation (S13: Yes), it returns to S03 and starts lighting the light in the next lighting cycle. When stopping the swing, forward, or backward operation (S13: No), the series of processes is terminated.

[0066] According to the present embodiment, since light is irradiated from the dead angle area of each camera toward the ground in the imaging area of the camera, when there is an object on the optical path from the light toward the ground in the imaging area, the presence of the object in the dead angle area can be estimated by capturing the shadow shown in the imaging area.

[0067] Furthermore, based on the size of the shadow, the size of the object is estimated. For example, objects that are not desired to be detected as objects, such as pylons, are excluded, and workers who have crawled under the body of the hydraulic excavator can be detected. Therefore, object detection in the dead angle area can be performed while preventing false detection.

[0068] Furthermore, since the imaging area includes the ground, and subject detection is performed based on this to determine the presence or absence of an object in the detection range of the camera, it is possible to prevent false detection of the unevenness of the ground and also include objects near the lower traveling body, which was a dead angle area in the conventional example, in the detection range.

[0069] Furthermore, compared with laser scanners and infrared devices, light and cameras are inexpensive, so even if they are damaged, they can be repaired at low cost.

[0070] In addition, in the first embodiment, when a shadow appears in the captured image without estimating the size of the object from the size of the shadow, the operation may be configured to stop by notifying that the object has been detected (S09). In that case, step S11 in FIG. 6 may be omitted.

[0071] <Second Embodiment> The second embodiment is an embodiment in which a plurality of lights are arranged with respect to one camera to more accurately detect the size of a shadow.

[0072] FIG. 7 is an explanatory diagram of a detection range in an object detection system of a construction machine according to the second embodiment (arranged in the left-right direction). In FIG. 7, an example in which three lights, a center rear light 25CB1, 25CB2, and 25CB3, are arranged along the left-right direction of the upper swing body 12 with respect to the center rear camera 20CB will be described.

[0073] As shown in FIG. 7(a), the center rear lights 25CB1, 25CB2, and 25CB3 are arranged side by side from left to right at the rear of the upper swing body 12. Each of the center rear lights 25CB1, 25CB2, and 25CB3 is attached to the hydraulic excavator 10 at positions and angles that irradiate light from different angles with respect to the ground 22CB included in the imaging region 21CB of one center rear camera 20CB.

[0074] It is assumed that the worker 91CB1 is working in a low posture behind the center of the hydraulic excavator 10 and is located in the dead zone 29CB (see FIG. 5) of the center rear camera 20CB.

[0075] FIG. 7(a) shows a state in which all of the center rear lights 25CB1, 25CB2, and 25CB3 are turned off. In this case, the shadow 92CB1 of the worker 91CB1 is not projected onto the ground.

[0076] Here, when only the leftmost center rear light 25CB1 among the three center rear lights 25CB1, 25CB2, and 25CB3 is lit, since the operator 91CB1 shields the optical path 26CB1 of the center rear light 25CB1, a shadow 92CB1 is projected as shown in FIG. 7(b).

[0077] Next, when only the center rear light 25CB2, which is arranged in the center among the three center rear lights 25CB1, 25CB2, and 25CB3, is lit, since the operator 91CB1 shields the optical path 26CB2 of the center rear light 25CB2, a shadow 92CB1 is projected at the position shown in FIG. 7(c).

[0078] Furthermore, when only the rightmost center rear light 25CB3 among the three center rear lights 25CB1, 25CB2, and 25CB3 is lit, since the operator 91CB1 shields the optical path 26CB3 of the center rear light 25CB3, a shadow 92CB1 is projected at the position shown in FIG. 7(d).

[0079] In this way, by arranging the three center rear lights 25CB1, 25CB2, and 25CB3 that illuminate the imaging area 21CB of the center rear camera 20CB from different angles, and repeating the sequence of lighting, imaging, extinguishing, lighting, imaging, and extinguishing while switching the center rear lights 25CB1, 25CB2, and 25CB3, the size and position of the shadow of the same obstacle change, so based on this, the size and position of the object can be estimated.

[0080] For example, similar to the first embodiment, if the pylon is not detected as an object and an object larger than the pylon, for example, a worker working in a crouched and low posture, is to be detected, the shadow of the worker working in a crouched and low posture is imaged by switching the rear center lights 25CB1, 25CB2, and 25CB3 in advance, and the size of the shadow of the worker is stored in the storage 114 as an object determination threshold value corresponding to each light. Then, in step S11, the length of the shadow reflected in the captured image obtained by lighting each light is compared with the object determination threshold value corresponding to that light, and if the length of the shadow reflected in any one of the captured images is equal to or greater than the object determination threshold value, it is determined that an object to be detected exists.

[0081] In FIG. 7, the positional relationship between the rear center camera 20CB and the rear center lights 25CB1, 25CB2, and 25CB3 will be described as an example, but the same applies to the left rear camera 20LB, the right rear camera 20RB, and the right camera 20R.

[0082] FIG. 8 is an explanatory diagram (arranged in the vertical direction) of the detection range in the object detection system of the working machine according to the second embodiment. In FIG. 8, an example in which two lights, the rear center lights 25CB4 and 25CB5, are arranged vertically with respect to the rear center camera 20CB will be described. Note that in FIG. 8, for the sake of illustration, the worker 91CB1 is shown in a right side view with the illustration of the right crawler 11R omitted.

[0083] When only the lower rear center light 25CB4 is lit, the worker 91CB1 blocks the optical path 26CB4, so a shadow 92CB1 is projected at the position shown in FIG. 8(a) with a length L_25CB4.

[0084] When only the upper rear center light 25CB5 is lit, the worker 91CB1 blocks the optical path 26CB5, so a shadow 92CB1 is projected at the position shown in FIG. 8(b) with a length L_25CB5.

[0085] In FIG. 8, the positional relationship between the rear center camera 20CB and the rear center lights 25CB4 and 25CB5 was described as an example. The same applies to the left rear camera 20LB, the right rear camera 20RB, and the right camera 20R.

[0086] In FIGS. 7 and 8, the blinking order of the plurality of rear center lights 25CB1, 25CB2, 25CB3, 25CB4, and 25CB5 may be changed and used as a warning light for the turning operation, forward movement, or backward movement of the hydraulic excavator 10. For example, in the example of FIG. 7, when the operation lever 18 receives an input operation for instructing the execution of the turning operation, the processor 111 receives the operation signal of the operation lever 18, and based on this, if it is a right turn, for example, a lighting signal may be output so as to repeat blinking in the order of the rear center lights 25CB3, 25CB2, and 25CB1.

[0087] Since the processing flow of the object detection system in the second embodiment is the same as the processing flow of the first embodiment shown in FIG. 6, the flowchart of FIG. 6 is reused, and only the differences from the first embodiment will be described.

[0088] In the lighting order of step S02, the lighting order is determined for each of the plurality of lights provided for one camera.

[0089] Also, in the shadow size determination of step S11, for each captured image obtained with the lights that irradiate light from different angles turned on for one camera, image analysis is performed. Then, when it is determined that the shadow size is equal to or greater than the object determination threshold in one or more captured images (S11: Yes), it is notified that an object has been detected and the operation is stopped (S09).

[0090] Also, in step S11, since the shadow of the same object is captured from a plurality of directions, the size of the object itself may be estimated from the size of the shadow, and it may be determined whether the object is the object to be detected. For example, in the example of FIG. 8, the height of the object may be calculated based on the shadow lengths L_25CB4 and L_25CB5 and the mounting angles of the rear center lights 25CB4 and 25CB5 to estimate the size of the object.

[0091] According to the second embodiment, since a plurality of lights are used to irradiate the imaging area of a single camera to detect an object lurking in the blind spot area of the camera, object detection can be performed over a wider range.

[0092] Also, depending on the relative position and orientation relationship between a certain light and the worker 91CB1, even if the shadow is accidentally projected short, the shadow of the worker 91CB1 is projected relatively long by other lights, thereby improving the accuracy of object detection. It is also possible to estimate the size of an object from the length of the shadow shown in different captured images.

[0093] Furthermore, since a shadow is created by the light and the luminance difference is captured and determined by the camera, even if the light is somewhat dirty, false detections are few. Also, by using a plurality of lights and changing the order in which they are lit to create a shadow, even if one light is dirty, it can be compensated for by other lights, thereby reducing false detections.

[0094] <Third Embodiment> The third embodiment is an embodiment in which a dimming light is used as the light for illuminating the blind spot area. The "dimming light" mentioned here is a light whose light quantity and wavelength can be changed. FIG. 9 is an explanatory diagram showing the processing content of the third embodiment. In FIG. 9, the center rear light 25CB is taken as an example for explanation, but the same applies to the lights provided corresponding to each of the left rear camera 20LB, the right rear camera 20RB, and the right camera 20R.

[0095] FIG. 9(a) shows a situation where the ambient light is relatively strong during the day. The controller 110 outputs a dimming signal to increase the light quantity of the center rear light 25CB or change the wavelength to emit light of a color different from the ambient light so that the contrast ratio between the shadow of the worker 91CB1 and the image of the ground captured under the ambient light is good in the captured image.

[0096] On the other hand, at night, there is little or almost no ambient light. Therefore, as shown in FIG. 9(b), the light quantity of the center rear light 25CB is reduced to suppress power consumption and it is lit.

[0097] In the evening, the ambient light approaches orange to red. Therefore, as shown in FIG. 9(c), the wavelength of the light of the rear center light 25CB is changed to emit light of a color that is easy to distinguish from the ambient light, for example, the complementary color light from orange to red. This prevents the light of the rear center light 25CB from being mixed into the ambient light, making it easier to identify the shadow.

[0098] FIG. 10 is a flowchart showing the processing flow of the object detection system according to the third embodiment. The same steps as those in the flowchart according to the first embodiment shown in FIG. 6 are denoted by the same reference numerals, and redundant descriptions are omitted.

[0099] In the third embodiment, following step S04, a dimming process is added. Specifically, following step S04, a determination step for starting the dimming sequence is executed (S31). When starting the dimming sequence (S31: Yes), the object detection unit 111d analyzes the light quantity and wavelength of the ambient light based on the image feature amounts of the captured image, such as the distribution (histogram) of luminance, lightness, and chroma, etc. (S32), and outputs it to the light control unit 111b. The light control unit 111b performs dimming according to the analysis result. Then, it proceeds to step S05.

[0100] In this embodiment, the ambient light is analyzed based on the captured image. However, when the hydraulic excavator 10 is equipped with an illuminance sensor, the light control unit 111b may acquire illuminance data from the illuminance sensor and determine the light quantity of the light based on the light quantity of the ambient light.

[0101] Alternatively, when the controller 110 is equipped with an RTC (Real Time Clock), the light control unit 111b may acquire time data from the RTC, increase the light quantity during the day, switch to the complementary color from orange to red in the evening, and switch to the power saving mode at night according to the time.

[0102] When performing dimming based on the illuminance sensor or time, the determination for starting the dimming sequence (S31) does not necessarily have to be after step S04, and it may be at any timing earlier than S03.

[0103] The dimming sequence may be configured to be executed once and then, for example, once every six hours.

[0104] According to the third embodiment, since the light is dimmed according to the ambient light conditions to image the shadow, it is possible to improve the detection accuracy of an object lurking in the blind spot area while suppressing the influence of the ambient light.

[0105] The above embodiments are not intended to limit the present invention, and various modified forms are also included in the present invention. For example, the mounting positions and numbers of the cameras and lights are not limited to the above.

[0106] Further, instead of an example in which a plurality of lights are arranged for one camera, one light whose irradiation angle can be changed may be provided.

Explanation of Reference Numerals

[0107] 1: Object detection system 10: Hydraulic excavator 11L: Left crawler 11R: Right crawler 12: Upper slewing body 13: Cab 14: Boom 15: Arm 16: Bucket 18: Operating lever 20CB: Rear center camera 20LB: Rear left camera 20R: Right camera 20RB: Rear right camera 21CB: Imaging area 21LB: Imaging area 21R: Imaging area 21RB: Imaging area 22CB: Ground 22LB: Ground 25CB: Rear center light 25CB1: Rear center light 25CB2: Rear center light 25CB3: Rear center light 25CB4: Rear Center Light 25CB5: Rear Center Light 25LB: Left Rear Light 25R: Right Light 25RB: Right Rear Light 26CB: Optical Path 26CB1: Optical Path 26CB2: Optical Path 26CB3: Optical Path 26CB4: Optical Path 26CB5: Optical Path 29CB: Blind Spot Area 60: Notification Device 91CB1: Operator 91CB3: Operator 91R: Operator 92CB1: Shadow 92CB3: Shadow 92R: Shadow 102: Pilot Pump 103: Main Pump 104: Electro-Hydraulic Proportional Valve 105: Control Valve 106: Actuator 107: Engine 108: ECU 110: Controller 111: Processor 111a: Operation Signal Judgment Unit 111b: Light Control Unit 111c: Camera Control Unit 111d: Object Detection Unit 111e: Motion Control Unit 112: RAM 113: ROM 114: Storage 115: Input I / F 116: Output I / F 117: Bus

Claims

1. An object detection system for a work machine that detects an object existing around the work machine including an upper slewing body mounted on a lower traveling body via a slewing mechanism, an imaging device provided on the upper slewing body, including the ground around the work machine as an imaging area, and generating an imaging image by imaging the imaging area; a light that emits light toward the ground included in the imaging area of the imaging device; a controller that detects an object around the work machine based on the imaging image acquired from the imaging device, and the controller determines, based on an image feature amount of the imaging image, whether a shadow of the object existing on the optical path from the light to the ground included in the imaging area is imaged in the imaging image, and detects the object based on the determination result. An object detection system for a work machine, characterized by the above.

2. In the object detection system for a work machine according to Claim 1, further comprising an operation lever for inputting any one of a slewing operation of the upper slewing body, a forward movement operation, or a backward movement operation of the work machine, and the controller when detecting that an operation signal indicating an execution instruction of any one of the slewing operation, the forward movement operation, or the backward movement operation is input from the operation lever, transmits a lighting signal to the light. An object detection system for a work machine, characterized by the above.

3. In the object detection system for a work machine according to Claim 2, further comprising a notification device for notifying a determination result by the controller, and the light is a plurality of lights that irradiate light from different angles to the ground included in one imaging area, and when the controller detects the object from the imaging image based on the optical path from any one of the plurality of lights, the controller sequentially blinks each of the plurality of lights one by one, acquires the imaging image generated each time each light lights up, estimates the size of the object based on the size of the shadow reflected in the plurality of imaging images, and outputs the estimation result to the notification device. An object detection system for a work machine, characterized by the above.

4. The object detection system for a work machine according to Claim 3, wherein the controller determines, based on the operation signal from the operation lever, which of the slewing operation, the forward movement operation, or the backward movement operation the execution instruction indicated by the operation signal indicates. Output a lighting signal for lighting a plurality of the lights in order along the direction in which the work machine operates according to the content of the execution instruction of the operation signal. An object detection system for a work machine, characterized by the above.

5. The object detection system for a work machine according to claim 3, wherein the plurality of the lights are arranged side by side in the vertical direction or the horizontal direction on the upper swing body. An object detection system for a work machine, characterized by the above.

6. The object detection system for a work machine according to claim 3, wherein the controller compares the estimated size of the object with an object determination threshold value that is a criterion for determining whether notification is required, and performs control to invalidate the operation of the operation lever when the size of the object is equal to or greater than the object determination threshold value. An object detection system for a work machine, characterized by the above.

7. The object detection system for a work machine according to claim 1, wherein the imaging device includes a rear imaging device attached above and behind the upper swing body at an angle including the ground behind the work machine as the imaging region. The light includes a rear light attached to the lower end of the upper swing body and in front of the rearmost part of the upper swing body. The attachment angle of the rear light is an angle at which the light emitted from the rear light reaches the ground behind. The shadow of the object to be detected existing between the rear light and the ground behind is projected onto the ground behind, and the shadow is imaged in the captured image. An object detection system for a work machine, characterized by the above.

8. The object detection system for a work machine according to claim 1, wherein the light is a dimmable light whose light quantity and wavelength can be changed. The controller analyzes the light quantity and wavelength of ambient light based on the image feature amount of the captured image, and outputs a dimming signal for changing at least one of the light quantity or wavelength of the dimmable light to the dimmable light when the contrast ratio between the shadow of the object and the ground image captured under the ambient light is less than a contrast threshold value for determining whether dimming is required. An object detection system for a work machine, characterized by the above.

Images