Control method for safety monitoring system and safety monitoring system

Abstract

To make an operator notice an object around a construction machine more easily.SOLUTION: A safety monitoring system 1 comprises a detection device 10, a display portion 22, and a control portion 32. The detection device 10 detects an object around a work vehicle 100. The display portion 22 displays a captured image around the work vehicle 100. The control potion 32 controls the detection device 10 and the display portion 22. When the detection device 10 detects an object, the control potion 32 controls the display portion 22 so as to display an edged image V1 in which edging processing E1 is performed to the surrounding image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention provides Control method for safety monitoring system and Regarding safety monitoring systems. [Background technology]

[0002] For example, in work machines and construction machines used at construction sites, cameras are used to capture images of surrounding obstacles and the like, and the captured images are displayed on a display device so that operators can confirm safety.

[0003] The work machine of Patent Document 1 uses a distance sensor to detect obstacles or people in the surrounding area in addition to capturing images, and when the distance sensor detects an obstacle, it displays on the display device that there is an obstacle other than a person in the surrounding area, and when the distance sensor detects a person, it displays on the display device that there is a person in the surrounding area. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-51156 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, an operator may not be able to keep his / her eyes on the display device while operating a construction machine, and may not notice the presence of an obstacle or person.

[0006] The present invention has been made in view of the above-mentioned problems, and its object is to provide a safety monitoring system that enables an operator to more easily notice objects around a construction machine. [Means for solving the problem]

[0007] The safety monitoring system according to the present invention comprises a detection device, a display device, and a control device. The detection device detects objects around a construction machine. The display device displays a captured image of the surroundings of the construction machine. The control device controls the detection device and the display device. When the detection device detects the object, the control device controls the display device to display a bordered image in which a border has been applied to the surrounding image. [Effects of the Invention]

[0008] The present invention allows an operator to more easily notice objects around the construction machine. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a perspective view showing an upper rotating body of the work vehicle. [Figure 2] 1 is a block diagram showing a work vehicle equipped with a safety monitoring system. [Figure 3] 10A and 10B are diagrams illustrating object detection and edge-delineation images by a detection device. [Figure 4] 10A and 10B are diagrams illustrating detection of the distance to an object by the detection device and a screen displayed on the display unit according to the distance. [Figure 5] FIG. 1 is a diagram illustrating the detection of a person or object by a detection device. [Figure 6] 10A and 10B are diagrams illustrating detection of a person by the detection device, a screen displayed on the display unit, and a warning sound output from the sound output unit depending on the distance of the person. [Figure 7] 1 is a top view of a work vehicle equipped with a safety monitoring system. [Figure 8] FIG. 1 illustrates a safety monitoring system. [Figure 9] FIG. 1 is a block diagram illustrating a safety monitoring system. [Figure 10] FIG. 1 illustrates a safety monitoring system. [Figure 11] 1 is a flowchart illustrating a safety monitoring method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference characters and description thereof will not be repeated.

[0011] First, a work vehicle 100 equipped with a safety monitoring system 1 according to an embodiment of the present invention will be described with reference to Figures 1 and 2. Figure 1 is a perspective view showing an upper rotating body 200 of the work vehicle 100. Note that the roof of the upper rotating body 200 has been omitted from Figure 1 to make the drawing easier to see. Figure 2 is a block diagram showing the work vehicle 100 equipped with the safety monitoring system 1. Below, this embodiment will be described using an example in which the work vehicle 100 is a hydraulic excavator. A hydraulic excavator is an example of construction machinery. The safety monitoring system 1 includes a detection device, a control device, and a display device.

[0012] As shown in Figure 1, the work vehicle 100 is equipped with an upper rotating body 200. The upper rotating body 200 rotates relative to a lower traveling body 400 (see Figure 2 below). The upper rotating body 200 is driven to rotate by a hydraulic pump 600 (see Figure 2 below) and a hydraulic motor (not shown). A work implement 300 (see Figure 2 below) is attached to the upper rotating body 200.

[0013] Specifically, the upper rotating body 200 includes a cockpit 205 , a seat 210 , a plurality of control levers 220 , and a positioning member 230 .

[0014] In the cockpit 205, an operating device 20, a detection device 10, a seat 210, a plurality of operating levers 220, and a positioning member 230 are arranged. An operator sits in the seat 210. The operator is the operator of the work vehicle 100. Each of the plurality of operating levers 220 accepts operation from the operator. The operating levers 220 are operating members for operating the work vehicle 100.

[0015] The operation device 20 receives operations from the operator, operates the work vehicle 100 , inputs various information to the work vehicle 100 , and displays various information related to the work vehicle 100 .

[0016] The detection device 10 detects the conditions around the work vehicle 100. For example, the detection device 10 has a camera 12 that captures images of the surroundings of the work vehicle 100, and infrared sensors 14A, 14B, and 14C. The detection device 10 is provided, for example, behind the seat 210 on the upper rotating body 200.

[0017] For example, in the detection device 10 provided behind the seat 210, the camera 12 captures an image of the area behind the work vehicle 100 and generates a camera image.

[0018] The infrared sensors 14A, 14B, and 14C are, for example, TOF (Time Of Flight) sensors. Specifically, the infrared sensors 14A, 14B, and 14C emit infrared light. If an object is present in the direction of travel of the infrared light emitted from the infrared sensors 14A, 14B, and 14C, the infrared light is reflected by the object. The reflected infrared light reflected by the object travels toward the infrared sensors 14A, 14B, and 14C. The infrared sensors 14A, 14B, and 14C receive the reflected infrared light. The infrared sensors 14A, 14B, and 14C measure the time from when the infrared light is emitted to when the reflected infrared light is received, and measure the distance to the object based on the measured time.

[0019] The operation device 20 displays various types of information and accepts pressing operations from the operator. The operation device 20 also displays an image of the surroundings of the construction machine. The operation device 20 is equipped with a display unit 22 and a plurality of push buttons 24. The display unit 22 displays various types of information related to the work vehicle 100, such as the status of the work vehicle 100 and a GUI (Graphical User Interface). The display unit 22 also displays camera images captured by the camera 12. The camera images are an example of an image of the surroundings of the construction machine.

[0020] The display unit 22 is an example of a display device, such as a liquid crystal display or an organic electroluminescence display. The display unit 22 may include a touch panel. In this case, each push button 24 may be displayed on the display unit 22 as a GUI widget.

[0021] As shown in Fig. 2, in addition to the operating device 20 and the upper rotating body 200, the work vehicle 100 further includes a work implement 300, a lower traveling body 400, a blade (not shown), an engine unit 500, a hydraulic pump 600, a control valve 700, and an oil tank 800. The operating device 20 further includes a main control device 30. The main control device 30 is disposed, for example, inside the upper rotating body 200, at a different position from the operating device 20. The main control device 30 is one example of a control device.

[0022] The work machine 300 performs work. Specifically, the work machine 300 includes a bucket (not shown), an arm (not shown), a boom (not shown), and a plurality of hydraulic cylinders (not shown).

[0023] The upper rotating body 200 is disposed above the lower traveling body 400 via a swivel joint (not shown). The lower traveling body 400 travels. Specifically, the lower traveling body 400 includes a pair of crawlers (not shown) and a pair of hydraulic motors (not shown).

[0024] The engine section 500 includes an engine 501 and an engine control device 503. The engine control device 503 controls the engine 501. The engine control device 503 is, for example, an ECU (Electronic Control Unit). Fuel is supplied to the engine 501 from a fuel tank (not shown).

[0025] The engine 501 drives the hydraulic pump 600. As a result, the hydraulic pump 600 sends out pressurized oil to the control valve 700. Pressurized oil is oil under pressure. The control valve 700 controls the flow of pressurized oil in accordance with the operation of each operating lever 220. The control valve 700 supplies pressurized oil to the hydraulic motor of the upper rotating body 200, the hydraulic cylinders of the work equipment 300, the hydraulic motors of the lower traveling body 400, and the hydraulic cylinder (not shown) that drives the blade.

[0026] The operation device 20 further includes a display control device 26 and a sound output unit 28. The display control device 26 controls the display unit 22. The display control device 26 is, for example, an ECU. The sound output unit 28 outputs sound. The sound output unit 28 is an example of an audio output device, and is, for example, a speaker or a buzzer.

[0027] Specifically, the display control device 26 includes a control unit 261 and a storage unit 263. The control unit 261 includes a processor such as a CPU (Central Processing Unit). The storage unit 263 includes a storage device and stores data and computer programs. Specifically, the storage unit 263 includes a main storage device such as a semiconductor memory, and an auxiliary storage device such as a semiconductor memory, a solid-state drive, and / or a hard disk drive. The storage unit 263 may include removable media. The storage unit 263 corresponds to an example of a non-transitory computer-readable storage medium.

[0028] The control unit 261 outputs to the main control device 30 various pieces of information that are input or set in response to operations on the screen displayed on the display unit 22.

[0029] The control unit 261 includes a display control unit 265. Specifically, the processor of the control unit 261 functions as the display control unit 265 by executing a computer program stored in the storage device of the storage unit 263. The display control unit 265 controls the display unit 22. For example, the display control unit 265 controls the screen displayed on the display unit 22 in accordance with operation information (on information or off information) indicating whether or not the push buttons 24A to 24F have been pressed. The display control unit 265 will be described in detail later.

[0030] The main control device 30 controls the detection device 10, the operation device 20, and the engine unit 500. The main control device 30 is, for example, an ECU.

[0031] Specifically, the main control device 30 includes a control unit 32 and a memory unit 34. The control unit 32 includes a processor such as a CPU. The memory unit 34 includes a storage device and stores data and computer programs. The hardware configuration of the memory unit 34 is similar to the hardware configuration of the memory unit 263, for example.

[0032] The control unit 32 acquires various pieces of information that have been input or set in response to operations on the screen displayed on the display unit 22 from the control unit 261 of the display control device .

[0033] The operator can know the condition behind the work vehicle 100 by looking at the camera image displayed on the display unit 22. However, the operator may not be able to pay attention to the display unit 22 while operating the work vehicle 100, and may not notice an object included in the camera image displayed on the display unit 22.

[0034] In contrast to this, in this embodiment, even when the operator cannot focus on the display unit 22, the operator can easily notice objects around the work vehicle 100.

[0035] [Object detection] Specifically, when detection device 10 detects an object, main control device 30 controls display unit 22 to display a bordered image in which border processing has been applied to the camera image.

[0036] Next, the detection of an object and a bordered image by the detection device 10 will be described with reference to Fig. 3. Fig. 3 is a diagram showing the detection of an object and a bordered image by the detection device 10.

[0037] 3, a triangular cone R1 as an example of an object is shown schematically as being detected by the infrared sensors 14A, 14B, and 14C in a comprehensive manner as detected by the infrared sensor 14. Details of detection by each of the infrared sensors 14A, 14B, and 14C will be described later.

[0038] Display unit 22 displays the camera image captured by camera 12 under the control of main control device 30. The camera image is an image showing the rear of work vehicle 100. The camera image includes an object corresponding to a triangular cone R1 located behind work vehicle 100.

[0039] The infrared sensor 14 emits infrared light. The infrared light emitted from the infrared sensor 14 reaches and is reflected by the triangular cone R1. The reflected infrared light reflected by the triangular cone R1 travels toward the infrared sensor 14. The infrared sensor 14 receives the reflected infrared light.

[0040] When the infrared sensor 14 receives reflected infrared light, it determines that an object exists in the direction of the infrared light emission. On the other hand, when the infrared sensor 14 cannot receive reflected infrared light, it determines that no object exists in the direction of the infrared light emission.

[0041] Main control device 30 acquires the determination result of infrared sensor 14 and, according to the determination result, controls display unit 22 to display bordered image V1 in which border processing E1 has been applied to the camera image. In other words, when infrared sensor 14 determines that an object exists in the direction in which infrared light is emitted, display unit 22 displays bordered image V1 in which border processing E1 has been applied to the camera image.

[0042] In this way, when an object is present around (behind) the work vehicle 100, the display on the entire screen of the display unit 22 changes, making it easier for the operator to notice the object.

[0043] Furthermore, for example, when infrared sensor 14 determines that an object is present in the direction of infrared light emission, main control device 30 controls display unit 22 to add a warning display M1 to the camera image in addition to border processing E1.

[0044] In this embodiment, the color of the edging and the color of the warning display may be changed depending on the distance to the object.

[0045] Next, detection of the distance to an object by the detection device 10 will be described with reference to Fig. 4. Fig. 4 is a diagram showing detection of the distance to an object by the detection device 10 and a screen displayed on the display unit 22 according to the distance.

[0046] The detection device 10 measures the distance to an object. Specifically, as shown in Fig. 4, the infrared sensor 14 measures the time from emitting infrared light to receiving the reflected infrared light reflected by the triangular cone R1, and calculates the distance to the triangular cone R1 based on the measured time.

[0047] The main control device 30 acquires the distance to the triangular cone R1 measured by the infrared sensor 14, and determines the colors of the edging E1 and the warning display M1 according to the acquired distance.

[0048] For example, if a triangular cone R1 is placed closer than a certain distance L2 from work vehicle 100, main control device 30 controls display unit 22 to add, for example, a "yellow" warning display M1 to the camera image. Under the control of main control device 30, display unit 22 displays camera image V2 with the "yellow" warning display M1 added. In Figure 4, the "yellow" is shown without hatching.

[0049] On the other hand, for example, if the triangular cone R1 is placed closer than a distance L1, which is shorter than the distance L2, from the work vehicle 100, the main control unit 30 controls the display unit 22 to, for example, apply a "red" border processing E1 to the camera image and display a border image V1 to which a "red" warning display M1 has been added.

[0050] In Figure 4, when the triangular cone R1 is placed closer than a certain distance L2 from the work vehicle 100, no edging is applied to the camera image V2, but for example, a "yellow" edging may be applied.

[0051] In this way, the screen displayed on the display unit 22 changes depending on the distance to the object, so that the operator is more likely to pay attention to closer objects.

[0052] [People Detection] Next, detection of a person by the detection device 10 will be described with reference to Fig. 5. Fig. 5 is a diagram showing detection of a person or an object by the detection device 10. Fig. 5 describes in detail detection of a person or an object by the infrared sensors 14A, 14B, and 14C.

[0053] Infrared sensors 14A, 14B, and 14C each emit infrared light in the same direction but at different angles. For example, infrared sensor 14A emits infrared light at an elevation angle (depression angle) α, infrared sensor 14B emits infrared light at an elevation angle (depression angle) β, and infrared sensor 14C emits infrared light at an elevation angle (depression angle) γ. Note that elevation angle (depression angle) α<elevation angle (depression angle) β<elevation angle (depression angle) γ.

[0054] The infrared light emitted from the infrared sensor 14A at an elevation angle (depression angle) α reaches and is reflected by the triangular cone R1, the person P1, or the ground. The reflected infrared light reflected by any of these is received by the infrared sensor 14A.

[0055] Infrared sensor 14A calculates distance LA to the point where the infrared light is reflected based on the time it takes for the emitted infrared light to be received as reflected infrared light. Infrared sensor 14A further calculates distance L between infrared sensor 14A and the reflection point based on the calculated distance LA and the elevation angle (depression angle) α.

[0056] Furthermore, infrared sensor 14A calculates the height from the ground of the point where the infrared light is reflected, based on the calculated distance LA and the elevation angle (depression angle) α. Specifically, infrared sensor 14A calculates the distance Ha that the emitted infrared light has descended from infrared sensor 14A, based on the calculated distance LA and the elevation angle (depression angle) α. Infrared sensor 14A calculates the height from the ground of the point where the infrared light is reflected, by subtracting the calculated distance Ha from the height HA set in infrared sensor 14A.

[0057] Similar to infrared sensor 14A, infrared sensors 14B and 14C calculate distances LB and LC between infrared sensors 14B and 14C and the reflection point, respectively. Infrared sensor 14B calculates distance L and the height of the reflection point from the ground based on distance LB and elevation angle (depression angle) β. Infrared sensor 14C calculates distance L and the height of the reflection point from the ground based on distance LB and elevation angle (depression angle) γ.

[0058] For example, when the calculated height of the reflection point from the ground is greater than zero, the infrared sensors 14A, 14B, and 14C detect that an object exists at a position that is a distance L away from the infrared sensors 14A, 14B, and 14C.

[0059] In this embodiment, the infrared sensors 14A, 14B, and 14C determine whether an object is a person based on the intensity of the reflected infrared light. For example, at a construction site where the work vehicle 100 is used, a person P1 is wearing a reflective vest W1 that has a high reflectivity for infrared light. Therefore, the intensity of the reflected infrared light reflected by the person P1 (reflective vest W1) is high. On the other hand, the intensity of the reflected infrared light reflected by an object (such as a traffic cone) is weaker than when reflected by the person P1 (reflective vest W1).

[0060] Infrared sensors 14A, 14B, and 14C measure the intensity of the received reflected infrared light, and if the intensity of the reflected infrared light is greater than a predetermined intensity, they determine that the object that reflected the infrared light is a person.On the other hand, if the intensity of the reflected infrared light is weaker than the predetermined intensity, infrared sensors 14A, 14B, and 14C determine that the object that reflected the infrared light is an object.

[0061] Here, the infrared sensors 14A, 14B, and 14C intermittently emit infrared light and receive reflected infrared light, and measure the intensity of the reflected infrared light each time they receive it. For example, the infrared sensors 14A, 14B, and 14C calculate the average (average intensity) of the intensities of the multiple reflected infrared lights, and if the average intensity is greater than a predetermined intensity, they determine that the object that reflected the infrared light is a person.

[0062] Furthermore, for example, if the triangular cone R1 has the same reflectance as the reflective vest W1, the infrared sensors 14A, 14B, and 14C will erroneously determine that the triangular cone R1 that reflects infrared light is a person.

[0063] To prevent this, the infrared sensors 14A, 14B, and 14C determine that the object that reflected the infrared light is an object, even if the intensity of the reflected infrared light is greater than a predetermined intensity, if the height from the ground of the reflection point of the reflected infrared light is lower than a predetermined height.

[0064] For example, the following describes a comparison between a case where infrared sensors 14A, 14B, and 14C detect person P1 and a case where infrared sensors 14A, 14B, and 14C detect a triangular cone R1 that has the same reflectance as reflective vest W1. In Fig. 5, person P1 and triangular cone R1 are assumed to be located at the same position, a distance L away from infrared sensors 14A, 14B, and 14C.

[0065] When infrared sensor 14A detects person P1, infrared sensor 14A calculates the distance L to the reflection point and the height H1 of the reflection point from the ground, as described above. Furthermore, infrared sensor 14A determines that the intensity of the reflected infrared light is greater than a predetermined intensity. Because the intensity of the reflected infrared light is greater than the predetermined intensity and the height H1 of the reflection point from the ground is greater than the predetermined height H0, infrared sensor 14A determines that the object that reflected the infrared light is a person.

[0066] On the other hand, when infrared sensor 14B detects cone R1, infrared sensor 14B calculates distance L to the reflection point and height H2 of the reflection point from the ground, similar to infrared sensor 14A described above. Furthermore, infrared sensor 14B determines that the intensity of the reflected infrared light is greater than a predetermined intensity. Because the intensity of the reflected infrared light is greater than the predetermined intensity and the height H2 of the reflection point from the ground is lower than the predetermined height H0, infrared sensor 14B determines that the object that reflected the infrared light is an object.

[0067] In this way, by determining whether the object that reflected the infrared light is a person or an object based on the intensity of the reflected infrared light and the height of the object that reflected the infrared light, the possibility of misjudging whether it is a person or an object is reduced.

[0068] Furthermore, in this embodiment, the intensity (threshold) of reflected infrared light at which the infrared sensor 14 determines that the object is a person may be different before and after the person P1 is detected.

[0069] For example, after detecting person P1, the intensity of reflected infrared light may decrease depending on the position and angle of person P1, which may result in erroneous determination of person P1 as an object. To prevent this, when infrared sensor 14 detects person P1, it lowers the threshold for reflected infrared light.

[0070] As a result, after detecting the person P1, the infrared sensor 14 continues to determine that the person P1 is a person until the intensity of the reflected infrared light falls below the lowered threshold. Therefore, after detecting the person P1, the possibility of erroneously determining that the person P1 is an object is reduced.

[0071] [warning sound] In this embodiment, when the detection device 10 detects a person, the sound output unit 28 may emit a warning sound.

[0072] Next, the warning sound output in the safety monitoring system according to this embodiment will be described with reference to Fig. 6. Fig. 6 is a diagram showing the detection of person P1 by the detection device 10, and the screen displayed on the display unit 22 and the warning sound output from the sound output unit 28 according to the distance of person P1.

[0073] Similar to the detection of the triangular cone R1 by the detection device 10 shown in Figure 4, when the detection device 10 detects a person P1, the main control device 30 obtains the distance to the person P1 measured by the infrared sensor 14, and determines the color of the edging E1 and the warning display M1 based on the obtained distance.

[0074] For example, when person P1 is farther away from work vehicle 100 than distance L1 but closer than distance L2, main control device 30 controls display unit 22 to add, for example, a "yellow" warning display M1 to the camera image. Under the control of main control device 30, display unit 22 displays camera image V2 with the "yellow" warning display M1 added. In Figure 6, the "yellow" is shown without hatching.

[0075] On the other hand, for example, if a person P1 is closer than a distance L1 from the work vehicle 100, the main control unit 30 controls the display unit 22 to, for example, apply a red border E1 to the camera image and display a border image V1 to which a red warning display M1 has been added.

[0076] Furthermore, main control device 30 controls sound output unit 28 to output a warning sound when person P1 is closer than distance L1.

[0077] Furthermore, the main control device 30 controls the sound output unit 28 to change the warning sound depending on the distance to the person P1 detected by the detection device 10. For example, as shown in Fig. 6, the main control device 30 controls the sound output unit 28 to output an intermittent warning sound when the distance to the person P1 measured by the detection device 10 is closer than distance L1 and farther than distance LS. On the other hand, the main control device 30 controls the sound output unit 28 to output a continuous warning sound when the distance to the person P1 is closer than distance LS.

[0078] In this embodiment, distances L1, L2, and LS are provided with hysteresis when person P1 approaches work vehicle 100 and when person P1 moves away from work vehicle 100. In other words, for example, the distance at which the state changes from a first state in which warning display M1 is not added to camera image V2 to a second state in which warning display M1 is added to camera image V2 is different from the distance at which the state changes from the second state to the first state. Specifically, the distance at which the state changes from the second state to the first state is longer than the distance at which the state changes from the first state to the second state.

[0079] Specifically, for example, suppose that distance L2 is 10 m, distance L1 is 5 m, and distance LS is 3 m. For example, if infrared sensor 14 determines that the distance from work vehicle 100 to person P1 exceeds 10 m, warning display M1 is not added to the camera image displayed on display unit 22.

[0080] In this state, when person P1 approaches work vehicle 100 and infrared sensor 14 determines that the distance from work vehicle 100 to person P1 is 10 m, display unit 22 displays camera image V2 (with warning display M1 added).

[0081] If hysteresis were not provided, when person P1 moved even slightly away from work vehicle 100 in this state, infrared sensor 14 would determine that the distance from work vehicle 100 to person P1 was greater than 10 m, and camera image without warning display M1 would be displayed on display unit 22. Furthermore, when person P1 moved even slightly closer to work vehicle 100, camera image V2 (with warning display M1 added) would be displayed on display unit 22.

[0082] In this way, if hysteresis is not provided, when person P1 is located before or after distance L2, the camera image on display unit 22 may change rapidly, and the operator may not be able to accurately grasp the conditions around work vehicle 100.

[0083] Therefore, when person P1 who is within 10 m of work vehicle 100 moves away from work vehicle 100 by more than 10 m, main control device 30 sets distance L2 to 11 m. The same applies to distances L1 and LS.

[0084] Next, a safety monitoring system 1A according to an embodiment of the present invention will be described with reference to Fig. 7. Fig. 7 is a diagram showing a work vehicle 100 equipped with the safety monitoring system 1A, viewed from above.

[0085] The safety monitoring system 1A includes detection devices 10L, 10B, and 10R, a control device (main control device 30), and a display device (display unit 22). The detection devices 10L, 10B, and 10R each detect objects in a corresponding direction. Specifically, the detection device 10L is provided, for example, on the left side of the seat 210 in the upper rotating body 200, and detects the status of the left side of the work vehicle 100. The detection device 10L includes a camera 12L that captures images of the left side of the work vehicle 100, and an infrared sensor 14L. The multiple infrared sensors 14 included in the detection device 10L are collectively referred to as the infrared sensor 14L. The camera 12L captures images of the left side of the work vehicle 100 and generates a camera image. The infrared sensor 14L detects a traffic cone R1L located on the left side of the work vehicle 100 and calculates the distance to the traffic cone R1L.

[0086] Detection device 10B is provided, for example, behind seat 210 on upper rotating body 200, and detects the condition behind work vehicle 100. Detection device 10B has camera 12B that captures images behind work vehicle 100, and infrared sensor 14B. Note that the multiple infrared sensors 14 possessed by detection device 10B are collectively referred to as infrared sensor 14B. Camera 12B captures images behind work vehicle 100 and generates camera images. Infrared sensor 14B detects a traffic cone R1B located behind work vehicle 100 and calculates the distance to traffic cone R1B.

[0087] The detection device 10R is provided, for example, on the right side of the seat 210 in the upper rotating body 200, and detects the state of the right side of the work vehicle 100. The detection device 10R has a camera 12R that captures images of the right side of the work vehicle 100, and an infrared sensor 14R. The multiple infrared sensors 14 possessed by the detection device 10R are collectively referred to as the infrared sensor 14R. The camera 12R captures images of the right side of the work vehicle 100 and generates a camera image. The infrared sensor 14R detects a triangular cone R1R located on the right side of the work vehicle 100, and calculates the distance to the triangular cone R1R.

[0088] The operation of each of the detection devices 10L, 10B, and 10R is the same as the operation of the detection device 10 in the safety monitoring system 1.

[0089] Under the control of main control device 30, display unit 22 displays on one screen the camera images taken from each direction by cameras 12L, 12B, and 12R.

[0090] Specifically, main control device 30 acquires camera images captured by cameras 12L, 12B, and 12R, and controls display unit 22 to display the acquired camera images side by side on one screen. For example, the camera image captured by camera 12L is displayed in the upper left of display unit 22, the camera image captured by camera 12B is displayed in the lower center of display unit 22, and the camera image captured by camera 12R is displayed in the upper right of display unit 22.

[0091] In addition, when the detection devices 10L, 10B, and 10R detect the cones R1L, R1B, and R1R, respectively, the main control device 30 applies a border to the corresponding camera image or adds a warning display depending on the distance to the cones R1L, R1B, and R1R.

[0092] Specifically, for example, when the detection device 10L detects a traffic cone R1L and the distance to the traffic cone R1L is equal to or greater than distance L1 but less than distance L2 (Figure 4), the main control device 30 controls the display unit 22 to display, in the upper left corner of the display unit 22, a camera image V2L captured by the camera 12L with a "yellow" warning display M1L added to it.

[0093] Furthermore, for example, when the detection device 10B detects a traffic cone R1B and the distance to the traffic cone R1B is less than the distance L1 (Figure 4), the main control device 30 controls the display unit 22 to, for example, apply a red border processing E1B to the camera image captured by the camera 12B, and further to display a border image V1B with a red warning display M1B added in the lower center of the display unit 22.

[0094] Also, for example, when the detection device 10R detects a traffic cone R1R and the distance to the traffic cone R1R is equal to or greater than distance L1 but less than distance L2 (Figure 4), the main control device 30 controls the display unit 22 to display a camera image V2R captured by the camera 12R with a "yellow" warning display M1R added to it in the upper right corner of the display unit 22.

[0095] In this way, by detecting objects in multiple directions and making it possible to check objects in multiple directions on a single screen, it becomes easier for the operator to check their surroundings.

[0096] Next, a safety monitoring system 1B according to an embodiment of the present invention will be described with reference to Fig. 8 and Fig. 9. Fig. 8 is a diagram showing the safety monitoring system 1B. Fig. 9 is a block diagram showing the safety monitoring system 1B. The safety monitoring system 1B includes a detection device 10, a control device 301, and a display device 201.

[0097] The detection device 10 is mounted on, for example, a work vehicle 100. The work vehicle 100 and the detection device 10 are the same as the work vehicle 100 and the detection device 10 on which the safety monitoring system 1 is mounted.

[0098] The control device 301 is, for example, a server, desktop personal computer, or laptop personal computer installed in a location remote from the construction site where the work vehicle 100 is used. The control device 301 includes a control unit 321 and a memory unit 341. The control unit 321 includes a processor such as a CPU. The memory unit 341 includes a storage device and stores data and computer programs. The hardware configuration of the memory unit 341 includes a main storage device such as a semiconductor memory, and an auxiliary storage device such as a semiconductor memory, a solid state drive, and / or a hard disk drive. The memory unit 341 may include removable media.

[0099] The display device 201 is, for example, a liquid crystal display or an organic electroluminescence display installed in the same location as the control device 301 or in a location different from the control device 301. The display device 201 includes a display unit 221, a control unit 266, and a storage unit 267.

[0100] The display unit 221 displays various information relating to the work vehicle 100 including, for example, camera images taken by the camera 12 provided on the work vehicle 100 .

[0101] The storage unit 267 includes a storage device and stores data and computer programs. Specifically, the storage unit 263 includes a main storage device such as a semiconductor memory, and an auxiliary storage device such as a semiconductor memory, a solid-state drive, and / or a hard disk drive. The storage unit 267 may also include removable media.

[0102] The control unit 266 includes a processor such as a CPU (Central Processing Unit). The control unit 266 includes a display control unit 268. Specifically, the processor of the control unit 266 functions as the display control unit 268 by executing a computer program stored in a storage device of the storage unit 267. The display control unit 268 controls the display unit 221.

[0103] The work vehicle 100, the control device 301, and the display device 201 transmit and receive information to and from each other via a network N1 such as a wired LAN (Local Area Network) or a wireless LAN.

[0104] For example, the main control device 30 of the work vehicle 100 transmits camera image data showing the camera image captured by the detection device 10 (camera 12), the detection result of the detection device 10 (infrared sensor 14), and the distance to the object measured by the detection device 10 (infrared sensor 14) to the control device 301. In the example shown in Figure 8, it is assumed that the detection device 10 detects a traffic cone R1, and the distance to the traffic cone R1 is less than distance L1 (Figure 4).

[0105] The control unit 321 of the control device 301 receives the camera image data transmitted from the main control device 30, the detection results of the detection device 10 (infrared sensor 14), and the distance to the object measured by the detection device 10 (infrared sensor 14).

[0106] The control unit 321 generates processed camera video data by applying a border processing to the camera video represented by the camera video data and adding a warning display based on the received camera video data, the detection result of the detection device 10 (infrared sensor 14), and the distance to the object measured by the detection device 10 (infrared sensor 14). In the example shown in Fig. 8, the control unit 321 applies a "red" border processing E3 to the received camera video, and generates processed camera video data VD including a border image V3 to which a "red" warning display M3 is further added.

[0107] The control unit 321 transmits the generated processed camera video data VD to the display device 201.

[0108] The display control unit 268 of the display device 201 receives the processed camera video data VD transmitted from the control device 301. The display control unit 268 controls the display unit 221 to display the camera video indicated by the received processed camera video data VD. In the example shown in Fig. 8, the display control unit 268 controls the display unit 221 to display an outlined image V3.

[0109] In this way, by making it easier to notice changes in the conditions around the work vehicle 100 from a location away from the construction site where the work vehicle 100 is used, safety monitoring can be performed more reliably at the construction site.

[0110] The work vehicle 100 in the safety monitoring system 1B may be, for example, a work vehicle 100 equipped with the safety monitoring system 1A shown in Fig. 7. In this case, the display device 201 displays the same screen as that displayed on the display unit 22 shown in Fig. 7.

[0111] Next, a safety monitoring system 1C according to an embodiment of the present invention will be described with reference to Fig. 10. Fig. 10 is a diagram showing the safety monitoring system 1C. The safety monitoring system 1C includes a plurality of detection devices 10L, 10B, and 10R, a control device 301, and a plurality of display devices 201L, 201B, and 201R. The detection devices 10L, 10B, and 10R are the same as the detection devices 10L, 10B, and 10R in the work vehicle 100 equipped with the safety monitoring system 1A shown in Fig. 7. The control device 301 is the same as the control device 301 shown in Figs. 8 and 9. Each of the display devices 201L, 201B, and 201R is the same as the display device 201 shown in Figs. 8 and 9.

[0112] Display devices 201L, 201B, 201R each display various information related to work vehicle 100, including camera footage captured by the corresponding camera 12 installed on work vehicle 100. Specifically, display device 201L displays various information related to work vehicle 100, including camera footage captured by camera 12L. Display device 201B displays various information related to work vehicle 100, including camera footage captured by camera 12B. Display device 201R displays various information related to work vehicle 100, including camera footage captured by camera 12R.

[0113] For example, the main control device 30 of the work vehicle 100 transmits to the control device 301 camera image data showing camera images captured by each detection device 10 (cameras 12L, 12B, 12R), the detection results of each detection device 10 (infrared sensors 14L, 14B, 14R), and the distances to objects measured by each detection device 10 (infrared sensors 14L, 14B, 14R). In the example shown in FIG. 10, it is assumed that detection device 10L detects a traffic cone R1L, and the distance to the traffic cone R1L is equal to or greater than distance L1 but less than distance L2 (FIG. 4), detection device 10B detects a traffic cone R1B, and the distance to the traffic cone R1B is less than distance L1 (FIG. 4), and detection device 10R detects a traffic cone R1R, and the distance to the traffic cone R1R is less than distance L1 (FIG. 4).

[0114] The control unit 321 of the control device 301 receives each camera image data transmitted from the main control device 30, the detection results of each detection device 10 (infrared sensors 14L, 14B, 14R), and the distance to the object measured by each detection device 10 (infrared sensors 14L, 14B, 14R).

[0115] The control unit 321 generates processed camera image data by applying border processing to the camera image shown by the corresponding camera image data or adding a warning display based on the detection result of the corresponding detection device 10 (infrared sensor 14) and the distance to the object measured by the detection device 10 (infrared sensor 14) for each received camera image data.

[0116] In the example shown in Figure 10, the control unit 321 generates processed camera video data VDL that includes, for example, camera video V3L from the received camera video data, in which a yellow warning display M3L has been added to the camera video captured by camera 12L.

[0117] In addition, the control unit 321 applies a "red" border processing E3B to the camera image captured by camera 12B from among the received camera image data, and generates processed camera image data VDB including a border image V3B to which a "red" warning display M3B is further added.

[0118] Furthermore, the control unit 321 generates processed camera video data VDR including camera video V3R, which is camera video captured by the camera 12R and has a yellow warning display M3R added thereto, from among the received camera video data.

[0119] The control unit 321 transmits the generated processed camera video data to the corresponding display devices 201L, 201B, and 201R.

[0120] The display control unit 268 of the display device 201L receives the processed camera video data VDL transmitted from the control device 301. The display control unit 268 controls the display unit 221 to display the camera video indicated by the received processed camera video data VDL. In the example shown in Fig. 10, the display control unit 268 controls the display unit 221 to display the camera video V3L.

[0121] The display control unit 268 of the display device 201B receives the processed camera video data VDB transmitted from the control device 301. The display control unit 268 controls the display unit 221 to display the camera video indicated by the received processed camera video data VDB. In the example shown in Fig. 10, the display control unit 268 controls the display unit 221 to display the bordered image V3B.

[0122] The display control unit 268 of the display device 201R receives the processed camera video data VDR transmitted from the control device 301. The display control unit 268 controls the display unit 221 to display the camera video indicated by the received processed camera video data VDR. In the example shown in Fig. 10, the display control unit 268 controls the display unit 221 to display the camera video V3R.

[0123] In this way, by using multiple display devices to enable checking of the surroundings of the work vehicle 100 at a location away from the construction site where the work vehicle 100 is used, safety monitoring can be performed more reliably, for example, when the work vehicle 100 is remotely operated.

[0124] Next, a safety monitoring method according to an embodiment of the present invention will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the safety monitoring method according to an embodiment of the present invention.

[0125] The detection device 10 (camera 12) captures a camera image of the surroundings (step S11).

[0126] The detection device 10 (infrared sensor 14) detects surrounding objects (step S12). If the detection device 10 does not detect surrounding objects (No in step S12), the display unit 22 displays the camera image captured by the camera 12 (step S22).

[0127] If the detection device 10 detects an object in the vicinity (Yes in step S12), the infrared sensor 14 measures the distance L to the detected object (step S13).

[0128] Main control device 30 applies a border to the camera image or adds a warning display according to distance L measured by infrared sensor 14. Main control device 30 determines whether distance L is equal to or greater than distance L2 (FIG. 4) (step S14).

[0129] If the distance L is equal to or greater than the distance L2 (Yes in step S14), the display unit 22 displays the camera image captured by the camera 12 (step S22).

[0130] On the other hand, if distance L is less than distance L2 (No in step S14), main control device 30 determines whether distance L is equal to or greater than distance L1 (FIG. 4) (and less than distance L2) (step S15). If distance L is equal to or greater than distance L1 (FIG. 4) (and less than distance L2) (Yes in step S15), main control device 30 adds, for example, a "yellow" warning display M1 to the camera image (step S16). Display unit 22 displays the camera image with the "yellow" warning display M1 added (step S22).

[0131] Also, if the distance L is less than the distance L1 (No in step S15), the main control unit 30 applies a "red" border processing E1 to the camera image captured by the camera 12, and generates a border image V1 to which a "red" warning display M1 is further added (step S17).

[0132] Furthermore, infrared sensor 14 determines whether the detected object is a person or not based on the intensity of the infrared light reflected from the detected object (step S18). If the object detected by infrared sensor 14 is not a person but an object (No in step S18), display unit 22 displays the camera image including bordered image V1 (step S22).

[0133] On the other hand, if the object detected by infrared sensor 14 is a person (Yes in step S18), main control device 30 determines whether distance L is equal to or greater than distance LS (FIG. 6) (and less than distance L1) (step S19).

[0134] If the distance L is equal to or greater than the distance LS (Yes in step S19), the sound output unit 28 outputs an intermittent warning sound (step S20). The display unit 22 displays the camera image including the outlined image V1 (step S22).

[0135] On the other hand, if the distance L is less than the distance LS (No in step S19), the sound output unit 28 outputs a continuous warning sound (step S21). The display unit 22 displays the camera image including the outlined image V1 (step S22).

[0136] In this embodiment, an object is detected by an infrared sensor 14 that uses infrared light, but this is not limiting, and an object may be detected by, for example, processing a camera image.

[0137] The embodiments of the present invention have been described above with reference to the drawings (FIGS. 1 to 11). However, the present invention is not limited to the above-described embodiments and can be embodied in various forms without departing from the spirit and scope of the present invention. The drawings mainly show each component in a schematic manner for ease of understanding, and the thickness, length, number, etc. of each component shown in the drawings may differ from the actual components due to the convenience of creating the drawings. Furthermore, the materials, shapes, dimensions, etc. of each component shown in the above-described embodiments are merely examples and are not particularly limited, and various modifications are possible within a scope that does not substantially depart from the effects of the present invention. [Industrial Applicability]

[0138] The present invention can be used in fields such as construction machines and work vehicles at construction sites. [Explanation of symbols]

[0139] 1, 1A, 1B, 1C: Safety monitoring system 10, 10B, 10L, 10R: Detector 12, 12B, 12L, 12R: Camera 14, 14A, 14B, 14C, 14L, 14R: Infrared sensor 20: Operating device 22, 221, 221B, 221L, 221R: Display section 26, 261: Display control device 28: Sound output section 30: Main control unit 32, 266, 321: Control section 100: Work vehicle 201, 201B, 201L, 201R: Display device 265, 268: Display control unit 301: Control device E1, E1B, E3, E3B: Hemming L, L1, L2, LA, LB, LC, LS: Distance M1, M1B, M1L, M1R, M3, M3B, M3L, M3R: Warning display P1 :Person R1, R1B, R1L, R1R: Triangular cone V1, V1B, V3, V3B: border images V2, V2L, V2R, V3L, V3R: Camera footage

Claims

1. a detection device for detecting objects around the construction machine; a display device that displays a captured image of the surroundings of the construction machine, When the detection device detects the object, the display device is controlled to display a bordered image in which a border is applied to at least a part of the periphery of the surrounding image; The border processing is applied to the top and bottom edges of the screen displayed on the display unit of the display device. A method for controlling a safety monitoring system.

2. The method for controlling a safety monitoring system according to claim 1 , further comprising the step of controlling the display device so as to add a warning display to the image of the surroundings when the detection device detects the object.

3. the detection device measures the distance to the object; The method for controlling a safety monitoring system according to claim 2 , further comprising changing a color of the edging and a color of the warning display in accordance with the distance measured by the detection device.

4. The method for controlling a safety monitoring system according to claim 3 , wherein the border processing includes the same color as the warning display.

5. The method for controlling a safety monitoring system according to claim 2 , wherein the warning display is superimposed on the image of the surroundings.

6. The method for controlling a safety monitoring system according to claim 1 , further comprising determining whether the object is a person.

7. The construction machine includes a lower traveling body and an upper rotating body rotatably disposed above the lower traveling body, 7. The method for controlling a safety monitoring system according to claim 1, wherein the image of the surroundings is an image taken by a camera installed on the upper rotating body.

8. a detection device for detecting objects around the construction machine; a display device that displays the captured image of the surroundings of the construction machine; a control device that controls the detection device and the display device; Equipped with the control device controls the display device to display a bordered image in which a border is applied to at least a part of the periphery of the surrounding image when the detection device detects the object; The border processing is applied to the top and bottom edges of the screen displayed on the display unit of the display device. Safety monitoring system.

Images