Video recording system

Abstract

A construction machine is provided with a camera that captures video of the surroundings of the construction machine and one or more types of sensors that detect the state of the construction machine. On the basis of the output of the sensors or the video of the surroundings of the construction machine captured by the camera, a sign that the construction machine will fall or a sign that part of the construction machine will come into contact with surrounding objects is detected, and the detection of this sign is used as a trigger to record video of the surroundings of the construction machine captured by the camera.

Description

Technical Field

[0001] The present invention relates to a video recording system, and is suitable for application to a video recording system that records the surrounding video of a construction machine, for example.

Background Art

[0002] In recent years, for the purpose of investigating the cause at the time of an accident, the number of drivers who attach a drive recorder to an automobile has been increasing. Here, a drive recorder refers to a video recording device that captures the front of the vehicle body during driving or while parked by a camera attached to the vehicle body and records the video obtained by the capture.

[0003] Also in recent years, not only the video of the front of the vehicle body of an automobile but also vehicle body data such as CAN (Controller Area Network) signals are associated with video data and stored in a video recording device. By storing not only video but also vehicle body data in this way, it becomes possible to verify a more detailed situation at the time of an accident based on these video data and vehicle body data.

[0004] Such a video recording device is considered to be effective not only for private automobiles but also for construction machines. By installing a video recording device on a construction machine and recording the surrounding video of the operating construction machine, it is considered possible to verify (for example, investigate the cause) of a so-called near miss accident based on the recorded surrounding video. Here, a "near miss accident" refers to a case that did not result in a serious accident but was a step before an accident that could have been directly related to the accident.

[0005] In addition, Patent Document 1 below discloses that, regarding a drive recorder, the scale of an accident is grasped based on acceleration in the X - Y - Z directions detected by a collision sensor, and the data amount of a file to be stored is variably set according to the grasped scale of the accident.

Prior Art Documents

Patent Documents

[0006] [Patent Document 1] Patent No. 4394780 specification [Overview of the project] [Problems that the invention aims to solve]

[0007] By the way, when installing a video recording device on construction machinery, if the device is to record video data and vehicle data for the entire duration of the machinery's operation, this data can overwhelm the storage capacity of the video recording device, potentially leading to the overwriting of necessary data or increasing the workload required to extract data when near-miss incidents occur.

[0008] Therefore, when installing video recording devices on construction machinery, it is desirable to set a trigger to start recording only when recording is necessary, thereby preventing the recording of unnecessary video data unrelated to near misses and facilitating the extraction of near miss cases.

[0009] In this regard, Patent Document 1 discloses saving recorded video using an impact on the vehicle body detected by an acceleration sensor as a trigger. However, construction machinery is constantly subjected to impacts and large vibrations due to its operation, and it is not uncommon for impacts equivalent to those of an accident to occur even during normal operation. Therefore, it is not appropriate to use the same trigger for starting recording in construction machinery as in automobiles.

[0010] This invention was made in consideration of the above points, and aims to propose a video recording system that facilitates the extraction of surrounding video footage when incidents such as near misses occur, thereby simplifying the verification process for near misses and similar incidents. [Means for solving the problem]

[0011] To solve these problems, the present invention provides a video recording system for recording images of the surroundings of a construction machine, comprising: a camera for photographing the surroundings of the construction machine; one or more types of sensors for detecting the state of the construction machine; a gate lock device for switching between a state in which the operation of the construction machine is permitted and a state in which it is prohibited; a controller that detects signs of the construction machine about to tip over or signs of a part of the construction machine about to come into contact with surrounding objects based on the output of the sensors or images of the surroundings of the construction machine taken by the camera, and generates a recording trigger signal when such signs are detected, provided that recording start conditions set based on the state of the construction machine based on body data including the output of the sensors are met and the operation of the construction machine is permitted by the gate lock device; and a video recording device that has a storage device for temporarily storing the body data and video data consisting of images of the surroundings of the construction machine taken by the camera when the power of the construction machine is turned on, and when it receives the recording trigger signal from the controller, it files the body data and video data temporarily stored in the storage device as signs detection data and stores it in the storage device.

[0012] According to the video recording system of the present invention, the timing of recording video of the surroundings of construction machinery can be restricted when an operation is performed that may cause the construction machinery to tip over or a part of the construction machinery to come into contact with surrounding objects, that is, when an operation that results in a near-miss occurs. [Effects of the Invention]

[0013] According to the present invention, it is possible to realize a video recording system that facilitates the extraction of surrounding video footage when incidents such as near misses occur, thereby simplifying the verification work for near misses and similar incidents. [Brief explanation of the drawing]

[0014] [Figure 1] This is a side view showing the configuration of a construction machine equipped with the operational information recording system according to this embodiment. [Figure 2]It is a block diagram showing the configuration of the operation information recording system according to this embodiment. [Figure 3] (A) is a diagram showing the state of the gate lock device, (B) is a diagram showing the recording trigger signal, and (C) is a diagram for explaining the on / off timing of the recording function. [Figure 4] It is a flowchart showing the processing procedure of the recording process. [Figure 5] It is a flowchart showing the processing procedure of the recording trigger signal generation process. [Figure 6] (A) and (B) are a top view and a side view respectively for explaining the operation restriction area of the construction machine. [Figure 7] It is a diagram showing an example of the screen configuration of the omen detection case list screen. [Figure 8] It is a diagram showing an example of the screen configuration of the case details screen.

Mode for Carrying Out the Invention

[0015] Hereinafter, based on the drawings, one embodiment of the present invention will be described in detail.

[0016] (1) Configuration of the operation information recording system according to this embodiment and the construction machine equipped with the same In FIG. 1, reference numeral 1 indicates a construction machine (hydraulic excavator) equipped with the operation information recording system according to this embodiment as a whole. This construction machine 1 includes a self-propelled lower traveling body 2, an upper revolving body 3 rotatably mounted on the lower traveling body 2, and a front device 4 connected to the front of the upper revolving body 3.

[0017] The upper revolving body 3 is equipped with an engine 5 as a power source and a main pump 6 driven by the engine 5. The hydraulic oil sent out by the main pump 6 can independently operate the lower traveling body 2, the upper revolving body 3, and the front device 4 respectively.

[0018] Actually, a pair of traveling hydraulic motors 7 driven by the hydraulic oil sent out from the main pump 6 are mounted on the lower traveling body 2 at the left and right rear sides. By driving these traveling hydraulic motors 7, the crawlers 8 provided on both sides in the left - right direction of the lower traveling body 2 can be independently rotationally driven. Thereby, the lower traveling body 2, and thus the entire construction machine 1, can be traveled in the front - rear direction or swiveled in the left - right direction.

[0019] Also, the upper slewing body 3 is connected to the lower traveling body 2 via a slewing hydraulic motor 9 operated by the hydraulic oil sent out from the main pump 6. By driving the slewing hydraulic motor 9, the upper slewing body 3 can be swiveled in the left - right direction with respect to the lower traveling body 2.

[0020] The front device 4 is a working device that directly performs operations such as excavation or leveling of the ground, or lifting of excavated soil or other loads. The front device 4 includes a boom 11 whose one end is connected to the upper slewing body 3 so as to be rotatable in the front - rear direction of the upper slewing body 3 via a rotating shaft 10, an arm 13 whose one end is rotatably connected to the other end of the boom 11 via a rotating shaft 12, a bucket 15 rotatably connected to the other end of the arm 13 via a rotating shaft 14, and boom cylinders 16, arm cylinders 17, and bucket cylinders 18 that can be telescoped by the hydraulic oil sent out from the main pump 6 respectively.

[0021] In this case, one end of the boom cylinder 16 is connected to the boom 11 via a rotating shaft 19, and the other end is connected to the upper slewing body 3 via a rotating shaft (not shown). Thus, by telescopically driving the boom cylinder 16, the boom 11 can be rotated around the rotating shaft 10.

[0022] Also, one end of the arm cylinder 17 is connected to the boom 11, and the other end is connected to the arm 13 via a rotating shaft 20. Thus, by telescopically driving the arm cylinder 17, the arm 13 can be rotated around the rotating shaft 12.

[0023] Furthermore, one end of the bucket cylinder 18 is connected to the arm 13, and the other end is connected to the bucket 15 via a link, so that the bucket 15 can be rotated around the rotation axis 14 by extending and retracting the bucket cylinder 18.

[0024] Furthermore, the travel hydraulic motor 7, slewing hydraulic motor 9, boom cylinder 16, arm cylinder 17, and bucket cylinder 18 can be switched between a state in which operation is permitted and a state in which operation is prohibited by a gate lock device 45 (see Figure 2) provided in the operator's cab of the upper slewing body 3.

[0025] Figure 2 shows the configuration of the operational information recording system 30 of this embodiment, which is mounted on the construction machine 1. This operational information recording system 30 is a system for recording video data of the surroundings of the construction machine 1 and vehicle data for the time before and after a near-miss incident is detected while the construction machine 1 is in operation. In the following, a "near-miss incident" will be described as an example in which the operational information recording system 30 detects signs that the posture of the construction machine 1 is changing and that may lead to it tipping over, or signs that the front device 4 may come into contact with an object in the surroundings.

[0026] The operational information recording system 30 is mounted on the construction machine 1 and consists of various sensors such as multiple surrounding surveillance cameras 31, a tilt sensor 32, a slewing angle sensor 33, a boom angle sensor 34, an arm angle sensor 35, and a load sensor 36, as well as a monitor control controller 37, a monitor 38, a vehicle control controller 39, a video recording device 40, and a communication terminal 41, each located in the driver's seat 3A (Figure 1) of the upper slewing body 3. The monitor control controller 37, monitor 38, vehicle control controller 39, and video recording device 40 are also freely connected via an in-vehicle network 42 such as a CAN network.

[0027] Each of the surrounding surveillance cameras 31 is a camera designed to photograph the area around the construction machine 1, and multiple cameras are installed on top of the upper rotating body 3 to photograph the left and right sides, the front, and the rear. Each surrounding surveillance camera 31 outputs video data of the surrounding image of the construction machine 1 obtained through photography to the monitor control controller 37.

[0028] The tilt sensor 32 and the rotation angle sensor 33 are each installed on the upper rotating body 3. The tilt sensor 32 detects the tilt angles of the upper rotating body 3 in the front-rear and left-right directions (i.e., the pitch angle and roll angle of the upper rotating body 3 with the front direction of the upper rotating body 3 as the roll axis direction) and outputs these detected tilt angles to the vehicle control controller 39. The rotation angle sensor 33 detects the rotation angle of the upper rotating body 3 relative to the lower traveling body 2 and outputs the detected rotation angle to the vehicle control controller 39.

[0029] The boom angle sensor 34, arm angle sensor 35, and load sensor 36 are each installed on the front device 4. The boom angle sensor 34 detects the posture of the boom 11 as an inclination angle from a reference state (hereinafter referred to as the boom inclination angle) and outputs the detected boom inclination angle to the vehicle control controller 39.

[0030] The arm angle sensor 35 also detects the posture of the arm 13 as an inclination angle from a reference state relative to the boom 11 (hereinafter referred to as the arm inclination angle), and outputs the detected arm inclination angle to the vehicle control controller 39. Furthermore, the load sensor 36 detects the load of the suspended load lifted by the front device 4, and outputs the detected load to the vehicle control controller 39.

[0031] The monitor control controller 37 generates a composite image based on the video data provided by each surrounding surveillance camera 31. This composite image is either a split image, where the surrounding images captured by each surrounding surveillance camera 31 are displayed in sections, or a surround-view image, where the boundaries of surrounding images captured by adjacent surrounding surveillance cameras 31 are blended. The monitor control controller 37 then transmits the generated composite image data to the monitor 38 and the video recording device 40 via the communication line 43.

[0032] The monitor 38 is composed of a display device such as a liquid crystal panel or an organic EL (Electro-Luminescence) panel. The monitor 38 displays a composite image based on video data provided from the monitor control controller 37 via the communication line 43.

[0033] The vehicle control controller 39 transmits sensor data from the tilt sensor 32, slewing angle sensor 33, boom angle sensor 34, arm angle sensor 35, and load sensor 36 to the video recording device 40 via the in-vehicle network 42. The vehicle control controller 39 also monitors this sensor data, and when the gate lock device 45 located in the driver's seat 3A is unlocked, if it detects signs that the construction machine 1 is about to tip over or that the front device 4 is about to come into contact with an object, it transmits a recording trigger signal to the video recording device 40 via the in-vehicle network 42 to trigger the start of recording. Details of "signs that the posture of the construction machine 1 is about to change and lead to tipping over or signs that the front device 4 is about to come into contact with an object" will be described later.

[0034] Although not shown in the diagram, the vehicle control controller 39 is also provided with sensor data representing the hydraulic pressure of the hydraulic fluid supplied to the travel hydraulic motor 7, slewing hydraulic motor 9, boom cylinder 16, arm cylinder 17, and bucket cylinder 18, respectively, as detected by hydraulic sensors attached to these components as described in Figure 1. The vehicle control controller 39 then transmits this sensor data from the hydraulic sensors to the video recording device 40 via the in-vehicle network 42.

[0035] The video recording device 40 is configured to include semiconductor memory such as flash memory, and storage devices such as a hard disk drive or SSD (Solid State Drive). As described above, the video recording device 40 is supplied with video data of the composite image from the monitor control controller 37 and sensor data from various sensors from the vehicle body control controller 39. In addition, the video recording device 40 is sequentially supplied with engine speed data representing the rotational speed of the engine 5 (Figure 1) from an ECU (Electronic Control Unit) (not shown).

[0036] The video recording device 40 then temporarily stores sequential video data of the composite image provided by the monitor control controller 37 for a certain period of time immediately prior to the date. At the same time, it stores sensor data from various sensors provided by the vehicle control controller 39 and engine speed data provided by the ECU as vehicle data, linking these to the composite image generated at the same time, and temporarily storing them sequentially for the same period of time immediately prior to the date to the date of the composite image.

[0037] Furthermore, when the video recording device 40 receives a recording trigger signal from the vehicle control controller 39, it saves the most recent predetermined time (e.g., 30 seconds) of video data and vehicle data that it has temporarily stored at that time, as well as the subsequent predetermined time (e.g., 30 seconds) of video data and vehicle data provided by the monitor control controller 37, vehicle control controller 39, and ECU, etc., as predictive detection data in a file. Hereinafter, this operation of saving predictive detection data for a predetermined time in a file will be referred to as recording.

[0038] At this time, the video recording device 40 also stores the current position of the construction machine 1 at that time, as notified by the communication terminal 41 as described later, in the same predictive detection data file. The video recording device 40 then uploads (transmits) the saved predictive detection data file to the external server device 47 via wireless communication through the communication terminal 41 and the antenna 46 provided on the upper rotating body 3 at appropriate timings.

[0039] The communication terminal 41 is a communication device capable of bidirectional communication with the external server device 47 via a wireless communication line such as a mobile phone line. This allows the operational information recording system 30 to upload the aforementioned files to the external server device 47, and also allows the external server device 47 to access the operational information recording system 30.

[0040] This communication terminal is equipped with a GPS (Global Positioning System) receiver. The communication terminal then sequentially detects its current location based on the GPS signal received by this GPS receiver and notifies the video recording device 40 of the detected current location. Thus, this current location is stored in the data file for predictive detection, as described above, and uploaded to the external server device 47.

[0041] The external server device 47 is a general-purpose server device equipped with a CPU (Central Processing Unit), a main memory device such as semiconductor memory, an auxiliary memory device such as a hard disk drive, an input device such as a mouse and keyboard, and a display device such as a liquid crystal display or an organic EL display, and is installed separately from the construction machine 1.

[0042] The external server device 47 stores and manages files of predictive detection data transmitted as appropriate from one or more construction machines 1 under its management. Based on the stored predictive detection data, the external server device 47 displays, in response to a predetermined operation by the user, a composite image of when the vehicle control controller 39 of each construction machine 1 detects a change in the posture of the construction machine 1 that could lead to an overturning or a sign that the front device 4 is about to come into contact with an object, as well as other necessary information.

[0043] In this embodiment, a single video recording system is constructed using an external server device 47 and an operation information recording system 30 for one or more construction machines 1 under the management of the external server device 47. The user of this video recording system can use the predictive detection data stored in the external server device 47 to more thoroughly verify the operator's actions that led to the detection of a change in the posture of the construction machine 1 that could lead to an overturning or a sign that the front device 4 would come into contact with a surrounding object, using statistical methods, and the verification results can be used for future training of the operators of the construction machine 1.

[0044] (2) Relationship between recording trigger signal and predictive detection data Figure 3 shows the state of the gate lock device 45 and the timing of recording data when an anomaly is detected. When the gate lock device 45 is unlocked and operation is permitted as shown in Figure 3(A), the vehicle control controller 39 detects an anomaly of the construction machine 1 tipping over or the front device 4 contacting an object in the surrounding area, and transmits a recording trigger signal SG to the video recording device 40, which is valid for a predetermined time t, as shown in Figure 3(B).

[0045] The video recording device 40 then starts recording when the recording trigger signal SG becomes active, and as shown in Figure 3(C), it records video data and vehicle data for a predetermined time T1 minutes before the recording trigger signal SG becomes active, and video data and vehicle data for a predetermined time T2 minutes after that time, as data recorded at the time of the anomaly detection. The value of the predetermined time T2 is set to a value greater than the predetermined time t.

[0046] The video recording device 40 continues recording one session ((T1+T2) hours of recording) even if the recording trigger signal SG switches back from disabled to enabled, ignoring this until the recording is complete.

[0047] (3) Processing of video recording device and vehicle control controller related to recording Figure 4 shows the flow of a series of processes (hereinafter referred to as the recording process) performed by the video recording device 40 in connection with the recording of surrounding images of the construction machine 1 as described above. This recording process starts when the power to the vehicle body is turned on, such as when the engine of the construction machine 1 is started, and the power to the entire operation information recording system 30 (Figure 2) is turned on accordingly, and the video recording device 40 is ready to record.

[0048] The video recording device 40 first begins to temporarily store video data provided by the monitor control controller 37, etc., and vehicle data provided by the vehicle control controller 39 and ECU, etc. (S1), and then determines whether or not the recording trigger signal provided by the vehicle control controller 39 is valid (S2).

[0049] If the video recording device 40 obtains a negative result in this determination, it determines whether or not the vehicle's power has been turned off (S6). If the video recording device 40 obtains a negative result in this determination, it returns to step S2, and thereafter repeats the loop of steps S2-S6-S2 until the recording trigger signal becomes valid or the vehicle's power is turned off.

[0050] Then, when the recording trigger signal becomes effective, the video recording device 40 starts saving the video data and vehicle data that it has temporarily stored from a time T1 prior to the time the recording trigger signal became effective (S3).

[0051] Next, the video recording device 40 waits for time T2 to elapse from the moment the recording trigger signal becomes active (S4). When time T2 has elapsed from the moment the recording trigger signal became active, the video recording device 40 completes saving the video data and vehicle data, and files the video data and vehicle data for the (T1+T2) hours saved up to that point, along with the location data of the vehicle's current position notified by the communication terminal 41 at that time, as predictive detection data (S5).

[0052] Next, the video recording device 40 determines whether or not the vehicle's power has been turned off (S6). If the video recording device 40 obtains a negative result in this determination, it returns to step S2, and thereafter repeats the processing from step S2 onward in the same manner as described above.

[0053] Furthermore, when the vehicle's power is eventually turned off, the video recording device 40 obtains a positive result in step S6 and turns off its own power (S7). This completes the recording process.

[0054] On the other hand, Figure 5 shows a series of processes performed by the vehicle control controller 39 in parallel with the recording process described above in Figure 4, regarding the recording of such video data and vehicle data. The vehicle control controller 39 generates a recording trigger signal according to the processing procedure shown in Figure 5 and transmits the generated recording trigger signal to the video recording device 40.

[0055] In practice, when the vehicle control controller 39 is powered on, it starts the process shown in Figure 5 (hereinafter referred to as the recording trigger signal generation process), and first determines whether predetermined conditions (hereinafter referred to as the recording start conditions) that allow this recording trigger signal generation process to start are met, such as the posture of the front device 4 and the rotational speed of the engine 5 (S10).

[0056] These recording start conditions can include, for example, whether the posture of the front device 4 and the rotation speed of the engine 5 are within a predetermined range, or whether the current position of the construction machine 1 is a pre-specified location. If the vehicle control controller 39 obtains a negative result from this determination, it waits for the recording start conditions to be met.

[0057] Then, when the recording start conditions are met, the vehicle control controller 39 first determines whether or not the gate lock device 45 (Figure 2) has been unlocked (S11). If the vehicle control controller 39 obtains a negative result in this determination, it returns to step S10, and thereafter executes the processing from step S10 onward in the same manner as described above.

[0058] In response, if the vehicle control controller 39 obtains a positive result in the judgment in step S11, it determines whether it has detected any signs that the construction machine 1 is about to tip over or that the front device 4 is about to come into contact with an object, based on the sensor data provided by the tilt sensor 32, slewing angle sensor 33, boom angle sensor 34, arm angle sensor 35, and load sensor 36, respectively (S12).

[0059] If the vehicle control controller 39 obtains a negative result in this determination, it returns to step S10 and repeats the processing from step S10 onward in the same manner as described above. Conversely, if the vehicle control controller 39 obtains a positive result in the determination in step S12, it activates the recording trigger signal for a time t (S13).

[0060] Next, the vehicle control controller 39 determines whether it is still detecting signs that the construction machine 1 is about to tip over or that the front device 4 is about to come into contact with an object (S14). If the vehicle control controller 39 obtains a positive result in this determination, it waits for the detection of such signs to cease.

[0061] The vehicle control controller 39 then stops detecting such warning signs, and when it obtains a positive result in the judgment of step S14, it returns to step S10, and thereafter repeats the processing from step S10 onwards in the same manner as described above.

[0062] Here, we will explain the "signs that indicate a change in the posture of the construction machine 1 leading to an overturning, or that indicate the front device 4 will come into contact with an object." Multiple such "signs" can be set for the vehicle control controller 39.

[0063] For example, one such "warning sign" can be set as "the attitude (tilt) of the upper rotating body 3 has reached a threshold angle set in advance for at least one of the tilt angles in the front-rear direction and the left-right direction."

[0064] In this case, the construction machine 1 of this embodiment is originally equipped with a vehicle control controller 39 that, when the inclination of the vehicle body exceeds a certain threshold angle, warns the operator in the driver's seat 3A by sound or by displaying an image on the monitor 38, indicating that the construction machine 1 may tilt and tip over. In this embodiment, when issuing a warning to the operator in this manner, the vehicle control controller 39 also activates a recording trigger signal.

[0065] The file of data recorded by the video recording device 40 upon detection of this "precursor" includes not only the sensor data output from the tilt sensor 32 at that time, but also the sensor data from the various hydraulic sensors mentioned above, which serve as the basis for estimating the operator's actions on the construction machine 1. This allows for later verification, based on this precursor detection data, of what actions by the operator caused the construction machine 1 to tilt.

[0066] A second "warning sign" can be set as "the magnitude of the load applied to the front device 4 by the suspended load detected by the load sensor 36 of the front device 4 has exceeded a predetermined threshold (hereinafter referred to as the load threshold) for that magnitude of load."

[0067] In this case, the construction machine 1 of this embodiment is originally equipped with a function in the vehicle control controller 39 that warns the operator in the driver's seat 3A by sound or by displaying an image on the monitor 38 if the load lifted by the front device 4 exceeds a load threshold and the tipping moment of the vehicle body becomes excessive due to the posture of the front device 4, indicating that the construction machine 1 may tip over.

[0068] Therefore, in this embodiment, when a warning is issued to the operator in this manner, the vehicle control controller 39 also activates a recording trigger signal. The file of data recorded by the video recording device 40 upon detection of this "precursor" includes not only the magnitude of the load from the suspended load detected by the load sensor 36 at that time, but also sensor data from various hydraulic sensors (in this case, hydraulic sensors attached to each cylinder such as the boom cylinder 16) which serve as the basis for estimating the operator's actions on the front device 4. This makes it possible to later verify, based on this precursor detection data, what actions by the operator caused a load exceeding the load threshold to be applied to the front device 4 of the construction machine 1.

[0069] Furthermore, a third "warning sign" can be set as "a part of the front device 4 has exceeded a pre-set operating limit range."

[0070] In this embodiment, the construction machine 1 allows the operator to set operating restriction areas for the front device 4 to the vehicle control controller 39 so that the vehicle body does not come into contact with surrounding objects. Specifically, as shown in Figure 6, the operator can set a forward restriction area 50 in front of the lower traveling body 2, a height restriction area 51 in the height direction, a depth restriction area 52 in the depth direction, a rightward restriction area 53 to the right, and a leftward restriction area 54 to the left, all with respect to the lower traveling body 2.

[0071] In this case, the vehicle control controller 39 is equipped with a function that constantly calculates the attitude of the front device 4 based on sensor data provided by the slewing angle sensor 33, boom angle sensor 34, and arm angle sensor 35, and if it determines that a part of the front device 4 has exceeded the forward restriction area 50, height restriction area 51, depth restriction area 52, rightward restriction area 53, or leftward restriction area 54, it issues a warning to the operator in the driver's seat 3A by sound or by displaying an image on the monitor 38, and also stops the operation of the front device 4.

[0072] Therefore, in this embodiment, when a warning is issued to the operator in this manner, the vehicle control controller 39 also activates a recording trigger signal. The file of data recorded by the video recording device 40 upon detection of this "precursor" includes information calculated by the vehicle control controller 39 regarding which direction the front device 4 has exceeded its operating limit range, as well as sensor data corresponding to the operation of rotating the upper rotating body 3 and various operations performed by the operator on the front device 4. This allows for later verification, based on this precursor detection data, of what operator operation caused a part of the front device 4 to exceed its operating limit range.

[0073] (4) Configuration of various screens displayed on the external server device Figure 7 shows the screen configuration of the predictive detection case list screen 60, which can be displayed on the external server device 47 (Figure 2) by a predetermined operation, based on the predictive detection data uploaded from the construction machine 1 to the external server device 47 (Figure 2) as described above.

[0074] This premonitory detection case list screen 60 displays a list of information regarding each case in which a premonitory sign leading to a fall due to a change in the posture of the construction machine 1 or a premonitory sign that the front device 4 will come into contact with a surrounding object was detected, based on the premonitory detection data that has been uploaded from each construction machine 1 to the external server device 47 up to that point.

[0075] Specifically, for each of these cases, the date and time of occurrence ("Date and Time of Occurrence"), the type of warning issued to the operator when the predictive detection data was saved ("Event"), the details of the operation performed by the operator at that time as recognized based on the sensor data from each hydraulic sensor ("Operation"), the model of the construction machine 1 in which the accident occurred ("Model"), and a thumbnail image 61 of the first still image of the composite video based on the predictive detection data are displayed.

[0076] For example, Figure 7 shows an instance where, on April 7, 2021, at 12:45 PM, a "vehicle tilt warning" was issued during the "travel" operation of construction machine 1, model "Hydraulic Excavator B," indicating that the vehicle's posture tilted beyond the permissible angle. Also, Figure 7 shows an instance where, on April 3, 2021, at 6:30 PM, a "front load warning" was issued during the "slewing" operation of construction machine 1, model "Hydraulic Excavator A," indicating that the load of the suspended load on the front device 4 exceeded the aforementioned load threshold. Furthermore, Figure 7 shows an instance where, on April 1, 2021, at 3:00 PM, a "region limit warning" was issued during the "excavation" operation of construction machine 1, model "Hydraulic Excavator C," indicating that a portion of the front device 4 of construction machine 1 exceeded the limit area (for example, the depth limit area 52 in Figure 6).

[0077] Furthermore, on the Precursor Detection Case List screen 60, by clicking on the strings 62A~62D labeled "Date and Time of Occurrence," "Event," "Operation," and "Model" displayed at the top, the user can select a desired date or item from the dropdown menu (not shown) that appears, thereby limiting the range of cases displayed on the Precursor Detection Case List screen 60 to cases corresponding to that date or item.

[0078] For example, clicking on the string "Date and Time of Occurrence" 62A will display a dropdown menu containing a calendar. By using the calendar displayed in this dropdown menu to specify a date or date range, the cases displayed on the list of predictive detection cases screen 60 can be limited to cases where predictive detection occurred within the specified date or date range.

[0079] Furthermore, clicking the string "Event" 62B will display a pull-down menu listing all types of warnings issued by construction machine 1. By selecting a desired warning from this pull-down menu, the cases displayed on the predictive detection case list screen 60 can be limited to cases in which the specified warning was issued.

[0080] Furthermore, clicking the string "Operation" 62C displays a pull-down menu listing all types of operations that an operator can perform on construction machine 1. By selecting a desired operation from this pull-down menu, the cases displayed on the predictive detection case list screen 60 can be limited to cases where the specified operation was performed.

[0081] Furthermore, clicking the string "Model" 62D displays a pull-down menu listing all the models of construction machinery 1 managed by the external server device 47. By selecting a desired model from this pull-down menu, the cases displayed on the predictive detection case list screen 60 can be limited to cases that occurred on the construction machinery 1 of the specified model.

[0082] On the other hand, on the Anomaly Detection Case List screen 60, by double-clicking on the row or thumbnail image 61 corresponding to the desired case, a screen like the one shown in Figure 8 (hereinafter referred to as the Case Details screen) 70, based on the Anomaly Detection Case List screen 60, can be displayed in place of or overlaid on the Anomaly Detection Case List screen 60 that was previously displayed.

[0083] This case details screen 70 is comprised of a case overview display area 71, a map display area 72, a video display area 73, and an operation information display area 74. The case overview display area 71 displays an overview of the corresponding case, and the map display area 72 displays a bird's-eye view map 75 of the location where the corresponding case occurred, based on the location data included in the data at the time of the anomaly detection of the corresponding case.

[0084] The video display area 73 also displays the composite video 76 saved when the corresponding incident occurred. Figure 8 shows an example where the composite video 76 is a split video. The composite video 76 displayed in the video display area 73 can be started or stopped by pressing (clicking) the play button 77 or stop button 78 displayed below the composite video 76. The playback position of the composite video 76 can also be moved to a position corresponding to the slider 80 of the slider bar 79 displayed below the composite video 76 by dragging it left or right.

[0085] Furthermore, the operation information display area 74 displays the details of various operations performed by the operator before and after the date and time of the corresponding incident in graph format. In Figure 8, the engine speed ("engine speed") before and after the date and time of the corresponding incident is displayed in graph format. Also in Figure 8, the hydraulic pressure of the hydraulic fluid supplied to the swing hydraulic motor 9 (Figure 1) ("swing operation") is displayed in graph format as part of the swing operation, and the hydraulic pressure of the hydraulic fluid supplied to the travel hydraulic motor 7 (Figure 1) ("travel operation") is also displayed in graph format as part of the travel operation. In addition, in Figure 8, the status of the gate lock device 45 (Figure 2) ("gate lock") is displayed in the same format as in Figure 3(A).

[0086] (5) Effects of this embodiment As described above, in the operational information recording system 30 of this embodiment, the detection of signs that the construction machine 1 may overturn or that the front device 4 may come into contact with surrounding objects is treated as a near-miss incident, and video data of the composite image and vehicle data for a predetermined time before and after the detection are saved. Therefore, even in environments where a large impact occurs while the construction machine 1 is in operation, only the video data and vehicle data at the time of the near-miss incident can be recorded.

[0087] Furthermore, since the operational information recording system 30 can record only video data and vehicle data at the time of a near-miss incident, unnecessary data is not stored in the video recording device 40. This effectively prevents unnecessary data from overwhelming the storage capacity of the video recording device and overwriting necessary data.

[0088] Therefore, the operational information recording system 30 facilitates the extraction of surrounding video footage when a near-miss incident occurs, thereby simplifying the verification process for near-miss incidents.

[0089] (6) Other embodiments In the above-described embodiment, the case where the construction machine is a hydraulic excavator was mentioned, but the present invention is not limited to this and can be broadly applied to various other construction machines such as cranes and loaders.

[0090] Furthermore, although the above-described embodiment described a case where the predictive detection case list screen 60 and the case details screen 70 are configured as shown in Figures 7 and 8, the present invention is not limited to this, and various other configurations can be widely applied.

[0091] Furthermore, while the above-described embodiment mentions a case where the list of predictive detection cases 60 and the case details screen 70 can only be displayed on the external server device 47, the present invention is not limited to this. For example, the list of predictive detection cases 60 and the case details screen 70 can also be displayed on client terminals connected to the external server device 47 via a network such as the Internet. [Industrial applicability]

[0092] This invention can be widely applied to various construction machines. [Explanation of symbols]

[0093] 1...Construction machine, 2...Lower traveling body, 3...Upper slewing body, 4...Front device, 30...Operation information recording system, 31...Surrounding camera, 32...Tilt sensor, 33...Slewing angle sensor, 34...Boom angle sensor, 35...Arm angle sensor, 36...Load sensor, 37...Monitor control controller, 38...Monitor, 39...Vehicle control controller, 40...Video recording device, 41...Communication terminal, 45...Gate lock device, 47...External server device, 50...Forward restricted area, 51...Height restricted area, 52...Depth restricted area, 53...Right restricted area, 54...Left restricted area, 60...Predictive detection case list screen, 70...Case details screen.

Claims

1. In a video recording system that records images of the surroundings of construction machinery, A camera that photographs the area around the aforementioned construction machine, One or more types of sensors for detecting the state of the construction machine, A gate lock device that switches between a state in which the operation of the construction machine is permitted and a state in which it is prohibited, A controller that, based on the output of the sensor or the image of the surroundings of the construction machine captured by the camera, detects signs that the construction machine is about to tip over or that a part of the construction machine is about to come into contact with an object in the surroundings, and generates a recording trigger signal when the signs are detected, provided that the recording start conditions set based on the state of the construction machine based on vehicle data including the output of the sensor are met and the operation of the construction machine is permitted by the gate lock device, A video recording device having a storage device that temporarily stores the vehicle data and video data consisting of images of the surroundings of the construction machine captured by the camera when the power of the construction machine is turned on, and when it receives the recording trigger signal from the controller, it files the vehicle data and video data temporarily stored in the storage device as predictive detection data and stores it in the storage device. A video recording system characterized by having the following features.

2. The aforementioned sensor includes a tilt sensor that detects the tilt of the construction machine's body, The aforementioned signs that lead to the construction machine tipping over are, The tilt of the construction machine's body, as detected by the tilt sensor, exceeds a preset first threshold. The video recording system according to feature 1.

3. The aforementioned construction machine is a hydraulic excavator having a front device for performing work, The aforementioned sensor includes a load sensor that detects the load of the suspended load applied to the front device. The aforementioned signs that lead to the construction machine tipping over are, The load of the suspended load on the front device, as detected by the load sensor, exceeds a preset second threshold. The video recording system according to feature 1.

4. The aforementioned construction machine is a hydraulic excavator having a front device for performing work, The aforementioned sensor includes an attitude sensor for detecting the attitude of the front device, The signs that a part of the aforementioned construction machine is about to come into contact with an object are, The part of the front device has exceeded a pre-set operating limit area around the construction machine. The video recording system according to feature 1.

5. The construction machinery is further equipped with an external server device, which is provided separately from the aforementioned construction machinery. The aforementioned video recording device is The video data of the surroundings of the construction machine that was recorded is uploaded to the external server device. The external server device is Display the surrounding image of the construction machine based on the uploaded video data. The video recording system according to feature 1.

6. The construction machinery is further equipped with an external server device, which is provided separately from the aforementioned construction machinery. The video data is uploaded to the external server device from multiple construction machines. The external server device is The system displays the surrounding video based on the video data that matches the specified conditions from among the uploaded video data. The video recording system according to feature 1.

Images