Operation record analysis system for construction machine

Abstract

This invention provides an operation record analysis system for construction machinery, capable of efficiently extracting work content that may be the main cause of reduced operating rates from the operation information recorded during the operation of construction machinery. The system includes an object detection device for detecting objects present around the vehicle body. A controller calculates the position of the object based on information from the object detection device, determines whether the vehicle body is close to the object based on information from a vehicle body position detection device and a vehicle body posture detection device, and appends this determination result to the operation information.

Description

Technical Field

[0001] This invention relates to a system for analyzing the operating records of construction machinery, such as excavators, used in mines and other mining operations, in order to efficiently monitor the operating status of the machinery. Background Technology

[0002] As background technology in this field, there is Patent Document 1. Patent Document 1 describes "an excavator, an excavator management system and a portable communication terminal that can provide more detailed management of the excavator in the work site".

[0003] In previous operation record analysis systems, the work content estimation unit estimated the work content based on the motion status information output by the motion status information acquisition unit. Work content refers to the type of operation, such as excavation, soil removal, or movement of the excavator. This work content is then associated with work position information (indicating the excavator's location) and work height information (indicating the excavator's digging height), and recorded at regular intervals in a recording device, or via a communication device to an external server, or displayed on a display device. This allows for more detailed management of the excavator at the work site, knowing exactly what kind of work the excavator was performing at what time.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent No. 6144373 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] In excavator operations, if other operators enter the vicinity of the excavator's boom travel range or the swing radius of the upper slewing body, a portion of the excavator may come into contact with other operators. Assuming this occurs, the entire construction operation, including that of other machinery, must be halted for cause analysis, countermeasure formulation, application, and verification, thus reducing the excavator's operating rate. Therefore, operation managers need to manage the entire operation to prevent phenomena that lead to reduced operating rates. With conventional operation record analysis systems, operation content can be recorded on recording devices or external servers. However, the recorded data includes both potentially major causes of reduced operating rates and other unrelated content, with the former generally having a very low probability of occurrence. Furthermore, the amount of recorded data becomes enormous, corresponding to the overall operation time and the number of construction machines used. System users require considerable time to detect and extract only the potentially major causes of reduced operating rates from such a large volume of data, and sometimes omissions occur.

[0009] The present invention was made in view of the above-mentioned problems, and its purpose is to provide an operation record analysis system for construction machinery, which can efficiently extract the operation content that may be the main cause of the decrease in operation rate from the operation information recorded during the operation of construction machinery.

[0010] Methods for solving problems

[0011] To achieve the above objectives, the present invention provides an operation record analysis system for construction machinery, comprising: a vehicle body position detection device for detecting the position of the vehicle body; a vehicle body posture detection device for detecting the posture of the vehicle body; an action state detection device for detecting the action state of the vehicle body; and a controller for calculating the working position of the vehicle body based on information from the vehicle body position detection device and the vehicle body posture detection device, estimating the work content of the vehicle body based on the working position and the action state, and outputting operation information including the working position and the work content. The system is characterized in that it further comprises: an object detection device for detecting objects existing around the vehicle body; the controller for calculating the position of the object based on information from the object detection device; determining whether the vehicle body is close to the object based on information from the vehicle body position detection device and the vehicle body posture detection device and the position of the object; and including the determination result in the operation information and outputting it.

[0012] According to the present invention configured as described above, the determination of whether the vehicle body is close to surrounding objects is included in the operation information. Therefore, the system user can quickly extract the operation content when the vehicle body is close to surrounding objects from a large amount of operation information. This improves the efficiency of verifying the operating status of construction machinery.

[0013] Effects of the Invention

[0014] The operation record analysis system for construction machinery according to the present invention can efficiently extract the work content that may be the main cause of reduced operating rate from the operation information recorded during operation. Attached Figure Description

[0015] Figure 1 This is a structural diagram of an operation record analysis system for engineering machinery.

[0016] Figure 2 This diagram illustrates the processing flow in an operation record analysis system for construction machinery.

[0017] Figure 3 This is a diagram showing the status of the excavator.

[0018] Figure 4 This is a diagram illustrating an example of an excavator's working position and operational status.

[0019] Figure 5 This is a diagram showing the camera mounting status and camera images in an excavator.

[0020] Figure 6 This diagram illustrates the process of calculating the difference region using camera images.

[0021] Figure 7 This diagram illustrates the projection transformation process performed on the differential regions within a camera image.

[0022] Figure 8 This diagram illustrates the processing of overlapping regions in calculations.

[0023] Figure 9 This is a diagram showing an example of runtime information.

[0024] Figure 10 This is a diagram illustrating the display content of the display device in the first embodiment of the present invention.

[0025] Figure 11 This is a diagram illustrating the display content of the display device in the second embodiment of the present invention.

[0026] Figure 12 This is a diagram illustrating the display content of the display device in the third embodiment of the present invention.

[0027] Figure 13 This is a diagram illustrating the LiDAR detection process in the fourth embodiment of the present invention.

[0028] Figure 14 This is a diagram illustrating the LiDAR detection process in the fifth embodiment of the present invention.

[0029] Figure 15 This is a diagram illustrating the display content of the display device according to the fifth embodiment of the present invention. Detailed Implementation

[0030] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, in each drawing, equivalent components are labeled with the same reference numerals, and repeated descriptions are omitted where appropriate.

[0031] Example 1

[0032] In this embodiment, an example of an operation record analysis system for construction machinery is described, which extracts content that meets certain conditions from the operation records and displays it preferentially.

[0033] Figure 1 This is a structural diagram of the operation record analysis system for construction machinery according to this embodiment. Here, we will use an excavator as the example of construction machinery for explanation.

[0034] The vehicle position detection device 101, for example, is composed of GPS or GNSS, which detects the position of the excavator and transmits the results.

[0035] The vehicle posture detection device 102 consists of angle sensors that detect the angles of the boom, stick, and bucket, and angle sensors that detect the rotation angle of the upper rotating body. It detects the positional relationship between the lower traveling body and the upper rotating body of the excavator, as well as the positional relationship between the boom, stick, and bucket, and transmits the results.

[0036] The motion status detection device 103 consists of a speed sensor that detects the engine speed and a pressure sensor that detects the load pressure of the actuator. It obtains the status related to the excavator's motion, such as the engine output and the hydraulic pressure of the movable part, and transmits the results.

[0037] The work position acquisition unit 104 acquires the location where the excavator is working and the positional relationship and status of each movable part of the excavator based on the information received from the vehicle position detection device 101 and the vehicle posture detection device 102, and transmits the results.

[0038] The operation status acquisition unit 105 acquires the status of the engine and the hydraulic system based on the information received from the operation status detection device 103, and transmits the results.

[0039] The work content estimation unit 106 estimates, based on the information received from the work position acquisition unit 104 and the operation status acquisition unit 105, whether the current excavator is digging, dumping soil, or moving, and then transmits the estimated work content as a result.

[0040] The operation information calculation unit 107 combines the operation content output by the operation content estimation unit 106 with information indicating whether objects existing around the excavator are close to the excavator. If the object is determined to be close, the unit adds a priority setting for display and transmits the result as operation information.

[0041] The communication device 108 is, for example, a portable communication device consisting of a wireless LAN terminal, which transmits the operation information generated by the operation information computing unit 107 to external devices through a communication line.

[0042] The recording device 109, for example, is composed of a memory or a disk, and records the operating information output by the operating information calculation unit 107, and outputs the information according to the requests of the operating information calculation unit 107 and the communication device 108.

[0043] The display device 110 consists of, for example, a console and a monitor, and displays the operating information output by the operating information calculation unit 107 on the monitor in accordance with the operation of the console.

[0044] Object detection device 111 is cameras 501 and 502 installed on the vehicle body. Figure 3 As shown, the camera captures the surroundings of the excavator and outputs the resulting camera footage.

[0045] The object detection unit 113 inputs the image output by the object detection device 111, detects objects existing around the excavator from it, and transmits the detection results.

[0046] The shape information holding unit 114 holds the 3D shape of the excavator and outputs the 3D shape in accordance with the requirements of the configuration calculation unit 112.

[0047] The configuration status determination unit 115 compares the current 3D configuration of each component of the excavator output by the configuration calculation unit 112 with the result of projecting the object detection result in the camera image output by the object detection unit 113 from 2D to 3D, determines whether the object detected by the object detection unit 113 exists near the current location of the excavator body, and sends the determination result to the operation information calculation unit 107.

[0048] In the aforementioned operation record analysis system for construction machinery, the following is used: Figures 2 to 10 This section shows examples of runtime information at the point in time when a phenomenon that could be the main cause of a decrease in runtime occurred.

[0049] Figure 2 This describes the processing flow of the operation record analysis system of the present invention. The processing flow is roughly divided into two parts. The processing during operation information recording is performed when the excavator is running. The processing during operation information verification is performed when the display device 110 verifies the excavator's operation information recorded by the recording device 109.

[0050] In the operation record analysis system for engineering machinery of the present invention, when the operation information recording begins, step 201 is performed first.

[0051] In step 201, the work position acquisition unit 104 acquires the work position based on the information input from the vehicle position detection device 101 and the vehicle posture detection device 102. Figure 3 This describes the state of the excavator in this example. The excavator 301 includes at least: a lower traveling body 302, an upper slewing body 303 rotatably mounted on the lower traveling body 302, and a front workpiece consisting of a boom 304, a stick 305, and a bucket 306 mounted on the upper slewing body 303. The excavator 301 exists on a sloped ground surface. Cameras 501 and 502 are mounted on the vehicle body to capture images around the vehicle body. The vehicle body position detection device 101 considers, for example, using GPS (Global Positioning System) to obtain latitude and longitude on Earth. Alternatively, it considers using a total station or similar device to obtain the relative position within the target area of ​​the construction plan. In the vehicle body posture detection device 102, angle sensors mounted on the excavator 301 are used to detect the posture of the vehicle body. In this case, the posture refers to information summarizing the direction of the lower traveling body 302, the difference in rotation angle between the lower traveling body 302 and the upper slewing body 303, and the difference in configuration angles between the various parts of the upper slewing body 303, boom 304, stick 305, and bucket 306. The processing then proceeds to step 202.

[0052] In step 202, the operation state acquisition unit 105 acquires the operation state based on the information output from the operation state detection device 103. The operation state detection device 103 outputs information from sensors such as engine output sensors, hydraulic sensors, speedometers, and anomaly detection sensors to the operation state acquisition unit 105. Then, the processing is transferred to step 203.

[0053] In step 203, based on the information output by the work position acquisition unit 104 and the action status acquisition unit 105, the current work content of the excavator is estimated by the work content estimation unit 106. Figure 4This section describes examples of the work position output by the work position acquisition unit 104 and the action state output by the action state acquisition unit 105. Here, the work position refers to the 3D coordinates of the position of the bottom surface of the excavator's rotation center. The action state refers to the angles θ1, θ2, θ3, and θ4 formed relatively by the various parts of the excavator. In estimating the work content, for example, based on the vehicle posture information output by the vehicle posture detection device 102, if the position of the bucket 306 is higher than the ground surface and the speedometer value is above a certain value, it is estimated that the work content is driving. Alternatively, if the angle of the bucket 306 continuously changes and the discharge pressure of the hydraulic pump is above a certain value, it is estimated that the work content is digging. The work content estimation unit 106 outputs the work content thus estimated to the operation information calculation unit 107. Then, the processing is transferred to step 204.

[0054] In step 204, the operation information calculation unit 107 combines the operation content input from the operation content estimation unit 106 with numerical information such as position, speed, and angle associated with the operation content. For example, if the operation content is estimated to be driving, the driving speed is combined with the operation content. Or, if the operation content is estimated to be excavation, the angle information, represented as posture, is combined. This result is stored within the operation information calculation unit 107. Then, the processing is transferred to step 205.

[0055] In step 205, the configuration calculation unit 112 constructs a 3D shape based on the current posture of the excavator 301. The basic 3D shape of the excavator 301, held by the shape information holding unit 114, is applied to it with the work position obtained from the work position acquisition unit 104 and the action state obtained from the action state acquisition unit 105. For example, based on the posture information, the shape is updated to match the direction of the lower traveling body 302, the difference in rotation angle between the lower traveling body 302 and the upper slewing body 303, and the difference in configuration angles between the upper slewing body 303, boom 304, stick 305, and bucket 306, so that the angle differences between the components such as the lower traveling body 302, upper slewing body 303, boom 304, stick 305, and bucket 306 in the 3D shape are equal. This result is output to the configuration state determination unit 115. Then, the processing is transferred to step 206.

[0056] In step 206, the images around the excavator captured by cameras 501 and 502 are processed for object detection in the object detection unit 113. Figure 5This example illustrates object detection processing. In this example, a front camera 501 and a rear camera 502 are mounted on the upper rotating body 303. The images captured by cameras 501 and 502 are as shown in front camera image 503 and rear camera image 504. Since there are no objects behind the excavator 301, only the ground surface and sky are shown in rear camera image 504. A person 505 is present in front of the excavator 301, therefore, this person 505 is shown in front camera image 503. Furthermore, depending on the vehicle's posture, the boom 305 and bucket 306 are shown in front camera image 503.

[0057] Figure 6 This illustrates an example of surrounding object detection in the foreground camera image 503. In the foreground camera image 503, the upper left corner represents the case at time ti-1, and the lower left corner represents the case at subsequent times ti. Here, person 505 approaches the excavator, and the image of person 505 moves downwards within the foreground camera image 503 over time. When the difference between the foreground camera images 503 at these times ti-1 and ti is obtained, the result is shown on the right side of the figure. Within the image, the difference is generated only at the location where person 505 moves. This is designated as the difference region 601 on the image. Thus, when a difference is generated, the difference region 601 on the image is output to the configuration state determination unit 115. Afterwards, the processing is transferred to step 207.

[0058] In step 207, in the configuration state determination unit 115, a projection transformation is performed between the coordinate system of the camera image and the coordinate system representing the shape of the excavator 301. The camera image is represented by a 2D coordinate system, and the shape of the excavator 301 is represented by a 3D coordinate system. A projection transformation is performed between them. The relative position of the difference region 601 detected in the camera image to the excavator is calculated. The shape of the excavator 301 is updated through the processing in step 205 to match the current posture of the excavator 301. The shape of the excavator 301 also maintains the setting position and shooting direction of the front camera 501. Furthermore, based on the information of the working position, the relative positional relationship between the ground surface and the front camera 501 is determined. Since the person 505 exists on the ground surface, it is possible to determine, through projection transformation, the location of the difference region 601 detected in the front camera image 503 on the ground surface in 3D space. Figure 7 This scenario is represented. Through projection transformation, the position of character 701 at time ti in 3D space is obtained. Then, the processing is transferred to step 208.

[0059] In step 208, the overlap area between the structural elements of excavator 301 and the figure 701 at time ti is calculated. For example, assuming from... Figure 7 As time progresses, character 701 moves further. Figure 8 This scenario is illustrated. At time ti, character 701 changes position by moving, as shown by character 801 at time ti+1, moving towards the vicinity of bucket 306. The overlapping area in this case is calculated as follows: The possible movement range of character 801 after time ti+1 is defined as character movement range 803, calculated as a 3D region. Character movement range 803 represents the range that the character could move at the possible movement speed before time ti+2. Similarly, the possible movement range of excavator 301 after the boom 305 and bucket 306 at time ti+1 is defined as excavator movement range 804, calculated as a 3D region. Excavator movement range 804 represents the range that could be moved before time ti+2 while moving at maximum acceleration from the movement speed of boom 305 and bucket 306 at time ti+1. The overlapping area 802 is calculated as the area where character movement range 803 and excavator movement range 804 overlap (represented by a diagonal line in the figure). Then, the processing is transferred to step 209.

[0060] In step 209, it is determined whether an overlapping region 802 has been generated by the processing in step 208. If it has, the processing proceeds to step 210; otherwise, the processing proceeds to step 211.

[0061] In step 210, the operation information calculation unit 107 adds the generated overlapping region 802 to the operation information calculated in step 204 to set it as new operation information. Then, the processing is transferred to step 211.

[0062] In step 211, the operation information calculation unit 107 records the operation information in the recording device 109. Alternatively, it records it in an external recording device via the communication device 108. Then, the process is transferred to step 212.

[0063] In step 212, it is determined whether the recording of operation information has ended. If the recording of operation information ends due to the excavator 301 stopping, etc., then the recording ends. Figure 2 The processing during runtime information recording, as shown on the left, is transferred to step 201 in all other cases. The processing described above is performed during runtime information recording.

[0064] Next, the process for verifying the operational information recorded in the recording device 109 will be described. The process flow is described in the section located at... Figure 2 The right side shows the part related to verifying the running information.

[0065] In the operation record analysis system for engineering machinery in this embodiment, when the operation information verification begins, step 213 is performed first.

[0066] In step 213, the operation information calculation unit 107 acquires the operation information recorded in the recording device 109. The point in time from which the operation information is acquired is set by input from the system user using the display device 110. Furthermore, there are cases where the acquired operation information is from a single point in time, or where operation information from multiple consecutive points in time is designated as a range. In the case of a range, the operation information verification process is performed continuously within that range. Afterwards, the process is transferred to step 214.

[0067] In step 214, additional information appended to the acquired operational information is checked. If no additional information is appended, nothing is checked. Then, the process proceeds to step 215.

[0068] In step 215, it is determined whether additional information has been added to the acquired runtime information. If so, the process proceeds to step 216; otherwise, the process proceeds to step 217.

[0069] In step 216, the obtained runtime information is appended to the candidate extraction list. Then, the processing is transferred to step 217.

[0070] In step 217, it is determined whether there is any remaining running information in the running information specified by the display device 110 that has not undergone the above processing. If there is, the processing is transferred to step 213; otherwise, the processing is transferred to step 218.

[0071] In step 218, the acquired operating information is displayed on the display device 110. Then, the processing is transferred to step 219.

[0072] In step 219, the running information added to the extraction candidates is highlighted. The process then ends.

[0073] Here, Figure 9Examples of operational information are shown here. The operational information at time ti and time ti+1 is displayed. At time ti, the excavator 301's operation is "digging," and the position and posture of the excavator 301 are recorded in the operational information. At this time point, no overlapping region 802 is generated; therefore, the additional information is set to "none." At time ti+1, the excavator 301's operation is "digging," and the position and posture of the excavator 301 are recorded in the operational information. At this time point, the worker 701 moves and exists in the position shown by worker 801 at time ti+1. The result of the overlapping region calculation is the calculation of overlapping region 802. Based on this result, the additional information is set to "yes," and overlapping region 802 is added as additional information to the operational information.

[0074] then, Figure 10 This example illustrates the emphasis in step 218. In this example, a graph as shown is displayed on the display screen of display device 110. The horizontal axis of the graph represents the passage of time. The row for work content shows the changes in work content that accompany the passage of time. Similarly, the row for work position shows the changes in work position. Here, for convenience, a graph based on a single line is used, but in reality, a graph showing changes in 3D coordinates, latitude and longitude, etc., is displayed. The row for posture shows the changes in the posture of excavator 301. Here, for convenience, a graph based on a single line is also used, but in reality, the graph shows changes in angles such as θ1, θ2, θ3, and θ4. When there is an overlapping area 802 that adds to the operating information, the presence of the overlapping area 802 is indicated by a band (represented by a diagonal line in the figure) and the text "Overlapping area generated". Thus, the proximity of excavator 301 and person 801, which may be the main reason for the decrease in the operating rate of excavator 301, is clearly shown in the display of operating information at multiple times.

[0075] In this embodiment, the operation record analysis system for construction machinery includes: a vehicle body 301, which has a front work machine with at least a boom 305 and a bucket 306; a vehicle body position detection device 101, which detects the position of the vehicle body 301; a vehicle body posture detection device 102, which detects the posture of the vehicle body 301; an action state detection device 103, which detects the action state of the vehicle body 301; and a controller 100, which calculates the working position of the vehicle body 301 based on information from the vehicle body position detection device 101 and the vehicle body posture detection device 102, infers the work content of the vehicle body 301 based on the working position and the action state, and outputs the work content of the vehicle body 301. The operation information includes the work location and the operation content. The operation record analysis system for construction machinery includes: an object detection device 111, which detects objects 505, 701, and 801 around the vehicle body 301; a controller 100 calculates the positions of objects 505, 701, and 801 based on information from the object detection device 111; and determines whether the vehicle body 301 is close to objects 505, 701, and 801 based on information from the vehicle body position detection device 101 and the vehicle body posture detection device 102 and the positions of objects 505, 701, and 801. The determination result is included in the operation information and output.

[0076] According to this embodiment configured as described above, the determination of whether the vehicle body 301 is close to surrounding objects 505, 701, and 801 is included in the operation information. Therefore, the system user can quickly extract the operation content when the vehicle body is close to surrounding objects from a large amount of operation information. As a result, the verification efficiency of the excavator 301's operating status can be improved.

[0077] Furthermore, the object detection device 111 consists of cameras 501 and 502 that capture images of the area surrounding the vehicle body 301. When at least a portion of the vehicle body 301's movement range 804 within the camera interval of the cameras 501 and 502 overlaps with at least a portion of the object 801's movement range 803 within the same camera interval, the controller 100 determines that the vehicle body 301 and the object 801 are close. Thus, it is possible to determine whether the vehicle body 301 and the object 801 are close based on their respective movement speeds.

[0078] Furthermore, the operation record analysis system for construction machinery in this embodiment also includes: a communication device 108 for transmitting operation information; a recording device 109 for recording the operation information; and a display device 110 for displaying the operation information. This increases the flexibility in determining the location for confirming operation information.

[0079] Example 2

[0080] In this embodiment, an example illustrating the main reasons for the occurrence of overlapping region 802 will be described.

[0081] Figure 11 The upper left corner represents an example of the front camera image 503 at time ti+1 in this embodiment. Here, due to the activity of the person 801 at time ti+1, a difference region 1101 at time ti+1 is generated. In subsequent processing, if it is determined that an overlap region 802 has been generated, the difference region 1101 at time ti+1 is recorded in the recording device 109. When an overlap region 802 is generated, a difference region 1101 must exist; therefore, in such a situation, the difference region 1101 must be recorded in the recording device 109.

[0082] exist Figure 11 The right side represents an example of displaying the image to the display device 110. When an overlapping area 802 exists, the differential area 1101 recorded in the recording device 109 and the overlapping area are displayed together with an emphasis display.

[0083] In this embodiment, when the controller 100 determines that the vehicle body 301 is close to the object 801, it includes the differential region 1101 in the images captured by the cameras 501 and 502 at the time of determination, which represents the range of movement of the object 801 within the shooting interval of the cameras 501 and 502, in the operation information and outputs it.

[0084] According to this embodiment configured as described above, when an overlapping region 802 is generated, the difference region 1101, which serves as the determination criterion for the generation, can be displayed as an image, allowing the system user to visually confirm which object generated the overlapping region 802. Based on the visually confirmed content, the system user can grasp the importance of the analysis of the overlapping region 802 and quickly execute subsequent actions.

[0085] Example 3

[0086] In this embodiment, an example is given showing the main reasons for the occurrence of overlapping region 802, as well as other states.

[0087] Figure 12The upper left corner represents an example of the front camera image 503 at time ti+1 in this embodiment. Here, due to the activity of the person 801 at time ti+1, a difference region 1101 at time ti+1 is generated. In subsequent processing, if it is determined that an overlap region 802 has been generated, the difference region 1101 at time ti+1 and the camera image 1201 at time ti+1 are recorded in the recording device 109. The camera image is used in the calculation of the overlap region 802 and the difference region 1101, therefore, in such a case, the camera image 1201 and the difference region 1101 must be recorded in the recording device 109.

[0088] exist Figure 12 The right side represents an example of displaying the image to display device 110. When an overlapping area 802 exists, the camera image 1201 recorded in recording device 109 and the differential area 1101 are displayed together with the emphasis generated by the overlapping area.

[0089] In this embodiment, when the controller 100 determines that the vehicle body 301 is close to the object 801, it includes the information of the differential region 1101 representing the range of movement of the object 801 within the shooting interval of the cameras 501 and 502 in the images captured by the cameras 501 and 502 at the time of determination, and the images captured by the cameras 501 and 502 at the time of determination in the operation information and outputs it.

[0090] According to this embodiment configured as described above, when an overlapping region 802 is generated, the surrounding conditions and the difference region 1101, which serves as the determination criterion for its generation, can be displayed as an image. The system user can visually confirm the conditions under which the overlapping region 802 was generated. Based on the visually confirmed content, the system user can grasp the importance of the analysis of the overlapping region 802 and quickly execute subsequent responses.

[0091] Example 4

[0092] In this embodiment, an example of object detection using LiDAR (Light Detection and Ranging) will be described.

[0093] Figure 13 This illustrates the structure of the excavator 301 in this embodiment. In this structure, the cameras 501 and 502 in the first embodiment are replaced with LiDARs. In this example, a front LiDAR 1301 is mounted to detect the front of the excavator 301, and a rear LiDAR 1302 is mounted to detect the rear of the excavator 301.

[0094] exist Figure 13The right side represents the front LiDAR output 1303 and the rear LiDAR output 1304. Within the detection range of the front LiDAR 1301, there is a person 505, a boom 305, and a bucket 306, resulting in waveforms in the graph of the front LiDAR output 1303. The waveform on the left is generated by the person 505, and the waveform on the right is generated by the boom 305 and the bucket 306. Furthermore, within the detection range of the rear LiDAR 1302, there are no objects other than the ground surface. Additionally, no characteristic waveforms are found in the rear LiDAR output 1304.

[0095] The object detection unit 113 uses these LiDAR outputs to perform object detection. The result is measurements of the person 505, the boom 305, and the bucket 306. Based on this result, in the configuration state determination unit 115, the waveforms generated by the boom 305 and bucket 306 are excluded from the comparison with the 3D shape of the excavator 301 output by the configuration calculation unit 112, and the waveform generated by the person 505 is obtained. Using this result, the overlapping region 802 is calculated.

[0096] In this embodiment, the object detection device 111 is a LiDAR 1301, 1302 that acquires distance information to the object. The controller 100 excludes the distance information of the vehicle body 301 from the distance information of the object acquired by the LiDAR 1301, 1302, thereby acquiring the distance information of the object 505.

[0097] According to this embodiment configured as described above, even when the object detection device 111 is composed of LiDAR 1301 and 1302, it records the determination result of whether the vehicle body 301 is close to the surrounding object 505 along with the operation information, just like in the first embodiment. Therefore, the system user can quickly extract the operation content when the vehicle body 301 is close to the surrounding object 505 from a large amount of operation information. As a result, the verification efficiency of the excavator 301's operation status can be improved.

[0098] Example 5

[0099] In this embodiment, an example of using LiDAR for object detection and displaying camera images during object detection will be described.

[0100] Figure 14This describes the structure of the excavator 301 in this embodiment. In this structure, in addition to cameras 501 and 502, a LiDAR is also provided, with its output set as the input to the object detection unit 113. The detection processing using the front LiDAR 1301 and the rear LiDAR 1302 is the same as in the fourth embodiment. In this embodiment, the area around the excavator 301 is also filmed using the front camera 501 and the rear camera 502. The object detection results output by the LiDAR are overlaid in the filmed content, and the results are recorded in the recording device 109. The display on the display device 110 in this case is as follows. Figure 15 As shown. At the moment when the occurrence of the overlapping region 802 is detected, the image recorded in the recording device 109 is also displayed.

[0101] In this embodiment, the object detection device 111 is a LiDAR 1301, 1302 that acquires distance information to an object. The controller 100 excludes the distance information of the vehicle body 301 from the distance information of the object acquired by the LiDAR 1301, 1302, thereby acquiring the distance information of the object 505. Furthermore, the operation record analysis system for construction machinery in this embodiment also includes cameras 501, 502, which capture images of the area around the vehicle body 301. When the controller 100 determines that the vehicle body 301 is close to the object 505, it includes the images captured by the cameras 501, 502 at the time of determination in the operation information and outputs them.

[0102] According to this embodiment configured as described above, when an overlapping region 802 is generated, the surrounding conditions and the object detection results that serve as the determination criteria for its generation can be displayed as an image. The system user can visually confirm under what circumstances the overlapping region 802 was generated. Based on the visually confirmed content, the system user can grasp the importance of the analysis of the overlapping region 802 and quickly execute subsequent responses.

[0103] The embodiments of the present invention have been described in detail above, but the present invention is not limited to the above embodiments and includes various modifications. For example, the above embodiments are described in detail for the purpose of easily understanding the present invention and are not limited to having all the structures described. In addition, a part of the structure of another embodiment may be added to the structure of a certain embodiment, a part of the structure of a certain embodiment may be deleted, or a part of the structure of another embodiment may be replaced.

[0104] Explanation of reference numerals in the attached figures

[0105] 100…Controller, 101…Vehicle position detection device, 102…Vehicle posture detection device, 103…Motion state detection device, 104…Work position acquisition unit, 105…Motion state acquisition unit, 106…Work content estimation unit, 107…Operation information calculation unit, 108…Communication device, 109…Recording device, 110…Display device, 111…Object detection device, 112…Configuration calculation unit, 113…Object detection unit, 114…Shape information holding unit, 115…Configuration state determination unit, 201-219…Steps, 301…Excavator (vehicle body), 302…Lower traveling body, 303…Upper rotating body, 304…Boom, 305…Stick, 306…Bucket, 501…Front-facing camera (object detection) 502… Rear camera (object detection device), 503… Front camera image, 504… Rear camera image, 505… Person (object), 601… Differential region, 701… Person (object) at time ti, 801… Person (object) at time ti+1, 802… Overlapping region, 803… Person movement range (object movement range), 804… Excavator movement range (vehicle movement range), 1101… Differential region at time ti+1, 1201… Camera image at time ti+1, 1301… Front LiDAR (object detection device), 1302… Rear LiDAR (object detection device), 1303… Front LiDAR output, 1304… Rear LiDAR output.

Claims

1. A system for analyzing operation records of engineering machinery, comprising: The vehicle body is equipped with at least a boom and a bucket for the front work machine; A vehicle position detection device that detects the position of the vehicle body; A vehicle posture detection device that detects the posture of the vehicle body; An action state detection device that detects the action state of the vehicle body; The controller calculates the working position of the vehicle body based on information from the vehicle body position detection device and the vehicle body posture detection device, infers the working content of the vehicle body based on the working position and the action state, and calculates operation information including the working position and the working content. Its features are, The operation record analysis system for construction machinery includes: an object detection device, which consists of a camera that captures images of the area around the vehicle body, detecting objects present around the vehicle body. The controller calculates the position of the object based on information from the object detection device. The controller calculates the vehicle's forward working machine's movement range within the camera's shooting interval in a three-dimensional region, and calculates the object's movement range within the shooting interval in a three-dimensional region. The overlapping portion of the vehicle's movement range and the object's movement range is calculated as the overlap area between the vehicle's forward working machine and the object, and this overlap area is added to the operating information at the time the overlap area is generated. The controller displays the existence of the overlapping area, the passage of time, and the work content together on the display device based on the operation information.

2. The operation record analysis system for engineering machinery according to claim 1, characterized in that, When at least a portion of the vehicle's movement range within the camera's shooting interval overlaps with at least a portion of the object's movement range within the camera's shooting interval, the controller determines that the vehicle is close to the object.

3. The operation record analysis system for engineering machinery according to claim 2, characterized in that, When the controller determines that the vehicle body is close to the object, it includes information about the differential region representing the range of movement of the object within the shooting interval in the image captured by the camera at the time of the determination in the operation information and performs calculations.

4. The operation record analysis system for engineering machinery according to claim 2, characterized in that, When the controller determines that the vehicle body is close to the object, it includes the image captured by the camera at the time of the determination in the operation information and performs calculations.

5. The operation record analysis system for engineering machinery according to claim 1, characterized in that, The operation record analysis system for construction machinery also includes: a communication device for transmitting the operation information; a recording device for recording the operation information; and a display device for displaying the operation information.

Images