External environment detection system and construction machine

Abstract

Provided are an external environment detection system and a construction machine that do not require overlapping of measurement regions of a plurality of external environment detection devices during calibration. The external environment detection system is mounted on a construction machine having a lower traveling body, an upper revolving body, and a work device, and comprises: an external environment detection device provided in the construction machine and configured to detect an object; an attitude detection device that detects the attitude of either the upper revolving body or the work device; and a calculation device that detects a deviation of either the position or attitude of the external environment detection device. The calculation device calculates the coordinates of an object estimated to be detected by the external environment detection device, on the basis of the attitude of either the upper revolving body or the work device detected by the attitude detection device, and detects the deviation of either the position or posture of the external environment detection device by comparing the calculated coordinates of the object with the coordinates of the object detected by the external environment detection device.

Description

External Detection System and Construction Machinery

[0001] The present invention relates to an external detection system and a construction machine.

[0002] In the calibration of the positions and attitudes of a plurality of external detection devices mounted on a construction machine, for example, a method of performing calibration by overlapping the measurement areas of a plurality of stereo cameras has been proposed in, for example, Patent Document 1.

[0003] Japanese Patent Application Laid-Open No. 2014-215039

[0004] By the way, construction machines for mines are huge, and when detecting the surroundings of the vehicle body without dead spots, it is necessary to mount a large number of external detection devices. However, in the method of Patent Document 1, a certain degree of overlap of the measurement areas between the external detection devices is required. Therefore, the number of external detection devices that need to be mounted may increase. Therefore, an external detection system for a construction machine that does not require overlap of the measurement areas of a plurality of external detection devices in calibration is desired.

[0005] An object of the present invention is to provide an external detection system and a construction machine that do not require overlap of the measurement areas of a plurality of external detection devices in calibration.

[0006] In order to solve the above problems, an external detection system according to the present invention is an external detection system mounted on a construction machine having a lower traveling body, an upper revolving body, and a working device, and includes an external detection device provided on the construction machine and configured to detect an object, an attitude detection device that detects the attitude of either the upper revolving body or the working device, and an arithmetic device that detects any deviation in the position and attitude of the external detection device. The arithmetic device calculates the coordinates of an object estimated to be detected by the external detection device based on the attitude of either the upper revolving body or the working device detected by the attitude detection device, and detects any deviation in the position and attitude of the external detection device by comparing the calculated coordinates of the object with the coordinates of the object detected by the external detection device.

[0007] According to the present invention, calibration is possible without requiring overlapping measurement areas of multiple external detection devices. Further features related to the present invention will become apparent from the description herein and the accompanying drawings. In addition, problems, configurations, and effects other than those described above will be revealed by the following description of embodiments.

[0008] A diagram showing the external environment detection system and construction machinery of Example 1. A diagram showing the external environment detection system and construction machinery of Example 1. A diagram showing a bracket structure for attaching the external environment detection device to the construction machinery. A diagram showing the bracket structure of Figure 3, with the external environment detection device and bracket in an exploded perspective view. A diagram showing one end of the external environment detection device when adjusting the pitch angle. A diagram showing the other end of the external environment detection device when adjusting the pitch angle. A diagram showing the misalignment of the bracket and the external environment detection device. A diagram showing the external environment detection device without positional and orientation misalignment. A diagram showing the bracket and external environment detection device with orientation misalignment in the pitch direction. A diagram showing the bracket and external environment detection device with positional misalignment in the translational direction as shown in Figure 7C, followed by orientation misalignment in the pitch direction as shown in Figure 7B. A functional block diagram of the external environment detection system of Example 1. A flowchart of misalignment detection and calibration information update in Example 1. A flowchart showing the details of the misalignment detection process in Example 1. A diagram showing object detection by the first external environment detection device. A diagram showing the detection of an object by the second external detection device. A diagram showing the content displayed on the display terminal in Example 1. A functional block diagram of the external detection system in Example 2. A functional block diagram of the displacement detection unit in Figure 14. A flowchart showing the details of the displacement detection process in Example 2. A diagram showing the detection of the work device by the external detection device. A functional block diagram of the external detection system in Example 3. A functional block diagram of the displacement detection unit in Figure 18. A diagram showing the change in posture when the operator lifts the boom from a predetermined posture in the work device of Example 3. A flowchart showing the details of the displacement detection process in Example 3. A diagram showing the trajectory of the coordinates of the work device and the estimated trajectory of the coordinates of the work device in Example 3. A functional block diagram of the external detection system in Example 4. A functional block diagram of the displacement detection unit in Figure 23. A diagram showing the operation of the construction machine in the displacement detection process of Example 4. A flowchart showing the details of the displacement detection process in Example 4. A diagram showing the trajectory of the coordinates of an object and the estimated trajectory of the coordinates of an object in Example 4.

[0009] This embodiment will be described below with reference to the attached drawings. In the attached drawings, functionally identical elements may be indicated by the same number. The attached drawings show embodiments and implementation examples in accordance with the principles of this disclosure, but they are for the purpose of understanding this disclosure and are not to be used in any way to restrict the interpretation of this disclosure. The descriptions in this specification are merely typical examples and do not limit the claims or applications of this disclosure in any way.

[0010] While this embodiment is described in sufficient detail for those skilled in the art to implement the disclosure, other implementations and forms are possible, and it should be understood that modifications to the configuration, structure, and various elements can be replaced without departing from the scope and spirit of the technical idea of ​​this disclosure. Therefore, the following description should not be construed as limiting to this.

[0011] [Example 1] [Overall Configuration] The configuration of the external environment detection system in Example 1 will be described below with reference to Figures 1 to 3. Figure 1 shows the construction machine 100 and the external environment detection system 400A in this embodiment. As shown in Figure 1, the construction machine 100 has a lower traveling body 101, an upper rotating body 102, and a working device 103. The construction machine 100 has an external environment detection system 400A. The external environment detection system 400A is mounted on the construction machine 100. The lower traveling body 101 moves the construction machine 100. The upper rotating body 102 is rotatable relative to the lower traveling body 101. The working device 103 is mounted so as to be rotatable relative to the upper rotating body 102. The working device 103 consists of a boom 104, an arm 105, and a bucket 106.

[0012] Furthermore, the construction machine 100 has various posture detection devices P1, P2, P3, and P4 for detecting the posture of either the upper slewing body 102 or the work device 103. In this embodiment, the construction machine 100 includes a posture detection device P1 for detecting the slewing angle of the upper slewing body 102 with respect to the lower traveling body 101. The construction machine 100 includes a posture detection device P2 for detecting the relative angle of the boom 104 with respect to the upper slewing body 102. The construction machine 100 includes a posture detection device P3 for detecting the relative angle of the arm 105 with respect to the boom 104. The construction machine 100 includes a posture detection device P4 for detecting the relative angle of the bucket 106 with respect to the arm 105. The posture detection devices P1, P2, P3, and P4 are, for example, angle sensors such as gyro sensors or potentiometers.

[0013] Hereinafter, the position of the work device 103 refers to, for example, the coordinates of any part of the work device 103, such as the boom 104, arm 105, and bucket 106, in a Cartesian coordinate system. The attitude of the work device 103 refers to, for example, the rotation angle of the upper slewing body 102 relative to the lower traveling body 101, the relative angle of the boom 104 relative to the upper slewing body 102, the relative angle of the arm 105 relative to the boom 104, and the relative angle of the bucket 106 relative to the arm 105. In this embodiment, if only the attitude detection device P1 that detects the rotation angle of the upper slewing body 102 relative to the lower traveling body 101 is available, it is possible to detect any deviation in position or attitude in either the external environment detection device E1 or the external environment detection device E2.

[0014] The upper rotating body 102 is equipped with a cab 107 on which an operating device 120 is provided for the operator to board and operate the construction machine 100. Inside the cab 107 is an operating seat (not shown) where the operator sits. Inside the cab 107 are also a display terminal 108 and a calculation device 200A.

[0015] The operating device 120 is, for example, a lever, pedal, or switch. By operating the operating device 120, the operator can rotate the upper rotating body 102 and turn the work device 103. The display terminal 108 is, for example, a touch panel display device that displays various information and accepts information input. The computing device 200A is, for example, a computer equipped with a CPU (Central Processing Unit) and memory, and its mounting location is not limited to inside the cab 107.

[0016] The construction machine 100 and the external environment detection system 400A are equipped with at least one external environment detection device E1. For example, as shown in Figures 1 and 2, the external environment detection system 400A is equipped with an external environment detection device (first external environment detection device) E1, an external environment detection device (second external environment detection device) E2, and an external environment detection device (third external environment detection device) E3. In this embodiment, each of the external environment detection devices E1, E2, and E3 is provided on the construction machine 100 and is configured to detect objects. Here, the objects detected by the external environment detection devices E1, E2, and E3 include other vehicles and the ground surface outside the construction machine 100, as well as the work equipment 103.

[0017] More specifically, in this embodiment, the external detection device E1 is mounted in front of the upper rotating body 102 and detects objects in front of the construction machine 100 (in the direction of the work device 103 relative to the upper rotating body 102). The external detection devices E2 and E3 are mounted on the left and right sides of the upper rotating body 102, respectively, and detect objects on the left and right sides of the upper rotating body 102. In other words, in this embodiment, the construction machine 100 and the external detection system 400A have sensors that detect objects in front of, to the right of, and to the left of the vehicle body.

[0018] Information regarding object detection from each of the external detection devices E1, E2, and E3 is transmitted to the computing device 200A. The computing device 200A outputs information regarding objects and terrain around the construction machine 100 as external information to a controller (not shown) that performs various controls on the body of the construction machine 100. For example, the construction machine 100 is configured to monitor its surroundings based on this external information and to perform automatic (semi-automatic) operations such as excavation, loading, and notification, taking into consideration safety and productivity.

[0019] The measurement areas R1, R2, and R3 of the external detection devices E1, E2, and E3, respectively, have a fan-shaped planar form, as schematically shown in Figure 2. In this embodiment, the measurement area R1 of the external detection device E1 is the area in front of the upper rotating body 102, which includes a part of the work device 103, and includes the area (work area) where the work device 103 performs work such as excavation and loading. Depending on the position of the upper rotating body 102 relative to the lower traveling body 101, i.e., the rotation angle of the upper rotating body 102, the measurement areas R1 to R3 of the external detection devices E1 to E3 may include a part of the lower traveling body 101.

[0020] The measurement areas R1 to R3 may or may not overlap. External detection devices E1 to E3 include, for example, imaging devices with CMOS sensors, LiDAR (Light Detection and Ranging), radar, and ultrasonic sensors.

[0021] In this embodiment, the number of external detection devices E1, E2, and E3 is set to three, but this is not the only option. For example, as will be described later, in this embodiment, if there are two external detection devices E1 and E2, it is possible to detect deviations in either the position or orientation of either external detection device E1 or external detection device E2. Also, if there are three external detection devices E1, E2, and E3, it is preferable to keep two of the three external detection devices E1, E2, and E3 constantly operational.

[0022] Here, Figure 3 shows the bracket structure for attaching the external detection devices E1, E2, and E3 to the construction machine in this embodiment. As shown in Figure 3, the external detection device E2 on the left side of the construction machine 100 is attached to the construction machine 100 by a bracket 109.

[0023] Figure 4 shows the bracket structure shown in Figure 3, and displays the external environment detection device E2 and bracket 109 in a disassembled state. As shown in Figure 4, the external environment detection device E2 (LiDAR in this figure) is equipped with a laser transmitting and receiving unit 110 on one side.

[0024] Furthermore, as shown in Figure 4, the bracket 109 is equipped with an external detection device mounting surface 114 and a roll adjustment mechanism 111, a yaw adjustment mechanism 112, and a pitch adjustment mechanism 113 for adjusting the installation angles thereof. These adjustment mechanisms consist of a U-shaped groove and bolts. Specifically, the installation angle refers to three angles: the rotation angle around the roll axis (roll angle) when the direction in which the laser beam is emitted from the laser transmitting / receiving unit 110 of the external detection device E2, i.e., the optical axis of the emitted laser beam, is defined as the front-to-back direction of the external detection device E2; the rotation angle around the yaw axis (yaw angle) when the axis along the vertical direction of the construction machine 100, one of the two axes perpendicular to the optical axis of the laser beam, is defined as the yaw axis; and the rotation angle around the pitch axis (pitch angle) when the axis perpendicular to both the roll axis and the yaw axis is defined as the pitch axis.

[0025] Figures 5A and 5B show the angle adjustment of the external detection device using the adjustment mechanism shown in Figure 4. Figures 5A and 5B show the pitch angle of the external detection device E2 being adjusted by operating the pitch adjustment mechanism 113. By adjusting within the range from one end shown in Figure 5A to the other end shown in Figure 5B, the orientation of the laser transmitting / receiving unit 110 of the external detection device E2 changes, and the measurement area R2 also changes.

[0026] Figure 6 shows the positional and orientation deviations of the bracket 109 and the external detection device E2. Hereafter, the positional deviation of the external detection devices E1, E2, and E3 refers to a state in which the coordinates of the external detection devices E1, E2, and E3 in the Cartesian coordinate system are shifted, for example, as shown in Figure 6. Furthermore, the orientation deviation of the external detection devices E1, E2, and E3 refers to a state in which the rotation angle around the roll axis (roll angle) when the optical axis of the laser beam of the external detection devices E1, E2, and E3 is defined as the roll axis, the rotation angle around the yaw axis (yaw angle) when the axis along the vertical direction of the construction machine 100 is defined as the yaw axis (one of two axes perpendicular to the optical axis of the laser beam), and the rotation angle around the pitch axis (pitch angle) when the axis perpendicular to both the roll axis and the yaw axis is defined as the pitch axis are shifted.

[0027] As described above, in this embodiment, the adjustment mechanisms of the bracket 109 make it possible to adjust the mounting surface 114 of the external detection device E2, which is the mounting position of the external detection device E2, around each of the three axes. However, if an unintended malfunction occurs during the adjustment of the bracket 109, the orientation of the external detection device E2 may shift from its design position around the three axes (in the rotational direction). Furthermore, this adjustment of the orientation of the bracket 109 is performed as needed during maintenance after the bracket 109 is installed on the construction machine 100, or when the design is changed, and each time this is done, a shift in the orientation of the external detection device E2 may occur.

[0028] Next, using Figures 7A to 7D, an example of a possible displacement of the position and orientation of the external detection device E2 after the bracket is installed will be explained. Figures 7A to 7D show the bracket 109 and the external detection device E2 as viewed from the pitch axis direction. First, Figure 7A shows the external detection device E2 in a state where the bracket 109 has been properly installed and no displacement of position or orientation has occurred. For example, immediately after the bracket 109 has been adjusted to the appropriate position, the pitch axis 113o of the pitch adjustment mechanism 113 and the pitch axis E2o of the external detection device E2 are in a positional relationship that coincides with each other, as shown in Figure 7A.

[0029] Next, Figures 7B to 7D show the bracket 109 and the external detection device E2 in a state where a misalignment of position or orientation has occurred. Figure 7B shows the bracket 109 and the external detection device E2 in a state where the pitch axis 113o of the pitch adjustment mechanism 113 and the pitch axis E2o of the external detection device E2 coincide with each other, but a misalignment of orientation in the pitch direction has occurred. Figure 7C shows the bracket 109 and the external detection device E2 in a state where a misalignment of position has occurred in the translational direction (parallel movement, for example, in the roll axis direction of the bracket 109). As shown in Figure 7C, when a misalignment of position in the translational direction occurs, the pitch axis 113o of the pitch adjustment mechanism 113 and the pitch axis E2o of the external detection device E2 are not in a positional relationship that coincides.

[0030] Figure 7D shows the bracket 109 and the external environment detection device E2 after a positional displacement in the translational direction, as shown in Figure 7C, has occurred, followed by a positional displacement in the pitch direction, as shown in Figure 7B. Figure 7D shows the pitch axis E2o of the external environment detection device E2 before the pitch direction positional displacement occurs, and the pitch axis E2o' of the external environment detection device E2 after the pitch direction positional displacement occurs. As shown in Figure 7D, when the roll axis direction of the bracket 109 is the X direction and the yaw axis direction of the bracket 109 is the Z direction, a displacement of ΔX occurs in the X direction and a displacement of ΔZ occurs in the Z direction between the pitch axis E2o and the pitch axis E2o'.

[0031] More specifically, in this embodiment, if the bracket 109 as a whole or each adjustment mechanism loosens due to aging deterioration or vibration caused by long-term operation of the construction machinery, it is necessary to consider not only the rotational positional displacement shown above and in Figure 7B, but also the translational positional displacement, i.e., positional offset, shown in Figure 7C.

[0032] For example, if the bracket 109 is offset in the direction of the roll axis, the pitch axis 113o of the pitch adjustment mechanism 113 and the pitch axis E2o of the external detection device E2 will be misaligned, as shown in Figure 7C. Furthermore, if both rotational and translational positional misalignments occur, the pitch axis E2o' of the external detection device E2 will be misaligned in a complex manner from the pitch axis E2o, as shown in Figure 7D. Thus, in this embodiment, for example, six types of misalignment were assumed: three rotational misalignments and three translational misalignments.

[0033] Figure 8 is a functional block diagram of the external environment detection system in this embodiment. As shown in Figure 8, the external environment detection system 400A in this embodiment includes a plurality of external environment detection devices E1, E2, E3, an attitude detection device P1, a calculation device 200A, and a display terminal 108. The calculation device 200A detects any deviation in either the position or attitude of the external environment detection devices E1, E2, and E3. Based on the attitude of the upper rotating body 102 detected by the attitude detection device P1, the calculation device 200A calculates the coordinates of an object that is estimated to be detected by the external environment detection devices E1, E2, and E3. By comparing the calculated coordinates of the object with the coordinates of the object detected by the external environment detection devices E1, E2, and E3, the calculation device 200A detects any deviation in either the position or attitude of the external environment detection devices E1, E2, and E3.

[0034] The computing unit 200A includes, for example, an information recording unit 301, a displacement detection unit 302A, and an information management unit 303 as functional units. When the CPU executes a program stored in the memory of the computing unit 200A, each of the above functional units performs the following functions. The information recording unit 301 records the position and orientation of the external detection devices E1, E2, and E3. The position and orientation of the external detection devices E1, E2, and E3 are calibration information, and the information recording unit 301 records this calibration information.

[0035] Calibration information refers to information indicating where and in what orientation each of the external detection devices E1, E2, and E3 is mounted on the construction machine 100. Specifically, the calibration information is, for example, information for six degrees of freedom, which combines the X, Y, and Z coordinates in the vehicle body coordinate system of the construction machine 100, and the roll, pitch, and yaw angles in the X, Y, and Z coordinates in the vehicle body coordinate system.

[0036] The vehicle body coordinate system is a coordinate system in which the three axes of the Cartesian coordinate system are, for example, the longitudinal axis of the lower traveling body 101 included in the ground contact surface of the lower traveling body 101, the lateral axis of the lower traveling body 101 included in the ground contact surface of the lower traveling body 101, and the vertical axis passing through the center of the lower traveling body. For example, the initial values ​​of the calibration information are information indicating the position and orientation of each of the external detection devices E1, E2, and E3 when they are not misaligned. In other words, the information recording unit 301 of the calculation unit 200A records either the position or orientation of the external detection devices E1, E2, and E3 of the construction machine 100.

[0037] The displacement detection unit 302A detects any displacement in either the position or orientation of the external detection devices E1, E2, and E3. Based on the orientation of the upper rotating body 102 detected by the orientation detection device P1, the displacement detection unit 302A calculates the coordinates of an object that is estimated to be detected by the external detection devices E1, E2, and E3. By comparing the calculated coordinates of the object with the coordinates of the object detected by the external detection devices E1, E2, and E3, the displacement detection unit 302A detects whether either the position or orientation of the external detection devices E1, E2, and E3 deviates from the position and orientation of the calibration information recorded by the information recording unit 301. The displacement detection unit 302A also includes a storage area for temporarily storing the detection results from the external detection devices E1, E2, and E3.

[0038] Based on the output of the displacement detection unit 302A, the information management unit 303 displays the detection result of any displacement of the position or orientation of the external detection devices E1, E2, and E3 on the display terminal 108. The information management unit 303 also updates the calibration information recorded by the information recording unit 301 based on the input from the operator to the display terminal 108 and the output of the displacement detection unit 302A.

[0039] More specifically, the displacement detection unit 302A of the calculation unit 200A detects the displacement of position and orientation in either external detection device E1 or external detection device E2 by comparing the object detection result of one of the external detection devices E1, E2, and E3, which is external detection device E1, with the object detection result of another external detection device E2, which is external detection device E1, E2, and E3.

[0040] The displacement detection unit 302A detects the displacement of position and orientation in either the external detection device E1 or the external detection device E2 by comparing, for example, the first estimated coordinates (calculated value) of an object that is estimated to be detected by the external detection device E1 based on the detection result of the same object by the external detection device E2 with the first coordinates (measured value) of the object detected by the external detection device E1.

[0041] When an object is detected by the external detection device E1, the displacement detection unit 302A calculates the first estimated coordinates of the object that is estimated to be detected by the external detection device E1, based on the first rotation angle of the upper rotating body 102 relative to the lower traveling body 101 detected by the attitude detection device P1, the position and attitude of the external detection device E1 recorded by the information recording unit 301, and the second coordinates of the object detected by the external detection device E2. When an object is detected by the external detection device E2, the displacement detection unit 302A calculates the first estimated coordinates of the object that is estimated to be detected by the external detection device E1, based on the second rotation angle of the upper rotating body 102 relative to the lower traveling body 101 detected by the attitude detection device P1, and the position and attitude of the external detection device E2 recorded by the information recording unit 301.

[0042] External detection device E1 is the first external detection device. External detection device E2 is the second external detection device. The position and orientation of external detection devices E1 and E2 recorded by the information recording unit 301 are the calibration information described above. The object detected by external detection device E1 and the object detected by external detection device E2 are the same object. The displacement detection unit 302A detects a displacement in either the position or orientation of either external detection device E1 or external detection device E2 by comparing the first estimated coordinates with the first coordinates of the object detected by external detection device E1.

[0043] Note that the initial value of the calibration information recorded by the information recording unit 301 may be the information on the drawing when the external detection device is attached, or the measurement results using a total station, GNSS (Global Navigation Satellite System), or the like.

[0044] [Operation] Hereinafter, the deviation detection and calibration information update process of the external detection system 400A of the construction machine 100 in this embodiment will be described with reference to FIGS. 9 to 13.

[0045] FIG. 9 shows the flow of deviation detection and calibration information update of the external detection system 400A in this embodiment. First, when the deviation detection unit 302A satisfies a predetermined condition, it starts monitoring the detection information by the external detection devices E1, E2, E3 and the attitude information by the attitude detection devices P1, P2, P3, P4 (S0). In this embodiment, the monitoring of the detection information and the attitude information is started by starting the engine of the vehicle body of the construction machine 100 and starting the system. Note that the monitoring of the detection information and the attitude information may be started when output information of any of the external detection devices E1, E2, E3 and the attitude detection devices P1, P2, P3, P4 is acquired.

[0046] When the deviation detection unit 302A satisfies a predetermined condition, it detects any deviation in the position and attitude of the external detection devices E1, E2, E3 (S1). Here, in this embodiment, the predetermined condition is that, within a predetermined period, when the same object is detected by two of the external detection devices E1, E2, E3, the deviation detection process is performed.

[0047] Also, the deviation detection process may be started, for example, when any of the external detection devices E1, E2, E3 detects an object after the engine of the construction machine 100 is started or when the attitude of the upper swing body 102 with respect to the lower traveling body 101 becomes a predetermined attitude, and then the same object is detected by another one of the external detection devices E_, E2, E3.

[0048] In this embodiment, it is assumed that the lower traveling body 101 does not move the construction machine 100 during the time between when one of the external detection devices E1, E2, or E3 detects an object and when another external detection device E1, E2, or E3 subsequently detects the same object.

[0049] Furthermore, the timing at which two of the external detection devices E1, E2, and E3 detect an object may be different or simultaneous. Alternatively, the timing at which each of the two external detection devices E1, E2, and E3 detects an object may be determined based on the operator's input on the display terminal 108 to start displacement detection and calibration information updating of the external detection system 400A, and then the displacement detection process may be started.

[0050] If the system detects a deviation in the position or orientation of the external detection devices E1, E2, or E3 in S1, the information management unit 303 notifies the operator by displaying the detection result on the display terminal 108 (S2). Next, the information management unit 303 receives input from the operator through the display terminal 108 (S3). After the operator has made an input through the display terminal 108, the information management unit 303 performs a process to update or maintain the calibration information according to the operator's input (S4).

[0051] The details of the displacement detection process S1 performed by the displacement detection unit 302A will be explained using the flowchart in Figure 10. As shown in Figure 10, the displacement detection unit 302A repeats the following processes until the displacement detection process is terminated (S10 to S15). The external detection devices E1, E2, and E3 perform the process of detecting objects outside the construction machine 100 (S11). Here, the objects to be detected by the external detection devices E1, E2, and E3 are, for example, dump trucks and the surrounding terrain. The displacement detection unit 302A determines whether an object has been detected by any of the external detection devices E1, E2, and E3 (S12).

[0052] If an object is detected by any of the external detection devices E1, E2, or E3, the displacement detection unit 302A proceeds to S13. If no object is detected by any of the external detection devices E1, E2, or E3, the displacement detection unit 302A continues to determine whether an object has been detected by any of the external detection devices E1, E2, or E3 until the displacement detection process is terminated (S10, S11, S12, S15).

[0053] Furthermore, in order to confirm that the objects detected by each of the external detection devices E1, E2, and E3 are the same object, the displacement detection unit 302A may perform a process to determine whether the objects detected by each of the external detection devices E1, E2, and E3 are the same object based on characteristic quantities such as the shape of the detected object.

[0054] When an object is detected by any of the external detection devices E1, E2, or E3, the displacement detection unit 302A stores the detection result in its memory area (S13). The detection result includes an identifier for identifying the external detection device E1, E2, or E3 that detected the object, the coordinates of the detected object, and the rotation angle of the upper rotating body 102 relative to the lower traveling body 101, indicated by the value of the attitude detection device P1 at the time of detection. If the memory area of ​​the displacement detection unit 302A already contains detection results for external detection devices E1, E2, or E3 with the same identifier, it updates the memory area with the newly acquired detection result.

[0055] Alternatively, the displacement detection unit 302A may store the detection result in S13 after confirming that the object detected by the external detection devices E1, E2, and E3 has not moved, based on the coordinates of the object detected by the same external detection devices E1, E2, and E3 within a predetermined period. Alternatively, the displacement detection unit 302A may determine the running state of the construction machine 100 (not shown) using a running state detection means, and discard the detection result stored in the memory if the construction machine 100 is running.

[0056] The displacement detection unit 302A determines whether detection results from two or more of the three external detection devices E1, E2, and E3 are stored in the memory area of ​​the displacement detection unit 302A (S14). If detection results from two or more external detection devices E1, E2, and E3 are stored, the displacement detection unit 302A proceeds to S16. If detection results from two or more external detection devices E1, E2, and E3 are not stored, the displacement detection unit 302A repeats the processes of S10 to S15 until detection results from two or more external detection devices E1, E2, and E3 are stored.

[0057] The displacement detection unit 302A uses the detection results stored in the memory area of ​​the displacement detection unit 302A to determine whether the position and orientation of any of the external detection devices E1, E2, and E3 are out of alignment with the calibration information recorded by the information recording unit 301 (S16). In this case, the calibration information recorded by the information recording unit 301 may be an initial value or a value other than the initial value, as will be described later.

[0058] If the position and orientation of any of the external detection devices E1, E2, or E3 are misaligned, the misalignment detection unit 302A proceeds to S17. If the position and orientation of any of the external detection devices E1, E2, or E3 are not misaligned, the misalignment detection unit 302A repeats the processes of S10 to S16 until it detects a misalignment in the position and orientation of any of the external detection devices E1, E2, or E3.

[0059] The process of detecting a deviation in either the position or attitude of any of the external detection devices E1, E2, or E3 in S16 will be explained using Figures 11 and 12. Figure 11 shows the detection of object 501 by external detection device E1. As shown in Figure 11, the coordinates of object 501 detected by external detection device E1 at this time (for example, measured values ​​in a coordinate system where the roll axis, pitch axis, and yaw axis of external detection device E1 are the three coordinate axes of a Cartesian coordinate system) are defined as the first coordinate (X1, Y1, Z1), and the detection result of attitude detection device P1 is defined as the first rotation angle θ1.

[0060] Figure 12 shows the detection of object 501 by the external environment detection device E2. As shown in Figure 12, the coordinates of object 501 detected by the external environment detection device E2 at this time (for example, measured values ​​in a coordinate system where the roll axis, pitch axis, and yaw axis of the external environment detection device E2 are the three coordinate axes of the Cartesian coordinate system) are defined as the second coordinate (X2, Y2, Z2), and the detection result of the attitude detection device P1 is defined as the second rotation angle θ2.

[0061] Here, among the calibration information recorded by the information recording unit 301, the information regarding the position of the external detection device E1 is defined as three-dimensional coordinate values ​​(XC1, YC1, ZC1). The information regarding the attitude of the external detection device E1 is defined as angular information θCR1 (roll), θCP1 (pitch), and θCY1 (yaw).

[0062] Of the calibration information recorded by the information recording unit 301, the information regarding the position of the external detection device E2 is defined as three-dimensional coordinate values ​​(XC2, YC2, ZC2). The information regarding the attitude of the external detection device E2 is defined as angular information θCR2 (roll), θCP2 (pitch), and θCY2 (yaw). In this case, the first estimated coordinates (X1', Y1', Z1') of the object 501 that is estimated to be detected by the external detection device E1 can be expressed by the following formula (1).

[0063]

[0064] In equation (1), R(0,0,θ2) for the second rotation angle is a rotation matrix representing a rotation defined by roll angle = 0, pitch angle = 0, and yaw angle = θ2. R(θCR2,θCP2,θCY2) for the attitude of the external detection device E2 is a rotation matrix representing a rotation defined by roll angle = θCR2, pitch angle = θCP2, and yaw angle = θCY2.

[0065] R with respect to the first rotation angle －１ (0,0,θ1) is the inverse of the rotation matrix representing the rotation defined by roll angle = 0, pitch angle = 0, and yaw angle = θ1. R related to the attitude of the external detection device E1 －１ (θCR1, θCP1, θCY1) is the inverse of the rotation matrix that represents the rotation defined by roll angle = θCR1, pitch angle = θCP1, and yaw angle = θCY1.

[0066] The displacement detection unit 302A can detect a displacement in either the position or orientation of either the external detection device E1 or the external detection device E2 by comparing the first estimated coordinates with the first coordinates of the object 501 detected by the external detection device E1.

[0067] In other words, if external detection device E1 experiences a deviation in either position or orientation, but external detection device E2 does not, the first coordinate is incorrect, resulting in a large difference between the first coordinate and the first estimated coordinate. On the other hand, if external detection device E1 does not experience a deviation in either position or orientation, but external detection device E2 experiences a deviation in either position or orientation, the first estimated coordinate is incorrect, resulting in a large difference between the first coordinate and the first estimated coordinate.

[0068] Specifically, the displacement detection unit 302A can calculate the difference between the calibration information of the external detection devices E1 and E2 recorded by the information recording unit 301 and the actual position and orientation of the external detection devices E1 and E2, by using, for example, the least squares method, based on the difference between the first estimated coordinates (X1', Y1', Z1') and the measured first coordinates (X1, Y1, Z1).

[0069] If the difference between the position and orientation of the external detection devices E1 and E2 exceeds a predetermined threshold, the displacement detection unit 302A determines that a displacement has been detected. Note that the method of comparing the first estimated coordinates with the first coordinates described above cannot determine whether a displacement of position or orientation has occurred in either the external detection device E1 or E2. However, it can detect that a displacement of position or orientation has occurred in either the external detection device E1 or E2.

[0070] Return to the flowchart in Figure 10. As shown in Figure 10, if the position and orientation of any of the external detection devices E1, E2, or E3 are misaligned, the misalignment detection unit 302A notifies the information management unit 303 of the detection result of the misalignment of the position and orientation of the external detection devices E1 and E2 in S16 (S17), and terminates the process. Steps S10 to S16 are repeated until the processing termination condition is met. Here, the processing termination condition may be the engine of the construction machine 100 being stopped, or it may be that the operator has entered an input to instruct the termination of the process through the display terminal 108.

[0071] Furthermore, by using the detection results from three or more external detection devices E1, E2, and E3, it becomes possible to identify which of the external detection devices E1, E2, and E3 has a deviation in either position or orientation. In this case, in S16, the deviation detection unit 302A performs the following processing in addition to the above processing.

[0072] In this case, the external environment detection system 400A is further equipped with an external environment detection device E3 on the upper rotating body 102 in addition to the external environment detection devices E1 and E2. The information recording unit 301 records the position and orientation of external environment detection devices E1, E2 and E3. The displacement detection unit 302A detects any displacement in the position or orientation of any of the external environment detection devices E1, E2 and E3.

[0073] The displacement detection unit 302A calculates the first estimated coordinates of the object 501 that is estimated to be detected by the external detection device E1, based on the first rotation angle, the position and orientation of the external detection device E1 recorded by the information recording unit 301, the third coordinates of the object 501 detected by the external detection device E3, the third rotation angle of the upper rotating body relative to the lower traveling body 101 detected by the orientation detection device P1 when the object 501 is detected by the external detection device E3, and the position and orientation of the external detection device E3 recorded by the information recording unit 301.

[0074] External detection device E3 is a third external detection device. The position and orientation of external detection device E3 recorded by the information recording unit 301 are the calibration information described above. The object detected by external detection device E1, the object detected by external detection device E2, and the object detected by external detection device E3 are the same object.

[0075] The coordinates of the object 501 detected by the external detection device E3 (for example, measured values ​​in a coordinate system where the roll axis, pitch axis, and yaw axis of the external detection device E3 are the three axes of the Cartesian coordinate system) are defined as the third coordinate (X3, Y3, Z3), and the detection result of the attitude detection device P1 when the object 501 is detected by the external detection device E3 is defined as the third rotation angle θ3. Of the calibration information recorded by the information recording unit 301, the information regarding the position of the external detection device E3 is defined as three-dimensional coordinate values ​​(XC3, YC3, ZC3). The information regarding the attitude of the external detection device E3 is defined as angle information θCR3 (roll), θCP3 (pitch), and θCY3 (yaw). In this case, the first estimated coordinates (X1', Y1', Z1') of the object 501 that is estimated to be detected by the external detection device E1 can be expressed by the following formula (2).

[0076]

[0077] In equation (2), R(0,0,θ3) for the third rotation angle is a rotation matrix representing a rotation defined by roll angle = 0, pitch angle = 0, and yaw angle = θ3. R(θCR3,θCP3,θCY3) for the attitude of the external detection device E3 is a rotation matrix representing a rotation defined by roll angle = θCR3, pitch angle = θCP3, and yaw angle = θCY3. The rest is the same as in equation (1) above.

[0078] The displacement detection unit 302A detects any displacement in the position and orientation of any of the external detection devices E1, E2, and E3 by comparing the first estimated coordinate calculated based on the second coordinate, second rotation angle, and position and orientation of the external detection device E2 with the first estimated coordinate calculated based on the third coordinate, third rotation angle, and position and orientation of the external detection device E3, and the first coordinate of the object detected by the external detection device E1.

[0079] In other words, if there is a deviation in either position or orientation of the external detection device E1, but there is no deviation in either position or orientation of the external detection devices E2 and E3, then the first coordinate is incorrect. As a result, the difference between the first estimated coordinate based on the second coordinate, etc., and the first coordinate becomes large, and the difference between the first estimated coordinate based on the third coordinate, etc., and the first coordinate becomes large. Therefore, it can be determined that there is a deviation in either the position or orientation of the external detection device E1.

[0080] Furthermore, if there is no deviation in either position or orientation of external detection devices E1 and E3, but there is a deviation in either position or orientation of external detection device E2, then the first estimated coordinate based on the second coordinate, etc., is incorrect. As a result, the difference between the first estimated coordinate based on the second coordinate, etc., and the first coordinate becomes larger, while the difference between the first estimated coordinate based on the third coordinate, etc., and the first coordinate becomes smaller. Therefore, it can be determined that there is a deviation in either the position or orientation of external detection device E2.

[0081] Furthermore, if there is no deviation in either position or orientation of external detection devices E1 and E2, but there is a deviation in either position or orientation of external detection device E3, then the first estimated coordinates based on the third coordinates, etc., are incorrect. As a result, the difference between the first estimated coordinates based on the third coordinates, etc., and the first coordinates will be large, while the difference between the first estimated coordinates based on the second coordinates, etc., and the first coordinates will be small. Therefore, it can be determined that there is a deviation in either the position or orientation of external detection device E3.

[0082] Having identified the external detection devices E1, E2, and E3 that have experienced a deviation in either position or orientation as described above, the deviation detection unit 302A, in S16 of Figure 10, notifies the information management unit 303 that it has detected a deviation, as well as the identifiers of the external detection devices E1, E2, and E3 that are experiencing the deviation and the amount of the detected deviation.

[0083] Figure 13 shows an example of the display content of the display terminal 108 shown to the operator. As shown in Figure 13, the display terminal 108 displays the positional or orientational deviation of the external detection devices E1, E2, and E3, which have been detected by the deviation detection unit 302A of the calculation unit 200A. Specifically, the display screen of the display terminal 108 displays, based on the information notified from the information management unit 303, the presence or absence of deviation detection, the identifiers of the external detection devices E1, E2, and E3 in which the deviation occurred, and the amount of deviation detected in the display item 1201. The display screen of the display terminal 108 displays a schematic diagram 1203 of the construction machine 100 and an indicator line 1202 that allows identification of which external detection device E1, E2, and E3 in the schematic diagram 1203 is experiencing the deviation. The display terminal 108 displays the above information at S2 in Figure 9.

[0084] Furthermore, as shown in Figure 13, operation buttons 1204 and 1205 are displayed on the display screen of the display terminal 108. In S3 of Figure 9, the display terminal 108 receives input to these operation buttons 1204 and 1205 from the operator's actions. The operator can refer to the information displayed on the display terminal 108 and, depending on the magnitude of the deviation in the position and orientation of the external detection devices E1, E2, and E3, choose whether to continue operating the construction machine 100 while accepting the current deviation, or to correct the deviation in the position and orientation of the external detection devices E1, E2, and E3 before resuming operation of the construction machine 100.

[0085] If the current misalignment is acceptable, the operator presses the operation button 1204 shown in Figure 13. Then, in S4 shown in Figure 9, the information management unit 303 overwrites the calibration information recorded in the information recording unit 301 based on the misalignment of either the position or orientation of the external detection devices E1, E2, and E3 calculated by the misalignment detection unit 302A, and hides the misalignment detection information displayed on the display terminal 108. In other words, the information management unit 303 of the calculation unit 200A updates either the position or orientation of the external detection devices E1, E2, and E3 recorded in the information recording unit 301 in response to input from the display terminal 108.

[0086] If the current deviation is not to be tolerated and a deviation in any of the position and orientation of the external detection devices E1, E2, and E3 is to be corrected, the operator adjusts the bracket 109 based on the above information to the state before any deviation in the position and orientation of the external detection devices E1, E2, and E3 occurred, and then presses the operation button 1205 shown in Figure 13. Then, in S4 shown in Figure 9, the information management unit 303 maintains the calibration information recorded in the information recording unit 301 without changing it and hides the information related to deviation detection displayed on the display terminal 108. The display terminal 108 may also display the history of detected deviations in any of the position and orientation of the external detection devices E1, E2, and E3. Furthermore, if the bracket is adjusted so that the position and orientation of the external detection devices E1, E2, and E3 return to their initial values, the display terminal 108 may accept input to reset the calibration information to its initial values.

[0087] In this way, the operator can choose whether to accept the current deviation and continue operating the construction machine 100, or to correct the deviation in the position and orientation of the external detection devices E1, E2, and E3 before resuming operation of the construction machine 100, depending on the magnitude of the deviation in either position or orientation of the external detection devices E1, E2, and E3.

[0088] In this embodiment, the case in which all components of the external environment detection system 400A are installed on the construction machine 100 has been described. However, some components of the external environment detection system 400A may be installed outside the construction machine 100. For example, if at least each of the external environment detection devices E1, E2, and E3 and each of the attitude detection devices P1, P2, P3, and P4 are installed on the construction machine 100, the calculation unit 200A (for example, the information recording unit 301, the displacement detection unit 302A, and the information management unit 303) and the display terminal 108 may be installed outside the construction machine 100, for example, on an external server that can communicate with the construction machine 100.

[0089] Furthermore, the amount of displacement detected by the displacement detection unit 302A and the log of the operator's input in S4 shown in Figure 9 may be recorded in a storage area (not shown) of the construction machine 100, and output as information that can be viewed by mine managers, maintenance workers, etc.

[0090] [Effects] As described above, in Embodiment 1, the calculation unit 200A calculates the coordinates of an object that is estimated to be detected by the external detection devices E1, E2, and E3 based on the attitude of the upper rotating body 102 detected by the attitude detection device P1. The calculation unit 200A detects any deviation in the position or attitude of the external detection devices E1, E2, and E3 by comparing the calculated coordinates of the object with the coordinates of the object detected by the external detection devices E1, E2, and E3. Therefore, calibration is possible without requiring overlap of the measurement areas of the multiple external detection devices E1, E2, and E3.

[0091] In the first embodiment, the displacement detection unit 302A calculates the first estimated coordinates of an object that is estimated to be detected by the external detection device E1 based on the first rotation angle of the upper rotating body 102 when an object is detected by the external detection device E1, the position and orientation of the external detection device E1 recorded by the information recording unit 301, the second coordinates of the object detected by the external detection device E2, the second rotation angle of the upper rotating body 102 when an object is detected by the external detection device E2, and the position and orientation of the external detection device E2 recorded by the information recording unit 301. The displacement detection unit 302A detects a displacement in either the position or orientation of either the external detection device E1 or the external detection device E2 by comparing the first estimated coordinates with the first coordinates of the object detected by the external detection device E1. Therefore, by simply rotating the upper rotating body 102 and detecting objects with the multiple external detection devices E1 and E2, calibration can be performed without requiring overlap of the measurement areas of the multiple external detection devices E1, E2, and E3.

[0092] In the first embodiment, the displacement detection unit 302A calculates the first estimated coordinates of the object that is estimated to be detected by the external detection device E1 based on the first rotation angle, the position and orientation of the external detection device E1 recorded by the information recording unit 301, the third coordinates of the object detected by the external detection device E3, the third rotation angle of the upper rotating body 102 when the object was detected by the external detection device E3, and the position and orientation of the external detection device E3 recorded by the information recording unit 301. The displacement detection unit 302A detects any displacement in the position and orientation of any of the external detection devices E1, E2, and E3 by comparing the first estimated coordinates calculated based on the second coordinates, etc., the first estimated coordinates calculated based on the third coordinates, etc., and the first coordinates of the object detected by the external detection device E1.

[0093] In Embodiment 1, the display terminal 108 displays the positional or orientational deviation of the external detection devices E1, E2, and E3, which has been detected by the arithmetic unit 200A. Therefore, it is possible to understand the positional or orientational deviation of the external detection devices E1, E2, and E3.

[0094] In Embodiment 1, the information recording unit 301 records either the position or orientation of the external detection devices E1, E2, and E3 of the construction machine 100. The information management unit 303 updates either the position or orientation of the external detection devices E1, E2, and E3 recorded by the information recording unit 301 in response to input from the display terminal 108. Therefore, depending on the magnitude of the deviation in either the position or orientation of the external detection devices E1, E2, and E3, it is possible to choose whether to continue operating the construction machine 100 while tolerating the current deviation, or to correct the deviation in either the position or orientation of the external detection devices E1, E2, and E3 before resuming operation of the construction machine 100.

[0095] In Example 1, a construction machine 100 having an external environment detection system 400A is provided. Therefore, calibration can be performed without requiring overlap of measurement areas of multiple external environment detection devices E1, E2, and E3, and it is not necessary to increase the number of external environment detection devices E1, E2, and E3 that need to be mounted on the construction machine 100 for calibration.

[0096] [Example 2] [Overall Configuration] Next, the external environment detection system 400B of the construction machine 100 according to Example 2 will be described using Figures 14 and 15. The overall configuration of the external environment detection system 400B of the construction machine 100 in Example 2 is the same as in Example 1, so redundant explanations will be omitted.

[0097] Figure 14 is a functional block diagram of the external environment detection system 400B of this embodiment. As shown in Figure 14, the external environment detection system 400B in this embodiment includes, for example, an external environment detection device E1, attitude detection devices P2, P3, P4, a calculation device 200B, and a display terminal 108. The calculation device 200B includes an information recording unit 301 and an information management unit 303, similar to those in Embodiment 1, as well as a displacement detection unit 302B. In this embodiment, at least one of the external environment detection devices E1, E2, and E3 includes the work device 103 of the construction machine 100 in the measurement areas R1, R2, and R3. For example, the external environment detection device E1 is attached to the front of the upper rotating body 102 and includes the work device 103 in the measurement area R1. The external environment detection device E1 detects the work device 103 as an object.

[0098] In this embodiment, the calculation unit 200B calculates the coordinates of an object that is estimated to be detected by the external environment detection device E1 based on the posture of the work device 103 detected by the posture detection devices P2, P3, and P4. The calculation unit 200B detects a deviation in either the position or posture of the external environment detection device E1 by comparing the calculated object coordinates with the coordinates of the object detected by the external environment detection device E1.

[0099] More specifically, the displacement detection unit 302B of the calculation unit 200B calculates the estimated coordinates of the work device 103, which are estimated to be detected by the external detection device E1, based on the posture of the work device 103 detected by the posture detection devices P2, P3, and P4, and the position and posture of the external detection device E1 recorded by the information recording unit 301. The displacement detection unit 302B detects a displacement in either the position or posture of the external detection device E1 by comparing the estimated coordinates with the coordinates of the work device 103 detected by the external detection device E1.

[0100] In Example 1, multiple external detection devices E1, E2, and E3 detected the same object 501, and the displacement of the external detection devices E1, E2, and E3 was detected from the difference between the first estimated coordinates and the first coordinates. In other words, in Example 1, the external detection devices E1, E2, and E3 were each used as a reference to detect the displacement of either the position or orientation of the external detection devices E1, E2, and E3 from each other.

[0101] On the other hand, in this embodiment, the coordinates of the work device 103 are calculated based on the detection results of the multiple posture detection devices P2, P3, and P4, and the position and posture of the external environment detection device E1 recorded by the information recording unit 301. The deviation of either the position or the coordinates of the external environment detection device E1 is then detected based on the calculated estimated coordinates and the measured coordinates of the work device 103 detected by the external environment detection device E1. In other words, in this embodiment, the deviation of either the position or the posture of the external environment detection device E1 is detected based on the estimated coordinates of the work device 103, which are estimated based on the detection results of the posture detection devices P2, P3, and P4, and the position and posture of the external environment detection device E1 recorded by the information recording unit 301.

[0102] Figure 15 is a functional block diagram of the displacement detection unit 302B shown in Figure 14. As shown in Figure 15, the displacement detection unit 302B consists of a work device position estimation unit 1001, a detection result management unit 1002, a storage area 1003, and a determination unit 1004. The work device position estimation unit 1001 calculates the estimated coordinates of the work device 103 that are estimated to be detected by the external detection device E1, based on the posture of the work device 103 detected by posture detection devices P2, P3, and P4, and the position and posture of the external detection device E1 recorded by the information recording unit 301, and outputs this to the detection result management unit 1002.

[0103] The detection result management unit 1002 stores the coordinates of the work device 103 detected by the external detection device E1 and the estimated coordinates of the work device 103 calculated by the work device position estimation unit 1001 in the storage area 1003, and controls the operation of the determination unit 1004 based on the information storage status in the storage area 1003. The determination unit 1004 determines the difference between the calibration information recorded by the information recording unit 301 and either the actual position or orientation of the external detection device E1, based on the coordinates of the work device 103 and the estimated coordinates of the work device 103 stored in the storage area 1003, and outputs the determination result to the information management unit 303.

[0104] [Operation] The process of detecting deviations and updating calibration information of the external environment detection system 400B of the construction machine 100 in this embodiment will be described. Descriptions of the processes in this embodiment that overlap with those in Embodiment 1 will be omitted.

[0105] As shown in Figure 9, when a predetermined condition is met, the displacement detection unit 302B detects a displacement in either the position or orientation of the external detection device E1 (S1). In this embodiment, the predetermined condition is that the operator inputs to the display terminal 108 to start detecting a displacement in either the position or orientation of the external detection device E1. In other words, the arithmetic unit 200B starts detecting a displacement in either the position or orientation of the external detection device E1 when it receives an input to the display terminal 108.

[0106] The details of S1 in Figure 9 in this embodiment will be explained using the flowchart in Figure 16. As shown in Figure 16, the displacement detection unit 302B repeats the following processes until the displacement detection process is terminated (S20 to S26). The external environment detection device E1 performs a process to detect the work device 103 (S21). The work device position estimation unit 1001 performs a process to calculate the estimated coordinates of the work device 103 that are estimated to be detected by the external environment detection device E1, based on the posture of the work device 103 detected by the posture detection devices P2, P3, and P4, and the position and posture of the external environment detection device E1 recorded by the information recording unit 301 (S22). The detection result management unit 1002 determines whether the external environment detection device E1 has detected the coordinates of the work device 103 and whether the work device position estimation unit 1001 has calculated the estimated coordinates of the work device 103 (S23).

[0107] If the external detection device E1 detects the coordinates of the work device 103 and the work device position estimation unit 1001 calculates the estimated coordinates of the work device 103, the detection result management unit 1002 proceeds to S24. If the external detection device E1 has not detected the coordinates of the work device 103 and the work device position estimation unit 1001 has not calculated the estimated coordinates of the work device 103, the detection result management unit 1002 continues to determine whether the external detection device E1 has detected the coordinates of the work device 103 and whether the work device position estimation unit 1001 has calculated the estimated coordinates of the work device 103 until the displacement detection process is terminated (S20, S21, S22, S23, S26).

[0108] Furthermore, in S21, the detection of the coordinates of the work device 103 by the external detection device E1 can be performed, for example, by using the coordinates 1601 of the center of the bucket 106 detected by the external detection device E1 as the coordinates of the work device 103, as shown in Figure 17. The external detection device E1 can, for example, detect the coordinates 1601 of the center of the bucket 106 in the external detection device coordinate system 1603 of the external detection device E1 as the coordinates of the work device 103. The external detection device coordinate system 1603 of the external detection device E1 is, for example, a coordinate system in which the roll axis, pitch axis, and yaw axis of the external detection device E1 are the three axes of the Cartesian coordinate system.

[0109] Furthermore, in S22, the calculation of the estimated coordinates of the work device 103 by the work device position estimation unit 1001 is performed by, for example, the work device position estimation unit 1001 calculating the coordinates 1601 of the center of the bucket 106 in the vehicle body coordinate system 1602 based on the detection results of the attitude detection devices P2, P3, and P4 and the dimensional values ​​of the boom 104, arm 105, and bucket 106 of the work device 103. The vehicle body coordinate system 1602 is a coordinate system in which the three coordinate axes of the Cartesian coordinate system are, for example, the longitudinal axis of the lower traveling body 101 included in the ground contact surface of the lower traveling body 101, the left-right axis of the lower traveling body 101 included in the ground contact surface of the lower traveling body 101, and the vertical axis passing through the center of the lower traveling body.

[0110] The work device position estimation unit 1001 calculates the coordinates 1601 of the center of the bucket 106 in the vehicle body coordinate system 1602 based on the calibration information of the external detection device E1 recorded by the information recording unit 301, i.e., the position and orientation of the external detection device E1, and transforms these coordinates to the coordinates 1601 of the center of the bucket 106 in the external detection device coordinate system 1603. The coordinates 1601 of the center of the bucket 106 in the external detection device coordinate system 1603, after the coordinate transformation, are the estimated coordinates of the work device 103.

[0111] Furthermore, the external environment detection device E1 may detect the coordinates of the tip of the bucket 106 as the coordinates of the work device 103. The work device position estimation unit 1001 may calculate the estimated coordinates of the tip of the bucket 106 as the estimated coordinates of the work device 103, similar to the center of the bucket 106. Alternatively, the external environment detection device E1 may detect the coordinates of the bottom surface of the bucket 106 detected by the external environment detection device E1 as the coordinates of the work device 103, and the work device position estimation unit 1001 may calculate the estimated coordinates of the bottom surface of the bucket 106 as the estimated coordinates of the work device 103.

[0112] Alternatively, the external environment detection device E1 may detect the coordinates of a portion of the arm 105 as the coordinates of the work device 103, and the work device position estimation unit 1001 may calculate the estimated coordinates of a portion of the arm 105 as the estimated coordinates of the work device 103. Alternatively, the external environment detection device E1 may detect the coordinates of a portion of the boom 104 as the coordinates of the work device 103, and the work device position estimation unit 1001 may calculate the estimated coordinates of a portion of the boom 104 as the estimated coordinates of the work device 103.

[0113] As shown in Figure 16, when the external detection device E1 detects the coordinates of the work device 103 and the work device position estimation unit 1001 calculates the estimated coordinates of the work device 103, the detection result management unit 1002 stores the coordinates of the work device 103 detected by the external detection device E1 and the estimated coordinates of the work device 103 calculated by the work device position estimation unit 1001 in the storage area 1003 (S24). The detection result management unit 1002 determines whether the number of coordinates and estimated coordinates of the work device 103 stored in the storage area 1003 is equal to or greater than a predetermined number (S25). If the number of coordinates and estimated coordinates of the work device 103 is equal to or greater than a predetermined number, the detection result management unit 1002 proceeds to S27. If the number of coordinates and estimated coordinates of the work device 103 is less than or equal to a predetermined number, the detection result management unit 1002 repeats the processes in S20 to S26.

[0114] The determination unit 1004 uses the coordinates and estimated coordinates of the work device 103 stored in the memory area 1003 to determine whether the position or orientation of the external detection device E1 is deviated from the calibration information recorded by the information recording unit 301 (S27). Specifically, for example, the determination unit 1004 uses each of the estimated coordinates of the work device 103 stored in the memory area 1003 as estimated coordinates approximated by the least squares method. The determination unit 1004 also uses each of the coordinates of the work device 103 stored in the memory area 1003 as coordinates approximated by the least squares method.

[0115] The determination unit 1004 can determine whether the position or orientation of the external detection device E1 is deviated from the calibration information recorded by the information recording unit 301, and the amount of the deviation, by comparing the estimated coordinates of the approximated work device 103 with the coordinates of the approximated work device 103. If the determination unit 1004 determines that the position or orientation of the external detection device E1 is deviated, the process proceeds to S28. If the deviation detection unit 302B determines that the position and orientation of the external detection device E1 are not deviated, the process from S20 to S27 is repeated.

[0116] In addition, in the processing of S20 to S27 described above, the work device position estimation unit 1001 does not need to convert the coordinates of the work device 103 in the calculated vehicle body coordinate system 1602 to the external detection device coordinate system 1603. The detection result management unit 1002 may store the coordinates of the work device 103 in the vehicle body coordinate system 1602 calculated by the work device position estimation unit 1001 in the storage area 1003. The determination unit 1004 may convert the coordinates of the work device 103 in the vehicle body coordinate system 1602 stored in the storage area 1003 to the external detection device coordinate system 1603 and determine whether the position or orientation of the external detection device E1 is shifted as described above.

[0117] As shown in S28 of Figure 16, the displacement detection unit 302B notifies the information management unit 303 that it has detected a displacement, as well as the identifier of the external detection device E1 where the displacement occurred and the amount of displacement detected. The information management unit 303 and the display terminal 108 perform the same processing as in the above embodiment 1.

[0118] [Effects] As described above, in Embodiment 2, the displacement detection unit 302B calculates estimated coordinates of the work device 103 that are estimated to be detected by the external detection device E1, based on the posture of the work device 103 detected by the posture detection devices P2, P3, and P4, and the position and posture of the external detection device E1 recorded by the information recording unit 301. The displacement detection unit 302B detects a displacement in either the position or posture of the external detection device E1 by comparing the estimated coordinates with the coordinates of the work device 103 detected by the external detection device E1. For this reason, it is not necessary for the measurement areas of the multiple external detection devices E1, E2, and E3 to overlap, and a displacement in either the position or posture of the external detection device E1 can be detected without operating the construction machine 100.

[0119] Furthermore, in Embodiment 2, the arithmetic unit 200B starts detecting deviations in either the position or orientation of the external environment detection device E1 in response to input from the display terminal 108. Therefore, the operator can detect deviations in either the position or orientation of the external environment detection device E1 at a timing of their choosing.

[0120] [Example 3] [Overall Configuration] Next, the external environment detection system 400C of the construction machine 100 according to Example 3 will be described using Figures 18 and 19. The overall configuration of the external environment detection system 400C of the construction machine 100 in Example 3 is the same as in Example 1, so redundant explanations will be omitted.

[0121] Figure 18 is a functional block diagram of the external environment detection system 400C of this embodiment. As shown in Figure 18, the external environment detection system 400C in this embodiment includes, for example, an external environment detection device E1, attitude detection devices P2, P3, P4, a calculation device 200C, and a display terminal 108. The calculation device 200C includes an information recording unit 301 and an information management unit 303, similar to those in Embodiment 1, as well as a displacement detection unit 302C. In this embodiment, at least one of the external environment detection devices E1, E2, and E3 includes the work device 103 of the construction machine 100 in the measurement areas R1, R2, and R3. For example, the external environment detection device E1 is attached to the front of the upper rotating body 102 and includes the work device 103 in the measurement area R1. The external environment detection device E1 detects the work device 103 as an object.

[0122] In this embodiment, the calculation unit 200C calculates the coordinates of an object that is estimated to be detected by the external environment detection device E1 based on the posture of the work device 103 detected by the posture detection devices P2, P3, and P4. The calculation unit 200C detects any deviation in either the position or posture of the external environment detection device E1 by comparing the estimated coordinates of the object with the coordinates of the object detected by the external environment detection device E1.

[0123] More specifically, when the attitude detection devices P2, P3, and P4 detect that the work device 103 is performing a predetermined operation, the displacement detection unit 302C of the calculation unit 200C calculates an estimated trajectory of the coordinates of the work device 103 that is estimated to be detected by the external environment detection device E1 when the work device 103 is performing a predetermined operation, based on the attitude of the work device 103 detected by the attitude detection devices P2, P3, and P4 and the position and attitude of the external environment detection device E1 recorded by the information recording unit 301. Note that the calculation of the estimated trajectory includes not only calculating a new estimated trajectory but also extracting a previously stored estimated trajectory from the storage area. The displacement detection unit 302C detects a displacement in either the position or attitude of the external environment detection device E1 by comparing the estimated trajectory with the trajectory of the coordinates of the work device 103 detected by the external environment detection device E1 when the work device 103 is performing a predetermined operation.

[0124] In this embodiment, the estimated trajectory of the coordinates of the work device 103 is calculated based on the detection results of the multiple posture detection devices P2, P3, and P4, and the position and posture of the external environment detection device E1 recorded by the information recording unit 301. The deviation of either the position or coordinates of the external environment detection device E1 is then detected from the calculated estimated trajectory and the measured trajectory of the coordinates of the work device 103 detected by the external environment detection device E1. In other words, in this embodiment, the deviation of either the position or posture of the external environment detection device E1 is detected based on the estimated trajectory of the coordinates of the work device 103, which is estimated based on the detection results of the posture detection devices P2, P3, and P4, and the position and posture of the external environment detection device E1 recorded by the information recording unit 301.

[0125] Figure 19 is a functional block diagram of the displacement detection unit 302C shown in Figure 18. As shown in Figure 19, the displacement detection unit 302C consists of a work device trajectory generation unit 2001, a work device trajectory estimation unit 2002, a detection result management unit 2003, a storage area 2004, and a determination unit 2005. The work device trajectory generation unit 2001 generates the trajectory of the coordinates of the work device 103 based on the detection result of the external detection device E1 and outputs it to the detection result management unit 2003. The work device trajectory estimation unit 2002 calculates the estimated trajectory of the coordinates of the work device 103 that is estimated to be detected by the external detection device E1, based on the posture of the work device 103 detected by the posture detection devices P2, P3, and P4, and the position and posture of the external detection device E1 recorded by the information recording unit 301, and outputs it to the detection result management unit 2003.

[0126] The detection result management unit 2003 stores the trajectory of the coordinates of the work device 103 generated by the work device trajectory generation unit 2001 and the estimated trajectory of the coordinates of the work device 103 estimated by the work device trajectory estimation unit 2002 in the memory area 2004, and controls the operation of the determination unit 2005 based on the information storage status in the memory area 2004. Based on the trajectory of the work device 103 and the estimated trajectory of the work device 103 stored in the memory area 2004, the determination unit 2005 determines the difference between the calibration information recorded by the information recording unit 301 and either the actual position or orientation of the external detection device E1, and outputs the determination result to the information management unit 303.

[0127] [Operation] The displacement detection and calibration information update process of the external environment detection system 400C of the construction machine 100 in this embodiment will be described. Descriptions of the processes in this embodiment that overlap with those of previously described embodiments will be omitted.

[0128] In this embodiment, the conditions for executing the process S1 in Figure 9 are that the work device 103 is in a predetermined position and the operator has performed a predetermined operation. An example of a predetermined position and predetermined operation will be explained using Figure 20. For example, as shown by the numeral (a) in Figure 20, the predetermined position is when the arm 105 and bucket 106 are held up to their stroke ends. The displacement detection unit 302C recognizes that this predetermined position has been achieved by the detection results of the position detection devices P2, P3, and P4 or by input from the operator to the display terminal 108. In other words, the calculation unit 200C can start detecting a displacement of either the position or the position of the external detection device E1 when it receives input from the display terminal 108. Alternatively, as shown by the numerals (b) and (c) in Figure 20, the displacement detection unit 302C may recognize that the operator has operated the control stick to lift the boom 104 and start executing the process S1 in Figure 9.

[0129] The details of the process in S1 of Figure 9 in this embodiment will be explained below with reference to Figures 20, 21, and 22. As shown in Figure 21, the displacement detection unit 302C repeats the following processes until the displacement detection process is terminated (S30 to S36). The work device trajectory generation unit 2001 executes a process to generate the trajectory of the work device 103 (S31). The work device trajectory estimation unit 2002 executes a process to calculate the estimated trajectory of the coordinates of the work device 103 that is estimated to be detected by the external detection device E1, based on the posture of the work device 103 detected by the posture detection devices P2, P3, and P4, and the position and posture of the external detection device E1 recorded by the information recording unit 301 (S32).

[0130] In S31, the work device trajectory generation unit 2001 of the displacement detection unit 302C acquires the coordinate detection result of the center of the bucket 106 by the external detection device E1 each time the center of the bucket 106 moves by a certain amount or more due to the lifting operation of the boom 104, and stores it in the memory area of ​​the work device trajectory generation unit 2001.

[0131] For example, the work device trajectory generation unit 2001 obtains the center coordinate 1801 of the bucket 106 in the external detection device coordinate system 1603 of the external detection device E1 when it is in the posture shown by the symbol (a) in Figure 20, based on the detection result of the external detection device E1. Next, the work device trajectory generation unit 2001 obtains the center coordinate 1802 of the bucket 106 in the external detection device coordinate system 1603 when it is in the posture shown by the symbol (b) in Figure 20, based on the detection result of the external detection device E1. Next, the work device trajectory generation unit 2001 obtains the center coordinate 1803 of the bucket 106 in the external detection device coordinate system 1603 when it is in the posture shown by the symbol (c) in Figure 20, based on the detection result of the external detection device E1. In this way, the work device trajectory generation unit 2001 acquires a predetermined number of center coordinates 1801, 1802, 1803, etc., of the buckets 106 in the external detection device coordinate system 1603, and stores them in the memory area of ​​the work device trajectory generation unit 2001.

[0132] The work device trajectory generation unit 2001 generates a trajectory 1901 of the bucket 106's center coordinates 1801, 1802, 1803, etc. in the external detection device coordinate system 1603, as shown in Figure 22, as a regression curve of the center coordinates 1801, 1802, 1803, etc. of the bucket 106 in the external detection device coordinate system 1603, etc.

[0133] Meanwhile, the work device trajectory estimation unit 2002 of the displacement detection unit 302C calculates the coordinates of the center of the bucket 106 based on the detection results of the posture detection devices P2, P3, and P4 and the dimensional values ​​of the boom 104, arm 105, and bucket 106 of the work device 103, whenever the center of the bucket 106 moves by a certain amount due to the lifting operation of the boom 104, and stores it in the memory area of ​​the work device trajectory estimation unit 2002.

[0134] Immediately after the start of processing S1 in Figure 9, the work device trajectory estimation unit 2002 calculates the center coordinate 1801 of the bucket 106 in the vehicle coordinate system 1602 for the posture shown by (a) in Figure 20, based on the detection results of the posture detection devices P2, P3, and P4 and the dimensional values ​​of the boom 104, arm 105, and bucket 106 of the work device 103. Next, the work device trajectory estimation unit 2002 calculates the center coordinate 1802 of the bucket 106 in the vehicle coordinate system 1602 for the posture shown by (b) in Figure 20, based on the detection results of the posture detection devices P2, P3, and P4 and the dimensional values ​​of the boom 104, arm 105, and bucket 106 of the work device 103. Next, the work device trajectory estimation unit 2002 calculates the center coordinate 1803 of the bucket 106 in the vehicle body coordinate system 1602 when the vehicle is in the posture shown by (c) in Figure 20, based on the detection results of the posture detection devices P2, P3, and P4 and the dimensional values ​​of the boom 104, arm 105, and bucket 106 of the work device 103. In this way, the work device trajectory estimation unit 2002 calculates a predetermined number of center coordinates 1801, 1802, 1803, etc., of the bucket 106 in the vehicle body coordinate system 1602 and stores them in the memory area of ​​the work device trajectory estimation unit 2002.

[0135] The work device trajectory estimation unit 2002 calculates regression curves such as the center coordinates 1801, 1802, 1803 of the bucket 106 in the vehicle coordinate system 1602, which are stored in the memory area. Based on the calibration information of the external detection device E1 recorded by the information recording unit 301, i.e., the position and orientation of the external detection device E1, the work device trajectory estimation unit 2002 transforms the estimated regression curves from the vehicle coordinate system 1602 to the external detection device coordinate system 1603. The work device trajectory estimation unit 2002 calculates the estimated trajectory 1902 such as the center coordinates 1801, 1802, 1803 of the bucket 106 in the external detection device coordinate system 1603, as shown in Figure 22.

[0136] The work device trajectory generation unit 2001 may acquire the trajectory of the coordinates of the tip of the bucket 106 as the trajectory of the coordinates of the work device 103. The work device trajectory estimation unit 2002 may calculate the estimated trajectory of the coordinates of the tip of the bucket 106 as the estimated trajectory of the work device 103, similar to the trajectory of the center of the bucket 106. Alternatively, the work device trajectory generation unit 2001 may acquire the trajectory of the coordinates of the bottom surface of the bucket 106 as the trajectory of the coordinates of the work device 103, and the work device trajectory estimation unit 2002 may calculate the estimated trajectory of the coordinates of the bottom surface of the bucket 106 as the estimated trajectory of the work device 103.

[0137] Alternatively, the work device trajectory generation unit 2001 may acquire the trajectory of some coordinates of the arm 105 as the trajectory of the coordinates of the work device 103, and the work device trajectory estimation unit 2002 may calculate the estimated trajectory of some coordinates of the arm 105 as the estimated trajectory of the work device 103. Alternatively, the work device trajectory generation unit 2001 may acquire some coordinates of the boom 104 as the trajectory of the coordinates of the work device 103, and the work device trajectory estimation unit 2002 may calculate the estimated trajectory of some coordinates of the boom 104 as the estimated trajectory of the work device 103.

[0138] As shown in Figure 21, the detection result management unit 2003 determines whether the work device trajectory generation unit 2001 has generated a trajectory of the coordinates of the work device 103 based on the detection results of the external detection device E1, and whether the work device trajectory estimation unit 2002 has calculated an estimated trajectory of the work device 103 based on the detection results of the attitude detection devices P2, P3, and P4 (S33). If the work device trajectory generation unit 2001 has generated a trajectory 1901 of the coordinates of the work device 103, and the work device trajectory estimation unit 2002 has calculated an estimated trajectory 1902 of the work device 103, the detection result management unit 2003 proceeds to S34. When the work device trajectory generation unit 2001 does not generate a trajectory of the coordinates of the work device 103, and when the work device trajectory estimation unit 2002 does not calculate an estimated trajectory of the work device 103, the detection result management unit 2003 repeats the processes of S30 to S33.

[0139] The detection result management unit 2003 stores the trajectory 1901 generated by the work device trajectory generation unit 2001 and the estimated trajectory 1902 calculated by the work device trajectory estimation unit 2002 in the storage area 2004 (S34). The determination unit 2005 uses the trajectory 1901 and estimated trajectory 1902 of the work device 103 stored in the storage area 2004 to determine whether the position or orientation of the external detection device E1 is deviated from the calibration information recorded by the information recording unit 301 (S35).

[0140] The determination unit 2005 can determine whether the position or orientation of the external detection device E1 is deviating from the calibration information recorded by the information recording unit 301, and the amount of the deviation, by comparing the trajectory 1901 with the estimated trajectory 1902. If the determination unit 2005 determines that either the position or orientation of the external detection device E1 is deviating, the process proceeds to S37. If the deviation detection unit 302C determines that the position and orientation of the external detection device E1 are not deviating, the process from S30 to S36 is repeated.

[0141] In addition, in the processing of S30 to S35 described above, the work device trajectory estimation unit 2002 does not need to perform coordinate transformation of the estimated trajectory of the coordinates of the work device 103 in the vehicle body coordinate system 1602 to the external detection device coordinate system 1603. The detection result management unit 2003 may store the trajectory of the coordinates of the work device 103 in the vehicle body coordinate system 1602 estimated by the work device trajectory estimation unit 2002 in the storage area 2004. The determination unit 2005 may perform coordinate transformation of the trajectory of the coordinates of the work device 103 in the vehicle body coordinate system 1602 stored in the storage area 2004 to the external detection device coordinate system 1603 to obtain the estimated trajectory 1902, and determine whether the position or orientation of the external detection device E1 is shifted as described above.

[0142] Furthermore, the determination unit 2005 may extract the estimated trajectory 1902, which has been previously stored in the memory area 2004, from the memory area 2004 and determine whether the position or orientation of the external detection device E1 is shifted. In other words, when the construction machine 100 is manufactured or during periodic maintenance, the work device trajectory estimation unit 2002 may store the estimated trajectory 1902 calculated by executing the above process in the memory area 2004 in advance. Also, when the construction machine 100 is manufactured or during periodic maintenance, the work device trajectory generation unit 2001 may store the trajectory 1901 generated by executing the above process in the memory area 2004 in advance as the estimated trajectory 1902.

[0143] As shown in S36 of Figure 21, the displacement detection unit 302C notifies the information management unit 303 that it has detected a displacement, as well as the identifier of the external detection device E1 where the displacement occurred and the amount of displacement detected. The information management unit 303 and the display terminal 108 perform the same processing as in the above embodiment 1.

[0144] [Effect] As described above, in Embodiment 3, when the posture detection devices P2, P3, and P4 detect that the work device 103 is performing a predetermined operation, the displacement detection unit 302C calculates an estimated trajectory 1902 of the coordinates of the work device that is estimated to be detected by the external environment detection device E1 when the work device 103 is performing a predetermined operation. The displacement detection unit 302C compares the estimated trajectory 1902 with the trajectory 1901 of the coordinates of the work device 103 detected by the external environment detection device E1 when the work device 103 is performing a predetermined operation, thereby detecting a displacement in either the position or posture of the external environment detection device E1. Therefore, without requiring overlapping measurement areas of multiple external detection devices E1, E2, and E3, the deviation of either the position or orientation of the external detection device E1 can be detected with higher accuracy by comparing the trajectory 1901 obtained from multiple center coordinates 1801, 1802, 1803, etc. of the bucket 106 with the estimated trajectory 1902.

[0145] [Example 4] [Overall Configuration] Next, the external environment detection system 400D of the construction machine 100 according to Example 4 will be described using Figures 23 and 24. The overall configuration of the external environment detection system 400D of the construction machine 100 in Example 4 is the same as in Example 1, so redundant explanations will be omitted.

[0146] Figure 23 is a functional block diagram of the external environment detection system 400D of this embodiment. As shown in Figure 23, the external environment detection system 400D of this embodiment includes, for example, external environment detection devices E1, E2, E3, attitude detection device P1, calculation device 200D, and display terminal 108. The calculation device 200D includes an information recording unit 301 and an information management unit 303 similar to those in Embodiment 1 above, as well as a displacement detection unit 302D.

[0147] In this embodiment, the external environment detection system 400D is equipped with external environment detection devices E1, E2, and E3 on the upper rotating body 102. In this embodiment, the external environment detection system 400D may be equipped with all of the external environment detection devices E1, E2, and E3, or it may be equipped with any one of the external environment detection devices E1, E2, and E3. In this embodiment, the external environment detection devices E1, E2, and E3 detect objects whose external coordinates are known. An object whose coordinates are known is, for example, a sign that is installed at a predetermined coordinate in the vehicle body coordinate system 1602 and has characteristics that make it easy to detect by the external environment detection devices E1, E2, and E3.

[0148] The calculation unit 200D calculates the coordinates of object 501, which is estimated to be detected by external detection devices E1, E2, and E3, based on the attitude of the upper rotating body 102 detected by the attitude detection device P1. By comparing the calculated coordinates of object 501 with the coordinates of object 501 detected by external detection devices E1, E2, and E3, it detects any deviation in the position or attitude of external detection devices E1, E2, and E3.

[0149] More specifically, when the attitude detection device P1 detects that the upper rotating body 102 is rotating relative to the lower traveling body 101, the displacement detection unit 302D calculates an estimated trajectory of the coordinates of an object that is estimated to be detected by the external detection devices E1, E2, E3 when the upper rotating body 102 is rotating relative to the lower traveling body 101, based on the rotation angle of the upper rotating body 102 relative to the lower traveling body 101 detected by the attitude detection device P1, the positions and attitudes of the external detection devices E1, E2, E3 recorded by the information recording unit 301, and the known coordinates of the object.

[0150] The displacement detection unit 302D detects any displacement in either the position or orientation of the external detection devices E1, E2, and E3 by comparing the estimated trajectory with the trajectory of the coordinates of the object detected by the external detection devices E1, E2, and E3 when the upper rotating body 102 is rotating relative to the lower traveling body 101.

[0151] In this embodiment, the estimated trajectory of an object such as a sign is calculated based on the rotation angle of the upper rotating body 102 relative to the lower traveling body 101 detected by the attitude detection device P1, the positions and attitudes of the external detection devices E1, E2, and E3 recorded by the information recording unit 301, and the coordinates of a known object such as a sign. The deviation of any of the positions and coordinates of the external detection devices E1, E2, and E3 is detected from the calculated estimated trajectory and the trajectory of the coordinates of the object such as a sign, which are measured values ​​detected by the external detection devices E1, E2, and E3. In other words, in this embodiment, the deviation of any of the positions and attitudes of the external detection devices E1, E2, and E3 is detected based on the estimated trajectory of the coordinates of the object such as a sign, which is estimated from the detection result of the attitude detection device P1, the positions and attitudes of the external detection devices E1, E2, and E3 recorded by the information recording unit 301, and the coordinates of a known object such as a sign.

[0152] Figure 24 is a functional block diagram of the displacement detection unit 302D shown in Figure 23. As shown in Figure 24, the displacement detection unit 302D consists of an object trajectory generation unit 3001, an object trajectory estimation unit 3002, a detection result management unit 3003, a storage area 3004, and a determination unit 3005. The object trajectory generation unit 3001 generates the trajectory of the coordinates of an object such as a sign based on the detection results of the external detection devices E1, E2, and E3, and outputs it to the detection result management unit 3003. The object trajectory estimation unit 3002 calculates the estimated trajectory of the coordinates of an object such as a sign that is estimated to be detected by the external detection devices E1, E2, E3, based on the rotation angle of the upper rotating body 102 relative to the lower traveling body 101 detected by the attitude detection device P1, the positions and attitudes of the external detection devices E1, E2, E3 recorded by the information recording unit 301, and the coordinates of a known object such as a sign, and outputs it to the detection result management unit 3003.

[0153] The detection result management unit 3003 stores the trajectory of the coordinates of an object such as a sign, generated by the object trajectory generation unit 3001, and the estimated trajectory of the coordinates of an object such as a sign, estimated by the object trajectory estimation unit 3002, in the storage area 3004, and controls the operation of the determination unit 3005 based on the information storage status in the storage area 3004. Based on the object trajectory and the estimated object trajectory stored in the storage area 3004, the determination unit 3005 determines the difference between the calibration information recorded by the information recording unit 301 and either the actual position or orientation of the external detection devices E1, E2, and E3, and outputs the determination result to the information management unit 303.

[0154] [Operation] The displacement detection and calibration information update process of the external environment detection system 400D of the construction machine 100 in this embodiment will be described. Descriptions of the processes in this embodiment that overlap with those of previously described embodiments will be omitted.

[0155] In this embodiment, the conditions for executing the process S1 in Figure 9 are triggered when an operator performs an operation such as placing an object such as a sign around the construction machine 100 at predetermined known coordinates, and causing the upper rotating body 102 to rotate relative to the lower traveling body 101. For example, as shown in Figure 25, an object such as a sign 502 is placed around the construction machine 100 at known coordinates. The object 502 is a sign that has characteristics that make it easily detectable by the external detection devices E1, E2, and E3. The object 502 is placed at predetermined known coordinates in the vehicle body coordinate system 1602 of the construction machine 100 using surveying equipment or the like. In Figure 25, an example is shown in which the external detection device E1 detects the object 502, but if the object 502 can be detected, it is possible to detect any deviation in the position or orientation of the external detection devices E1, E2, and E3.

[0156] For the sake of explanation, an example of detecting a deviation in either the position or attitude of the external detection device E1 will be described below. The deviation detection unit 302D can recognize that the upper rotating body 102 is rotating relative to the lower traveling body 101 based on the detection result of the attitude detection device P1 or input from the operator to the display terminal 108. In other words, the calculation unit 200D can start detecting a deviation in either the position or attitude of the external detection device E1 in response to input from the display terminal 108. Alternatively, as shown in Figure 25, the deviation detection unit 302D may recognize that the operator has operated the control stick to rotate the upper rotating body 102 relative to the lower traveling body 101, and may start executing the process of S1 in Figure 9.

[0157] The details of the process at S1 in Figure 9 in this embodiment will be explained below with reference to Figures 26 and 27. As shown in Figure 26, the displacement detection unit 302D repeats the following processes until the displacement detection process is terminated (S40 to S46). The object trajectory generation unit 3001 executes a process to generate the trajectory of the coordinates of the object 502 (S41). The object trajectory estimation unit 3002 executes a process to calculate the estimated trajectory of the coordinates of the object 502 that is estimated to be detected by the external detection device E1, based on the rotation angle of the upper rotating body 102 relative to the lower traveling body 101 detected by the attitude detection device P1, the position and attitude of the external detection device E1 recorded by the information recording unit 301, and the known coordinates of the object 502 (S42).

[0158] In S41, the object trajectory generation unit 3001 of the displacement detection unit 302D acquires the coordinate detection result of the object 502 by the external detection device E1 each time the upper rotating body 102 rotates by a certain angle or more relative to the lower traveling body 101, and stores it in the memory area of ​​the object trajectory generation unit 3001. The object trajectory generation unit 3001 acquires the coordinates of the object 502 in the external detection device coordinate system 1603 of a predetermined number of external detection devices E1. The object trajectory generation unit 3001 generates a trajectory 2901 of the coordinates of the object 502 in the external detection device coordinate system 1603, as shown in Figure 27, as a regression curve of the coordinates of the object 502 in the external detection device coordinate system 1603 of the external detection device E1 stored in the memory area.

[0159] On the other hand, each time the upper rotating body 102 rotates by a certain angle or more relative to the lower traveling body 101, the object trajectory estimation unit 3002 of the object 502 in the external detection device coordinate system 1603 of the external detection device E1 calculates the estimated trajectory 2902 of the object 502 in the external detection device coordinate system 1603 of the external detection device E1, as shown in Figure 27, by performing a coordinate transformation from the vehicle body coordinate system 1602 to the external detection device coordinate system 1603 of the external detection device E1, based on the detection result of the attitude detection device P1, the position and attitude of the external detection device E1 recorded by the information recording unit 301, and the coordinates of the object 502 which are known in the vehicle body coordinate system 1602.

[0160] As shown in Figure 26, the detection result management unit 3003 determines whether the object trajectory generation unit 3001 has generated a trajectory of the coordinates of object 502 based on the detection result of the external detection device E1, and whether the object trajectory estimation unit 3002 has calculated an estimated trajectory of the coordinates of object 502 based on the detection result of the attitude detection device P1, the position and attitude of the external detection device E1 recorded by the information recording unit 301, and the known coordinates of object 502 (S43). If the object trajectory generation unit 3001 has generated a trajectory 2901 of the coordinates of object 502, and the object trajectory estimation unit 3002 has calculated an estimated trajectory 2902 of the coordinates of object 502, the detection result management unit 3003 proceeds to S34. When the object trajectory generation unit 3001 does not generate a trajectory of the object 502's coordinates, and when the object trajectory estimation unit 3002 does not calculate an estimated trajectory of the object 502's coordinates, the detection result management unit 3003 repeats the processes of S40 to S43.

[0161] The detection result management unit 3003 stores the trajectory 2901 generated by the object trajectory generation unit 3001 and the estimated trajectory 2902 calculated by the object trajectory estimation unit 3002 in the storage area 3004 (S44). The determination unit 3005 uses the trajectory 2901 and estimated trajectory 2902 of the object 502 stored in the storage area 3004 to determine whether the position or orientation of the external detection device E1 is deviated from the calibration information recorded by the information recording unit 301 (S45).

[0162] The determination unit 3005 can determine whether the position or orientation of the external detection device E1 is deviated from the calibration information recorded by the information recording unit 301, and the amount of the deviation, by comparing the trajectory 2901 with the estimated trajectory 2902. If the determination unit 3005 determines that either the position or orientation of the external detection device E1 is deviated, the process proceeds to S47. If the deviation detection unit 302D determines that the position and orientation of the external detection device E1 are not deviated, the process from S40 to S46 is repeated.

[0163] In the above example, a deviation in either the position or orientation of the external detection device E1 was detected. However, by performing the same process for external detection devices E2 and E3, a deviation in either the position or orientation of external detection devices E2 and E3 can also be detected. By performing a coordinate transformation from the vehicle body coordinate system 1602 to the respective external detection device coordinate systems of external detection devices E2 and E3, the same estimated trajectory can be calculated. The respective external detection device coordinate systems of external detection devices E2 and E3 are, for example, coordinate systems in which the roll axis, pitch axis, and yaw axis of external detection devices E2 and E3 are the three axes of the Cartesian coordinate system.

[0164] As shown in S47 of Figure 26, the displacement detection unit 302D notifies the information management unit 303 that it has detected a displacement, as well as the identifiers of the external detection devices E1, E2, and E3 where the displacement occurred, and the amount of displacement detected. The information management unit 303 and the display terminal 108 perform the same processing as in the above embodiment 1.

[0165] [Effect] In Embodiment 4, the rotation angle of the upper rotating body 102 relative to the lower traveling body 101 detected by the attitude detection device P1, the positions and attitudes of the external detection devices E1, E2, and E3 recorded by the information recording unit 301, and the coordinates of the known object 502 such as a sign are used to calculate the estimated trajectory of the object 502 such as a sign. The deviation of any of the positions and coordinates of the external detection devices E1, E2, and E3 is detected from the calculated estimated trajectory and the trajectory of the coordinates of the object 502 such as a sign detected by the external detection devices E1, E2, and E3, which are measured values. For this reason, although it is necessary to place the object 502 such as a sign at predetermined coordinates, the deviation of any of the positions and attitudes of the multiple external detection devices E1, E2, and E3 can be detected simply by performing an operation to rotate the upper rotating body 102 relative to the lower traveling body 101.

[0166] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are explained in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations. In addition, some or all of the above configurations, functions, processing units, processing means, etc., may be realized in hardware, for example, by designing them as integrated circuits.

[0167] 100 Construction machine 101 Lower traveling body 102 Upper rotating body 103 Working device 104 Boom 105 Arm 106 Bucket 107 Cab 108 Display terminal 109 Bracket 110 Laser transmitting / receiving unit 111 Roll adjustment mechanism 112 Yaw adjustment mechanism 113 Pitch adjustment mechanism 120 Operating device 200A, 200B, 200C, 200D Calculation unit 301 Information recording unit 302A, 302B, 302C, 302D Shift detection unit 303 Information management unit 400A, 400B, 400C, 400D External detection system 501, 502 Object 1001 Working device position estimation unit 1002 Detection result management unit 1003 Memory area 1004 Judgment unit 1201 Display items 1202 Instruction lines 1203 Schematic diagram 1204, 1205 Operation buttons 1601 Coordinates 1602 Vehicle body coordinate system 1603 External detection device coordinate system 1801, 1802, 1803 Center coordinates 1901 Trajectory 1902 Estimated trajectory 2001 Work device trajectory generation unit 2002 Work device trajectory estimation unit 2003 Detection result management unit 2004 Memory area 2005 Judgment unit 2901 Trajectory 2902 Estimated trajectory 3001 Object trajectory generation unit 3002 Object trajectory estimation unit 3003 Detection result management unit 3004 Memory area 3005 Judgment unit P1, P2, P3, P4 Attitude detection device E1, E2, E3 External detection device R1, R2, R3 Measurement area

Claims

1. An external environment detection system mounted on a construction machine having a lower traveling body, an upper rotating body, and a working device, comprising: an external environment detection device provided on the construction machine and configured to detect objects; a posture detection device for detecting the posture of either the upper rotating body or the working device; and a calculation device for detecting a deviation in either the position or posture of the external environment detection device, wherein the calculation device calculates the coordinates of the object estimated to be detected by the external environment detection device based on the posture of either the upper rotating body or the working device detected by the posture detection device, and detects a deviation in either the position or posture of the external environment detection device by comparing the calculated coordinates of the object with the coordinates of the object detected by the external environment detection device.

2. An external environment detection system according to claim 1, wherein the upper rotating body is provided with a first external environment detection device and a second external environment detection device, the calculation device comprises: an information recording unit for recording the position and orientation of the first external environment detection device and the second external environment detection device; and a displacement detection unit for detecting a displacement in either the position or orientation of either the first external environment detection device or the second external environment detection device, wherein the displacement detection unit records, when the object is detected by the first external environment detection device, a first rotation angle of the upper rotating body relative to the lower traveling body detected by the orientation detection device; the position and orientation of the first external environment detection device recorded by the information recording unit; the second coordinates of the object detected by the second external environment detection device; and when the object is detected by the second external environment detection device, a second rotation angle of the upper rotating body relative to the lower traveling body detected by the orientation detection device. An external environment detection system characterized by calculating first estimated coordinates of an object that is estimated to be detected by the first external environment detection device based on the position and orientation of the second external environment detection device recorded by the information recording unit, and detecting a deviation in either the position or orientation of either the first external environment detection device or the second external environment detection device by comparing the first estimated coordinates with the first coordinates of the object detected by the first external environment detection device.

3. An external environment detection system according to claim 1, wherein the external environment detection device detects the work device as the object, the calculation device comprises an information recording unit for recording the position and orientation of the external environment detection device, and a deviation detection unit for detecting a deviation in either the position or orientation of the external environment detection device, the deviation detection unit calculates estimated coordinates of the work device that are estimated to be detected by the external environment detection device based on the orientation of the work device detected by the orientation detection device and the position and orientation of the external environment detection device recorded by the information recording unit, and detects a deviation in either the position or orientation of the external environment detection device by comparing the estimated coordinates with the coordinates of the work device detected by the external environment detection device.

4. An external environment detection system according to claim 1, wherein the external environment detection device detects the work device as the object, the calculation device comprises an information recording unit for recording the position and orientation of the external environment detection device, and a deviation detection unit for detecting a deviation in either the position or orientation of the external environment detection device, and the deviation detection unit, when the orientation detection device detects that the work device is performing a predetermined operation, calculates an estimated trajectory of the coordinates of the work device that is estimated to be detected by the external environment detection device when the work device is performing the operation, based on the orientation of the work device detected by the orientation detection device and the position and orientation of the external environment detection device recorded by the information recording unit, and detects a deviation in either the position or orientation of the external environment detection device by comparing the estimated trajectory with the trajectory of the coordinates of the work device detected by the external environment detection device when the work device is performing the operation.

5. An external environment detection system according to claim 1, wherein the external environment detection device is provided on the upper rotating body, the external environment detection device detects an object whose coordinates are known, the calculation device comprises an information recording unit for recording the position and orientation of the external environment detection device, and a deviation detection unit for detecting a deviation in either the position or orientation of the external environment detection device, the deviation detection unit, when the orientation detection device detects that the upper rotating body is rotating relative to the lower traveling body, calculates an estimated trajectory of the coordinates of the object that is estimated to be detected by the external environment detection device when the upper rotating body is rotating relative to the lower traveling body, based on the rotation angle of the upper rotating body relative to the lower traveling body detected by the orientation detection device, the position and orientation of the external environment detection device recorded by the information recording unit, and the coordinates of the known object, An external environment detection system characterized by detecting a deviation in either the position or orientation of the external environment detection device by comparing the estimated trajectory with the trajectory of the coordinates of the object detected by the external environment detection device when the upper rotating body is rotating relative to the lower traveling body.

6. An external environment detection system according to claim 1, further comprising a display terminal, wherein the display terminal displays the positional deviation or orientation deviation of the external environment detection device detected by the computing device.

7. An external environment detection system according to claim 6, further comprising: an information recording unit that records either the position or the orientation of the external environment detection device of the construction machine; and an information management unit that updates either the position or the orientation of the external environment detection device recorded by the information recording unit in response to input from the display terminal.

8. An external environment detection system according to claim 6, characterized in that the computing device starts detecting a deviation in either the position or orientation of the external environment detection device in response to input to the display terminal.

9. An external environment detection system according to claim 2, further comprising a third external environment detection device on the upper rotating body, wherein the information recording unit records the position and orientation of the first external environment detection device, the second external environment detection device, and the third external environment detection device, the displacement detection unit detects a displacement of either the position or orientation of any of the first external environment detection device, the second external environment detection device, and the third external environment detection device, and when the object is detected by the first external environment detection device, the first rotation angle of the upper rotating body relative to the lower traveling body detected by the orientation detection device, the position and orientation of the first external environment detection device recorded by the information recording unit, the third coordinates of the object detected by the third external environment detection device, and when the object is detected by the third external environment detection device, the third rotation angle of the upper rotating body relative to the lower traveling body detected by the orientation detection device, An external detection system characterized by: calculating the first estimated coordinates of the object that is estimated to be detected by the first external detection device based on the position and orientation of the third external detection device recorded by the information recording unit; comparing the first estimated coordinates calculated based on the second coordinates, the second rotation angle and the position and orientation of the second external detection device, the first estimated coordinates calculated based on the third coordinates, the third rotation angle and the position and orientation of the third external detection device, and the first coordinates of the object detected by the first external detection device, thereby detecting any deviation in the position and orientation of any of the first external detection device, the second external detection device and the third external detection device.

10. A construction machine having the external environment detection system described in claim 1.

Images