Construction management system, data processing device, and construction management method

The construction management system uses topographic data creation and AI-enhanced object recognition to monitor construction progress, improving efficiency by providing real-time visualizations of site changes and object positions.

JP7578529B2Active Publication Date: 2024-11-06KOMATSU LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2021061691
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-03-31
Publication Date
2024-11-06
Estimated Expiration
2041-03-31

AI Technical Summary

Technical Problem

There is a need to improve construction efficiency by effectively monitoring and checking the progress of construction at a site.

Method used

A construction management system that includes a current topographic data creation unit, detection data acquisition, reflection, and output units to generate and display reflected data on a display device, utilizing detection devices mounted on flying objects to capture three-dimensional data of construction sites, and integrate artificial intelligence for object recognition and data reflection.

Benefits of technology

Enables real-time monitoring of construction progress, allowing administrators to track the latest topography and object positions, enhancing construction efficiency by providing accurate and up-to-date visualizations of construction sites.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007578529000001
    Figure 0007578529000001
  • Figure 0007578529000002
    Figure 0007578529000002
  • Figure 0007578529000003
    Figure 0007578529000003
Patent Text Reader

Abstract

To allow for checking the progress of construction.SOLUTION: A construction management system is provided, comprising: a current topographical data generation unit configured to generate current topographical data of a construction site having a work machine operating thereon; a detection data acquisition unit configured to acquire detection data from a detection device for detecting the construction site; a reflection unit configured to generate reflected data representing the detection data reflected on the current topographical data; and an output unit for outputting the reflected data onto a display device.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a construction management system, a data processing device, and a construction management method. [Background technology]

[0002] 2. Description of the Related Art In the technical field relating to construction management systems, a construction management system as disclosed in Patent Document 1 is known. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 012993 Summary of the Invention [Problem to be solved by the invention]

[0004] In order to prevent a decrease in construction efficiency at a construction site, it is preferable to be able to properly check the progress of construction.

[0005] The purpose of this disclosure is to check the progress of construction. [Means for solving the problem]

[0006] According to the present disclosure, a construction management system is provided which includes a current terrain data creation unit which creates current terrain data of a construction site where a work machine is operating, a detection data acquisition unit which acquires detection data from a detection device which detects the construction site, a reflection unit which generates reflection data which reflects the detection data in the current terrain data, and an output unit which outputs the reflection data to a display device. Effect of the Invention

[0007] According to the present disclosure, it is possible to check the progress of construction. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram showing a construction management system according to an embodiment. [Diagram 2] FIG. 2 is a perspective view showing the hydraulic excavator according to the embodiment. [Diagram 3] FIG. 3 is a perspective view showing the crawler dump truck according to the embodiment. [Figure 4] FIG. 4 is a functional block diagram showing the construction management system according to the embodiment. [Diagram 5] FIG. 5 is a flowchart showing the construction management method according to the embodiment. [Figure 6] FIG. 6 is a diagram showing current topographical data according to the embodiment. [Figure 7] FIG. 7 is a diagram showing reflected data according to the embodiment. [Figure 8] FIG. 8 is a diagram showing reflected data according to the embodiment. [Figure 9] FIG. 9 is a diagram showing reflected data according to the embodiment. [Figure 10] FIG. 10 is a diagram showing reflected data according to the embodiment. [Figure 11] FIG. 11 is a diagram showing reflected data according to the embodiment. [Figure 12] FIG. 12 is a diagram showing reflected data according to the embodiment. [Figure 13] FIG. 13 is a block diagram illustrating a computer system according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiment. The components of the embodiments described below can be appropriately combined. In addition, some components may not be used.

[0010] [Construction management system] FIG. 1 is a schematic diagram showing a construction management system 1 according to an embodiment. The construction management system 1 manages construction at a construction site 2. A plurality of work machines 20 operate at the construction site 2. In the embodiment, the work machines 20 include a hydraulic excavator 21, a bulldozer 22, and a crawler dump truck 23. The work machines 20 may also include a wheel loader. In addition, a person WM is present at the construction site 2. An example of the person WM is a worker who works at the construction site 2. In addition, the person WM may be a supervisor who manages the construction. The person WM may also be an observer.

[0011] As shown in FIG. 1, the construction management system 1 includes a management device 3, a server 4, an information terminal 5, a detection device 9, and a detection device 12.

[0012] The management device 3 includes a computer system arranged at the construction site 2. The management device 3 is supported by a traveling device 6. The management device 3 can travel at the construction site 2 by the traveling device 6. Examples of the traveling device 6 include aerial work vehicles, trucks, and traveling robots. The server 4 is a data processing device including a computer system. The server 4 may be arranged at the construction site 2 or at a remote location from the construction site 2. The information terminal 5 is a computer system arranged at a remote location 13 from the construction site 2. Examples of the information terminal 5 include a personal computer or a smartphone. The management device 3, the server 4, and the information terminal 5 communicate with each other via a communication system 10. Examples of the communication system 10 include the Internet, a local area network (LAN), a mobile phone communication network, and a satellite communication network.

[0013] The detection device 9 detects the construction site 2. The detection device 9 acquires three-dimensional data of the construction site 2. Examples of detection targets of the detection device 9 include the topography of the construction site 2 and objects present at the construction site 2. The objects include one or both of movable and stationary objects. Examples of movable objects include a work machine 20 and a person WM. Examples of stationary objects include wood or materials.

[0014] The three-dimensional data acquired by the detection device 9 includes image data of the construction site 2. The image data acquired by the detection device 9 may be video data or still image data. An example of the detection device 9 is a stereo camera. The detection device 9 may include a monocular camera and a three-dimensional measuring device. An example of the three-dimensional measuring device is a laser sensor (LIDAR: Light Detection and Ranging) that detects a detection target by emitting laser light. The three-dimensional measuring device may be an infrared sensor that detects an object by emitting infrared light or a radar sensor (RADAR: Radio Detection and Ranging) that detects an object by emitting radio waves.

[0015] The detection device 9 is mounted on an aircraft 8. An example of the aircraft 8 is an unmanned aerial vehicle (UAV: Unmanned Aerial Vehicle) such as a drone. The detection device 9 detects the construction site 2 from above the construction site 2.

[0016] In the embodiment, the flying object 8 and the management device 3 are connected by a cable 7. The detection data of the detection device 9 is transmitted to the management device 3 via the cable 7. The detection data of the detection device 9 transmitted to the management device 3 is transmitted to the server 4 via the communication system 10.

[0017] In an embodiment, the management device 3 includes a power source or generator. The management device 3 can supply power to the air vehicle 8 via a cable 7.

[0018] The detection device 12 detects the construction site 2. Similar to the detection device 9, the detection device 12 acquires three-dimensional data of the construction site 2. The three-dimensional data acquired by the detection device 12 includes image data of the construction site 2.

[0019] The detection device 12 is mounted on the aircraft 11. The detection device 12 detects the construction site 2 from above the construction site 2. Detection data from the detection device 12 is transmitted to the server 4 via the communication system 10.

[0020] The aircraft 11 can fly higher in the sky than the aircraft 8. The aircraft 11 can fly in a wider range than the aircraft 8. The detection device 12 can detect a wider range of the construction site 2 than the detection device 9. In the embodiment, the detection device 12 detects the entire construction site 2. The detection device 9 detects a part of the construction site 2.

[0021] [Work machinery] Fig. 2 is a perspective view showing a hydraulic excavator 21 according to an embodiment. As shown in Fig. 2, the hydraulic excavator 21 includes a traveling body 24, a rotating body 25 supported by the traveling body 24, a work implement 26 supported by the rotating body 25, and a hydraulic cylinder 27 that drives the work implement 26.

[0022] The running body 24 has a pair of tracks. The hydraulic excavator 21 can travel on the construction site 2 by the running body 24. The rotating body 25 rotates while being supported by the running body 24. The work machine 26 includes a boom 26A connected to the rotating body 25, an arm 26B connected to the boom 26A, and a bucket 26C connected to the arm 26B. The hydraulic cylinder 27 includes a boom cylinder 27A that operates the boom 26A, an arm cylinder 27B that operates the arm 26B, and a bucket cylinder 27C that operates the bucket 26C.

[0023] The hydraulic excavator 21 operates. Examples of the operations of the hydraulic excavator 21 include a traveling operation of the traveling body 24, a rotating operation of the rotating body 25, a raising operation and a lowering operation of the boom 26A, an excavation operation and a dumping operation of the arm 26B, and an excavation operation and a dumping operation of the bucket 26C.

[0024] 3 is a perspective view showing the crawler dump truck 23 according to the embodiment. As shown in FIG. 3, the crawler dump truck 23 has a traveling body 28, a vehicle body 29, and a dump truck body 30.

[0025] The running body 28 has a pair of tracks. The crawler dump 23 can travel on the construction site 2 by the running body 28. The dump body 30 is a member onto which a load is loaded. The hydraulic excavator 21 can load a load into the dump body 30 by using the work machine 26. The dump body 30 can be raised by a hoist cylinder (not shown) to discharge the load.

[0026] The crawler dump truck 23 operates. Examples of the operation of the crawler dump truck 23 include a traveling operation of the traveling body 28, and a lowering operation and a dump operation of the dump body 30.

[0027] [server] 4 is a functional block diagram showing the construction management system 1 according to the embodiment. As shown in FIG. 4, the construction management system 1 includes an aircraft 8, an aircraft 11, a management device 3 arranged at a construction site 2, a server 4, and an information terminal 5 arranged at a remote location 13 of the construction site 2.

[0028] The flying object 8 has a position sensor 14 , an attitude sensor 15 , and a detection device 9 .

[0029] The position sensor 14 detects the position of the aircraft 8. The position sensor 14 detects the position of the aircraft 8 by using a global navigation satellite system (GNSS). The position sensor 14 includes a GNSS receiver (GNSS sensor) and detects the position of the aircraft 8 in a global coordinate system. The attitude sensor 15 detects the attitude of the aircraft 8. An example of the attitude sensor 15 is an inertial measurement unit (IMU).

[0030] The flying object 11 has a position sensor 16, an attitude sensor 17, and a detection device 12.

[0031] The position sensor 16 includes a GNSS receiver and detects the position in a global coordinate system of the aircraft 11. The attitude sensor 17 detects the attitude of the aircraft 11. An example of the attitude sensor 17 is an inertial measurement unit (IMU).

[0032] The server 4 includes a current topographical data creation unit 41, a detection data acquisition unit 42, a recognition unit 43, a reflection unit 44, an output unit 45, and a storage unit 46.

[0033] The current state terrain data creation unit 41 creates current state terrain data that shows the current state terrain of the construction site 2 where the work machine 20 is operating. The current state terrain data is three-dimensional terrain data that shows the current state terrain of the construction site 2. The current state terrain includes a reference topography before a specified construction work is started.

[0034] The current topography data creation unit 41 creates current topography data based on the detection data of the detection device 12. As described above, the detection device 12 detects the entire construction site 2. The current topography data indicates the current topography of the entire construction site 2.

[0035] In an embodiment, the detection device 12 detects the construction site 2 at a first frequency. The detection device 12 detects the construction site 2 only once, for example, before the start of work in a day. The detection device 12 detects the current topography indicating the reference topography before a specified construction is started at the construction site 2. The current topography data creation unit 41 creates current topography data at a first frequency. The current topography data created by the current topography data creation unit 41 is stored in the memory unit 46. The current topography data stored in the memory unit 46 is updated at the first frequency.

[0036] The timing when the detection device 12 detects the construction site 2 is not limited to before the start of work in a day, and may be any timing.

[0037] The detection data acquisition unit 42 acquires detection data from the detection device 9 that detected the work machine 20 and the construction area around the work machine 20.

[0038] As described above, the detection device 9 detects a portion of the construction site 2. The portion of the construction site 2 detected by the detection device 9 includes the work machine 20 performing work. The portion of the construction site 2 detected by the detection device 9 includes the construction area around the work machine 20 performing work. An example of the construction area detected by the detection device 9 is a construction area where construction is progressing by the work machine 20.

[0039] In an embodiment, the detection device 9 detects the construction site 2 at a second frequency higher than the first frequency. The detection device 9 continuously detects the construction site 2 for a fixed period of time, for example. The detection device 9 continuously detects the construction site 2 for a period during which work is being performed by the work machine 20. The detection device 9 may constantly detect the construction site 2. The detection data acquisition unit 42 acquires detection data of the detection device 9 at the second frequency. The detection data acquired by the detection data acquisition unit 42 is updated more frequently than the current topography data.

[0040] The recognition unit 43 recognizes objects at the construction site 2 based on the detection data acquired by the detection data acquisition unit 42. As described above, examples of objects include the work machine 20 and the person WM.

[0041] The recognition unit 43 recognizes an object by using artificial intelligence (AI) that analyzes input data by an algorithm and outputs output data. The input data is image data of the construction site 2 acquired by the detection device 9, and the output data is an object.

[0042] The recognition unit 43 holds a learning model generated by learning the feature amounts of an object. The learning model includes a learning model generated by learning the feature amounts of the work machine 20 and a learning model generated by learning the feature amounts of the person WM. In generating the learning model, machine learning is performed using a learning image including an object as teacher data, thereby generating a learning model that inputs the feature amounts of an object and outputs the object. The recognition unit 43 can input image data of the construction site 2 acquired by the detection device 9 into the learning model to recognize the object.

[0043] The reflection unit 44 generates reflection data that reflects the detection data acquired by the detection data acquisition unit 42 in the current topography data. The current topography data is three-dimensional topography data of the entire construction site 2 detected by the detection device 12 at a first frequency. The detection data includes three-dimensional topography data of a construction area that is a part of the construction site 2 detected by the detection device 9 at a second frequency. The detection data acquired by the detection data acquisition unit 42 is sequentially updated at the second frequency. The reflection unit 44 sequentially reflects the detection data acquired by the detection data acquisition unit 42 in a part of the current topography data at the second frequency. At least a part of the current topography data is sequentially updated by the detection data acquired by the detection data acquisition unit 42.

[0044] The detection data acquired by the detection data acquisition unit 42 includes updated terrain data indicating the updated terrain of the construction area. The updated terrain includes the latest terrain during or after construction. The updated terrain data is updated at a second frequency. The reflection unit 44 reflects the updated terrain data acquired by the detection data acquisition unit 42 in the current terrain data stored in the memory unit 46.

[0045] The detection data acquired by the detection data acquisition unit 42 includes non-terrain data indicating objects at the construction site 2. The non-terrain data is three-dimensional data indicating objects at the construction site 2. The non-terrain data is updated at a second frequency. The reflection unit 44 reflects the non-terrain data acquired by the detection data acquisition unit 42 in the current terrain data stored in the memory unit 46.

[0046] In the embodiment, the reflecting unit 44 reflects updated terrain data of the construction area, in which non-terrain data has been removed from the detection data acquired by the detection data acquiring unit 42, in the current terrain data. The non-terrain data indicating an object is recognized by the recognizing unit 43. The reflecting unit 44 removes the non-terrain data from the detection data to generate updated terrain data. In addition, when an object is present at the construction site 2, the reflecting unit 44 generates reflection data in which the object is reflected.

[0047] The reflection data generated by the reflection unit 44 includes updated topographical data of the construction area.

[0048] The reflection data generated by the reflection unit 44 includes the object recognized by the recognition unit 43 .

[0049] The reflection data generated by the reflection unit 44 includes a three-dimensional model of the work machine 20 recognized by the recognition unit 43. The three-dimensional model of the work machine 20 includes computer graphics (CG) of the work machine 20.

[0050] The 3D model of the work machine 20 is a 3D model that represents the work machine 20, and is constructed for each part that constitutes the work machine 20, such as the traveling body 24, the rotating body 25, and the work implement 26. The 3D model of the work machine 20 is stored in advance in the memory unit 46.

[0051] The reflection data generated by the reflection unit 44 includes a symbol image indicating the position of an object. The symbol image is image data that emphasizes the position of an object. The reflection unit 44 generates the symbol image based on the recognition result of the recognition unit 43.

[0052] The output unit 45 outputs the reflection data generated by the reflection unit 44 to the information terminal 5. The output unit 45 transmits the reflection data to the information terminal 5 via the communication system 10.

[0053] The information terminal 5 has an input device 51 and a display device 52 .

[0054] The input device 51 is operated by an administrator present at the remote location 13. The input device 51 generates input data based on the administrator's operation. Examples of the input device 51 include a touch panel, a computer keyboard, a mouse, and an operation button. The input device 51 may be a non-contact input device including an optical sensor, or may be a voice input device.

[0055] The display device 52 displays the display data. The administrator at the remote location 13 can check the display data displayed on the display device 52. In the embodiment, the output unit 45 outputs the reflection data generated by the reflection unit 44 to the display device 52. The display device 52 displays the reflection data generated by the reflection unit 44 as the display data. Examples of the display device 52 include a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OELD).

[0056] [Relationship between detection devices and 3D data] As described above, the flying object 8 is equipped with a position sensor 14 and an attitude sensor 15. The position sensor 14 can detect the position of the detection device 9. The attitude sensor 15 can detect the attitude of the detection device 9. The attitude includes, for example, a roll angle, a pitch angle, and a yaw angle. The yaw angle may be calculated based on detection data from two GNSS sensors provided on the flying object 8. The detection device 9 detects three-dimensional data of the construction site 2. The three-dimensional data of the construction site 2 includes a relative distance and a relative position between the detection device 9 and each of a plurality of detection points defined in the detection target. The recognition unit 43 and the reflection unit 44 can calculate, for example, the position of the three-dimensional data of the detection target in a global coordinate system based on the detection data of the position sensor 14, the detection data of the attitude sensor 15, and the detection data of the detection device 9. In addition, the recognition unit 43 and the reflection unit 44 can perform a predetermined coordinate conversion to calculate, for example, the position of the three-dimensional data of the detection target in a local coordinate system defined in the construction site 2. The detection target of the detection device 9 includes the updated terrain and object.

[0057] Similarly, the current terrain data creation unit 41 can calculate the position of the three-dimensional data of the detection target in, for example, a local coordinate system defined for the construction site 2, based on the detection data of the position sensor 16, the detection data of the attitude sensor 17, and the detection data of the detection device 12. The detection target of the detection device 12 includes the current terrain.

[0058] The recognition unit 43 can recognize the presence or absence of an object and the position of the object based on the detection data acquired by the detection data acquisition unit 42.

[0059] For example, when recognizing the position of the person WM, the recognition unit 43 recognizes the person WM based on a two-dimensional image acquired by the monocular camera of the detection device 9. In addition, the recognition unit 43 acquires three-dimensional topographical data of the construction site 2. The recognition unit 43 can recognize the position of the person WM at the construction site 2 based on the person WM recognized based on the two-dimensional image. For example, the person WM recognized based on the two-dimensional image acquired by the two monocular cameras constituting the stereo camera is subjected to image processing based on the principle of triangulation to calculate the three-dimensional position of the person WM and correspond it to the three-dimensional topographical data of the construction site 2, thereby recognizing the position of the person WM at the construction site 2. In addition, the position of the person WM at the construction site 2 may be recognized by calculating the three-dimensional position of the person WM recognized based on the two-dimensional image using the detection value of a laser sensor or a radar sensor and corresponding it to the three-dimensional topographical data of the construction site 2. The recognition unit 43 may recognize the position of the person WM from the three-dimensional data of the construction site 2. In addition, the recognition unit 43 may recognize the position of the person WM based on the detection data of a position sensor held by the person WM. For example, if the person WM carries a smartphone equipped with a GNSS sensor, the recognition unit 43 can recognize the position of the person WM based on the detection data of the GNSS sensor of the smartphone. The position sensor carried by the person WM may be a beacon. The position of the person WM at the construction site 2 may be estimated by geometric calculation based on the coordinates on the two-dimensional image of the person WM recognized based on the two-dimensional image, the three-dimensional position and posture of the detection device 9, and the three-dimensional topographical data.

[0060] Furthermore, the recognition unit 43 can recognize the operation of the work machine 20 based on the detection data acquired by the detection data acquisition unit 42. The hydraulic excavator 21 can operate the running body 24, the rotating body 25, and the work implement 26. The crawler dump 23 can operate the running body 28 and the dump body 30. The reflection unit 44 can move the three-dimensional model in synchronization with the work machine 20.

[0061] [Construction management method] 5 is a flowchart showing a construction management method according to the embodiment. The current topography data creation unit 41 creates current topography data of the construction site 2 where the work machine 20 operates (step S1).

[0062] 6 is a diagram showing current state topography data according to the embodiment. The current state topography data is three-dimensional topography data showing the current state topography of the entire construction site 2. The current state topography data creation unit 41 acquires detection data from the detection device 12. The current state topography data creation unit 41 creates current state topography data based on the detection data from the detection device 12.

[0063] The current terrain data created by the current terrain data creation unit 41 is stored in the storage unit 46. The output unit 45 can transmit the current terrain data to the information terminal 5. The display device 52 can display the current terrain as shown in FIG.

[0064] When the work machine 20 starts working at the construction site 2, the topography of the construction site 2 changes. The detection device 9 mounted on the aircraft 8 detects the construction site 2. The detection device 9 detects, for example, the work machine 20 performing work and the construction area around the work machine 20. The detection data of the detection device 9 is transmitted to the management device 3 via the cable 7. The management device 3 transmits the detection data of the detection device 9 to the server 4. The detection data acquisition unit 42 acquires the detection data of the detection device 9 that detected the work machine 20 and the construction area around the work machine 20 (step S2).

[0065] The recognition unit 43 determines whether or not an object exists in the construction site 2 based on the detection data acquired by the detection data acquisition unit 42. In an embodiment, the recognition unit 43 determines whether or not an object exists in the construction area (step S3).

[0066] In step S3, if it is determined that no object exists in the construction area (step S3: No), the reflection unit 44 reflects the detection data acquired by the detection data acquisition unit 42 in the current topographical data stored in the storage unit 46 to generate reflection data. The detection data acquired by the detection data acquisition unit 42 includes updated topographical data of the construction area showing a part of the construction site 2. The reflection unit 44 reflects the updated topographical data in the current topographical data (step S4).

[0067] For example, when a part of the construction site 2 is excavated by the hydraulic excavator 21, the detection data including the excavation location is acquired by the detection data acquisition unit 42 as updated terrain data. The updated terrain data includes the excavation location excavated by the hydraulic excavator 21. The reflection unit 44 combines a part of the current terrain data with the updated terrain data. The reflection unit 44 applies the updated terrain data to a part of the current terrain data. As a result, reflection data in which the updated terrain data including the excavation location is reflected is generated.

[0068] In step S3, if it is determined that an object exists in the construction area (step S3: Yes), the reflection unit 44 removes the non-terrain data indicating the object recognized by the recognition unit 43 from the detection data and generates updated terrain data (step S5).

[0069] After the updated terrain data is generated, the reflecting unit 44 reflects the updated terrain data in the current terrain data to generate reflected data. Also, if an object exists, the reflecting unit 44 generates reflected data in which the object is reflected (step S6).

[0070] Even if an object exists, the reflection unit 44 may generate reflection data in which the object is not reflected.

[0071] The output unit 45 outputs the reflected data generated in at least one of step S4 and step S6 to the information terminal 5. The display device 52 displays the reflected data transmitted from the output unit 45 (step S7).

[0072] FIG. 7 is a diagram showing reflected data according to an embodiment. FIG. 7 shows reflected data when no object exists in the construction area. The reflected data includes image data of the construction area. For example, as construction in the construction area progresses, at least a part of the terrain of the construction site 2 changes as shown in FIG. 7. In the example shown in FIG. 7, an excavation point is generated in the construction area by the excavation work of the hydraulic excavator 21. The detection data acquired by the detection data acquisition unit 42 includes updated terrain data of the construction area. The reflection unit 44 reflects the change in the terrain of the construction site 2 in real time to the current terrain. The reflection unit 44 reflects the updated terrain data in real time to the current terrain data. As shown in FIG. 7, the display device 52 can display the reflected data in which the excavation point is reflected. The manager of the remote location 13 can recognize the progress of construction at the construction site 2 in real time by checking the reflected data displayed on the display device 52.

[0073] Fig. 8 is a diagram showing reflected data according to the embodiment. Fig. 8 shows reflected data when a hydraulic excavator 21, a crawler dump truck 23, and a person WM are present as objects in a construction area. The reflected data includes three-dimensional terrain data in which updated terrain data is reflected in part of the current terrain data. The reflected data includes a three-dimensional model of the work machine 20 recognized by the recognition unit 43. In the example shown in Fig. 8, the three-dimensional model of the work machine 20 includes a three-dimensional model 21D of the hydraulic excavator 21 and a three-dimensional model 23D of the crawler dump truck 23.

[0074] When the three-dimensional model of the work machine 20 is displayed on the display device 52, the recognition unit 43 calculates the position (three-dimensional position) and attitude of the work machine 20 based on the detection data of the detection device 12 acquired by the detection data acquisition unit 42. The attitude of the work machine 20 includes the inclination of the revolving body 25 with respect to the horizontal plane and the revolving angle of the revolving body 25 with respect to the traveling body 24. The attitude of the work machine 20 also includes the angle of the work implement 26. The angle of the work implement 26 includes the angle of the boom 26A, the angle of the arm 26B, and the angle of the bucket 26C. The detection data of the detection device 12 includes images acquired by a stereo camera. Therefore, the recognition unit 43 can calculate the three-dimensional position and attitude of the work machine 20 based on the detection data of the detection device 12. The reflection unit 44 adjusts the three-dimensional model stored in the memory unit 46 so that the three-dimensional model is placed at the position calculated by the recognition unit 43 and has the attitude calculated by the recognition unit 43, and generates reflection data.

[0075] As described above, the 3D model of the work machine 20 is constructed for each part that constitutes the work machine 20, such as the traveling body 24, the rotating body 25, and the work implement 26. The reflection unit 44 adjusts the 3D model by changing the angles of corresponding parts of the 3D model based on the angle of the boom 26A, the angle of the arm 26B, the angle of the bucket 26C, and the rotation angle of the rotating body 25.

[0076] The output unit 45 outputs the reflection data including the three-dimensional model generated by the reflection unit 44 to the display device 52.

[0077] If two GNSS sensors are mounted on the revolving structure 25 of the work machine 20, the recognition unit 43 may calculate the position of the work machine 20 based on detection data from one GNSS sensor. The recognition unit 43 may also calculate the inclination and rotation angle of the revolving structure 25 based on detection data from each of the two GNSS sensors. If stroke sensors are provided on each of the boom cylinder 27A, the arm cylinder 27B, and the bucket cylinder 27C, the recognition unit 43 may calculate the angle of the work implement 26 based on detection data from the stroke sensors.

[0078] FIG. 9 is a diagram showing reflection data according to an embodiment. The reflection data includes a symbol image indicating the position of an object. In the example shown in FIG. 9, the reflection data includes a symbol image 31 indicating the position of the person WM. The reflection unit 44 generates the symbol image 31 based on the position of the person WM recognized by the recognition unit 43. The reflection unit 44 generates the reflection data so that the person WM and the symbol image 31 are displayed superimposed on each other. In the example shown in FIG. 9, the symbol image 31 is in the shape of a frame (box shape) surrounding the person WM. The shape of the symbol image 31 is arbitrary as long as it can highlight the person WM. The symbol image 31 may be displayed adjacent to the person WM. By highlighting the person WM by the symbol image 31, the manager at the remote location 13 can smoothly recognize the presence of the person WM.

[0079] Fig. 10 is a diagram showing reflection data according to the embodiment. In the example shown in Fig. 10, the reflection data includes a symbol image 32 indicating the position of the hydraulic excavator 21. The reflection unit 44 can generate the symbol image 32 based on the position of the hydraulic excavator 21 recognized by the recognition unit 43. In the example shown in Fig. 10, the symbol image 32 is in a frame shape (box shape) surrounding the three-dimensional model 21D. The reflection unit 44 generates the reflection data such that the three-dimensional model 21D of the hydraulic excavator 21 and the symbol image 32 are displayed superimposed on each other. The shape of the symbol image 32 is arbitrary as long as it can emphasize the hydraulic excavator 21.

[0080] FIG. 11 is a diagram showing reflection data according to the embodiment. In the example shown in FIG. 11, the reflection data includes a three-dimensional model 21D that moves in synchronization with the hydraulic excavator 21. As described above, the recognition unit 43 can recognize the operation of the work machine 20. The reflection unit 44 can move the three-dimensional model 21D on the display device 52 so as to synchronize with the operation of the hydraulic excavator 21 based on the operation of the hydraulic excavator 21 recognized by the recognition unit 43. FIG. 11 shows an example in which the boom of the three-dimensional model 21D moves up in synchronization with the raising operation of the boom 26A of the hydraulic excavator 21. Similarly, the reflection unit 44 can move the three-dimensional model 23D on the display device 52 so as to synchronize with the operation of the crawler dump truck 23 based on the operation of the crawler dump truck 23 recognized by the recognition unit 43.

[0081] Fig. 12 is a diagram showing reflection data according to the embodiment. In the example shown in Fig. 12, the reflection data includes a symbol image 31 indicating the position of the person WM, a symbol image 32 indicating the position of the hydraulic excavator 21, and a symbol image 33 indicating the position of the crawler dump truck 23. The reflection unit 44 generates the reflection data such that, for example, the three-dimensional model 23D of the crawler dump truck 23 and the symbol image 33 are displayed superimposed on each other.

[0082] The reflection unit 44 determines whether or not the construction of the construction area has been completed (step S8).

[0083] If it is determined in step S8 that the construction is not completed (step S8: No), the processes from step S2 to step S7 are repeated. If it is determined in step S8 that the construction is completed (step S8: Yes), the construction management method according to the embodiment is terminated.

[0084] [Computer System] FIG. 13 is a block diagram showing a computer system 1000 according to an embodiment. The above-mentioned server 4 includes the computer system 1000. The computer system 1000 has a processor 1001 such as a CPU (Central Processing Unit), a main memory 1002 including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory), a storage 1003, and an interface 1004 including an input / output circuit. The functions of the above-mentioned server 4 are stored in the storage 1003 as a computer program. The processor 1001 reads the computer program from the storage 1003, loads it in the main memory 1002, and executes the above-mentioned processing according to the program. The computer program may be distributed to the computer system 1000 via a network.

[0085] In accordance with the above-described embodiment, the computer program or computer system 1000 can execute the following operations: storing current terrain data of the construction site 2 where the work machine 20 is operating, acquiring detection data from the detection device 9 that detects the construction site 2, generating reflection data that reflects the detection data in the current terrain data, and displaying the reflection data on the display device 52.

[0086] [effect] As described above, according to the embodiment, current topography data is created which indicates the reference topography of the construction site 2 before construction. During construction, the work machine 20 and the construction area around the work machine 20 are detected by the detection device 9. Reflection data in which the detected data is reflected in the current topography data is displayed on the display device 52. The manager at the remote location 13 can check the progress of construction in the construction area in real time by checking the display device 52. The manager at the remote location 13 can remotely monitor the progress of construction.

[0087] The detection data reflected in the current topographical data is updated more frequently than the current topographical data, so that the manager at the remote location 13 can check the latest status of the construction area.

[0088] The detection data reflected in the current topographical data includes updated topographical data of the construction area, so that the manager at the remote location 13 can check the latest topographical data of the construction area during or after construction.

[0089] The detection data reflected in the current topographical data includes non-topographical data showing objects at the construction site 2. The reflecting unit 44 reflects updated topographical data, in which the non-topographical data has been removed from the detection data acquired by the detection data acquiring unit 42, in the current topographical data. This allows the manager at the remote location 13 to check the latest topographical data of the construction area in which the influence of objects has been suppressed.

[0090] The detection data reflected in the current topographical data includes non-topographical data showing objects at the construction site 2. This allows the manager at the remote location 13 to check not only the latest topographical data of the construction site, but also the status of objects at the construction site 2 in real time.

[0091] The reflected data displayed on the display device 52 includes the objects recognized by the recognition unit 43. The reflected data in which the latest topography and the objects are distinguished is displayed on the display device 52. This allows the manager at the remote location 13 to properly check both the latest topography of the construction site and the status of the objects at the construction site 2 in real time.

[0092] The reflected data displayed on the display device 52 includes a three-dimensional model of the work machine 20. By checking the three-dimensional model displayed on the display device 52, the manager at the remote location 13 can properly check the status of the work machine 20 in real time.

[0093] The recognition unit 43 recognizes the operation of the work machine 20 based on the detection data acquired by the detection data acquisition unit 42. The reflection unit 44 can move the three-dimensional model in synchronization with the work machine 20 based on the recognition results of the recognition unit 43. The manager at the remote location 13 can properly check the status of the work machine 20 in real time by checking the three-dimensional model that moves in synchronization with the actual work machine 20.

[0094] The recognition unit 43 recognizes the person WM based on the detection data acquired by the detection data acquisition unit 42. The manager at the remote location 13 can properly check the situation of the person WM in real time by checking the person WM displayed on the display device 52.

[0095] The reflected data displayed on the display device 52 includes a symbol image indicating the position of the object. The manager at the remote location 13 can properly confirm the position of the object by checking the symbol image displayed on the display device 52.

[0096] The detection device 9 is mounted on the aircraft 8. This allows the detection device 9 to collectively detect the work machine 20 and the construction area from the sky.

[0097] [Other embodiments] In the above-described embodiment, the recognition unit 43 recognizes the position of the work machine 20 based on detection data from the detection device 9. For example, a position sensor that detects the position of the work machine 20 may be provided on the work machine 20, and the recognition unit 43 may recognize the position of the work machine 20 based on the detection data of the position sensor.

[0098] In the above-described embodiment, the recognition unit 43 recognizes the operation of the work machine 20 based on the detection data of the detection device 9. For example, a motion sensor that detects the operation of the work machine 20 may be provided on the work machine 20, and the recognition unit 43 may recognize the operation of the work machine 20 based on the detection data of the motion sensor. Examples of the motion sensor include an angle sensor that detects the operation of the work implement 26 or a stroke sensor that detects the amount of extension and contraction of the hydraulic cylinder 27.

[0099] In the above-described embodiment, the recognition unit 43 may recognize an object based on, for example, a pattern matching method without using artificial intelligence. The recognition unit 43 can recognize an object by matching a template representing a person WM with image data of the construction site 2.

[0100] In the above-described embodiment, the detection device 9 does not have to be mounted on the aircraft 8. The detection device 9 may be mounted on, for example, the work machine 20, or may be attached to a structure present at the construction site 2. The same applies to the detection device 12.

[0101] In the above-described embodiment, the current terrain data creation unit 41, the detection data acquisition unit 42, the recognition unit 43, the reflection unit 44, the output unit 45, and the storage unit 46 may each be configured as separate hardware. For example, at least one of the functions of the current terrain data creation unit 41, the detection data acquisition unit 42, the recognition unit 43, the reflection unit 44, the output unit 45, and the storage unit 46 may be provided in the management device 3 or in a server other than the server 4.

[0102] In the above-described embodiment, the management device 3 is supported by the traveling device 6 and can travel within the construction site 2. The management device 3 may be mounted on the work machine 20, or may be installed at a predetermined position within the construction site 2.

[0103] In the above embodiment, the detection device 12 detects the entire construction site 2, and the detection device 9 detects a part of the construction site 2. The detection device 9 may detect the entire construction site 2.

[0104] In the above-described embodiment, the detection targets of the detection device 9 are not limited to the topography, the work machine 20, and the person WM of the construction site 2. In other embodiments, the detection device 9 may be configured to detect construction materials.

[0105] In the above-described embodiment, the information terminal 5 does not have to be located at the remote location 13 of the construction site 2. The information terminal 5 may be mounted on the work machine 20, for example. Furthermore, the information terminal 5 may be omitted. The progress of the construction may be output from a monitor on the work machine 20. The monitor may include not only a display device but also an input device.

[0106] In the above-described embodiment, the work machine 20 is not limited to include the hydraulic excavator 21, the bulldozer 22, and the crawler dump truck 23. In other embodiments, the work machine 20 may include parts of the hydraulic excavator 21, the bulldozer 22, and the crawler dump truck 23. The work machine 20 may also include other types of work machines.

[0107] In the above-described embodiment, the position of the aircraft 8 is not limited to being detected using a Global Navigation Satellite System (GNSS) and the attitude of the aircraft 8 is not limited to being detected using an inertial measurement unit. In other embodiments, the position and attitude of the aircraft 8 may be detected using Simultaneous Localization and Mapping (SLAM). Similarly, the position and attitude of the aircraft 11 and the work machine 20 may be detected using SLAM. [Explanation of symbols]

[0108] 1...construction management system, 2...construction site, 3...management device, 4...server (data processing device), 5...information terminal, 6...traveling device, 7...cable, 8...aircraft, 9...detection device, 10...communication system, 11...aircraft, 12...detection device, 13...remote location, 14...position sensor, 15...attitude sensor, 16...position sensor, 17...attitude sensor, 20...working machine, 21...hydraulic excavator, 21D...three-dimensional model, 22...bulldozer, 23...crawler dump, 23D...three-dimensional model, 24...traveling body, 25...rotating body, 26...working machine, 26A...boom, 26B...arm, 26C... Bucket, 27...hydraulic cylinder, 27A...boom cylinder, 27B...arm cylinder, 27C...bucket cylinder, 28...running body, 29...vehicle body, 30...dump body, 31...symbol image, 32...symbol image, 33...symbol image, 41...current terrain data creation unit, 42...detection data acquisition unit, 43...recognition unit, 44...reflection unit, 45...output unit, 46...memory unit, 51...input device, 52...display device, 1000...computer system, 1001...processor, 1002...main memory, 1003...storage, 1004...interface, WM...person.

Claims

1. a current status topography data creation unit that creates current status topography data of a construction site where a work machine is operating; A detection data acquisition unit that acquires detection data of a detection device that detects a portion of the construction site; a reflection unit that generates reflection data that reflects the detection data in the current topographical data; A recognition unit that recognizes an object at the construction site based on the detection data; an output unit that outputs the reflected data to a display device; The reflection data includes updated terrain data indicating a portion of an updated terrain of the construction site, and non-terrain data indicating objects of the construction site; The reflection unit is When the recognition unit determines that no object exists in the updated terrain data, the updated terrain data is applied to a part of the current terrain data; when the recognition unit determines that an object is present in the updated terrain data, the updated terrain data in which the non-terrain data has been removed from the detection data is applied to the current terrain data, and then the non-terrain data is applied; Construction management system.

2. The detection data is updated more frequently than the current terrain data. The construction management system according to claim 1 .

3. the object includes the work machine, the reflection data includes a three-dimensional model of the work machine; The construction management system according to claim 1 .

4. The recognition unit recognizes an operation of the work machine, The reflection unit moves the three-dimensional model in synchronization with the work machine. The construction management system according to claim 3.

5. The object includes a person. The construction management system according to any one of claims 1 to 4.

6. The recognition unit recognizes a position of the object, The reflected data includes a symbol image indicating a position of the object. The construction management system according to any one of claims 1 to 5.

7. The detection device is mounted on an aircraft. The construction management system according to any one of claims 1 to 6.

8. The detection device detects the work machine and a construction area around the work machine. The construction management system according to any one of claims 1 to 7.

9. a current status topography data creation unit that creates current status topography data of a construction site where a work machine is operating; A detection data acquisition unit that acquires detection data of a detection device that detects a portion of the construction site; a reflection unit that generates reflection data that reflects the detection data in the current topographical data; A recognition unit that recognizes an object at the construction site based on the detection data; an output unit that outputs the reflected data to a display device; The reflection data includes updated terrain data indicating a portion of an updated terrain of the construction site, and non-terrain data indicating objects of the construction site; The reflection unit is When the recognition unit determines that no object exists in the updated terrain data, the updated terrain data is applied to a part of the current terrain data; when the recognition unit determines that an object is present in the updated terrain data, the updated terrain data in which the non-terrain data has been removed from the detection data is applied to the current terrain data, and then the non-terrain data is applied; Data processing device.

10. A computer system comprising: Storing current topographical data of a construction site where a work machine is operating; Obtaining detection data from a detection device that detects a portion of the construction site; generating reflection data that reflects the detection data in the current topographical data; Recognizing objects at the construction site based on the detection data; and displaying the reflected data on a display device; The reflection data includes updated terrain data indicating a portion of an updated terrain of the construction site, and non-terrain data indicating objects of the construction site; If it is determined that no object exists in the updated terrain data, the updated terrain data is applied to a part of the current terrain data; if it is determined that an object is present in the updated terrain data, the updated terrain data, in which the non-terrain data has been removed from the detection data, is applied to the current terrain data, and then the non-terrain data is applied; Construction management method.

11. The detection data is updated more frequently than the current terrain data. The construction management method according to claim 10.

Citation Information

Patent Citations

  • Precise installation support system and program for construction, and precise installation method using the system and the program

    JP2003253661A

  • Detecting and processing device and detecting and processing method for work machine

    JP2018059268A

  • Site management system and work machine

    JP2021038571A

  • Work management system for construction machine, and construction machine

    WO2017170651A1

  • Operation information transmission device, construction management system, operation information transmission method, and program

    WO2019012993A1