Work support system, work support method, and work support program

Abstract

To provide a work support system, a work support method, and a work support program, for supporting efficient and accurate setting-out in a work site.SOLUTION: A work support system A1 includes: a soil information storage unit 22 that stores soil conditions recorded on a three-dimensional model with depths set for sections; and a work support apparatus 30 which displays a three-dimensional model of elements arranged in a virtual space. The work support apparatus 30 specifies, in a real space, a field of view captured by a camera 30b, specifies a three-dimensional model of a section included in the field of view, and specifies an excavation status of the section. The work support apparatus 30 uses the soil information storage unit 22 to specify soil condition in accordance with the excavation status, and displays information indicating the soil condition on an image of the field of view.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a work support system, a work support method, and a work support program that support marking out work at a construction site. [Background technology]

[0002] Conventionally, the degree of soil contamination has been measured at former industrial waste disposal sites and the like, with the aim of replacing contaminated soil with healthy soil. For this reason, technology for managing the results of soil contamination measurements has been studied (see Patent Document 1). The soil contamination status management system described in this document imports the shape of the yard, which is the area of ​​soil to be managed, as well as contamination data and soil quality data for that yard, and plots the contamination status, excavation status, and backfill status diagrams, which are then displayed on a screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-008102 Summary of the Invention [Problem to be solved by the invention]

[0004] When replacing contaminated soil with sound soil, marking out was performed to identify the location of the contaminated soil. For example, because it was necessary to identify and treat excavated soil according to the contamination level, it was necessary to consider the depth direction. Here, at the work site, marking out the area was necessary to excavate according to the different contamination levels in the depth direction. In this case, if a total station was used for marking out, it would be time-consuming to survey and mark out the area. [Means for solving the problem]

[0005] A work support system for solving the above problems includes a soil information storage unit that records soil conditions in a 3D model in which depths are set for each section, and a work support device that displays the 3D model of elements arranged in a virtual space. The work support device identifies a field of view captured by a camera in real space, identifies a 3D model of the section included in the field of view, identifies the excavation status of the section, uses the soil information storage unit to identify soil conditions corresponding to the excavation status, and displays information indicating the soil conditions on an image of the field of view. [Effects of the Invention]

[0006] The present disclosure can support efficient and accurate marking work at work sites. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is an explanatory diagram of a system according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram of a hardware configuration of the embodiment. [Figure 3] FIG. 2 is an explanatory diagram of a soil information storage unit according to the embodiment. [Figure 4] FIG. 2 is an explanatory diagram of an excavation information storage unit according to the embodiment. [Figure 5] FIG. 2 is an explanatory diagram of a processing procedure according to an embodiment. [Figure 6] FIG. 2 is an explanatory diagram of a processing procedure according to an embodiment. [Figure 7] FIG. 2 is an explanatory diagram of a processing procedure according to an embodiment. [Figure 8] FIG. 1 is an explanatory diagram showing a soil condition of an embodiment in a three-dimensional model. [Figure 9] FIG. 1 is an explanatory diagram showing a soil condition of an embodiment in a three-dimensional model. [Figure 10] FIG. 2 is an explanatory diagram of a display screen according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] An embodiment of a work support system, a work support method, and a work support program will be described below with reference to Figures 1 to 10. In this embodiment, the work support system, the work support method, and the work support program will be described as being used when work is performed at a work site where contaminated soil is excavated while comparing a three-dimensional image of the contamination situation (soil situation) with the site situation. As shown in FIG. 1, a work support system A1 of this embodiment includes a management device 10, a support server 20, and a work support device 30, which are interconnected via a network.

[0009] (Hardware configuration description) 2, the hardware configuration of the information processing device H10 that constitutes the management device 10, the support server 20, and the work support device 30 will be described. The information processing device H10 includes a communication device H11, an input device H12, a display device H13, a storage device H14, and a processor H15. Note that this hardware configuration is an example, and it can also be realized by other hardware.

[0010] The communication device H11 is an interface that establishes a communication path with another device and executes data transmission and reception, and is, for example, a network interface or a wireless interface.

[0011] The input device H12 is a device that accepts input of various information, such as a mouse, a keyboard, etc. The display device H13 is a display that displays various information, etc. Note that a touch panel display may be used as the input device H12 and the display device H13.

[0012] The storage device H14 is a storage device that stores data and various programs for executing various functions of the management device 10, the support server 20, and the task support device 30. Examples of the storage device H14 include a ROM, a RAM, and a hard disk.

[0013] The processor H15 uses programs and data stored in the storage device H14 to control each process in the management device 10, the support server 20, and the work support device 30. Examples of the processor H15 include a CPU and an MPU. The processor H15 loads programs stored in a ROM or the like into a RAM and executes various processes for each process.

[0014] The processor H15 is not limited to a processor that performs all of its processing using software. For example, the processor H15 may include a dedicated hardware circuit (e.g., an application-specific integrated circuit (ASIC)) that performs hardware processing for at least some of the processing it performs. That is, the processor H15 may be configured with the following:

[0015] [1] One or more processors that operate according to a computer program (software). [2] One or more dedicated hardware circuits that perform at least some of the various processes [3] Circuits containing combinations of these The processor includes a CPU and memory, such as RAM and ROM, that stores program code or instructions configured to cause the CPU to perform processes. Memory, or computer-readable media, includes any available media that can be accessed by a general-purpose or special-purpose computer.

[0016] (System Configuration) Next, each function of the work support system A1 will be described with reference to FIG. The management device 10 is a computer terminal for performing modeling processing (3D CAD processing) of the contamination status of a work site using 3D CAD (computer-aided design). This modeling can manage not only the shapes of elements but also the attributes (properties) of the elements. The management device 10 functions as a modeling unit 11 using a 3D CAD application that runs on an operating system.

[0017] The modeling unit 11 uses 3D CAD technology to represent each element as a 3D model (object) and place it in a 3D virtual space. Each 3D model holds attribute information of each element (e.g., contamination information indicating the contamination level).

[0018] The support server 20 is a computer system that uses mixed reality (MR) to execute processing to support management of on-site conditions. The support server 20 includes a control unit 21, a soil information storage unit 22, an excavation information storage unit 23, and a display model storage unit 24.

[0019] The control unit 21 performs the processes described below (processes including a conversion stage, a management stage, etc.) By executing a processing program for this purpose, the control unit 21 functions as a conversion unit 211, a management unit 212, etc.

[0020] The conversion unit 211 converts the 3D CAD data into data that can be displayed in a mixed reality 3D virtual space. Here, the 3D CAD data designed using the 3D CAD is reduced in size, converted into polygons, and converted into a format that can be used by the display application of the work assistance device 30. The management unit 212 acquires the excavation status at the site from the work support device 30 and records it in the excavation information storage unit 23.

[0021] 3, three-dimensional model information 220 created using three-dimensional CAD is recorded in the soil information storage unit 22. This three-dimensional model information 220 is recorded when the contamination level of a work site is registered using three-dimensional CAD. The three-dimensional model information 220 includes three-dimensional model information (object ID, element model, placement information, attribute information) for project information.

[0022] The project information includes information such as the project ID, the name of the work site, the longitude and latitude, and the direction of the construction site. The object ID is information about an identifier for identifying the three-dimensional shape (three-dimensional model) that constitutes the soil at the work site.

[0023] The element model is information about the three-dimensional shape (three-dimensional model) that constitutes the soil, etc., of the work site. In this embodiment, soil element objects and marker objects are used as elements. The soil element objects are objects related to the soil of the work site. The marker objects are objects used to align the three-dimensional virtual space with the real space. In this embodiment, as shown in FIG. 8, a three-dimensional model of soil is used in which the work site is divided into rectangular parallelepiped elements with a planar section of 10 m×10 m and a depth of 1 m. The placement information includes information about the placement of the three-dimensional model (coordinates in the three-dimensional virtual space).

[0024] The attribute information includes the contents of each element model. The attribute information of a soil element object includes a soil block ID. The soil block ID includes an area ID for identifying the section (area) of the work site and depth information indicating the depth from the ground surface before excavation. Furthermore, the attribute information of a soil element object includes property information related to the contamination level. The attribute information of a marker object includes a marker ID.

[0025] 4, excavation management information 230 relating to the excavation status at the work site is recorded in the excavation information storage unit 23. This excavation management information 230 is recorded when excavation status information is acquired from the work support device 30. The excavation management information 230 includes information relating to the project ID, area ID, and progress.

[0026] The project ID is an identifier for identifying each project. The area ID is an identifier for identifying the section (area) where excavation will be carried out at the work site of this project. The progress information includes information about the depth (depth) that has been excavated in this area.

[0027] The display model storage unit 24 stores three-dimensional models (MR model information) that can be displayed in a mixed reality three-dimensional virtual space. This MR model information is recorded when a conversion process is performed on the three-dimensional model information 220. Like the three-dimensional model information 220, this MR model information also stores placement information and attribute information in the three-dimensional virtual space.

[0028] The work support device 30 shown in Fig. 1 is a computer terminal (mobile terminal) that supports marking out on site. For example, a head-mounted display (HMD) terminal such as HoloLens (registered trademark) is used as the work support device 30. The work support device 30 includes a display unit 30a, a camera 30b, an operation unit 30c, and a position identification unit 30d. Furthermore, the work support device 30 executes a work support program to realize an MR processing unit 31 and a work support unit 32, and also includes a display model storage unit 35.

[0029] The display unit 30a functions as an output unit for outputting various information such as a three-dimensional model to the display device H13. The camera 30b is a photographing unit that photographs a subject.

[0030] The operation unit 30c is used for inputting various information, for example, by voice recognition of voices collected by a microphone or by gestures based on images captured by the camera 30b.

[0031] The position identification unit 30d executes a process of identifying the current location of the work assistance device 30 by self-position estimation. For this self-position estimation, for example, a visual odometry (VO) technique can be used, which estimates the movement distance according to the relative change in images sequentially captured by the camera 30b. With this technique, the position identification unit 30d estimates the movement of the camera 30b according to the detection of feature points in the captured images and the relative positional relationship of the feature points. Note that the method of identifying the position is not limited to a method using the visual odometry technique. For example, a visual-inertial odometry (VIO) technique may be performed by using or in combination with an IMU (Inertial Measurement Unit) such as a 3-axis acceleration sensor or a gyro sensor.

[0032] The MR processing unit 31 executes a process of superimposing the MR model recorded in the display model storage unit 35 on the image captured by the camera 30b and displaying it. The work support unit 32 executes a process for supporting marking out at a work site by superimposing and displaying the MR model on the image captured by the camera 30b. The display model storage unit 35 stores the display model acquired from the support server 20.

[0033] [Work support processing] The work support process will be described with reference to FIGS.

[0034] In this embodiment, as shown in Fig. 5, an element model 410 including soil information (contamination information) configured as a three-dimensional model is narrowed down by the excavation depth in the excavation situation 420. As a result, a surface contamination situation display 430 according to the excavation situation is output at the work site. Then, based on this contamination situation display 430, it is possible to identify marking positions 440 for areas where excavation of contaminated soil is required according to the contamination level.

[0035] Below, the soil information registration process (FIG. 6) and the marking process (FIG. 7) will be explained in that order. (Soil information registration process) First, the soil information registration process will be described with reference to Fig. 6. Here, the contamination status (soil information) is measured at multiple locations in the work site.

[0036] Then, the modeling unit 11 of the management device 10 executes a process of acquiring soil information (step S101). Specifically, the user inputs project information using the modeling unit 11 of the management device 10. Then, the user inputs the contamination level for each depth relative to the plane coordinates to the modeling unit 11.

[0037] Next, the modeling unit 11 of the management device 10 executes a three-dimensional model generation process (step S102). Specifically, the modeling unit 11 of the management device 10 records three-dimensional model information 220 including project information in the soil information storage unit 22 of the support server 20. In this case, the contamination level is set as attribute information in the element model to which the object ID and the area ID corresponding to the plane coordinates are assigned.

[0038] In this case, as shown in Fig. 8, the work site is divided into 10m x 10m grids (elements) in a three-dimensional virtual space 500. Each grid is made up of a three-dimensional model with a depth of 1m. In the three-dimensional virtual space 500, grids with darker colors represent areas with higher contamination levels, and grids with lighter colors represent areas with higher contamination levels.

[0039] Next, the modeling unit 11 of the management device 10 executes a process for installing a position correction marker (step S103). Specifically, the user uses the modeling unit 11 of the management device 10 to install a position correction marker (first position correction marker) at a predetermined position on the work site in a predetermined orientation. This position correction marker is also an object on the 3D CAD. In this case, the modeling unit 11 sets "marker," "marker ID," and "marker dimensions" as attribute information of this object. In this embodiment, the position correction marker is set at a location on the work site that is easy to position and place, such as a triangulation point that serves as a reference for longitude, latitude, and elevation.

[0040] Next, the control unit 21 of the support server 20 executes MR data conversion processing (step S104). Specifically, when setting completion is input to the management device 10, the conversion unit 211 of the control unit 21 converts the 3D model of the 3D model information 220 recorded in the soil information storage unit 22 into an MR model that can be displayed in a mixed reality 3D virtual space. Here, the 3D model is lightened and converted into polygons, and an MR model in a format that can be displayed by the MR processing unit 31 of the work support device 30 is generated. This MR model includes 3D models of soil element objects and marker objects.

[0041] (Marking support processing) Next, the marking support process will be described with reference to FIG. 7. Here, a marker-printed sheet on which a position correction marker (second position correction marker) is printed in advance is placed at a position corresponding to the position correction marker set in the 3D CAD at the work site. The position correction marker is printed on this marker-printed sheet in a predetermined size created in the 3D CAD. A two-dimensional code image encoding a marker ID is used as the position correction marker. This two-dimensional code image has a rectangular outer frame of a predetermined size designed in the 3D CAD, and is placed in the orientation that corresponds to the arrangement in the three-dimensional virtual space. Then, the work support program is executed in the work support device 30, and the MR processing unit 31 is started.

[0042] First, the work assistance device 30 executes a project selection process (step S201). Specifically, the user selects a project recorded in the display model storage unit 24 of the assistance server 20. In this case, the MR processing unit 31 acquires the MR model of the selected project from the display model storage unit 24 and records it in the display model storage unit 35.

[0043] Next, the work assistance device 30 executes a position correction marker photographing process (step S202). Specifically, the user moves to the location where the position correction marker is located. Furthermore, the user checks the position correction marker on the display unit 30a.

[0044] The sheet placed at the work site is captured by the camera 30b of the work support device 30. When the work support device 30 recognizes a marker in the image captured by the camera 30b, it displays a rectangular frame identifying the marker on the display unit 30a.

[0045] Next, the task assistance device 30 executes a decoding process of the position correction marker (step S203). Specifically, the MR processing unit 31 decodes the position correction marker to obtain the marker ID and the marker dimensions.

[0046] Next, the task support device 30 executes a registration process (step S204). Specifically, the MR processing unit 31 matches the arrangement (shape, size, and orientation) of the captured 2D code image in the captured image with the arrangement of the position correction marker in the 3D virtual space of the decoded marker ID. This correspondence identifies the captured position (coordinates) and orientation, aligns the coordinate axes in the 3D virtual space with the coordinate axes in the real space, and determines the scale according to the marker dimensions. As a result, using the acquired MR model, registration is achieved so that the field of view image in the 3D virtual space, with the viewpoint of the camera 30b of the task support device 30, can be mapped onto the captured image of the camera 30b.

[0047] Next, the work support device 30 executes a process of identifying the excavation status (step S205). Specifically, the MR processing unit 31 identifies the excavated depth (excavation depth) in each area as the excavation status from the excavation information storage unit 23.

[0048] Next, the task support device 30 executes an object identification process (step S206). Specifically, the MR processing unit 31 identifies an MR model that is placed below the already excavated depth.

[0049] As shown in Figure 9, in the virtual three-dimensional space 510, the MR models placed in the excavated area are deleted, and the MR models placed below the excavation depth remain. The soil block ID (area ID, depth) of the attribute information is placed at a display position a predetermined distance from the corner of the MR model. In this case, the display position of the soil block ID of adjacent MR models is changed so that the distance from the corner does not overlap within the field of view.

[0050] Next, the work assistance device 30 executes a display process of the object according to the degree of superimposition (step S207). Specifically, the MR processing unit 31 displays, on the display unit 30a, a mixed reality image in which the image captured by the camera 30b and the MR model image of the display target within the field of view of the 3D virtual space are superimposed. Furthermore, the MR processing unit 31 displays a slider on the display unit 30a. By using this slider, the degree of superimposition of the MR model image displayed on the display unit 30a can be adjusted.

[0051] As shown in a display image 520 in FIG. 10, the MR model becomes transparent, and a real image 521 of the scene and an MR model image 522 are displayed in a superimposed form.

[0052] Next, the work support device 30 executes a process for displaying the marking position (step S208). Specifically, the work support unit 32 identifies the boundary position of MR models with different contamination levels in adjacent MR models in an unexcavated surface layer excavation state. Then, the work support unit 32 displays the end of the identified boundary position (a corner of the MR model) as the marking position. 10 shows a marking position 523. Marking is performed on site while referring to the marking position 523.

[0053] As will be described later, the user carries the work support device 30 with him or her and performs marking while moving around. Furthermore, after moving, a new marker-printed sheet may be found. In this case, the work supporting device 30 again executes the position correction marker photographing process (step S202) to the position alignment process (step S204).

[0054] According to this embodiment, the following effects can be obtained. (1) In this embodiment, the modeling unit 11 of the management device 10 executes a soil information acquisition process (step S101) and a three-dimensional model generation process (step S102), thereby making it possible to grasp the contamination level that varies depending on the depth.

[0055] (2) In this embodiment, the modeling unit 11 of the management device 10 executes a process of setting a position correction marker (step S103). Then, the work support device 30 executes a process of photographing the position correction marker (step S202), a process of decoding the position correction marker (step S203), and a process of aligning the position (step S204). This enables alignment between an object designed by 3D CAD and an object in real space.

[0056] (3) In this embodiment, the modeling unit 11 of the management device 10 executes a process of setting a position correction marker (step S102). Here, the position correction marker is set at a location on the work site where it is easy to position and place. This allows the marker for alignment to be placed using the reference position at the work site.

[0057] (4) In this embodiment, the work support device 30 executes an excavation situation identification process (step S205) and an object identification process (step S206), thereby making it possible to change the MR image according to the excavation situation.

[0058] (5) In this embodiment, the work support device 30 executes a display process of the object according to the degree of superimposition (step S207). As a result, the three-dimensional virtual image is superimposed on the real image, so that the excavation area can be identified in the real space and marked out.

[0059] (6) In this embodiment, the work support device 30 executes a process for displaying the marked positions (step S208), thereby enabling areas with different contamination levels to be identified and excavation to be carried out.

[0060] (7) In this embodiment, when a new marker-printed sheet is found, the task assistance device 30 executes the position correction marker photographing process (step S202) to the position alignment process (step S204). This allows the task assistance device 30 to correct errors that occur in the self-position estimation.

[0061] This embodiment can be modified as follows: This embodiment and the following modifications can be combined and implemented within the scope of technical compatibility. In the above embodiment, the present disclosure is used when marking out a work site of contaminated soil while comparing a 3D image with the site conditions. The application of the present disclosure is not limited to work sites of contaminated soil. For example, marking out may be performed taking into account the soil type classification (soil conditions), and excavation may be performed for each soil type classification.

[0062] In the above embodiment, the management device 10, the support server 20, and the work support device 30 are used. However, the hardware configuration is not limited to this. For example, the management device 10 or the work support device 30 may convert 3D model information into MR model information. In this case, the conversion unit 211 is provided in each terminal.

[0063] The excavation information storage unit 23 may be provided in a site management server different from the support server 20 that manages the three-dimensional information. In this case, the work support device 30 uploads the excavation results to the site management server.

[0064] In the above embodiment, a head-mounted terminal is used as the task support device 30. Instead of this, a tablet terminal having a touch panel display that has both the functions of the display unit 30a and the operation unit 30c may be used.

[0065] In the above embodiment, the excavation information storage unit 23 stores excavation management information 230 related to the excavation status at the work site. Here, the excavation status may be identified during the marking support process. For example, an image is taken of an area where no excavation has been performed so that the step in the excavated area is visible. Then, the work support unit 32 of the work support device 30 calculates the height of the step contained in the captured image through image processing. Then, after the work support unit 32 identifies the area of ​​the step using the field of view (photographing direction, photographing distance), the excavation status of this area is updated in the excavation information storage unit 23.

[0066] In the above embodiment, the position identification unit 30d of the work assistance device 30 executes a process of identifying the current location of the work assistance device 30 by self-position estimation. The method of identifying the current location of the work assistance device 30 is not limited to this. For example, a global positioning system (GPS) may be used. Alternatively, light detection and ranging (LIDAR) technology may be used.

[0067] In the above embodiment, the modeling unit 11 of the management device 10 executes a process for placing a position correction marker (step S102). The position correction marker used for alignment is not limited to a 2D code image. It may be a 3D shape, not just a 2D image, as long as it can be recognized as a unique identifier. In this case, by assigning a marker attribute to a marker object in the 3D CAD, alignment is possible in the same way as with a 2D code image. For example, an existing object already placed at the construction site may be used as the position correction marker. An example of an existing object may be a construction signboard or the like that is placed early on. In this case, the position identification unit 30d recognizes the existing object through image recognition processing. The position identification unit 30d then compares the recognized object position and shape with the position (shape, size, orientation) of the element model in the 3D model information to perform alignment.

[0068] In the above embodiment, the work support device 30 executes the excavation status identification process (step S205) and the object identification process (step S206). The processing method is not limited as long as it can display a 3D virtual space image of the unexcavated soil model in the 3D image. For example, the modeling unit 11 of the management device 10 may execute the MR data conversion process (step S104) after deleting the 3D model up to the excavated depth in the 3D model generation process (step S102).

[0069] Next, the technical ideas that can be understood from the above-described embodiment and other examples will be described below. (a) The work support system according to claim 1, wherein the soil condition is a contamination condition.

[0070] (b) A work support system according to claim 1 or (a), characterized in that a marking position is displayed on the image in a boundary area where the soil condition changes on the unexcavated ground surface in the excavation situation. (c) The work support device according to claim 1, characterized in that the excavation depth according to the excavation situation is identified from a captured image. [Explanation of symbols]

[0071] A1...work support system, 10...management device, 11...modeling unit, 20...support server, 21...control unit, 211...conversion unit, 212...management unit, 22...soil information storage unit, 23...excavation information storage unit, 24...display model storage unit, 30...work support device, 30a...display unit, 30b...camera, 30c...operation unit, 30d...position identification unit, 31...MR processing unit, 32...work support unit, 35...display model storage unit.

Claims

1. a soil information storage unit that records soil conditions in a three-dimensional model in which depths are set for the plots; a work support device that displays a three-dimensional model of an element arranged in a virtual space, The work support device includes: In real space, a field of view captured by a camera is identified; Identifying a three-dimensional model of a section included in the field of view; Identifying the excavation status of the section; Using the soil information storage unit, identify the soil conditions according to the excavation conditions; A work support system characterized in that information indicating the soil condition is displayed on the image of the field of view.

2. a soil information storage unit that records soil conditions in a three-dimensional model in which depths are set for the plots; a work support device that displays a three-dimensional model of an element arranged in a virtual space, The work support device includes: In real space, a field of view captured by a camera is identified; Identifying a three-dimensional model of a section included in the field of view; Identifying the excavation status of the section; Using the soil information storage unit, identify the soil conditions according to the excavation conditions; A work support method characterized in that information indicating the soil condition is displayed on the image of the field of view.

3. a soil information storage unit that records soil conditions in a three-dimensional model in which depths are set for the plots; a work support device that displays a three-dimensional model of an element arranged in a virtual space; The work support device, In real space, a field of view captured by a camera is identified; Identifying a three-dimensional model of a section included in the field of view; Identifying the excavation status of the section; Using the soil information storage unit, identify the soil conditions according to the excavation conditions; A work support program characterized by functioning as a means for displaying information indicating the soil condition on the image of the field of view.

Images