Process display system, process display method, and process display program

Abstract

The present invention relates to a process display system, a process display method, and a process display program. SOLUTION: The process display system 1 includes a storage unit, an acquisition unit, and a display processing unit. The storage unit stores the 3D model information and the process information associated with the 3D model information. The acquisition unit acquires video information captured at a predetermined shooting position and shooting time. The display processing unit, based on the input of time information, video information, and process information corresponding to the process received from the user terminal, displays the video information regarding the process corresponding to the time information in the space of the 3D model in correspondence with the shooting position, and transmits the results of the display processing.

Description

[Technical field]

[0001] The present invention relates to a process display system, a process display method, and a process display program. [Background technology]

[0002] Traditionally, video and images taken at construction sites have been used to check the status of construction work and conduct inspections.

[0003] Patent Document 1 discloses an inspection processing system that can efficiently inspect structures by superimposing a BIM image generated under specified conditions onto a captured image and outputting the image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-27855 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, at an actual construction site, the positions and angles at which images are taken vary depending on the work content, etc., and the position and viewpoint from which the object to be photographed (structure) is photographed are very important factors in checking the situation and inspecting it. However, the technology of Patent Document 1 does not take such viewpoints into consideration, and it is not possible to grasp the position and angle from which the structure is photographed.

[0006] The present invention has been made in consideration of the problems with the conventional technology described above, and has an object to realize a process display system that enables history checking and inspection at a construction site simply and effectively. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides a process display system for displaying a history of processes at a construction site, The process display system includes a storage unit, an acquisition unit, and a display processing unit, The storage unit stores 3D model information and process information associated with the 3D model information, The acquisition unit acquires video information captured at a predetermined shooting position and shooting time, Based on the input of time information corresponding to the process received from the user terminal, the video information, and the process information, the display processing unit displays the video information regarding the process corresponding to the time information in the space of the 3D model in correspondence with the shooting position, and transmits the results of the display processing.

[0008] The present invention also provides a process display method executed by a process display system for displaying a history of processes at a construction site, comprising: The process display system includes a storage unit, an acquisition unit, and a display processing unit, The storage unit stores 3D model information and process information associated with the 3D model information; The acquiring unit acquires video information captured at a predetermined shooting position and at a predetermined shooting time; The display processing unit displays the image information relating to the process corresponding to the time information in the space of the 3D model in correspondence with the shooting position based on the input of time information corresponding to the process history received from the user terminal, the image information, and the process information, and transmits the result of the display processing.

[0009] The present invention also provides a process display program for displaying a history of a process at a construction site, A computer is configured to function as a storage unit, an acquisition unit, and a display processing unit; The storage unit stores 3D model information and process information associated with the 3D model information, The acquisition unit acquires video information captured at a predetermined shooting position and shooting time, Based on input of time information corresponding to the process history received from a user terminal, the video information, and the process information, the display processing unit displays the video information regarding the process corresponding to the time information in the space of the 3D model in correspondence with the shooting position, and transmits the results of the display processing.

[0010] With this configuration, history checking and inspection can be carried out simply and effectively at the construction site.

[0011] In a preferred embodiment of the present invention, the display processing unit, when there are a plurality of steps corresponding to the input of the time information in the input of the time information, displays the plurality of steps and transmits a result of the display processing to the user terminal, The image information corresponding to the process selected by the input from the user terminal is displayed.

[0012] With this configuration, image information can be easily and appropriately displayed according to the user's desire, allowing the user to carry out an examination or the like more quickly and accurately than ever before.

[0013] In a preferred embodiment of the present invention, when the input of the time information includes a plurality of steps corresponding to the input of the time information, the display processing section displays a plurality of pieces of video information corresponding to the steps.

[0014] With this configuration, multiple pieces of image information can be displayed simply and appropriately, allowing the user to carry out an examination or the like more quickly and accurately than ever before.

[0015] In a preferred embodiment of the present invention, the photographing position has time-dependent position information indicating that the photographing position changes over time, The display processing unit displays the video information within the space of the 3D model based on the temporal position information.

[0016] With this configuration, even in a situation where the shooting position changes over time, appropriate display processing can be performed according to the change. For example, when a person in charge at a site takes pictures while patrolling the site, the image information can be displayed at an appropriate position (coordinate position) according to the passage of time within the space of the 3D model based on the position information.

[0017] In a preferred embodiment of the present invention, the display processing unit displays the plurality of pieces of image information corresponding to the plurality of steps within the space of the 3D model based on chronological position information of each piece of image information.

[0018] With this configuration, multiple pieces of video information can be appropriately displayed in the 3D model space according to the passage of time.

[0019] In a preferred embodiment of the present invention, the display processing section processes and displays each piece of image information corresponding to the plurality of steps in an identifiable manner.

[0020] With this configuration, the user can easily distinguish between the steps and can view a highly convenient display of image information.

[0021] In a preferred embodiment of the present invention, the process display system further includes an image position adjustment unit, The video information has a shooting direction indicating a direction in which the video was shot at the construction site, The image position adjustment unit determines an overlap between the image information and a shooting target model in the space of the 3D model based on the shooting direction, the shooting position, and the process information, When the subject model and the video information overlap, the display position of the video information is adjusted.

[0022] With this configuration, the subject model in the 3D model space and the video information can be displayed side by side on the same screen, regardless of the direction from which the subject was photographed. For example, even if the subject is photographed from multiple locations at the same time, the positional relationship between the subject model in the 3D model space and the video information can be automatically adjusted.

[0023] In a preferred embodiment of the present invention, the adjustment process changes the display position of the video information to a position that does not overlap with the subject model in a viewing direction from a user using the user terminal.

[0024] By configuring in this way, it is possible to realize an optimal display that is easy for the user to view. For example, if the 3D model displayed on the user's terminal can be changed to various angles, the subject model and video information can be displayed appropriately according to the changed angle. Effect of the Invention

[0025] According to the present invention, a new technique for a process display system can be provided by corresponding information relating to time and executing a predetermined process. [Brief description of the drawings]

[0026] [Figure 1] 1 shows a block diagram of a system configuration according to an embodiment of the present invention. [Diagram 2] 1A and 1B are schematic diagrams illustrating an example of a hardware configuration of an information processing device and a terminal according to an embodiment of the present invention. [Diagram 3] 2 shows a flowchart of a processing procedure in a process display system according to an embodiment of the present invention. [Figure 4] 3 shows an example of each piece of information used in the process display system according to one embodiment of the present invention. [Diagram 5] 11 shows an example of a display processing result in a user terminal according to an embodiment of the present invention. [Figure 6]11 shows an example of a display processing result in a user terminal according to an embodiment of the present invention. [Figure 7] 11 shows an example of a display processing result in a user terminal according to an embodiment of the present invention. [Figure 8] 11 shows an example of a display processing result in a user terminal according to an embodiment of the present invention. [Figure 9] 11 shows an example of a display processing result in a user terminal according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] The present invention will now be described more fully with reference to the accompanying drawings, in which preferred embodiments are shown, but which may be embodied in many different forms and are not limited to the embodiments set forth herein.

[0028] For example, although the configuration, operation, etc. of the process display system will be described in this embodiment, the same effects can be achieved by an executed method (steps), device, computer program, etc. The program in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server.

[0029] In addition, in this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by circuits in a broad sense and software information processing that can be specifically realized by these hardware resources.

[0030] In this embodiment, "information" is represented by, for example, the physical value of a signal representing a voltage or current, the high or low value of a signal as a collection of binary bits consisting of 0 or 1, or a quantum superposition (so-called quantum bit), and communication and calculations can be performed on a circuit in a broad sense.

[0031] A circuit in the broad sense is a circuit realized by appropriately combining a circuit, circuitry, a processor, memory, etc. In other words, it includes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), etc.

[0032] <System configuration> Fig. 1 is a block diagram showing a system configuration according to an embodiment of the present invention. As shown in Fig. 1, the process display system 1 includes an information processing device 10 and a database DB. The process display system 1 is configured to be able to communicate with a plurality of user terminals 2 (in Fig. 1, reference numerals 2(a) to 2(c)) and image capture devices 3 (reference numerals 3(a) to 3(c)) via a network NW.

[0033] The information processing device 10 operates as a server, the user terminal 2 is a terminal used by users such as general contractors and site supervisors at the construction site, and the imaging device 3 is a terminal used to capture videos and footage (and images, etc.) at the construction site.

[0034] In this embodiment, the network NW is an IP (Internet Protocol) network, but there is no restriction on the type of communication protocol, and further, there is no restriction on the type and scale of the network.

[0035] It should be noted that a general-purpose server computer, a personal computer, or the like can be used as the information processing device 10. It is also possible to configure the process display system 1 by implementing the functional components described below in a plurality of computers.

[0036] The user terminal 2 is a terminal used by a prime contractor or the like at a construction site. A smartphone, a tablet terminal, a personal computer, a wearable device, or the like can be used as the user terminal 2. The user terminal 2 stores a process display application program for a user, and this application program is configured to have a function for displaying the work history (e.g., 3D models and images) of each process at the construction site.

[0037] The user terminal 2 may be configured not to have the process display application program. In this case, the user terminal 2 can display and transmit each piece of information using a web browser or the like.

[0038] The photographing device 3 is a camera device or the like for photographing the work history at the construction site as images or videos. A known camera device can be used as the photographing device 3. The photographing device 3 is configured to be capable of transmitting the photographing position, the record start date and time, the record end date and time (photographing time), video information, and the like to the process display system 1. In addition, a smartphone, a tablet terminal, a wearable device, and the like can also be used as the photographing device 3. The photographing device 3 may be any device that can photograph images such as the history of each process at the construction site and transmit the video information, photographing position report, and the like to the process display system 1. There are multiple photographing devices 3 at the construction site, and each is arranged at a predetermined position.

[0039] <Hardware configuration> 2(a) is a diagram showing an example of a hardware configuration of the information processing device 10. The information processing device 10 includes a control unit 11, a storage unit 12, and a communication unit 13 as the hardware configuration.

[0040] The control unit 11 includes one or more processors such as a CPU, and controls the overall operation and processing of the information processing device 10 by executing the process display program according to the present invention, an OS, and other applications.

[0041] The storage unit 12 is an HDD, SSD, ROM, RAM, etc., and stores the process display program according to the present invention and data used when the control unit 11 executes processing based on the program. The control unit 11 executes processing based on the process display program stored in the storage unit 12, thereby realizing a functional configuration described below.

[0042] The communication unit 13 executes communication control with the network NW, and performs input required to operate the information processing device 10 and output related to the operation results.

[0043] 2(b) is a diagram showing an example of a hardware configuration of a terminal 90 (user terminal 2 in FIG. 1). The terminal 90 includes, as its hardware configuration, a control unit 91, a storage unit 92, a communication unit 93, an input unit 94, and an output unit 95.

[0044] The control unit 91 of the terminal 90 includes one or more processors such as a CPU, and controls the overall operation and processing of the terminal 90. The storage unit 92 of the terminal 90 is an HDD, SSD, ROM, RAM, or the like, and stores the above-mentioned performance management application program, as well as data and the like used when the control unit 91 executes processing based on the program.

[0045] A communication unit 93 of the terminal 90 controls communication with the network. An input unit 94 of the terminal 90 is a touch panel, a mouse, a keyboard, or the like, and inputs operation requests from a user to the control unit 91. An output unit 95 of the terminal 90 is a display, or the like, and displays the results of processing by the control unit 91, etc.

[0046] <Functional configuration> 2(a), the information processing device 10 includes, as its functional configuration, a reception unit 101, an acquisition unit 102, a display processing unit 103, and an image position adjustment unit 104. These are information processing performed by software (stored in a storage unit 12) that is specifically realized by hardware (such as a control unit 11).

[0047] The reception unit 101 receives and registers various information from the user terminal 2, an administrator, etc. The reception unit 101 receives user information, etc. from the user terminal 2, etc., and stores it in the storage unit 12. The reception unit 101 also receives 3D model information, on-site information, etc. from the user terminal 2 or an administrator terminal (not shown), and stores it in the storage unit 12. The reception unit 101 can also receive input of time information corresponding to the process history (for example, input by operating a time axis bar, which will be described later), from the user terminal 2, the administrator terminal, etc.

[0048] The acquiring unit 102 acquires various information. In this embodiment, the acquiring unit 102 acquires video information captured at a predetermined shooting position and shooting time. For example, the acquiring unit 102 can acquire information on the shooting position where the video information was captured and the video time from a server or the like other than the imaging device 3. In this embodiment, the shooting time is collectively expressed as a predetermined date and time (a predetermined period, the time captured, for example, 2 hours, 12 hours, etc.) from the recording start date and time to the recording end date and time in the video information, as well as the shooting start date and time, the shooting end date and time, etc.

[0049] The display processing unit 103 performs display processing of various information. In this embodiment, the display processing unit 103 performs display processing of video information related to a process corresponding to the time information in association with a shooting position within the space of the 3D model based on input of time information (operation input such as a time axis bar) corresponding to the history of the process, video information, and process information received from a user terminal, and transmits the result of the display processing.

[0050] Furthermore, when there is a history of multiple steps corresponding to the input of time information, the display processing unit 103 displays the multiple steps, transmits the display processing result to the user terminal 2, and displays video information corresponding to the step selected by the input from the user terminal 2. When there is a history of multiple steps corresponding to the input of time information, the display processing unit 103 can display multiple pieces of video information corresponding to the multiple steps. The display processing unit 103 can also display video information within the space of the 3D model based on temporal position information in which the shooting position changes over time.

[0051] Furthermore, the display processing unit 103 displays and processes the multiple pieces of video information within the space of the 3D model based on the temporal positional information of each piece of video information corresponding to the multiple processes, and in addition, the display processing unit 103 can display and process each piece of video information corresponding to the multiple processes in an identifiable manner.

[0052] The image position adjustment unit 104 adjusts the display position of the image information based on a predetermined condition. In this embodiment, the image position adjustment unit 104 determines whether the image information overlaps with the image-captured model (e.g., a standalone model such as an elevated bridge) in the space of the 3D model based on the image capture direction, image capture position, and process information indicating the direction in which the image was captured at the construction site, and adjusts the display position of the image information when the image capture target model and the image information overlap.

[0053] In addition, the image position adjustment unit 104 changes the display position of the video information to a position that does not overlap with the model being photographed in the viewing direction from the user using the user terminal 2 (if the model being photographed and the video information overlap when viewed from the direction viewed by the user, the display position of the video information is adjusted).

[0054] <Database> 1 stores user information, 3D model information, site information, process information, video information, and other information necessary for process display, etc. Some or all of these may be stored in the storage unit 12, etc., or some of these may be stored in another database, etc.

[0055] Hereinafter, the process display system 1 and the processing contents of each functional component will be described with reference to FIGS.

[0056] <Outline of the process display system> The process display system 1 according to this embodiment is a system for displaying images (history) of the implementation history of each process carried out at a construction site, etc., in the space of a 3D model corresponding to the construction site (process) based on predetermined conditions. Specifically, the image can be displayed in the 3D model space at the same position (coordinates) as the shooting position where the image was taken at the construction site (or the image can be displayed so that the shooting position can be grasped in the 3D model space). By displaying in this way, the user can appropriately carry out inspection and history management for each process.

[0057] Furthermore, in the case where the photographing position changes over time, the process display system 1 can display an image in the space of the 3D model while changing the photographing position over time. The process display system 1 will now be described in detail.

[0058] <Registering various information> 3 shows a flowchart of a processing procedure in a process display system according to an embodiment of the present invention. In S201, users such as a prime contractor and a site supervisor (and workers) at a construction site register various information.

[0059] Specifically, the user can register information such as user information, 3D model information corresponding to the construction site or process, site information of the construction site, process information related to the process at the construction site, etc. via the user terminal 2, etc. In this embodiment, the term "user" refers to not only the prime contractor (the prime contractor company and its personnel), but also site supervisors, workers, and other people who manage the process and work at the construction site.

[0060] A user registers user information such as information about the company in order to use the process display system 1. As shown in Fig. 4(a), the user information includes information such as the company name, such as XXX Corporation, contact information, company address, types of work that can be done, and registration date, and is managed by user ID. The user information can also include the name of the person actually in charge of the construction site.

[0061] In this embodiment, users (prime contractors and their staff) whose user information is registered can use services related to the process display system 1, but registration of user information is not necessarily required, and the system may be configured so that users who have not registered user information can also use the services.

[0062] As shown in Fig. 4(b), the 3D model information includes information such as the model storage destination and is managed by a model ID. The 3D model information includes multiple various models, and the model here refers to a 3D model for constructing a 3D space, and for example, each structure or part, such as an viaduct or building, is managed by a model ID (or a structure ID or part ID, not shown), and the 3D space of the construction site can be constructed as a 3D model space by combining each part or structure (according to the progress status). Also, as shown in Fig. 4(c), the site information includes information such as the site name and address of the construction site, and is managed by a site ID.

[0063] As shown in FIG. 4(d), the process information includes information such as the name of the process such as preparatory work and crane installation, the planned start date and time and the planned end date and time of each process, a model ID, a site ID, and the current progress rate of each process, and is managed by a process ID. 3D model information is associated with the process information, and the process information is used to display a 3D model corresponding to the date and time of each process. That is, which 3D model is displayed at which position at which date and time is associated in advance (the 3D model and the process correspond on the time axis). In this specification, a predetermined date and time from the planned start date and time to the planned end date and time of a process may be expressed as working time. In this embodiment, a 3D model at a specific date and time can also be constructed using the progress rate and the like included in the process information.

[0064] <Acquisition of video information> In S202, the acquisition unit 102 acquires video information captured by the camera 3. The video information includes information about the shooting position. A plurality of camera devices 3 are placed at predetermined positions at the construction site, and are used to capture the work content (work history, process history) in each process. For example, when work related to a bridge girder occurs in a process at the construction site, the camera device 3 captures the bridge girder.

[0065] As shown in Figure 4(e), the video information is managed by a video ID, including information such as the name of the imaging device 3, such as a fixed camera A, the video storage location, the recording start date and time, the recording end date and time (shooting time), the site ID, the shooting position, and the shooting direction.

[0066] In addition, at the construction site, a plurality of image capturing devices 3 capture images of the bridge girder from different directions, and the acquisition unit 102 can acquire the plurality of pieces of image information. The image information is stored in the storage unit 12 in association with the above-mentioned process information and 3D model information.

[0067] <Display processing of 3D models and schedules> In S203, the 3D model and the construction site schedule are displayed. The display processing unit 103 can display the schedule based on the schedule information. The display processing unit 103 displays the 3D model and the schedule side by side based on the 3D model information and the schedule information, and transmits the result of the display processing to the user terminal 2.

[0068] For example, as shown in FIG. 5, the display processing unit 103 displays a 3D model W10 and a schedule W20 at a specific construction site side by side. In this embodiment, the 3D model W10 is displayed on the upper side and the schedule W20 is displayed on the lower side, but the 3D model W10 and the schedule W20 may be displayed upside down or side by side. The 3D model W10 and the schedule W20 may be displayed in any positional relationship, not limited to upside down and left to right, as long as they are displayed side by side so that the user can view them. In addition, the schedule W20 shown in FIG. 5 displays only the date, but for example, it can be displayed as a date and time display, or a date and time can be selected by selecting a specific position of the date display (selection by a time axis bar described later).

[0069] For example, in this embodiment, multiple 3D models W10 with different scales and display ranges of the construction site and the corresponding schedule W20 can be displayed side by side. By displaying multiple 3D models W10 with different scales and display ranges and the corresponding schedule W20 side by side, the user can grasp the overall image of the construction site while also grasping the details of the structure that he or she wants to check.

[0070] The process schedule W20 in Fig. 5 displays a time axis bar W40 for selecting a specific date and time (day or time) for process W30 (a process showing the period from the planned start date and time to the planned end date and time in Fig. 4(d)). The user can select the process W30 of a specific date and time by moving this time axis bar W40 left or right. The user can move the time axis bar W40 by performing a touch operation (operation input) on the screen of the user terminal 2.

[0071] At an actual construction site, multiple tasks may be carried out simultaneously during a given period (a given date and time), and multiple processes are displayed overlapping each other during the given period on the schedule W20. Since the construction period for each process W30 differs on the schedule W20, each process W30 is displayed with a different horizontal length. Note that in this specification, the date (date and time) on which the time axis bar W40 is located may also be referred to as time information.

[0072] <Display processing of video information> In S204, the reception unit 101 receives time information corresponding to the process chart W20. Specifically, by moving the time axis bar W40 in FIG. 5 left and right (by inputting an operation from the user terminal 2), a date or date and time of the process W30 that the time axis bar W40 touches (or overlaps on the screen) is selected. At this time, the history of the process W30 at the selected time information (the history of the process based on the time information) is displayed as the 3D model W10. As described above, multiple tasks may be carried out simultaneously, in which case there may be multiple processes that the time axis bar W40 touches.

[0073] In this embodiment, a process corresponding to a specific date and time is selected by operating the time axis bar W40. However, the selection of a process corresponding to a specific date and time is not necessarily limited to operating the time axis bar W40. For example, the specific date and time may be entered as text, or may be displayed on a calendar or time display. In this case, the display processing unit 103 can display the calendar display or the time display based on general time information and process information held by the information processing device 10, and transmit the result of the display processing to the user terminal 2.

[0074] Then, in S205, the video information of the process W30 at the selected date and time is displayed within the space of the 3D model W10. The reception unit 101 receives predetermined time information based on the movement of the time axis bar W40 from the user terminal 2. The display processing unit 103 displays the video information related to the process W30 corresponding to the time information in the space of the 3D model W10 in association with the shooting position based on the input of the time information corresponding to the history of the process W30 received from the user terminal 2, the shooting position of the video information, and the process information stored in the storage unit 12, and transmits the result of the display processing.

[0075] 6(a), based on the time information (and the 3D model information and the video information associated with the 3D model) at which the time axis bar W40a is located in the process chart W20, video information W50a relating to the process W30a corresponding to the time information is displayed within the space of the 3D model W10. In this embodiment, this video information W50a is displayed in correspondence with the space of the 3D model W10 at the shooting position where the shooting device 3 shot the actual construction site.

[0076] For example, the shooting position of the process information in Fig. 4 can be converted to the coordinate position of the 3D model information (global coordinate system), and the video information W50 can be displayed at the coordinate position in the 3D model space that corresponds to the converted shooting position. In Fig. 6(a), the shooting position is displayed as a circle, and the video information W50a is displayed above it.

[0077] 6(b), when the position of the time axis bar W40a is moved leftward in the process chart W20, that is, when the time axis bar W40a is moved from the position in FIG. 6(a) toward the past date to select a specific date and time (time axis bar W40b in FIG. 6(b)), the time axis bar W40b comes into contact with another process W30b, and video information W50b based on that process W30b is displayed. At this time, in FIG. 6(b), the video information W50b of the specific date and time selected in the newly selected process W30b is displayed within the space of the 3D model W10 corresponding to the shooting position of the shooting device 3 used to shoot that process W30b (the 3D model of the selected date and time is also displayed for the 3D model W10).

[0078] In this way, in this embodiment, by utilizing the shooting position and shooting time, it is possible to express images shot at the same position and time as the actual construction site in the 3D model space. For example, assuming a scene of viaduct construction, the user can confirm the image information together with the viaduct model displayed in the 3D model space while knowing from which direction the viaduct was shot in the 3D model, so that more accurate inspection and appropriate history management can be performed than before.

[0079] 7, when the time axis bar W40 is in contact with two processes W30c and W30d, the image information W50c and image information W50d for the processes W30c and W30d can be simultaneously displayed in the space of the 3D model W10. When there are multiple processes corresponding to the input of time information, the display processing unit 103 can display multiple pieces of image information corresponding to the multiple processes. The display processing unit 103 can also display each piece of image information corresponding to the multiple processes in a distinguishable manner. For example, the color or shape of the frame portion of each piece of image information can be set to be distinguishable.

[0080] The user can appropriately grasp the progress of the two processes while simultaneously viewing the two images. For example, when related work occurs in two processes, the progress of processes W30c and W30d can be quickly grasped (W50c and W50d can be viewed simultaneously), and the subsequent process plan can be appropriately changed. Although FIG. 7 describes a case where two processes occur, there is no limit to the number of processes, and two or more processes can be displayed simultaneously in the space of the 3D model W10.

[0081] For example, when the time axis bar W40 is in contact with multiple steps W30, the display processing unit 103 processes the display of the multiple steps and transmits the display processing results to the user terminal. Then, the display processing unit 103 may process and display video information corresponding to a step selected by input from the user terminal. In this case, the video related to the step desired by the user can be appropriately displayed.

[0082] The above-mentioned photographing device 3 is like a fixed camera, and is fixed at a predetermined photographing position to photograph each process, but in reality, a person in charge of the main contractor may patrol the construction site and photograph each process (for example, wearable B in Figure 6(e)).

[0083] In such a case, in this embodiment, as shown in Fig. 8, if a person in charge takes pictures related to the process W30e while patrolling the construction site at the date and time when the process W30e and the time axis bar W40 are in contact with each other, the video information W50e can be displayed in the space of the 3D model W10 in correspondence with the shooting position that changes over time. Specifically, the video information in this embodiment has time-series position information in which the shooting position changes over time. Then, the display processing unit 103 displays the video information in the space of the 3D model based on the time-series position information.

[0084] Furthermore, the display processing unit 103 can also display multiple pieces of image information in the space of the 3D model based on the positional information over time of each piece of image information corresponding to multiple processes, as shown in Fig. 9. This type of display processing allows a user checking the process history to properly understand how the shooting position has changed, and by simultaneously checking multiple changes in position over time and each piece of image information, it can be used to consider appropriate measures when some kind of trouble or irregularity occurs in multiple processes that were being performed simultaneously.

[0085] <Adjusting the position of image information in 3D model space> As described above, in this embodiment, the 3D model information, process information, and image information each share information regarding time and position, and this time and position information can be used to display the image information at a specified position within the space of the 3D model (the position where the imaging device 3 is actually installed, or a corresponding position).

[0086] For example, in FIGS. 6 to 9, the shooting position is displayed as a circle and the image information is displayed above it, but there may be cases where the display position of this image information hides the target object in the 3D model space.

[0087] Therefore, in the process display system 1 according to this embodiment, the display position of the image information W50 in the space of the 3D model W10 can be appropriately adjusted. Specifically, the image position adjustment unit 104 determines whether a predetermined photographed subject model (e.g., a model of an viaduct) overlaps with the image information W50 in the space of the 3D model based on information (directional information) related to the photographing direction contained in the image information, the photographing position of the photographing device 3, and the 3D model information, and can adjust the display position of the image information if the predetermined photographed subject model overlaps with the image information W50. The photographing direction is angle information from a predetermined reference, detailed directional information showing east, west, south, etc. in more detail, etc.

[0088] For example, when the shooting target model and the video information W50 overlap, the video position adjustment unit 104 can adjust (change) the display position of the video information W50 from an upper position to a lower position or a left / right position with respect to the circle shown in Figs. 5 to 7. In this manner, in this embodiment, the shooting target of the 3D model and the video information can be displayed side by side on the same screen without impairing convenience by appropriately adjusting the display position of the video information by the video position adjustment unit 104. For example, when the 3D model displayed on the user terminal 2 can be changed to various angles according to the operation input from the user, the shooting target model and the video information can be appropriately displayed according to the changed angle. The adjustment process in the video position adjustment unit 104 can change the display position of the video information to a position that does not overlap with the shooting target model in the viewing direction from the user using the user terminal.

[0089] As described above, according to the process display system 1 of the present invention, by performing a predetermined display process by associating the position information and time information of each piece of information such as 3D model information, process information, and video information, it is possible to simply and appropriately display video information according to the user's desire within the space of the 3D model. As a result, it is possible to improve the convenience of inspection and process management at the construction site.

[0090] In addition, in this embodiment, a specified display process at a construction site (display of image information within a 3D model space) has been described, but the same effects as the present invention can be obtained when the process display system 1 is used in places other than a construction site. [Explanation of symbols]

[0091] 1. Process display system 2. User terminal 3. Imaging Equipment 4. Elevator 10. Information processing device 11 Control section 12 Storage section 13. Communications Department 90 Terminal (User Terminal 2) 91 Control Unit 92 Memory section 93 Communications Department 94 Input section 95 Output section 101 Reception 102 Acquisition Department 103 Display processing unit 104 Image position adjustment section NW Network W10 3D model W20 process chart W30 process W40 Time Axis Bar W50 Video Information

Claims

1. A process display system that displays a history of a process at a construction site, The process display system includes a storage unit, an acquisition unit, and a display processing unit, The storage unit stores 3D model information and process information associated with the 3D model information, The acquisition unit acquires video information captured at a predetermined shooting position and shooting time, The display processing unit, based on the input of time information corresponding to the process, the video information, and the process information received from the user terminal, displays the video information related to the process corresponding to the time information in the space of the 3D model in association with the shooting position, and transmits the result of the display processing. Process display system.

2. the display processing unit, when there are a plurality of steps corresponding to the input of the time information in the input of the time information, performs a display process of the plurality of steps and transmits a result of the display process to the user terminal; displaying image information corresponding to the step selected by the input from the user terminal; The process display system according to claim 1 .

3. When the input of the time information includes a plurality of steps corresponding to the input of the time information, the display processing unit displays a plurality of pieces of video information corresponding to the plurality of steps. The process display system according to claim 1 .

4. The photographing position has time-varying position information indicating that the photographing position changes over time, The display processing unit displays the image information within the space of the 3D model based on the time-series position information. The process display system according to claim 1 .

5. The display processing unit displays the plurality of pieces of image information in the space of the 3D model based on chronological position information of each piece of image information corresponding to the plurality of steps. The process display system according to claim 4.

6. The display processing unit processes and displays each of the image information corresponding to the plurality of steps in an identifiable manner. The process display system according to claim 3 or 5.

7. The process display system further includes an image position adjustment unit, The video information has a shooting direction indicating a direction in which the video was shot at the construction site, The image position adjustment unit determines an overlap between a shooting target model and the image information in the space of the 3D model based on the shooting direction, the shooting position, and the process information, If the subject model and the video information overlap, a display position of the video information is adjusted. The process display system according to claim 1 .

8. The adjustment process changes a display position of the video information to a position that does not overlap with the subject model in a viewing direction from a user using the user terminal. The process display system according to claim 7.

9. A process display method executed by a process display system that displays a history of processes at a construction site, The process display system includes a storage unit, an acquisition unit, and a display processing unit, The storage unit stores 3D model information and process information associated with the 3D model information; The acquisition unit acquires video information captured at a predetermined shooting position and shooting time, The display processing unit displays the image information related to the process corresponding to the time information in association with the shooting position within the space of the 3D model based on the input of time information corresponding to the process, the image information, and the process information received from the user terminal, and transmits a result of the display processing. Process display method.

10. A process display program for displaying a history of a process at a construction site, A computer is configured to function as a storage unit, an acquisition unit, and a display processing unit; The storage unit stores 3D model information and process information associated with the 3D model information, The acquisition unit acquires video information captured at a predetermined shooting position and shooting time, The display processing unit, based on the input of time information corresponding to the process, the video information, and the process information received from the user terminal, displays the video information related to the process corresponding to the time information in the space of the 3D model in association with the shooting position, and transmits the result of the display processing. Process display program.

Images