Terminal device, control device, device for integrating data, work vehicle, image recording system, image recording method

Abstract

Terminal device (200) comprising: a capture data receiving unit (202) that receives first capture data at recurring time intervals captured by a single image capture device (125, 1251) included in a work vehicle (100) and capable of generating 3D data; a display unit (RP) that displays the first recording data received by the recording data receiving unit (202) at recurring time intervals; and a unit (206) for transmitting a capture command which, in response to a check of the first capture data displayed on the display unit (RP), transmits a capture command which causes the work vehicle (100) to capture second capture data which are recorded with a plurality of image recording devices (125, 1251, 1252, 1253, 1254) included in the work vehicle (100) and serve to generate 3D data, wherein the check comprises determining whether a time width between a most recent recording time of the first recording data and a closest recording time of the second recording data is shorter than a predetermined time.

Description

Technical area

[0001] The present invention relates to a terminal device, a control device, a data integration device, a work vehicle, an image pickup system and an image pickup method Technical background

[0002] As disclosed in JP 2013 - 36 243 A, a technology for acquiring image data of a construction site using a stereo camera included in a construction machine is known.

[0003] JP 2010 - 60 344 A discloses a spatial information display device designed to assist operators of work machines or robots by capturing and displaying three-dimensional (3D) information. The system uses stereo cameras mounted on the work vehicle to capture image data of the work area. These images are processed to generate spatial information, which is then displayed on a screen for the operator. Summary of the inventionTechnical problem

[0004] As an image pickup device capable of generating 3D data, a stereo camera is known. A technology for generating 3D data using image data acquired with a stereo camera is known. By generating 3D data of the topography of a construction site using image data acquired by imaging the topography of the construction site using a stereo camera installed on a work machine, process management at the construction site can be performed. However, a line of sight of a construction machine operator and a line of sight of an image pickup device do not necessarily coincide. Furthermore, there is a concern that a shot range captured by a photographer and a shot range of an image pickup device may be different.For example, the photographer may be an operator of a construction machine with a stereo camera installed. Therefore, it is possible that image data captured according to the operator's instructions may not be captured by imaging a desired location.

[0005] An object of the present invention is to provide a terminal device, a control device, a data integrating device, a work vehicle, an image pickup system and an image pickup method with which 3D data relating to a desired location can be generated. Solution to the problem

[0006] According to a first aspect of the present invention, a terminal device comprises: a shot data receiving unit that receives first shot data at recurring time intervals, which are shot by a single image pickup device included in a work vehicle and are capable of generating 3D data; a display unit that displays the first shot data received by the shot data receiving unit at recurring time intervals; and an acquisition command transmission unit that, in response to a check of the first shot data displayed on the display unit, transmits an acquisition command for causing the work vehicle to acquire second shot data, which are shot by a plurality of image pickup devices included in the work vehicle and are used to generate 3D data.The test includes determining whether a time width between a most recent recording time of the first recording data and a closest recording time of the second recording data is shorter than a predetermined time.

[0007] According to a second aspect of the present invention, an image pickup system comprises: a work vehicle; and a terminal device. The work vehicle includes: a vehicle body; an image pickup device comprising a plurality of image pickup devices, including a single image pickup device included in the vehicle body and capable of acquiring information about a depth direction; a pickup data transmission unit that transmits first pickup data acquired by the single image pickup device to the terminal device at recurring time intervals; an acquisition command reception unit that receives a command for acquiring second pickup data used to generate 3D data;and a shot data acquisition unit that acquires the second shot data from the plurality of image pickup devices according to the acquisition command, wherein the terminal device includes: a shot data receiving unit that receives the first shot data acquired by the single image pickup device from the work vehicle at recurring time intervals; a display unit that displays the first shot data received by the shot data receiving unit at recurring time intervals;and an acquisition command transmission unit that, in response to a check of the first image data displayed on the display unit, sends an acquisition command for causing the work vehicle to acquire the second image data captured by the plurality of image pickup devices included in the work vehicle. The check includes determining whether a time width between a most recent capture time of the first image data and a most recent capture time of the second image data is shorter than a predetermined time.

[0008] According to a third aspect of the present invention, an image capturing method comprises: receiving, at recurring time intervals, first image data captured by a single image capturing device included in a work vehicle and capable of capturing information about a depth direction; displaying the received first image data at recurring time intervals on a terminal device; and sending, in response to a check of the first image data displayed on the terminal device, an acquisition command for causing the work vehicle to acquire second image data captured by a plurality of image capturing devices included in the work vehicle and used to generate 3D data.The test includes determining whether a time width between a most recent recording time of the first recording data and a closest recording time of the second recording data is shorter than a predetermined time.

[0009] According to a fourth aspect of the present invention, a control device comprises: a shot data transmission unit that transmits, at recurring time intervals, first shot data captured by a single image pickup device included in a work vehicle and capable of acquiring information about a depth direction to a terminal device; an acquisition command reception unit that receives a command to acquire second shot data for generating 3D data; and a shot data acquisition unit that, in response to a check of the first shot data displayed on the terminal device, acquires second shot data from a plurality of image pickup devices according to the acquisition command, the second shot data being used to generate 3D data.The test includes determining whether a time width between a most recent recording time of the first recording data and a closest recording time of the second recording data is shorter than a predetermined time.

[0010] According to a fifth aspect of the present invention, a data integrating device comprises: a 3D data acquiring unit that acquires a plurality of pieces of 3D data generated based on image data acquired by performing image capturing of a construction site using an image capturing device included in a work vehicle, each piece of the plurality of pieces of 3D data being associated with an indication of a capture time at which the image of the respective piece of 3D data was captured; a command receiving unit that receives an integration command that selects a portion of the plurality of pieces of 3D data for integration;and a 3D data integrating unit that determines whether a time width between a most recent acquisition time and a closest acquisition time of the selected part of the plurality of pieces of 3D data is shorter than a predetermined time, and generates integrated 3D data when the time width is shorter than the predetermined time by integrating the selected part of the plurality of pieces of 3D data acquired by the 3D data acquiring unit.;

[0011] According to a sixth aspect of the present invention, a work vehicle comprises: a vehicle body; an image pickup device included in the vehicle body; a 3D data generation unit that generates a plurality of pieces of 3D data based on a plurality of pieces of image data acquired with the image pickup device, each piece of the plurality of pieces of 3D data being associated with an indication of an acquisition time at which the image of the respective piece of 3D data was acquired;and a 3D data integrating unit that determines whether a time width between a most recent acquisition time point and a closest acquisition time point of the plurality of pieces of 3D data is shorter than a predetermined time, and generates integrated 3D data when the time width is shorter than the predetermined time by integrating a plurality of pieces of 3D data generated by the 3D data generating unit.;

[0012] According to a seventh aspect of the present invention, an image acquisition system comprises: a work vehicle; and a data integrating device. The work vehicle includes: a vehicle body; an image acquisition device included in the vehicle body; a 3D data generation unit that generates a plurality of pieces of 3D data based on a plurality of pieces of image data acquired with the image acquisition device, wherein each piece of the plurality of pieces of 3D data is associated with an indication of an acquisition time at which the image of the respective piece of 3D data was acquired; and a data transmission unit that transmits the plurality of pieces of 3D data to the data integrating device, wherein the data integrating device includes: a data reception unit that receives the plurality of pieces of 3D data from the work vehicle;and a 3D data integrating unit that determines whether a time width between a most recent acquisition time and a closest acquisition time of the plurality of pieces of 3D data is shorter than a predetermined time, and generates integrated 3D data when the time width is shorter than the predetermined time by integrating the plurality of pieces of 3D data received by the data receiving unit; Advantageous effects of the invention

[0013] According to at least one aspect of the aspects described above, a terminal device, a control device, a data integrating device, a work vehicle, an image capturing system and an image capturing method are provided with which 3D data relating to a desired location can be generated. Short description of the drawings Fig. 1 is a schematic diagram showing the configuration of a construction monitoring system according to a first embodiment. Fig. 2 is a perspective view illustrating the structure of a construction machine according to the first embodiment. Fig. 3 is a block diagram illustrating the configuration of a control device of a construction machine according to the first embodiment. Fig. 4 is a block diagram illustrating the configuration of a terminal device according to the first embodiment. Fig. 5 is a block diagram illustrating the configuration of a server device according to the first embodiment. Fig. 6 is a flowchart illustrating the operation of the terminal device according to the first embodiment. Fig. 7 is a flowchart illustrating an example of a process selection screen displayed on the terminal device according to the first embodiment. Fig. 8 is a first flowchart illustrating a process for controlling the generation of 3D data according to the first embodiment. Fig. 9 is a schematic diagram illustrating an example of a screen for checking a recording area displayed on the terminal device according to the first embodiment. Fig. 10 is a second flowchart illustrating a process for controlling the generation of 3D data according to the first embodiment. Fig. 11 is a flowchart illustrating a process for managing 3D data according to the first embodiment. Fig. Figure 12 is a schematic diagram illustrating a process for generating integrated 3D data from 3D data. Fig. 13 is a flowchart illustrating a process for integrating according to the first embodiment. Fig. 14 is a schematic diagram showing the configuration of a construction monitoring system according to a second embodiment. Fig. 15 is a schematic block diagram illustrating the configuration of a terminal device according to the second embodiment. Fig. 16 is a flowchart illustrating the operation of the terminal device according to the second embodiment. Fig. 17 is a first flowchart illustrating a process for controlling the generation of 3D data according to a third embodiment. Fig. 18 is a second flowchart illustrating a process for controlling the generation of 3D data according to the third embodiment. Fig. 19 is a schematic block diagram showing the configuration of a control device of a construction machine according to a fifth embodiment. Fig. 20 is a first flowchart illustrating a process for controlling the generation of 3D data according to the fifth embodiment. Fig. 21 is a second flowchart illustrating a process for controlling the generation of 3D data according to the fifth embodiment. Fig. 22 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment. Description of embodimentsFirst embodimentConstruction monitoring system

[0014] Fig. 1 is a schematic diagram showing the configuration of a construction monitoring system according to a first embodiment.

[0015] The construction monitoring system 1 includes a plurality of construction machines 100, a plurality of terminal devices 200, and a server device 300. Although Fig. 1 illustrates the construction monitoring system 1 including the plurality of construction machines 100 and the plurality of terminal devices 200. The construction monitoring system may include a construction machine 100 and a plurality of terminal devices 200, or a plurality of terminal devices 200 and a terminal device 200. Furthermore, the construction monitoring system 1 may include a construction machine 100 and a terminal device 200. The construction monitoring system 1 is a system that monitors the progress of construction work using the construction machine 100 or human labor. That is, the construction monitoring system 1 generates 3D data of a construction site from image data captured by a stereo camera included in the construction machine 100. The construction monitoring system 1 is an example of an image acquisition system. A driver D of the construction machine 100 is a photographer who can operate the terminal device 200.

[0016] A control device 126, described below, is included in the construction machine 100, receives a command from the terminal device 200, and performs image acquisition using a stereo camera according to the command. The control device 126 of the construction machine 100 generates 3D data from image data acquired with the stereo camera. Examples of the 3D data include point group data, polygon data, and voxel data. The construction machine 100 according to the first embodiment is a hydraulic excavator as a work machine.

[0017] A mobile phone, a smartphone, or a portable computer, for example, can be used as the terminal device 200. The terminal device 200 displays image data (first image data) acquired using a first camera 1251 described below from stereo cameras 125 included in the construction machine 100. The terminal device 200 sends an acquisition command for acquiring image data (second image data) used to generate 3D data to the construction machine 100 in accordance with an operator's input. The second image data is image data acquired with all cameras (a first camera 1251, a second camera 1252, a third camera 1253, and a fourth camera 1254) of the stereo cameras 125 described below.The second acquisition data may either include the first acquisition data acquired in the past or not include the first acquisition data acquired in the past. That is, when the second acquisition data includes the first acquisition data acquired in the past, second acquisition data, which is the second acquisition data acquired at the same time and includes the first acquisition data, may be stored in a storage unit (not shown in the drawing) of the control device 126 described below, and 3D data may be generated using this acquisition data.On the other hand, when second shot data does not include the first shot data acquired in the past, as described below, 3D data can be generated using second shot data including other first shot data than first shot data used by a photographer to check a shot range and a shot content of the stereo camera 125. .

[0018] The server device 300 receives the recording data and the 3D data from the construction machine 100. The server device 300 can generate integrated 3D data representing the entire construction site by integrating a plurality of elements of 3D data. The construction machines 100, the terminal devices 200, and the server devices 300 are connected to each other via a network N. The network N includes a mobile communication network or a satellite communication network. Construction machine

[0019] Fig. 2 is a perspective view illustrating the structure of a construction machine according to the first embodiment.

[0020] The construction machine 100 includes a working equipment 110 operated by hydraulic pressure, a vehicle body 120 as an upper rotating body supporting the working equipment 110, and a traveling device 130 serving as a lower traveling device supporting the vehicle body 120.

[0021] The vehicle body 120 includes a driver's cab 121 in which a driver D sits, a position detector 122 that detects the position of the vehicle body 120, an azimuth detector 123 that detects an azimuth of the orientation of the vehicle body 120, a posture detector 124 that detects a posture of the vehicle body 120, a stereo camera 125 that performs image capturing of the side in front of the vehicle body 120 (+Y direction), and a control device 126 that controls the construction machine 100. Furthermore, the azimuth detector 123 also serves as a part of the position detector that detects the position of the vehicle body 120. The driver's cab 121 is installed on the side in front of the vehicle body 120 (+Y direction) and on the left side of the work equipment 110 (-X side).

[0022] The position detector 122 includes a first receiver 1221 that receives positioning signals from satellites forming a global navigation satellite system (GNSS). The position detector 122 detects, for example, an installation position of the first receiver 1221 based on positioning signals received by the first receiver 1221 via a second receiver 1231 and a third receiver 1232, which are antennas. An example of the satellite navigation system includes a global positioning system (GPS).

[0023] The azimuth detector 123 includes a second receiver 1231 and a third receiver 1232, which receive positioning signals from the satellites constituting the satellite navigation system. The second receiver 1231 and the third receiver 1232 are installed at different positions on the vehicle body 120. The azimuth detector 123 detects the installation position of the second receiver 1231 based on the positioning signals received by the second receiver 1231. The azimuth detector 123 detects the installation position of the third receiver 1232 based on the positioning signals received by the third receiver 1232. The azimuth detector 123 detects an azimuth in which the vehicle body 120 is oriented based on the detected installation position of the second receiver 1231 and the detected installation position of the third receiver 1232.

[0024] The attitude detector 124 measures an acceleration and an angular velocity of the vehicle body 120 and detects the attitude (e.g., pitch, yaw, and roll) of the vehicle body 120 based on the measurement result. The attitude detector 124 is installed, for example, on a lower surface of the driver's cab 121. The attitude detector 124 may use, for example, an inertial measurement unit (IMU) as an inertial measurement device.

[0025] When a key switch (not shown in the drawing) is operated, a motor (not shown in the drawing) installed in the construction machine 100 starts operating, and devices such as the control device 126 including the stereo camera 125 and the like are supplied with power from a battery (not shown in the drawing) and start operating. The stereo cameras 125 are installed at the front (+Y direction) and the top (+Z direction) inside the operator's cab 121. That is, the viewpoint of the stereo camera 125 is higher than the viewpoint of the operator D. The stereo cameras 125 capture an image of the side in front of the operator's cab 121 through a windshield arranged on the side in front of the operator's cab 121. The stereo cameras 125 include two pairs of cameras. That is, the stereo cameras 125 include four cameras.Specifically, the stereo cameras 125 include, in order from the right side (+X side), a first camera 1251, a second camera 1252, a third camera 1253, and a fourth camera 1254. The first camera 1251 and the third camera 1253 are cameras that form a pair. The first camera 1251 and the third camera 1253 are installed in a vehicle width direction with a gap between them so that their optical axes are approximately parallel to the floor surface of the driver's cab 121. The second camera 1252 and the fourth camera 1254 are cameras that form a pair.The second camera 1252 and the fourth camera 1254 are installed in the vehicle width direction with a gap between them so that their optical axes are approximately parallel to each other, and the optical axes are inclined downward (-Z direction) from the side in front of the driver's cab 121 (+Y direction) with respect to the floor surface of the driver's cab 121. The stereo camera 125 is an example of an image pickup device capable of generating 3D data. That is, the control device 126 can generate 3D data from a pair of pieces of image data captured by at least one of the stereo cameras 125. Here, "an image pickup device capable of generating 3D data" does not have to generate 3D data itself, but may acquire image data used to generate 3D data. Control device of the construction machine

[0026] Fig. 3 is a block diagram showing the configuration of the control device of the construction machine according to the first embodiment.

[0027] The control device 126 includes a unit 1261 for transmitting beacon signals, a unit 1262 for receiving commands, a unit 1263 for acquiring image data, a unit 1264 for acquiring vehicle information, a unit 1265 for generating 3D data, and a unit 1266 for transmitting data.

[0028] The beacon signal transmission unit 1261 transmits a beacon signal containing a vehicle identifier as vehicle identification information used to identify the construction machine 100. A transmission range of the beacon signal is, for example, approximately 50 m. Furthermore, as a method for transmitting a vehicle identifier, a communication form capable of near-field communication may be used. For example, a signal using Bluetooth Low Energy (BLE; Bluetooth is a registered trademark) may be used instead of a beacon signal.

[0029] The command receiving unit 1262 receives a recording command from the server device 300. The recording command is sent from the terminal device 200 to the server device 300 and is transmitted through the server device 300 to the control device 126. Included in the recording command is a terminal identifier as terminal identification information, which serves to identify a terminal device 200 that is a transmission source. An example of the terminal identifier includes an IP address of the terminal device 200. The command receiving unit 1262 receives an acquisition command for acquiring shot data (second shot data) used to generate 3D data from the server device 300. The command receiving unit 1262 is an example of a shot command receiving unit and a capture command receiving unit.

[0030] When the command receiving unit 1262 receives a capture command, the capture data acquiring unit 1263 starts a process of acquiring capture data (first capture data) captured by the first camera 1251 from the stereo camera 125 at predetermined time intervals. For example, after receiving the capture command, the capture data acquiring unit 1263 may acquire first capture data every 5 seconds. When the command receiving unit 1262 receives an acquisition command, the capture data acquiring unit 1263 acquires capture data (second capture data) captured by each of the stereo cameras 125 (the first camera 1251, the second camera 1252, the third camera 1253, and the fourth camera 1254).

[0031] The vehicle information acquisition unit 1264 acquires vehicle information such as position information, azimuth information, and attitude information from the position detector 122, the azimuth detector 123, and the attitude detector 124.

[0032] The 3D data generation unit 1265 generates 3D data representing a region imaged by the stereo cameras 125 based on the image data acquired by the image data acquisition unit 1263 and the information acquired by the vehicle information acquisition unit 1264. That is, the 3D data generation unit 1265 generates 3D data in the order shown below. The 3D data generation unit 1265 determines a position and a line-of-sight direction of the stereo cameras 125 in a global coordinate system based on the position information, the azimuth information, and the attitude information acquired by the vehicle information acquisition unit 1264.The 3D data generation unit 1265 generates 3D data in a local coordinate system using the positions of the stereo cameras 125 as a reference point from image data acquired by the first camera 1251 and the third camera 1253 using triangulation. The 3D data generation unit 1265 generates 3D data in a local coordinate system using the positions of the stereo cameras 125 as a reference point from a pair of items of image data acquired by the second camera 1252 and the fourth camera 1254 using triangulation. The 3D data in the local coordinate system is 3D data representing an overlap area in a pair of items of image data.The 3D data generation unit 1265 generates 3D data representing areas imaged by the stereo cameras 125 by applying the 3D data in a local coordinate system to the positions and line-of-sight directions of the stereo cameras 125 in a global coordinate system.

[0033] The data transmission unit 1266 transmits the first shot data acquired by the shot data acquisition unit 1263 to the server device 300. The first shot data is transmitted by the server device 300 to the terminal device 200. Furthermore, the data transmission unit 1266 transmits the 3D data generated by the 3D data generation unit 1265 and the second shot data to the server device 300. The 3D data and the second shot data are stored in the server device 300. The data transmission unit 1266 is an example of the shot data transmission unit. terminal device

[0034] Fig. 4 is a block diagram illustrating the configuration of the terminal device according to the first embodiment.

[0035] The terminal device 200 includes a beacon signal receiving unit 201, a data receiving unit 202, a display controlling unit 203, a command receiving unit 204, a target vehicle storing unit 205, a command sending unit 206, and a display unit RP.

[0036] The beacon signal receiving unit 201 receives a beacon signal transmitted from the construction machine 100. The beacon signal receiving unit 201 identifies a construction machine 100 based on a vehicle identifier contained in the beacon signal.

[0037] The data receiving unit 202 receives image data captured by stereo cameras 125 of the construction machine 100 or 3D data representing a construction site from the server device 300. The data receiving unit 202 is an example of a image data receiving unit and a 3D data receiving unit.

[0038] The display control unit 203 displays the received data on a display, which is a display unit RP.

[0039] Examples of the display of the terminal device 200 include a touchscreen, a liquid crystal display, and an OLED display. That is, the display control unit 203 displays a list of vehicles represented by the vehicle ID received with the beacon signal receiving unit 201, image data captured by the stereo camera 125, or 3D data representing a construction site on the display.

[0040] The input receiving unit 204 receives an input operation corresponding to a display content from the display control unit 203 from the driver D via an input device. Examples of the input device of the terminal device include a touch screen, a button, and a keyboard. That is, the input receiving unit 204 accepts inputs of trigger pulses for transmitting a capture command, a capture command, a transmission command, an integration command, and a reflection command. The input receiving unit 204 is an example of a capture command receiving unit. The capture command is a command for causing the control device 126 of the construction machine 100 to capture capture data. The acquisition command is a command that causes the control device 126 of the construction machine 100 to acquire recording data that is used to generate 3D data.The send command is a command that causes the terminal device 200 to send captured data and 3D data to the server device 300. The integrate command is a command that causes the server device 300 to integrate a plurality of pieces of 3D data into one piece of integrated 3D data. The reflect command is a command that causes the server device 300 to compare one piece or a plurality of pieces of 3D data with 3D data of a completed shape and reflect a result of the comparison into information (construction progress information) representing a progress of the construction work based on the result of the comparison. The 3D data of the completed shape is information representing a topography after the completion of the construction work at the construction site. The target vehicle storing unit 205 stores a vehicle ID of a construction machine 100 to which a pickup command and the acquisition command are directed.

[0041] The command transmission unit 206 transmits, in accordance with an input from driver D, a pickup command or a capture command for a construction machine 100 represented by a vehicle identifier stored by the target vehicle storage unit 205 to the server device 300. The pickup command and the capture command are transmitted through the server device 300 to the construction machine 100. That is, the command transmission unit 206 transmits, via the server device 300, a pickup command and a capture command to a construction machine 100 represented by a vehicle identifier stored by the target vehicle storage unit 205. The command transmission unit 206 transmits, in accordance with an input from driver D, a transmission command for transmitting 3D data or capture data to the server device 300.The command sending unit 206 is an example of a recording command sending unit and a capture command sending unit.

[0042] The display unit RP is a display arranged in the terminal device 200. A liquid crystal display (LCD) can be used as the display unit. Server device

[0043] Fig. 5 is a block diagram illustrating the configuration of the server device according to the first embodiment.

[0044] The server device 300 includes a unit 301 for sending commands, a unit 302 for storing connection relationships, a unit 303 for sending data, a unit 304 for receiving data, a unit 305 for storing data, a unit 306 for integrating 3D data, a unit 307 for receiving commands, a unit 308 for sending data, and a unit 309 for storing location information.

[0045] The command transmission unit 301 transmits a pickup command or a capture command received from the terminal device 200 to a construction machine 100 represented by a vehicle identifier included in the command. The command transmission unit 301 stores the vehicle identifier included in the pickup command and the terminal identifier included in the pickup command in the connection relationship storage unit 302, linked to each other.

[0046] The data transmission unit 303 transmits the recording data received from the construction machine 100 to the terminal device 200 represented by the terminal ID associated with the vehicle ID of the construction machine 100 in the connection relationship storage unit 302.

[0047] The data receiving unit 304 receives 3D data and recording data from the construction machine 100.

[0048] The data storage unit 305 stores the 3D data and the shot data received by the data receiving unit 304, as well as the 3D data representing the completed shape. The data storage unit 305 stores the 3D data and the shot data in association with a vehicle identifier used to identify a construction machine 100 that is a generation source of this data.

[0049] The 3D data integration unit 306 integrates two or more pieces of 3D data stored by the data storage unit 305 into one piece of 3D data. The 3D data integration unit 306 stores the integrated 3D data in the data storage unit 305. The server device 300 is an example of a data integration device.

[0050] The command receiving unit 307 receives a command to send 3D data or capture data from the terminal device 200.

[0051] The data transmitting unit 308 transmits, based on a transmitting command received by the command receiving unit 307, 3D data or shot data stored by the data storing unit 305 to the terminal device 200, which is a transmission source of the transmitting command.

[0052] The location information storage unit 309 stores a vehicle identifier used to identify a construction machine 100 performing work at the construction site, linked to a location identifier used to identify a construction site. function

[0053] Next, an image acquisition method using the construction monitoring system 1 according to the first embodiment will be described.

[0054] Fig. 6 is a flowchart illustrating the operation of the terminal device according to the first embodiment.

[0055] A driver D causes the terminal device 201 to execute an image capture control program for performing an image capture method according to the first embodiment by operating the terminal device 200. When the terminal device starts executing the image capture control program, the beacon signal receiving unit 201 starts a process of receiving a beacon signal (step S101). When the terminal device 200 is within a range of the beacon signal of the construction machine 100, the beacon signal receiving unit 201 can receive a beacon signal transmitted by the construction machine 100. That is, the beacon signal receiving unit 201 receives a beacon signal from a neighboring construction machine 100.The beacon signal receiving unit 201 determines a vehicle ID included in a beacon signal whose reception strength is the highest among the received beacon signals (step S102). The beacon signal receiving unit 201 stores the determined vehicle ID in the target vehicle storage unit 205 (step S103). Since a driver D carries a terminal device 200 into the operator's cab 121 of the construction machine 100, the strength of a beacon signal transmitted by the construction machine 100 is the highest among the beacon signals received by the terminal device 200. Accordingly, the terminal device 200 can determine the construction machine 100 driven by the driver D as a target vehicle.By determining the target vehicle, a pickup command and a capture command issued in accordance with an operation of the terminal device 200 are transmitted to the construction machine 100 represented by the vehicle ID stored by the target vehicle storage unit 205. Furthermore, a pickup command and a capture command may be transmitted to the target vehicle simultaneously, or when a pickup command is transmitted to the target vehicle, a capture command may be automatically transmitted to the target vehicle.

[0056] Fig. Fig. 7 is a flowchart illustrating an example of a process selection screen displayed on the terminal device according to the first embodiment. That is, Fig. 7 is a schematic diagram showing an example of a screen displayed on a display that is the display unit RP. When a vehicle ID is stored in the target vehicle storage unit 205, the display control unit 203 displays a process selection screen including an execution button B1 for executing a 3D data generation control process and an execution button B2 for executing a 3D data management process on the display that is the display unit RP (step S104). The input receiving unit 204 receives a press input on the execution button B1 for executing the generation control process or the execution button B2 for executing the management process that is displayed (step S105).When the execution button B1 is pressed to execute the generation control process (step S105: generation control process), the terminal device 200 executes the 3D data generation control process (step S106). On the other hand, when the execution button B2 is pressed to execute the management process (step S105: management process), the terminal device 200 executes the 3D data management process (step S107).

[0057] Fig. 8 is a first flowchart illustrating the process for controlling the generation of 3D data according to the first embodiment. The first flowchart illustrates a process from a step of sending a capture command using the terminal device 200 to a step of displaying capture data using the terminal device 200. When the terminal device 200 starts a process for controlling the generation of 3D data in step S106, the command transmission unit 206 transmits a capture command containing a vehicle ID stored by the target vehicle storage unit 205 to the server device 300 (step S211). When a capture command is received from the terminal device 200, the command transmission unit 301 of the server device 300 stores a vehicle ID included in the capture command and a terminal ID representing the terminal device 200 that is the transmission source of the capture command in the connection relationship storage unit 302, linked to each other (step S221).The relationship between the vehicle ID and the terminal ID stored in the connection relationship storage unit 302 is maintained until a connection between at least the terminal device 200 and the server device 300 is interrupted. Subsequently, the command transmission unit 301 transmits the received pickup command to a construction machine 100 represented by the vehicle ID included in the pickup command (step S222).

[0058] When the command receiving unit 1262 of the construction machine 100 receives a shooting command from the server device 300, the shooting data acquiring unit 1263 acquires shooting data (first shooting data) from the first camera 1251 (step S231). The first camera 1251 is located closest to the work equipment 110 among the cameras constituting the stereo cameras 125 and is arranged so that the optical axis is approximately parallel to the floor surface of the operator's cab 121. Therefore, the shooting data acquired by the first camera 1251 is data for imaging the topography in front of the work equipment 110. The data transmitting unit 308 transmits the first shooting data acquired by the shooting data acquiring unit 1263 to the server device 300 (step S232).Thereafter, until the connection between the terminal device 200 and the server device 300 is interrupted, the construction machine 100 performs acquisition and transmission of the first recorded data at predetermined time intervals (e.g., every 5 seconds). By receiving a disconnection notification from the server device 300, the construction machine 100 can detect the interruption of the connection between the terminal device 200 and the server device 300.

[0059] When the first shot data is received from the construction machine 100, the data transmission unit 303 of the server device 300 determines a terminal ID associated with the vehicle ID of the construction machine 100, which is a transmission source of the first shot data, in the connection relationship storage unit 302 (step S223). The data transmission unit 303 transmits the first shot data to a terminal device 200 represented by the determined terminal ID (step S224).When the data receiving unit 202 of the terminal device 200 receives first shot data, the display control unit 203 displays a shot area check screen including a transmission button for transmitting a command to acquire second shot data used to generate 3D data, as well as the received first shot data, on the display, which is the display unit RP (step S212). At this time, the data transmission unit 303 of the server device 300 transmits the first shot data received from the construction machine 100 to the terminal device 200 at predetermined time intervals (e.g., every 5 seconds).The data receiving unit 202 of the terminal device 200 receives first capture data from the data transmitting unit 303 of the server device 300 at predetermined time intervals, and the display control unit 203 displays the received first capture data on the display at predetermined time intervals. Furthermore, it is possible for the data transmitting unit 303 to transmit only first capture data first transmitted from the construction machine 100 to the terminal device 200, and for the display control unit 203 to display the first capture data on the display. Pressing the transmit button to transmit a capture command is an example of a trigger pulse for transmitting a capture command. That is, after receiving first capture data, the input receiving unit 204 starts receiving an input of a trigger pulse for transmitting a capture command.

[0060] Fig. Fig. 9 is a schematic diagram illustrating an example of the screen for checking a recording area displayed on the terminal device according to the first embodiment. That is, Fig. 9 is a schematic diagram showing an example of a screen displayed on a display that is the display unit RP.

[0061] The screen for checking a recording area closes as shown in Fig. 9, a display area R of first captured data and a send button B3 for sending a capture command. In the display area A of the first captured data, a frame F is displayed, which represents an area used to generate 3D data. The area used to generate 3D data is an overlapping portion of capture areas of cameras that form a pair, and is an area used to measure a topography. That is, in the display control unit 203, a frame F representing a portion of the capture area of ​​the first camera 1251 that overlaps the capture area of ​​the third camera 1253 is displayed on the display as measurement area information. The size of the frame F is based on a known capture area and is stored, for example, in the display control unit 203 in the form of graphic data. Although in the Fig. 9, the frame F is displayed using broken lines, in another embodiment, the frame F may be highlighted using, for example, solid red lines or the like instead of the broken lines as the measurement area information. Further, the display control unit 203 may display a character indicating a shooting area together with the frame F as the measurement area information. Accordingly, a driver D who is a photographer, after checking whether a desired topography is imaged within a region for generating 3D data (a shooting area or shooting content), can press a send button to send an acquisition command for acquiring shooting data for generating 3D data.

[0062] Fig. 10 is a second flowchart illustrating a process for controlling the generation of 3D data according to the first embodiment. The second flowchart illustrates a process from a step in which an acquisition command for acquiring shot data is sent using the terminal device 200 to a step in which the server device 300 is caused to execute various processes (sending, integrating, or reflecting 3D data).

[0063] When the terminal device 200 displays a transmission button for transmitting an acquisition command and a driver D presses the transmission button, the command transmission unit 206 transmits an acquisition command for acquiring shot data used to generate 3D data to the server device 300 (step S213). The acquisition command includes a vehicle ID stored by the target vehicle storage unit 205. The command transmission unit 301 of the server device 300 transmits the received acquisition command to a construction machine 100 represented by a vehicle ID included in the acquisition command (step S225).

[0064] When the command receiving unit 1262 of the construction machine 100 receives an acquisition command, the image data acquisition unit 1263 acquires image data (second image data) from all cameras (the first camera 1251, the second camera 1252, the third camera 1253, and the fourth camera 1254) of the stereo cameras 125 (step S233). A timestamp representing a time of capture is included in the image data. Furthermore, in another embodiment, a timestamp may be acquired as a separate piece of data associated with the image data instead of being included in the image data. The vehicle information acquisition unit 1264 acquires vehicle information such as position information, azimuth information, and posture information from the position detector 122, the azimuth detector 123, and the posture detector 124, respectively (step S234). The 3D data generation unit 1265 generates 3D data of the capture area corresponding to the stereo camera 125 based on the second capture data, the position information, the azimuth information, and the posture information that have been acquired (step S235). As with the capture data, a timestamp is included in the generated 3D data. A value of the timestamp may be a time of capture of capture data used to generate 3D data or a time of generation of 3D data.

[0065] The data transmission unit 1266 transmits the generated 3D data, the second image data, the position information, the location identifier, and the vehicle identifier of the construction machine 100 to the server device 300 (step S236). If the timestamp is acquired as a separate data item alongside the image data and 3D data, the data transmission unit 1266 also transmits the timestamp to the server device 300. The position information transmitted in this case may be position information determined using a GNSS or position information represented by an address of a construction site. When 3D data and second shot data are received, the data receiving unit 304 of the server device 300 stores the 3D data, the second shot data, a vehicle identifier, position information, and a location identifier in the data storing unit 305 (step S226).When the timestamp is acquired as a separate data item besides the capture data and the 3D data, the data receiving unit 304 also records the timestamp in the data storing unit 305. Accordingly, after a driver D has checked a shot area and a shot content used to generate 3D data, the terminal device 200 can send an acquisition command for acquiring shot data used to generate 3D data to the construction machine 100.

[0066] Fig. 11 is a flowchart showing a process for managing 3D data ( Fig. 6, step S107) according to the first embodiment. When the terminal device 200 instructs a 3D data management process in step S107, the command transmission unit 206 sends a transmission command to transmit a list of 3D data and shot data to the server device 300 (step S311). The acquisition command includes a vehicle ID stored by the target vehicle storage unit 205. When the command reception unit 307 of the server device 300 receives a transmission command, the data transmission unit 308 generates a list of 3D data and shot data stored by the data storage unit 305 and transmits the generated list to the terminal device 200 (step S321).Here, the data transmission unit 308 extracts data associated with the vehicle ID included in the transmission command from the data stored by the data storage unit 305, thereby generating a list. The list includes a combination of 3D data and shot data used to generate the 3D data as a basic component. When the data reception unit 202 of the terminal device 200 receives a list, the display control unit 203 displays the received list on the display, which is the display unit RP (step S312).

[0067] The input receiving unit 204 accepts a selection of a combination of 3D data and recording data contained in the received list (step S313). A plurality of combinations can be selected. If the selection of a combination of 3D data and recording data is accepted, the input receiving unit 204 accepts the selection of a functional step for the selected combination (step S314). Possible functional steps include reading data, integrating data, and reflecting on information about the progress of construction work.

[0068] When reading data is selected (step S314: Read), the command transmission unit 206 transmits a transmission command for transmitting selected data to the server device 300 (step S315). The transmission command includes identification information of data that is a transmission destination. When the command reception unit 307 of the server device 300 receives a transmission command, the data transmission unit 308 reads 3D data and capture data represented by the transmission command from the data storage unit 305 and transmits the read data to the terminal device 200 (step S322). When the data receiving unit 202 of the terminal device 200 receives the 3D data and the shot data, the display control unit 203 displays the 3D data and the shot data that have been received on the display (step S316).Furthermore, the 3D data received by the data receiving unit 304 includes integrated 3D data. Accordingly, a driver D can check whether the 3D data representing a corresponding area has been generated by the control device 126 of the construction machine 100. Furthermore, a timestamp associated with these data can be displayed by the display control unit 203 along with the 3D data and the recording data.

[0069] If integrating data is selected in step S314 (step S314: Integrate), the command sending unit 206 sends an integration command for integrating selected data (a combination of 3D data and captured data) to the server device 300 (step S317). Furthermore, if integrating data is selected, it is necessary to select a plurality of pieces of data. Included in the integration command are identification information of a plurality of pieces of data that are objects to be integrated. When the command receiving unit 307 of the server device 300 receives an integration command, the 3D data integrating unit 306 reads a plurality of pieces of 3D data represented by identification information included in the integration command.The 3D data integrating unit 306 integrates the plurality of pieces of read 3D data into one integrated 3D data piece (step S323). Fig. Figure 12 is a schematic diagram illustrating a process for generating integrated 3D data from 3D data. The 3D data, which is an object to be integrated, is as shown in Fig. 12 above, 3D data represented in an absolute coordinate system that has a common reference position as the origin. The 3D data integration unit 306 integrates 3D data that have the common origin and thus generates, as shown in Fig. 12 below, integrated 3D data. Details of the integration process will be described below. The 3D data integration unit 306 stores the generated integrated 3D data in the data storage unit 305 (step S324). Accordingly, the integrated 3D data is newly added to the 3D data list generated in step S321. Accordingly, the driver D can then check the integrated 3D data by causing the terminal device 200 to execute processes of steps S312 to S316. That is, the driver D selects integrated 3D data from the list displayed by the terminal device 200 in step S312 and selects data reading in step S314, so that the integrated 3D data is sent from the server device 300 to the terminal device 200 and the integrated 3D data is displayed on the terminal device 200.Accordingly, driver D can check the integrated 3D data.

[0070] If reflection on information about the progress of construction work is selected in step S314 (step S314: Reflect), the command transmission unit 206 sends a reflection command for reflecting selected data to the server device 300 (step S318). Included in the reflection command is identification information of data that is an object to be reflected. The data that is an object to be reflected may be integrated 3D data. When the command reception unit 307 of the server device 300 receives a reflection command, the 3D data integration unit 306 reads 3D data (data that is an object to be reflected) represented by identification information included in the reflection command and 3D data representing a completed shape.The 3D data representing the completed shape is information generated in advance by a construction company or the like, and represents a topography after the completion of construction work at the construction site. As the 3D data representing the completed shape, for example, data displayed in a polygon such as a triangle or the like can be used. The 3D data integrating unit 306 acquires a piece of 3D data representing the completed shape, whose plane position overlaps the data that is an object to be reflected, as data (progress data) representing a topography in the progress of construction work, or data (work extent data) of an excavation extent or an excavation extent in the progress of construction work (step S325).That is, the 3D data integrating unit 306, at a position where the plane positions of the pre-reflecting progress data stored in advance and the 3D data of an object to be reflected overlap (a position where the x- and y-coordinates are the same), substitutes the height data of the 3D data of an object to be reflected for the height data (z-coordinate) of the progress data, thereby updating the progress data. Furthermore, at a position where the plane positions of the pre-reflecting progress data and the 3D data of an object to be reflected overlap, the 3D data integrating unit 306 acquires work amount data based on a difference between the height data of the pre-reflecting progress data and the 3D data of the object to be reflected.Furthermore, when a plurality of pieces of data (3D data, integrated 3D data, or a combination thereof) are selected as data that is an object to be reflected, for a position where plane positions of the data overlap with each other, the 3D data integrating unit 306 generates progress data using height information of the most recent data (data whose timestamp is the most recent). The 3D data integrating unit 306 stores the updated progress data in the data storing unit 305 (step S326). Accordingly, progress data is newly added to the list of 3D data generated in step S321. Accordingly, the driver D can then check the updated progress data by causing the terminal device 200 to execute processes of steps S312 to S316.That is, the driver D selects progress data from the list displayed by the terminal device 200 in step S312 and selects data reading in step S314, so that the progress data is sent from the server device 300 to the terminal device 200 and the progress data is displayed on the terminal device 200. Accordingly, the driver D can check the progress data. Integration process

[0071] The integration process shown in step S323 is described in detail below. Fig. 13 is a flowchart illustrating the process of integrating according to the first embodiment.

[0072] The 3D data integration unit 306 reads 3D data represented by identifier information included in the integration command received by the command reception unit 307 (step S401). Then, the 3D data integration unit 306 calculates a time represented by a timestamp associated with the read 3D data and a time width of the timestamp (a time difference between a most recent time point and a most recent time point) (step S402). Then, the 3D data integration unit 306 judges whether the calculated time width is shorter than a predetermined time (step S403).If the time width is equal to or longer than the predetermined time (step S403; No), the 3D data integrating unit 306 informs the terminal device 200 that it is unable to integrate the 3D data (step S404) and terminates the integration process. This is because, if the generation times of 3D data that is an object to be integrated are separated by a predetermined time or more, certain construction operations such as excavation or backfilling may have been performed until new 3D data was generated after old 3D data was generated, and the topography may have changed.

[0073] If the time latitude is shorter than the predetermined time (step S403: Yes), the 3D data integrating unit 306 determines a sum of groups of plane coordinates where height data exists from each piece of 3D data that has been read (step S405). As an example, a case will be described where first 3D data is a point group formed by coordinates (X1, Y1, Z1), coordinates (X2, Y2, Z2), and coordinates (X3, Y3, Z3), and second 3D data is a point group formed by coordinates (X3, Y3, Z1), coordinates (X4, Y4, Z4), and coordinates (X5, Y5, and Z5). The coordinates (X, Y, Z) are coordinates determined by latitude X, longitude Y, and height Z. That is, plane coordinates are coordinates represented by a combination of latitude X and longitude Y.In this case, a sum of groups of plane coordinates where height data exists in the first 3D data and the second 3D data are coordinates (X1, Y1), coordinates (X2, Y2), coordinates (X3, Y3), coordinates (X4, Y4), and coordinates (X5, Y5). Then, the 3D data integration unit 306 generates empty integrated 3D data (step S406).

[0074] Then, the 3D data integrating unit 306 selects plane coordinates included in the sum of groups determined in step S405 one at a time and executes processes of steps S408 to S411 shown below (step S407). First, the 3D data integrating unit 306 judges whether a plurality of pieces of data including height information for the selected plane coordinates are present in the 3D data read in step S401 (step S408). That is, the 3D data integrating unit 306 judges whether height information related to the same plane coordinates is included in the other 3D data. If the height information related to the selected plane coordinates is included in only one piece of the 3D data (step S408: No), the height information at the selected plane coordinates is reflected in the integrated 3D data (step S409).Accordingly, the 3D data is recorded in the integrated 3D data.

[0075] On the other hand, if the height information related to the selected plane coordinates is included in a plurality of pieces of 3D data (step S408: Yes), the 3D data integrating unit 306 calculates an average value of heights from each piece of height information (step S410). Then, the 3D data integrating unit 306 reflects the average value of heights at the selected plane coordinates in the integrated 3D data as height information (step S411). Accordingly, the 3D data is recorded in the integrated 3D data.

[0076] When the processes of steps S408 to S411 described above are executed for all of the plane coordinates included in the sum of groups determined in step S407, the 3D data integrating unit 306 can generate integrated 3D data. Function and effects

[0077] Thus, according to the first embodiment, the terminal device 200 receives image data captured by the stereo cameras 125, displays the received image data on the display unit RP, and sends an acquisition command to the construction machine 100 to acquire the image data for generating 3D data. Accordingly, when the acquisition command is sent, the terminal device 200 can enable a driver D, who is a photographer, to check a capture area and capture content corresponding to the stereo cameras 125. Accordingly, the driver D can cause the control device 126 to generate three-dimensional data related to a desired location.

[0078] Furthermore, according to the first embodiment, the terminal device 200 receives an input of a trigger pulse for transmitting a capture command after the capture data is displayed on the display unit RP. Accordingly, by inputting the trigger pulse for transmission, when it can be confirmed by checking that the stereo cameras 125 capture a desired capture range and capture content, the driver D can cause the control device 126 to generate 3D data related to a desired location.

[0079] Furthermore, according to the first embodiment, the terminal device 200 sends a shooting command including a vehicle ID and receives shooting data from the construction machine 100 represented by the vehicle ID. Accordingly, even when a plurality of construction machines 100 are located at a construction site, the terminal device 200 can receive shooting data from a desired construction machine 100.

[0080] Furthermore, according to the first embodiment, the terminal device 200 receives 3D data from the server device 300 and displays the 3D data. Accordingly, a driver D can check whether the construction machine 100 is generating corresponding 3D data related to the desired location.

[0081] Furthermore, the terminal device 200 according to the first embodiment displays the measurement area information representing an area used for generating 3D data along with the captured data on the display unit RP. Accordingly, a driver D can check whether a desired location depicted in the captured data is included in the area used for generating 3D data.

[0082] Furthermore, the server device 300 according to the first embodiment can generate integrated 3D data acquired by integrating a plurality of pieces of 3D data. Accordingly, by reading the integrated 3D data using the terminal device 200, a driver D can easily check the presence / absence of a construction area missing from the image and an area that has not generated 3D data. That is, when there is no integrated 3D data, the driver D needs to individually check pieces of image data or 3D data and check whether there is a construction area missing from the image or an area that has not generated 3D data. Furthermore, even if pieces of image data or 3D data are individually checked, it is possible that the driver D may misjudge the presence / absence of a construction area missing from the image.Furthermore, by reading the integrated 3D data, the driver D can easily recognize the entire topography after the work as well as his position. That is, the purpose of the integrated 3D data is to be able to easily check whether there is a construction area missing in the image or an area that has not generated 3D data, or to easily recognize the overall topography after the work.

[0083] According to the first embodiment, the server device 300 performs integration of 3D data when a time width of time points associated with a plurality of pieces of 3D data that are objects to be integrated is equal to or shorter than a predetermined time. Accordingly, the server device 300 can generate the integrated 3D data in a state where there is a low probability that the topography will change due to certain construction work, such as excavation or backfilling, until new 3D data is generated after the generation of old 3D data. Modified example

[0084] Although in the first embodiment, a vehicle ID included in a beacon signal whose reception strength is the highest is determined when the terminal device 200 determines a target vehicle in step S102, the determination of a vehicle ID in another embodiment is not limited to this. For example, in another embodiment, the beacon signal receiving unit 201 may determine a vehicle ID included in a beacon signal whose reception strength is the highest among beacon signals whose reception strengths are continuously at a predetermined strength or higher for a predetermined time.Furthermore, in another embodiment, the beacon signal receiving unit 201 may display a list of vehicle IDs included in beacon signals whose reception strengths are at a predetermined strength or higher on the terminal device 200 or the like, and a selected vehicle ID may be determined from them.

[0085] Although in the first embodiment, in the integration process, when there are a plurality of pieces of height information for the same plane coordinates, the server device 300 acquires an average value thereof and reflects the average value to the integrated 3D data, the method of reflecting the same in another embodiment is not limited to this. For example, in another embodiment, the server device 300 may reflect height information, which from a plurality of pieces of 3D data relates to 3D data whose timestamp represents the closest time point (the time point of 3D data generation is the closest), to the integrated 3D data.In this case, instead of the processes of steps S407 to S411, the 3D data integrating unit 306 of the server device 300 may update the integrated 3D data by reading 3D data in order from the earliest to the most recent generation time and sequentially overwriting the height information. Furthermore, in another embodiment, the construction machine 100 may include a 3D data integrating unit 306, and the integrating process may be performed by the construction machine 100. In this case, the unit 1265 for generating 3D data of the construction machine 100 generates in step S2235 shown in Fig. 18, a plurality of pieces of 3D data, and sends the data sending unit 1266 after the 3D data integrating unit 306 of the construction machine 100 has processed them using the Fig. 13, in step S2236, the control device 126 of the construction machine 100 integrates the second acquisition data, the 3D data, and the integrated 3D data. In this case, the control device 126 of the construction machine 100 is an example of a data integration device.

[0086] Although in the first embodiment, in the integration process, the server device 300 judges whether a time width of time points associated with a plurality of pieces of 3D data that are objects to be integrated is equal to or shorter than a predetermined time, another embodiment is not limited to this. For example, in another embodiment, the server device 300 may generate integrated 3D data independently of time points associated with a plurality of pieces of 3D data that are objects to be integrated. Further, for example, in another embodiment, the terminal device 200 may judge whether a time width of time points associated with 3D data that are objects to be integrated is equal to or shorter than a predetermined time. In this case, the terminal device 200 does not send an integration command if a time width of the time points associated with the 3D data selected in step S313 is equal to or longer than a predetermined time. For example, if a time width of the time points associated with the three-dimensional data selected in step S313 is equal to or longer than a predetermined time, the terminal device 200 may disable the selection of the data in step S314. Second embodimentConstruction monitoring system

[0087] Fig. 14 is a schematic diagram showing the configuration of a construction monitoring system according to a second embodiment. In the construction monitoring system 1 according to the first embodiment, a driver D instructs the construction machine 100 to perform image acquisition and 3D data generation by operating a terminal device 200. In the construction monitoring system 1 according to the second embodiment, however, a site supervisor A of a construction site is a photographer, and the site supervisor A instructs the construction machine 100 to perform image acquisition and 3D data generation by operating the terminal device 200. The terminal device 200 according to the second embodiment may be, for example, a smartphone, a mobile phone, or a portable computer. Furthermore, the terminal device 200 may be, for example, a personal computer (PC) including a display unit and located in an office far from a construction site.

[0088] When the site manager A operates the terminal device 200, since a target vehicle may not include the beacon signal transmission unit 1261, or since a beacon signal of the target vehicle may not reach the terminal device 200 even if the target vehicle includes the beacon signal transmission unit 1261, it may be difficult to perform judgment of the target vehicle based on the beacon signal as in the first embodiment. Therefore, the terminal device 200 according to the second embodiment judges a target vehicle without using a beacon signal.

[0089] Fig. 15 is a schematic block diagram illustrating the configuration of the terminal device according to the second embodiment.

[0090] The terminal device 200 according to the second embodiment includes a vehicle list storage unit 207 instead of the beacon signal receiving unit 201 of the terminal device 200 according to the first embodiment. The vehicle list storage unit 207 stores a list that links a plurality of location identifiers used to identify construction sites to vehicle identifiers of construction machines 100 operating at the construction sites. The construction machine 100 according to the second embodiment may not include the beacon signal transmitting unit 1261. function

[0091] Fig. 16 is a flowchart illustrating the operation of the terminal device according to the second embodiment. A driver D causes the terminal device 200 to execute an image pickup control program for performing an image pickup method according to the second embodiment by operating the terminal device 200. When the terminal device 200 starts executing the image pickup control program, a display control unit 203 displays a list of construction sites stored by a vehicle list storage unit 207 on a display (step S1101). For example, a list representing construction sites based on addresses of the construction sites is displayed on the display. The input receiving unit 204 accepts an input of selecting a location ID from the displayed list (step S1102). The input receiving unit 204 displays a list of construction machines 100 associated with an input location ID on the display (step S1103).For example, a list representing construction machines 100 based on a combination of a type and a serial number of each construction machine 100 is displayed on the display. The input receiving unit 204 accepts an input of selecting a vehicle ID from the displayed list (step S1104). The input receiving unit 204 stores the input vehicle ID in a target vehicle storage unit 205 (step S1105). Accordingly, the terminal device 200 can determine a target vehicle without receiving a beacon signal.

[0092] When the input receiving unit 204 stores a vehicle ID in the target vehicle storing unit 205, the display control unit 203 displays an execution button for a 3D data generation control process and an execution button for a 3D data management process on the display (step S1106). The input receiving unit 204 accepts a press input to the execution button for the generation control process or the execution button for the management process displayed (step S1107). When the execution button for the generation control process is pressed (step S1107: generation control process), the terminal device 200 executes the 3D data generation control process (step S1108).On the other hand, when the execution button for the management process is pressed (step S1107: management process), the terminal device 200 executes displaying data, integrating data, and reflecting data (the 3D data management process) in the manner shown in FIG. 1 as in the first embodiment. Fig. 11 (step S1109). That is, a site manager A selects data to be checked from the list displayed by the terminal device 200 in step S312 and selects data reading in step S314, so that the data is sent from the server device 300 to the terminal device 200 and the data is displayed on the terminal device 200. Accordingly, the site manager A can check whether the 3D data representing a corresponding area has been generated using the control device 126 of the construction machine 100. Function and effects

[0093] Thus, according to the second embodiment, the terminal device 200 accepts an input of selecting a vehicle ID from the prestored list and thus designates a target vehicle. Accordingly, the terminal device 200 can designate a target vehicle without receiving a beacon signal. Accordingly, the site manager A, who is a photographer, can check a shooting range and shooting content captured by the stereo cameras 125 and can cause the control device 126 to generate three-dimensional data related to a desired location. Modified example

[0094] Although in the second embodiment, the terminal device displays the list stored by the vehicle list storage unit 207, another embodiment is not limited to this. For example, in another embodiment, the terminal device 200 may transmit a location identifier of a construction site that is a destination to the server device 300, and the server device 300 may transmit a list of construction machines 100 associated with the location identifier from the location information storage unit 309 to the terminal device 200. That is, in a construction monitoring system 1 according to another embodiment, if a site manager A or a driver D can select a construction site that is a destination from among a plurality of construction sites, a construction machine 100 can be selected.

[0095] Furthermore, a terminal device 200 according to another embodiment may select a construction machine 100 based on a beacon signal when a beacon signal can be received, as in the first embodiment, and may determine a construction machine 100 based on a list when a beacon signal cannot be received, as in the second embodiment. Third embodiment: Construction monitoring system

[0096] In the construction monitoring system 1 according to the first embodiment, the construction machine 100 and the terminal device 200 send and receive commands and data via the server device 300. However, in a construction monitoring system 1 according to a third embodiment, in a process for controlling the generation of 3D data, a construction machine 101 and a terminal device 200 send and receive commands and data by direct communication rather than via a server device 300. The terminal device 200 is operated by a site manager A or a driver D. function

[0097] Fig. 17 is a first flowchart illustrating a process for controlling the generation of 3D data according to the third embodiment.

[0098] When the terminal device 200 starts a process for controlling the generation of 3D data in step S106, a command transmission unit 206 transmits a shooting command to a construction machine 100 represented by a vehicle ID stored by a target vehicle storage unit 205 using near-field communication (step S2211). Examples of near-field communication include a wireless local area network (WLAN), Bluetooth (registered trademark), and near-field radio communication (NFC), which can be used by a driver D or a site manager A without special authentication or authorization. When a command reception unit 1262 of the construction machine 100 receives a shooting command from the terminal device 200, a shooting data acquisition unit 1263 acquires shooting data (first shooting data) from a first camera 1251 (step S2231).A data transmission unit 1266 transmits the first shot data acquired by the shot data acquisition unit 1263 to the terminal device 200, which is a transmission source of the shot command (step S2232). Furthermore, in another embodiment, a vehicle ID, which is information about a vehicle ID assigned to each construction machine 100, may be stored in a predetermined storage device of the construction machine 100, and the data transmission unit 1266 may read a vehicle ID from the predetermined storage device and transmit the vehicle ID together with the first shot data to the terminal device 200. In this case, a target vehicle storage unit 205 of the terminal device 200 stores the received vehicle ID.

[0099] When the capture command executed in step S2211 described above is issued, a vehicle ID acquired in the past may be selected from the target vehicle storage unit 205 of the terminal device 200, and a capture command may be sent to a construction machine 100 corresponding to the selected vehicle ID. Thereafter, until a connection with the terminal device 200 is interrupted, the construction machine 100 performs capture and transmission of the first capture data every time a predetermined time (e.g., 5 seconds) elapses. When a data receiving unit 202 of the terminal device 200 receives the first capture data, a display control unit 203 displays a capture area check screen including a capture command sending button and the received first capture data on a display (step S2212).

[0100] Fig. 18 is a second flowchart illustrating the process for controlling the generation of 3D data according to the third embodiment.

[0101] When the terminal device 200 displays a capture command transmission button and a driver D presses the transmission button, the command transmission unit 206 transmits a capture command to a construction machine 100 represented by a vehicle ID stored by the target vehicle storage unit 205 (step S2213). When the command reception unit 1262 of the construction machine 100 receives the capture command, the shot data acquisition unit 1263 acquires shot data (second shot data) from all cameras of the stereo camera 125 (step S2233). The vehicle information acquiring unit 1264 acquires vehicle information such as position information, azimuth information, and posture information from the position detector 122, the azimuth detector 123, and the posture detector 124 (step S2234).The 3D data generating unit 1265 generates 3D data of a capture area corresponding to the stereo cameras 125 based on the second capture data, the position information, the azimuth information, and the posture information that have been acquired (step S2235).

[0102] The data transmission unit 1266 transmits the generated 3D data and the second shot data to the server device 300 (step S2236). When the data reception unit 304 of the server device 300 receives the 3D data and the second shot data, the 3D data and the second shot data are stored in the data storage unit 305 (step S2226). Furthermore, as in the first embodiment, the terminal device 200 can be configured according to the Fig. 11 (3D data management process). That is, a site manager A or a driver D selects data to be checked from a list displayed by the terminal device 200 in step S312 and selects data reading in step S314, so that the data is sent from the server device 300 to the terminal device 200 and the data is displayed on the terminal device 200. Accordingly, the site manager A or the driver D can check whether the 3D data representing a corresponding area has been generated using the control device 126 of the construction machine 100. Function and effects

[0103] Thus, according to the third embodiment, the terminal device 200 communicates with the construction machine 100 using near-field communication. Accordingly, a response time in communication between the terminal device 200 and the construction machine 100 can be shortened to be lower than when communication is performed via the network N and the server device 300. Furthermore, in the third embodiment as well, the site manager A or the driver D, who is a photographer, can check a shooting range and shooting content captured with the stereo cameras 125, and can cause the control device 126 to generate three-dimensional data related to a desired location. Fourth embodiment

[0104] The terminal device 200 according to the third embodiment is connected to the construction machine 100 via near-field communication. A construction machine 100 according to a fourth embodiment, on the other hand, includes a terminal device 200, and the terminal device 201 and a control device 126 are connected via cables. The terminal device 200 is installed on the side in front of a seat (+Y direction) inside a driver's cab 121. Accordingly, a driver D can monitor the terminal device 200 while driving the construction machine 100.

[0105] The control device 126 according to the fourth embodiment may not include the beacon signal transmission unit 1261. The terminal device 200 according to the fourth embodiment may not include the beacon signal reception unit 201 and the target vehicle storage unit 205. The target vehicle of the terminal device 200 according to the fourth embodiment is a construction machine 100 in which the terminal device 200 is permanently installed. Fifth embodiment

[0106] The construction machine 100 according to the first embodiment acquires second image data from the stereo cameras 125 when a capture command is received. A construction machine 100 according to a fifth embodiment, on the other hand, acquires first image data to be used as second image data from a plurality of pieces of stored first image data.

[0107] Fig. 19 is a block diagram showing the configuration of a control device of a construction machine according to the fifth embodiment. The control device 126 of the construction machine 100 according to the fifth embodiment further includes, in addition to the components according to the first embodiment, a shot data recording unit 1267 and a shot data storage unit 1268. The shot data recording unit 1267 stores first shot data acquired from the stereo camera 125 by the shot data acquisition unit 1263, a shot ID as information about an image ID used to determine the first shot data, and vehicle information acquired by the vehicle information acquisition unit 1264 in the shot data storage unit 1268 in association with each other. Examples of a shot ID include the shot date and time and a serial number.

[0108] Fig. 20 is a first flowchart illustrating a process for controlling the generation of 3D data according to the fifth embodiment.

[0109] When the terminal device 200 is in the Fig. 6 begins a process for controlling the generation of 3D data, the command transmission unit 206 transmits a pickup command containing a vehicle ID stored by the target vehicle storage unit 205 to the server device 300 (step S3211). When the pickup command is received from the terminal device 200, the command transmission unit 301 of the server device 300 stores a vehicle ID included in the pickup command and a terminal ID representing the terminal device 200 that is a transmission source of the pickup command in the connection relationship storage unit 302, linked to each other (step S3221). Then, the command transmission unit 301 transmits the received pickup command to a construction machine 100 represented by a vehicle ID included in the pickup command (step S3222).

[0110] When the construction machine 100 command receiving unit 1262 receives a shooting command from the server device 300, the shooting data acquiring unit 1263 acquires shooting data from all cameras (the first camera 1251, the second camera 1252, the third camera 1253, and the fourth camera 1254) of the stereo cameras 125 (step S3231). The vehicle information acquiring unit 1264 acquires vehicle information, such as position information, azimuth information, and posture information, from the position detector 122, the azimuth detector 123, and the posture detector 124 (step S3232). The shot data recording unit 1267 stores the shot data and the vehicle information that has been acquired, associated with a shot ID in the shot data storing unit 1268 (step S3233).The data transmission unit 308 transmits, among the shot data to which the shot ID is assigned, shot data (first shot data) shot by the first camera 1251 to the server device 300 (step S3234).

[0111] When the first recording data is received from the construction machine 100, the data transmission unit 303 of the server device 300 determines a terminal identifier associated with a vehicle identifier of the construction machine 100, which is a transmission source of the first recording data, in which Fig. 5 (step S3223). The data transmission unit 303 transmits first capture data to a terminal device 200 represented by the determined terminal ID (step S3224). When the data reception unit 202 of the terminal device 200 receives the first capture data, the display control unit 203 displays a capture area check screen including a transmission button B3 for transmitting a capture command to the construction machine 100 and the received first capture data on the display (step S3212).

[0112] Fig. 21 is a second flowchart illustrating a process for controlling the generation of 3D data according to the fifth embodiment. When the terminal device 200 displays the send button B3 and a driver D presses the send button B3, the command sending unit 206 determines a shot ID appended to the first shot data most recently received by the data receiving unit 202 (step S3213). Accordingly, a shot ID of the first shot data displayed on the display when the send button B3 is pressed can be determined. The command sending unit 206 sends a command to acquire the second shot data to the server device 300 (step S3214). The acquisition command includes the determined shot ID and the vehicle ID stored by the target vehicle storage unit 205.The command sending unit 301 of the server device 300 sends the received acquisition command to a construction machine 100 represented by the vehicle identifier included in the acquisition command (step S3225).

[0113] When the command receiving unit 1262 of the construction machine 100 receives an acquisition command, the shot data acquisition unit 1263 extracts the first shot data associated with the shot ID included in the acquisition command, as well as shot data captured by the second camera 1252, the third camera 1253, and the fourth camera 1254 at the same time as the first shot data, from the shot data storage unit 1268, and acquires the shot data as second shot data (step S3235). Furthermore, the shot data acquisition unit 1263 acquires vehicle information associated with the shot ID included in the acquisition command from the shot data storage unit 1268.The 3D data generating unit 1265 generates 3D data of a capture area corresponding to the stereo cameras 125 based on the second capture data and the vehicle information that has been acquired (step S3236).

[0114] The data transmission unit 1266 transmits the generated 3D data and the second shot data to the server device 300 (step S3237). When the 3D data and the second shot data are received, the data reception unit 304 of the server device 300 stores the 3D data and the second shot data in the data storage unit 305 (step S3226). Accordingly, the terminal device 200 can send the acquisition command for acquiring recording data displayed when the driver D presses the send button B3 to the construction machine 100. Other embodiments

[0115] Although the embodiments have been described above in detail with reference to the drawings, a concrete configuration is not limited to those described above, and various structural changes and the like may be made. For example, although a terminal ID of the transmission source is included in a capture command sent by the terminal device 200 according to the above-described embodiment, the capture command is not limited thereto. For example, in a capture command according to another embodiment, a terminal ID representing a device other than the terminal device 200 may be included as a terminal ID of a transmission destination for capture data. That is, a terminal device 200 that sends a capture command and a terminal device 200 that receives capture data and displays the capture data may be different from each other.

[0116] Although the construction machine 100 according to the above-described embodiment transmits the captured data when a predetermined time has elapsed upon receiving a capture command, the function is not limited to this. For example, a construction machine 100 according to another embodiment may start streaming transmission of captured data captured by the first camera 1251 upon receiving a capture command. That is, a construction machine 100 according to another embodiment may transmit captured data as a moving image.

[0117] Although the construction machine 100 according to the above-described embodiment includes the stereo cameras 125 as image pickup devices capable of acquiring information about the depth direction, the image pickup device is not limited thereto. For example, the construction machine 100 according to another embodiment may include a 3D scanner, a depth sensor, or any other image pickup device as an image pickup device capable of acquiring information about the depth direction.

[0118] Although the construction machine 100 according to the above-described embodiment transmits image data captured by the first cameras 1251 when a capture command is received, the transmission is not limited to this. For example, a construction machine 100 according to another embodiment may transmit image data captured by the second camera 1252, image data captured by the third camera 1253, or image data captured by the fourth camera 1254 when a capture command is received. Furthermore, a construction machine 100 according to another embodiment may transmit image data captured by two or more of the first camera 1251, the second camera 1252, the third camera 1253, and the fourth camera 1254 when a capture command is received.

[0119] Although the control device 126 of the construction machine 100 according to the above-described embodiment generates 3D data by acquiring image data from the stereo cameras 125 when receiving a capture command, the generation of 3D data is not limited thereto. For example, in a construction monitoring system 1 according to another embodiment, a construction machine 100 may send image data acquired by the stereo cameras 125 and vehicle information acquired by the vehicle information acquisition unit 1264 to a server device 300, and the server device 300 may generate 3D data from the image data. That is, a construction machine 100 according to another embodiment may not include the 3D data generation unit 1265.

[0120] Although the processing machine 100 according to the above-described embodiment includes the beacon signal transmission unit 1261 as the function of the control device 126, the configuration is not limited thereto. For example, a construction machine 100 according to another embodiment may include a signal transmission device separate from the control device 126 as the beacon signal transmission unit 1261.

[0121] Although the construction machine 100 according to the embodiment described above is a hydraulic excavator as a work vehicle, the work vehicle is not limited to this. For example, a construction machine 100 according to another embodiment may be a wheel loader, a bulldozer, a motor grader, a dump truck, or any other construction machine as a work vehicle.

[0122] Although the terminal device 200 according to the above-described embodiment accepts input of a capture command by pressing the send button when recording data is displayed, the acceptance of the input is not limited to this. For example, the terminal device 200 according to another embodiment may accept input of a capture command by pressing the send button after clearing the display of recording data from the display. In this case, too, a driver D may send a capture command after checking the recording area.

[0123] Although the terminal device 200 according to the above-described embodiment accepts input of a capture command by pressing the send button when capture data is displayed, the process is not limited to this. For example, a terminal device 200 according to another embodiment may continuously send a capture command while displaying capture data. In this case, a driver D can also check the recording area for generating 3D data.

[0124] Although the terminal device 200 according to the embodiment described above receives 3D data and displays the received 3D data on the display, the display is not limited to this. For example, in another embodiment, the terminal device 200 may not perform a function of displaying 3D data. In this case, the 3D data generated according to an operation of the terminal device 200 is checked using another device.

[0125] Although the terminal device 200 according to the above-described embodiment displays a frame representing the 3D data generation area along with the captured data on the display as measurement area information, the display is not limited to this. For example, a terminal device 200 according to another embodiment may perform cropping of captured data in a 3D data generation area and display the cropped captured data on the display. Furthermore, captured data inside and outside the 3D data generation area may have different colors. Furthermore, in another embodiment, "measurement area" may be displayed as text in the frame. Furthermore, in another embodiment, a terminal device may not display the 3D data generation area.

[0126] Although in the terminal device 200 according to the above-described embodiment, a trigger for a capture command, a capture command, a transmission command, an integration command, and a reflection command is pressing the transmission button, the trigger for transmission is not limited to this. For example, a trigger for transmission according to another embodiment may be a predetermined voice input, detection of acceleration of a predetermined value or more, reception of an electric wave, or any other trigger. Furthermore, in another embodiment, a trigger for transmitting a capture command may not be a process for the terminal device 200, but rather pressing a capture button inside the operator's cab 121 of the construction machine 100.That is, in another embodiment, a driver D may acquire second shot data by pressing the button on the construction machine 100 after visually perceiving the first shot data displayed on the terminal device 200.

[0127] Although the server device 300 according to the above-described embodiment transmits first capture data to a terminal device 200 represented by a terminal ID stored in the connection relationship storage unit 302 linked to the ID of the vehicle that is a transmission source of the first capture data, the transmission is not limited thereto. For example, a server device 300 according to another embodiment may transmit first capture data to a terminal device 200 represented by a terminal ID appended to the first capture data received from the construction machine 100. In this case, the server device 300 transmits a capture command including the ID of the terminal that is a transmission source to the construction machine 100.Furthermore, in this case, the control device 126 of the construction machine 100 assigns the terminal identifier contained in the recording command to the first recording data and sends resulting data to the server device 300.

[0128] The server device 300 according to the above-described embodiment transmits image data acquired by the construction machine 100 connected to the terminal device 200 and a list of 3D data, but the function is not limited to this. For example, a server device 300 according to another embodiment may transmit a list of image data acquired by the construction machine 100 connected to the same construction site as the construction machine 100 connected to the terminal device 200 and 3D data.That is, the server device 300 can determine a vehicle identifier associated with the same location identifier as the vehicle identifier included in the transmission command received from the terminal device 200 via the location information storage unit 309, and transmit a list of data stored by the data storage unit 305 associated with the vehicle identifier to the terminal device 200. In this case, the construction monitoring system 1 can perform data reading, data integration, and reflection of information on the progress of construction work in units of the construction sites. Computer configuration

[0129] Fig. 22 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment.

[0130] The computer 99 contains a processor 991, a main memory 992, a memory 993 and an interface 994.

[0131] The control device 126, the terminal device 200, and the server device 300 according to the above-described embodiment include the computer 99. The function of each processing unit described above is stored in the memory 993 as a program. The processor 991 reads a program from the memory 993, extracts the program into the main memory 992, and executes the above-described process according to the program.

[0132] The processor 991 reserves a memory area corresponding to each storage unit described above in the main memory 992 according to the program. The memory 993 is an example of a medium in a non-volatile form. Other examples of the medium in a non-volatile form include an optical disk, a magnetic disk, a magneto-optical disk, and a semiconductor memory connected via the interface 994.

[0133] A program may be distributed to computer 99 over a network. In this case, computer 99 extracts the distributed program into main memory 992 and executes the process described above. A program may be used to implement some of the functions described above. For example, the program can realize the above-described functions by combining it with another program stored in advance in the memory 993 or by combining it with a program installed in another device. Furthermore, some of the above-described functions can be executed by another device connected via a network. That is, the above-described functions can be realized using cloud computing, grid computing, cluster computing, or any other parallel computing.

[0134] Computer 99 may include a programmable logic device (PLD) in addition to, or instead of, the configuration described above. Examples of PLDs include a programmable array logic (PAL), a generic array logic (GAL), a complex programmable logic device (CPLD), or a field programmable gate array (FPGA). Industrial applications

[0135] According to at least one aspect of the aspects described above, a terminal device, a control device, a data integrating device, a work vehicle, an image capturing system and an image capturing method are provided with which 3D data relating to a desired location can be generated. List of reference symbols 1 construction monitoring system 100 construction machines 125 stereo camera 126 Control device 1261 Unit for transmitting beacon signals 1262 Unit for receiving commands 1263 Unit for capturing recording data 1264 Unit for capturing vehicle information 1265 Unit for generating 3D data 1266 Unit for sending data 200 terminal devices 201 Unit for receiving beacon signals 202 Unit for receiving data 203 Unit for controlling display 204 Input receiving unit 205 Unit for storing target vehicles 206 Unit for sending commands

Claims

[1] Terminal device (200) comprising: a capture data receiving unit (202) that receives first capture data at recurring time intervals captured by a single image capture device (125, 1251) included in a work vehicle (100) and capable of generating 3D data; a display unit (RP) that displays the first recording data received by the recording data receiving unit (202) at recurring time intervals; and a unit (206) for transmitting a capture command which, in response to a check of the first capture data displayed on the display unit (RP), transmits a capture command which causes the work vehicle (100) to capture second capture data which are recorded with a plurality of image recording devices (125, 1251, 1252, 1253, 1254) contained in the work vehicle (100) and serve to generate 3D data, wherein the check comprises determining whether a time width between a most recent recording time of the first recording data and a closest recording time of the second recording data is shorter than a predetermined time. [2] The terminal device (200) of claim 1, further comprising: an input receiving unit (204) that accepts an input of a trigger pulse for sending a capture command after the first capture data is displayed, wherein the detection command sending unit (206) sends the detection command when the trigger pulse for transmission is input. [3] The terminal device (200) according to claim 1 or 2, further comprising: a pickup command sending unit (206) that sends a pickup command including identification information of a work vehicle (100), wherein the recording data receiving unit (202) receives the first recording data from the work vehicle (100) represented by the identification information included in the recording command. [4] The terminal device (200) according to any one of claims 1 to 3, further comprising: a 3D data receiving unit (202) that receives 3D data generated on the basis of image data captured by the image capture device (125), wherein the display unit (RP) displays the 3D data received by the 3D data receiving unit (202). [5] The terminal device (200) according to any one of claims 1 to 4, wherein the display unit (RP) displays measurement area information representing an area used to generate the 3D data together with the first acquisition data. [6] Terminal device (200) according to one of claims 1 to 5, wherein the image pickup device consists of a plurality of image pickup devices (125), and the first recording data is recording data recorded with an image recording device (125) of the plurality of image recording devices (125). [7] Image acquisition system (1) comprising: a work vehicle (100); and a terminal device (200), wherein the work vehicle (100) contains: a vehicle body (120); an image pickup device comprising a plurality of image pickup devices (125, 1251, 1252, 1253, 1254), including a single image pickup device (125, 1251) included in the vehicle body (120) and capable of acquiring information about a depth direction; a capture data transmission unit (1266) that transmits first capture data captured by the single image capture device (125, 1251) to the terminal device (200) at recurring time intervals; a capture command receiving unit (1262) that receives a command to capture second acquisition data used to generate 3D data; and a capture data acquisition unit (1263) that acquires the second capture data according to the acquisition command from the plurality of image pickup devices (125, 1251, 1252, 1253, 1254), and wherein the terminal device (200) includes: a capture data receiving unit (202) that receives the first capture data captured by the single image capture device (125, 1251) at recurring time intervals from the work vehicle (100); a display unit (RP) that displays the first recording data received by the recording data receiving unit (202) at recurring time intervals; and a unit (206) for transmitting a capture command which, in response to a check of the first capture data displayed on the display unit (RP), transmits a capture command causing the work vehicle (100) to capture the second capture data captured by the plurality of image capture devices (125, 1251, 1252, 1253, 1254) included in the work vehicle (100), wherein the check comprises determining whether a time width between a most recent recording time of the first recording data and a closest recording time of the second recording data is shorter than a predetermined time. [8] The image pickup system (1) according to claim 7, wherein the work vehicle (100) further includes a 3D data generating unit (1265) that generates 3D data using the second pickup data. [9] Image acquisition method that includes: Receiving, at recurring time intervals, first image data acquired with a single image acquisition device (125, 1251) included in a work vehicle (100) and capable of acquiring information about a depth direction; Displaying the received first recording data at recurring time intervals on a terminal device (200); and Sending a capture command, in response to a check of the first capture data displayed on the terminal device (200), which causes the work vehicle (100) to capture second capture data, which are captured by a plurality of image capture devices (125, 1251, 1252, 1253, 1254) included in the work vehicle (100) and used to generate 3D data, wherein the check comprises determining whether a time width between a most recent recording time of the first recording data and a closest recording time of the second recording data is shorter than a predetermined time. [10] Control device (126) comprising: a capture data transmission unit (1266) that transmits, at recurring time intervals, first capture data captured by a single image capture device (125, 1251) included in a work vehicle (100) and capable of acquiring information about a depth direction to a terminal device (200); a capture command receiving unit (1262) that receives a command to capture second acquisition data used to generate 3D data; and a unit (1263) for acquiring image data which, in response to a check of the first image data displayed on the terminal device (200), acquires the second image data according to the acquisition command from a plurality of image acquisition devices (125, 1251, 1252, 1253, 1254), the second image data being used to generate 3D data, and wherein the check comprises determining whether a time width between a most recent recording time of the first recording data and a closest recording time of the second recording data is shorter than a predetermined time. [11] Device (300) for integrating data, comprising: a 3D data acquisition unit (306) that acquires a plurality of pieces of 3D data generated on the basis of acquisition data acquired by performing an image acquisition of a construction site using an image acquisition device (125) included in a work vehicle (100), each piece of the plurality of pieces of 3D data being associated with an indication of an acquisition time at which the image of the respective piece of 3D data was acquired; a command receiving unit (307) that receives an integration command that selects a portion of the plurality of pieces of 3D data for integration; and a 3D data integrating unit (306) that determines whether a time width between a most recent acquisition time and a closest acquisition time of the selected part of the plurality of pieces of 3D data is shorter than a predetermined time, and generates integrated 3D data when the time width is shorter than the predetermined time by integrating the selected part of the plurality of pieces of 3D data acquired by the 3D data acquiring unit (306). [12] The data integrating device (300) according to claim 11, wherein, when a plurality of pieces of common 3D data, which are a plurality of pieces of 3D data having height information for the same plane coordinates, are included in the plurality of pieces of 3D data, the 3D data integrating unit (306) calculates an average value of heights related to the plane coordinates in the common 3D data and generates the integrated 3D data using the average value. [13] The data integrating device (300) according to claim 11, wherein, when a plurality of pieces of common 3D data, which are a plurality of pieces of 3D data having height information for the same plane coordinates, are included in the plurality of pieces of 3D data, the 3D data integrating unit (306) generates the integrated 3D data using a height related to the latest common 3D data among the plurality of pieces of common 3D data. [14] The data integrating device (300) according to any one of claims 11 to 13, wherein the 3D data integrating unit (306) generates the integrated 3D data by integrating the plurality of pieces of 3D data generated based on the image data acquired by the image acquisition devices (125) included in a plurality of work vehicles (100). [15] Work vehicle (100), which includes: a vehicle body (120); an image pickup device (125) contained in the vehicle body (120); a 3D data generation unit (1265) that generates a plurality of pieces of 3D data based on a plurality of pieces of image data acquired by the image acquisition device (125), wherein each piece of the plurality of pieces of 3D data is associated with an indication of an acquisition time at which the image of the respective piece of 3D data was acquired; and a 3D data integration unit (306) that determines whether a time width between a most recent acquisition time and a closest acquisition time of the plurality of pieces of 3D data is shorter than a predetermined time, and generates integrated 3D data if the time width is shorter than the predetermined time by integrating a plurality of pieces of 3D data generated by the 3D data generation unit (1265). [16] Image acquisition system (1) comprising: a work vehicle (100); and a device (300) for integrating data, wherein the work vehicle (100) contains: a vehicle body (120); an image pickup device (125) contained in the vehicle body (120); a 3D data generating unit (1265) that generates a plurality of 3D data elements based on a plurality of elements of image data acquired by the image acquisition device (125), each element of the plurality of 3D data elements being linked to an indication of an acquisition time at which the image of the respective 3D data element was acquired; and a data transmission unit (1266) that transmits the plurality of pieces of 3D data to the data integrating device (300), and wherein the device (300) for integrating data includes: a data receiving unit (304) that receives the plurality of pieces of 3D data from the work vehicle (100); and a 3D data integrating unit (306) that determines whether a time width between a most recent acquisition time and a closest acquisition time of the plurality of pieces of 3D data is shorter than a predetermined time, and generates integrated 3D data when the time width is shorter than the predetermined time by integrating the plurality of pieces of 3D data received by the data receiving unit (304).

Images