Information processing system and information processing method

Abstract

An information processing system includes: a first processor configured to receive position information indicating a position of a construction vehicle located within a construction site; receiving local image data of an area around the building vehicle, where the area around the building vehicle includes a portion of the construction site less than the entire construction site; generating local three-dimensional (3D) information based on the location information and the local image data; and outputting the local 3D information to a server, and a server wherein the server stores global 3D information of the construction site, the global 3D information including a topography of the entire construction site, and wherein the server is configured to update the global 3D information of the construction site based on the local 3D information output by the first processor.

Description

[0001] Related Applications

[0002] This application claims the benefit of Japanese Priority Patent Application JP 2022-081460 filed May 18, 2022, which is hereby incorporated by reference herein in its entirety. Technical Field

[0003] The present disclosure relates to an information processing system and an information processing method. Background Art

[0004] In order to realize digital conversion (DX) of the construction state of a construction site, etc., a system for generating three-dimensional information related to the topography of a construction area or the shape of an arranged object has been proposed. For example, the proposed system includes a system in which each control device provided in a plurality of construction vehicles generates three-dimensional information (three-dimensional data) from imaging data and transmits the generated three-dimensional data to a server device, and the server device integrates a plurality of three-dimensional data with each other (for example, refer to Patent Document 1). In this system, the server device instructs the control device on an imaging range to generate desired three-dimensional data.

[0005] [Citation List]

[0006] [Patent Document]

[0007] [Patent Document 1] JP 2017-186870 A Summary of the invention

[0008] [Technical issues]

[0009] However, in the above-mentioned known technology, since each control device generates three-dimensional information separately, there is a problem that the processing becomes complicated.

[0010] Therefore, the present disclosure proposes an information processing system and an information processing method that simplify the generation of three-dimensional information of a building area.

[0011] [Solution to the problem]

[0012] The information processing system according to the present disclosure includes: a server device, which maintains three-dimensional information of the entire building area in the three-dimensional information of the terrain of the building area including the shape of the arranged object, and updates the maintained three-dimensional information of the entire building area based on local three-dimensional information, the local three-dimensional information being three-dimensional information of the area where construction work is carried out by construction devices in the building area; a first information processing device, including a three-dimensional information generating unit and a three-dimensional information sending unit, the three-dimensional information generating unit generates local three-dimensional information based on position information of the construction devices and an image of the building area, and the three-dimensional information sending unit sends the generated local three-dimensional information to the server device; and a second information processing device, which generates position information and sends the generated position information to the first information processing device.

[0013] The information processing method according to the present disclosure includes: maintaining three-dimensional information of the entire building area in the three-dimensional information of the terrain of the building area including the shape of the arranged object; generating position information of the construction equipment performing construction work in the building area; generating local three-dimensional information based on the position information and the image of the building area, the local three-dimensional information being the three-dimensional information of the area where the construction work is performed by the construction equipment; and updating the maintained three-dimensional information of the entire building area based on the generated local three-dimensional information.

[0014] An information processing system includes: a first processor configured to: receive position information indicating the position of a construction vehicle located within a construction site; receive local image data of an area around the construction vehicle, wherein the area around the construction vehicle includes a portion that is smaller than the entire construction site; generate local three-dimensional (3D) information based on the position information and the local image data; and output the local 3D information to a server; and a server, wherein the server stores global 3D information of the construction site, the global 3D information including the terrain of the entire construction site, and wherein the server is configured to update the global 3D information of the construction site based on the local 3D information output by the first processor.

[0015] A method for updating global three-dimensional (3D) information of a construction site, comprising: receiving position information indicating the position of a construction vehicle located within the construction site; receiving local image data of an area around the construction vehicle, wherein the area around the construction vehicle includes a portion that is less than the entire construction site; generating local 3D information based on the position information and the local image data; sending the local 3D information to a server; and updating global 3D information of the construction site based on the local 3D information sent to the server, the global 3D information including a terrain of the entire construction site stored by the server.

[0016] At least one non-transitory computer-readable medium has instructions encoded thereon, which when executed by at least one processor cause the at least one processor to perform a method for updating global three-dimensional (3D) information of a construction site, the method comprising: receiving position information indicating the position of a construction vehicle located within the construction site; receiving local image data of an area around the construction vehicle, wherein the area around the construction vehicle includes a portion that is less than the entire construction site; generating local 3D information based on the position information and the local image data; sending the local 3D information to a server; and updating the global 3D information of the construction site based on the local 3D information sent to the server, the global 3D information including a topography of the entire construction site stored by the server. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1A is a diagram showing a configuration example of an information processing system according to the first embodiment of the present disclosure.

[0018] Figure 1B is a diagram showing a configuration example of an information processing system according to the first embodiment of the present disclosure.

[0019] Figure 2 is a diagram showing a configuration example of a third information processing device according to the first embodiment of the present disclosure;

[0020] Figure 3 : is a diagram showing a configuration example of a second information processing device according to the first embodiment of the present disclosure.

[0021] Figure 4 is a diagram showing a configuration example of an information processing device according to the first embodiment of the present disclosure.

[0022] Figure 5A : is a diagram illustrating an example of processing of construction project information and a construction area according to the first embodiment of the present disclosure.

[0023] Figure 5B : is a diagram illustrating an example of processing of construction project information and a construction area according to the first embodiment of the present disclosure.

[0024] Figure 5C : is a diagram illustrating an example of processing of construction project information and a construction area according to the first embodiment of the present disclosure.

[0025] Figure 6 is a diagram showing a configuration example of a server device according to the first embodiment of the present disclosure.

[0026] Figure 7 : is a diagram illustrating an example of a processing procedure in a process of the third information processing apparatus according to the first embodiment of the present disclosure.

[0027] Figure 8 : is a diagram illustrating an example of a processing procedure of a process of the second information processing apparatus according to the first embodiment of the present disclosure.

[0028] Fig. 9 : is a diagram illustrating an example of a processing procedure in the processing of the information processing device according to the first embodiment of the present disclosure.

[0029] Fig.10 : is a diagram illustrating an example of a processing procedure regarding a process of the server device according to the first embodiment of the present disclosure.

[0030] Fig.11 : is a diagram illustrating an example of a processing procedure in a process of the server device according to the first embodiment of the present disclosure.

[0031] Fig.12 : is a diagram illustrating an example of generation of three-dimensional information according to the first embodiment of the present disclosure.

[0032] Fig.13 is a diagram illustrating a configuration example of an information processing device according to a second embodiment of the present disclosure.

[0033] Fig.14 : is a diagram showing a configuration example of a second information processing device according to the third embodiment of the present disclosure.

[0034] Fig.15 : is a diagram illustrating an example of a processing procedure regarding a second information processing device according to a third embodiment of the present disclosure.

[0035] Fig.16 : is a diagram illustrating an example of a processing procedure of an information processing device according to a third embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. The description will be given in the following order. It should be noted that in each of the following embodiments, the same parts are represented by the same reference symbols, and their repeated description will be omitted.

[0037] 1. First Implementation

[0038] 2. Second Implementation

[0039] 3. Third Implementation

[0040] 4. Modifications

[0041] (1. First Embodiment)

[0042] <Configuration of Information Processing System>

[0043] Figure 1A and Figure 1B is a diagram showing a configuration example of an information processing system according to the first embodiment of the present disclosure. Figure 1A is a diagram showing a configuration example of an information processing system 10 of the present invention. The information processing system 10 generates three-dimensional information of a topography of a construction area including the shapes of objects arranged at a construction site, etc. The three-dimensional information can be formed using point cloud data, for example. The information processing system 10 includes a first information processing device 200, a plurality of second information processing devices 400, a third information processing device 100, a server device 300, and a display terminal 500. The information processing system 10 in the accompanying drawings shows an example including a second information processing device 400a and second information processing devices 400b to 400c.

[0044] The third information processing device 100 generates an image of the entire building area. For example, the third information processing device 100 is installed on a device such as a drone, and shoots the building area from above, thereby generating an image of the entire building area. The third information processing device 100 records the generated image and the position information at the time of shooting. During or after the drone flight, the image and the position information are provided to the first information processing device 200 set on the ground through wireless communication, wired communication, or a recording medium such as a semiconductor memory.

[0045] The second information processing device 400 is installed on a construction device performing construction work in a construction area and records location information during the construction work. During or after the construction work, the location information is supplied to the first information processing device 200 via wireless communication, wired communication, or a recording medium such as a semiconductor memory.

[0046] The first information processing device 200 generates and records 3D information of the entire building area or a part of the building area based on the image and position information provided by the third information processing device 100 and the position information provided by the second information processing device 400. The first information processing device 200 sends the recorded 3D information to the server device 300.

[0047] The server device 300 holds the three-dimensional information from the first information processing device 200 and updates the held three-dimensional information. The server device 300 transmits all or part of the three-dimensional information to the display terminal 500 in response to a request from the display terminal 500.

[0048] Based on the user's operation, the display terminal 500 displays the three-dimensional information on the display device installed on the terminal. The display terminal 500 obtains the three-dimensional information of the entire construction area or a part of the construction area from the server device 300 via wireless communication or wired communication, and displays the obtained three-dimensional information. The user can use the display terminal 500 to confirm the progress of the construction project and can use tasks such as budget management.

[0049] Figure 1B 1 is a diagram showing a construction site. The accompanying drawings are diagrams showing the configuration of a second information processing device 400 and a third information processing device 100 in a construction site 1. Second information processing devices 400a and 400b are respectively arranged in construction vehicles 2 and 3. Construction vehicles 2 and 3 perform civil engineering work and the like in a construction area 20. Construction vehicles 2 and 3 are excavators and bulldozers, respectively. Construction vehicles 2 and 3 are examples of the above-mentioned construction devices. Construction vehicles 2 and 3 have a power unit and a control unit. The control unit performs control of driving and construction work on a route indicated by a user. Construction work by construction vehicle 2 and the like causes a change in the terrain of the construction area 20. Construction vehicles 2 and 3 may also be referred to herein as construction engineering vehicles.

[0050] The third information processing device 100 is provided in the drone 4. The drone 4 flies above the building area 20 and generates an image of the entire building area 20. The drone 4 includes a power unit and a control unit, which are dedicated to flight. The control unit performs control to achieve flight at a height and along a route indicated by a user. In addition, the third information processing device 100 can also be configured in place of the drone 4 and at a position where the entire building area can be ignored, such as on the roof of a high-rise building.

[0051] <Configuration of Third Information Processing Device>

[0052] Figure 2 1 is a diagram showing a configuration example of a third information processing device according to the first embodiment of the present disclosure; the accompanying drawings are block diagrams showing a configuration example of a third information processing device 100. The third information processing device 100 includes a camera 101, an image data generating unit 102, a satellite positioning unit 103, a position information generating unit 104, an inertial measurement unit 105, a posture information generating unit 106, a shooting information integrating unit 107, and a shooting information retaining unit 108. In addition, the third information processing device 100 also includes a transmission unit 109 and a control unit 110.

[0053] The camera 101 photographs the building area 20. The camera 101 includes an imaging element such as a complementary metal oxide semiconductor (CMOS) image sensor, and photographs under the control of the control unit 110.

[0054] The image data generating unit 102 generates an image based on the photographing result of the camera 101. The image data generating unit 102 generates an image of the entire building area 20. The image data generating unit 102 outputs the generated image to the photographing information integrating unit 107.

[0055] The satellite positioning unit 103 acquires a position (latitude, longitude, and altitude). The satellite positioning unit 103 includes, for example, a global navigation satellite system (GNSS), and acquires a position under the control of the control unit 110. The satellite positioning unit 103 outputs the acquired position to the position information generation unit 104.

[0056] The position information generation unit 104 generates position information, which is information about the position of the third information processing device 100, based on the position acquired by the satellite positioning unit 103. The position information generation unit 104 outputs the generated position information to the photographing information integration unit 107.

[0057] The inertial measurement unit 105 acquires the posture of the camera 101. The inertial measurement unit 105 includes, for example, an inertial measurement unit (IMU), and acquires yaw, pitch, and roll each indicating a three-dimensional rotation amount under the control of the control unit 110. The inertial measurement unit 105 outputs the acquired posture to the posture information generation unit 106.

[0058] The posture information generation unit 106 generates posture information of the camera 101 based on the posture acquired by the inertial measurement unit 105. The posture information generation unit 106 outputs the generated posture to the shooting information integration unit 107.

[0059] The shooting information integration unit 107 generates shooting information including shot image data, position information, posture information, and time information indicating synchronization of each piece of data. The shooting information may include, for example, image data, position information, and posture information of one frame as one file. The shooting information integration unit 107 outputs the generated shooting information to the shooting information holding unit 108.

[0060] The photographing information holding unit 108 holds the photographing information generated by the photographing information integration unit 107. The photographing information holding unit 108 includes a recording medium such as a semiconductor memory or a hard disk, and can hold photographing information generated during the flight of the drone 4.

[0061] The transmission unit 109 sends the photographing information held in the photographing information holding unit 108 to the first information processing device 200. After the drone 4 flies or lands, the transmission unit 109 can read the photographing information held in the photographing information holding unit 108, and transmit the read photographing information to the first information processing device 200 through wired communication or wireless communication.

[0062] The control unit 110 performs overall control of the third information processing apparatus 100 .

[0063] <Configuration of Second Information Processing Device>

[0064] Figure 3 1 is a diagram showing a configuration example of a second information processing device according to a first embodiment of the present invention. 1 is a block diagram showing a configuration example of a second information processing device 400. The second information processing device 400 includes a satellite positioning unit 401, a position information generating unit 402, a vehicle state information acquiring unit 403, a construction project information generating unit 404, a transmission unit 405, and a control unit 406.

[0065] The satellite positioning unit 401 acquires a position (latitude, longitude, and altitude) similarly to the satellite positioning unit 103 . The satellite positioning unit 401 outputs the acquired position to the position information generation unit 402 .

[0066] Similar to the position information generating unit 104, the position information generating unit 402 generates position information, which is information about the position of the second information processing device 400, based on the position acquired by the satellite positioning unit 401. The position information generating unit 402 outputs the generated position information to the construction project information generating unit 404.

[0067] The vehicle state information acquisition unit 403 acquires vehicle state information indicating the operating state of the construction vehicle 2. The operating state indicates the state of the construction vehicle 2, such as driving, stopping, and working. In an excavator, the operating state of a bucket or a dipper arm corresponds to the working state. In a bulldozer, the operating state of a shovel corresponds to the working state. Of course, by acquiring construction project information only on the working state of the construction vehicle 2, it is also possible to acquire construction project information when the terrain shape changes. The vehicle state information acquisition unit 403 outputs the acquired vehicle state information to the construction project information generation unit 404.

[0068] The construction work information generating unit 404 generates and holds information obtained by adding the position information and the time information synchronized with the vehicle state information as construction work information.

[0069] The transmission unit 405 transmits the construction project information to the first information processing device 200. The transmission unit 405 can transmit the construction project information stored in the construction project information generation unit 404 during driving or after the construction project of the construction vehicle 2 is completed.

[0070] The control unit 406 performs overall control of the second information processing device 400. In addition, the control unit 406 controls the satellite positioning unit 401 to acquire the position, and controls the vehicle state information acquisition unit 403 to acquire the vehicle state information. The control unit 406 can perform these controls when the receiving unit (not shown) receives an acquisition instruction signal. The acquisition instruction signal can be, for example, a signal that is repeated at a predetermined interval. In this case, the vehicle state information is acquired at a predetermined interval, and the construction project information is generated by the construction project information generation unit 404.

[0071] <Configuration of Information Processing Device>

[0072] Figure 4 1 is a diagram showing a configuration example of a first information processing device according to a first embodiment of the present disclosure. The accompanying figure is a block diagram showing a configuration example of a first information processing device 200. The first information processing device 200 includes a shooting information receiving unit 201, an image frame selecting unit 202, a construction project information receiving unit 205, a construction project information integrating unit 206, a building area determining unit 207, a three-dimensional information generating unit 203, a three-dimensional information transmitting unit 204, and a control unit 208.

[0073] The photographing information receiving unit 201 receives photographing information from the third information processing apparatus 100. The photographing information receiving unit 201 outputs the received photographing information to the image frame selecting unit 202.

[0074] The construction project information receiving unit 205 receives construction project information from the second information processing device 400. The construction project information receiving unit 205 outputs the received construction project information to the construction project information synthesizing unit 206. When there is one second information processing device 400, the construction project information receiving unit 205 outputs the construction project information to the construction area determining unit 207.

[0075] The construction project information integration unit 206 integrates the construction project information. The construction project information integration unit 206 stores the construction project information respectively transmitted from the plurality of second information processing devices 400. When all the construction project information is obtained, the construction project information integration unit 206 outputs the construction project information to the building area determination unit 207.

[0076] The construction area determination unit 207 determines a construction area where construction work is performed by the construction vehicle 2 or the like from the construction area 20. The construction area corresponds to an area having a terrain shape changed by the construction vehicle 2 or the like. The construction area determination unit 207 can determine the area around the construction vehicle 2 or the like as the construction area where construction work is performed. That is, the construction area determination unit 207 can detect the area around the construction vehicle 2 or the like based on the position information included in the construction work information, and output it as the construction area. When outputting the construction area, the construction area determination unit 207 adds the position information to the construction area.

[0077] When the construction areas of a plurality of construction vehicles are close to each other or overlap each other, the construction area determination unit 207 may determine one area combining the plurality of construction areas as the construction area. On the other hand, when the construction area is separated from other construction areas, the construction area determination unit 207 determines the construction areas as separate construction areas. The construction area determination unit 207 notifies the construction area of ​​the image frame selection unit 202 of the determination result including its position information.

[0078] In addition, the construction area determination unit 207 may also narrow the construction area whose terrain shape changes due to the construction work of the construction vehicle 2, etc., based on the vehicle state information included in the construction project information. Specifically, the construction area determination unit 207 may detect an area in which the vehicle state information of the construction vehicle 2, etc. indicates the construction state in the construction area based on the above-mentioned position information, and may output the detected area to the image frame selection unit 202.

[0079] Based on the building area output from the building area determination unit 207, the image frame selection unit 202 selects the photographic information required for generating the three-dimensional information from the plurality of photographic information provided by the photographic information receiving unit 201. For example, when the position information included in the received photographic information is included in the range of the building area notified from the building area determination unit 207, the image frame selection unit 202 selects the image frame including the photographic information as the photographic information for generating the three-dimensional information of the building area. Here, the image frame is an image of an area obtained by dividing the image of the entire building area 20. The image frame selection unit 202 provides the selected image frame to the three-dimensional information generating unit 203.

[0080] On the other hand, when the building area is not notified from the building area determination unit 207, the image frame selection unit 202 selects all image frames and provides the selected image frames to the three-dimensional information generation unit 203. For example, when the building area is photographed for the first time, three-dimensional information (also referred to as global three-dimensional information herein) is generated using all photographing information acquired by the third information processing device 100. In this case, since the building area is not notified by the building area determination unit 207, the image frame selection unit 202 selects all image frames and provides the selected image frames to the three-dimensional information generation unit 203.

[0081] The three-dimensional information generating unit 203 generates local three-dimensional information based on the construction project information and the image of the construction area, and the local three-dimensional information is three-dimensional information of the area where the construction project is performed by the construction vehicle 2 or the like that performs the construction project in the construction area 20. Specifically, the three-dimensional information generating unit 203 generates local three-dimensional information from the shooting information provided by the image frame selecting unit 202. When the construction area is first shot, the three-dimensional information generating unit 203 generates global three-dimensional information of the entire construction area 20, rather than local three-dimensional information. The three-dimensional information generating unit 203 can generate three-dimensional information based on point cloud data. The point cloud data can be generated by, for example, the structure from motion (SfM) method. The three-dimensional information generating unit 203 outputs the generated global three-dimensional information of the entire construction area 20 and the generated local three-dimensional information to the three-dimensional information transmitting unit 204.

[0082] In this way, the three-dimensional information generating unit 203 can generate three-dimensional information specifically for the building area 20. In addition, the three-dimensional information generating unit 203 first generates global three-dimensional information of the entire building area 20, and then generates local three-dimensional information. The local three-dimensional information is an area having a terrain shape that is highly likely to be changed by the construction vehicle 2 or the like. This makes it possible to reduce the processing time compared to the case of generating the three-dimensional information of the entire building area 20.

[0083] The three-dimensional information transmission unit 204 transmits the three-dimensional information to the server device 300. The three-dimensional information transmission unit 204 may transmit the three-dimensional information through wired communication or wireless communication.

[0084] The control unit 208 performs overall control of the first information processing apparatus 200 .

[0085] <Construction project information and handling of construction areas>

[0086] FIG. 5A to FIG. 5C 1 is a diagram showing an example of processing of construction project information and building areas according to the first embodiment of the present disclosure. This figure is a diagram showing processing of the construction project information integration unit 206 and the building area determination unit 207.

[0087] Figure 5A 2 is a diagram showing processing performed by the construction project information integration unit 206. The figure is a diagram showing an example in which three construction vehicles (AC) perform construction works on the construction area 20 in the areas 41 to 43, respectively. The construction project information integration unit 206 waits until construction project information about the areas 41 to 43 is transmitted from the second information processing device 400 provided in each construction vehicle. Upon receiving the construction project information of the areas 41 to 43, the construction project information integration unit 206 outputs the construction project information to the construction area determination unit 207.

[0088] Figure 5B is a diagram showing an example of determination of a building area by the building area determination unit 207. As described above, the building area determination unit 207 combines building areas that overlap or are close to each other. In the figure, the building area determination unit 207 combines the area 41 and the area 42 that are close to each other, and outputs a building area 45 (dashed rectangle) including the combined areas as a determination result. In addition, the building area determination unit 207 outputs a building area 46 (dashed circle) including the area 43 separated from other areas as a determination result. Although the building area can use any shape, forming the building area into a rectangular shape or a circular shape makes it possible to simplify the inside-outside determination processing.

[0089] Figure 5C is a diagram showing an example of image frame selection by the image frame selection unit 202. This shows building areas 45 and 46 in the image 30 representing the entire building area 20. The image 30 is divided into a plurality of image frames 31. The image frame selection unit 202 selects the image frame 31 including the building areas 45 and 46. The shaded area in the figure represents the area of ​​the image frame 31 selected by the image frame selection unit 202.

[0090] <Configuration of server device>

[0091] Figure 6 1 is a block diagram showing a configuration example of a server device 300 according to a first embodiment of the present disclosure. The server device 300 includes a three-dimensional information receiving unit 301, a three-dimensional information updating unit 302, a three-dimensional information holding unit 303, a transmission unit 304, a data request receiving unit 305, and a control unit 306.

[0092] The three-dimensional information receiving unit 301 receives three-dimensional information from the first information processing apparatus 200 . The three-dimensional information receiving unit 301 outputs the received three-dimensional information to the three-dimensional information updating unit 302 .

[0093] The three-dimensional information updating unit 302 updates the three-dimensional information maintained in the three-dimensional information maintaining unit 303 described below. The updating is a process of referring to the three-dimensional information maintained in the three-dimensional information maintaining unit 303 and replacing the past three-dimensional information with the newly received three-dimensional information. The updating process can be performed based on the position information included in the received three-dimensional information. Specifically, when there is past three-dimensional information at the position indicated by the position information included in the received three-dimensional information, the updating process can be performed. On the contrary, when there is no past three-dimensional information at the position indicated by the received three-dimensional information, the three-dimensional information updating unit 302 performs the process of adding the received three-dimensional information. This makes it possible to keep the three-dimensional information of the entire building area 20 in the latest state.

[0094] The three-dimensional information holding unit 303 holds the three-dimensional information of the entire building area 20. The three-dimensional information holding unit 303 adds or updates the held three-dimensional information under the control of the three-dimensional information updating unit 302. For example, when the three-dimensional information is not held, such as when photographing the building area for the first time, all the three-dimensional information is recorded and held.

[0095] The data request receiving unit 305 receives a three-dimensional information request from the display terminal 500. Based on the received three-dimensional information request, the data request receiving unit 305 instructs the three-dimensional information holding unit 303 to output the three-dimensional information.

[0096] The transmission unit 304 transmits the three-dimensional information output from the three-dimensional information holding unit 303 to the display terminal 500 .

[0097] The control unit 306 performs overall control of the server device 300 .

[0098] When the data request receiving unit 305 has received a request for three-dimensional information from the display terminal 500, the control unit 306 performs control to read the three-dimensional information about the requested position from the three-dimensional information holding unit 303. The read three-dimensional information is provided to the transmission unit 304. The transmission unit 304 transmits the three-dimensional information provided from the three-dimensional information holding unit 303 to the display terminal 500 via wired communication or wireless communication. This makes it possible to browse the three-dimensional information about the entire building area or a specific position on the display terminal 500.

[0099] <Processing by the Third Information Processing Device>

[0100] Figure 7 100 is a diagram showing an example of a processing procedure regarding the third information processing apparatus according to the first embodiment of the present disclosure.

[0101] When the control unit 110 of the third information processing device 100 receives a shooting instruction indicating the moment to obtain shooting information (step S101), the camera 101 shoots, and the image data generation unit 102 generates image data (step S102). At this time, the amount of data can be reduced by encoding the image data. Next, the satellite positioning unit 103 acquires the position, and the position information generation unit 104 generates the position information (step S103). Here, the position information includes, for example, latitude, longitude, and altitude. Subsequently, the inertial measurement unit 105 acquires the attitude, and the attitude information generation unit 106 generates attitude information (step S104). Here, the attitude information includes, for example, the rotation vectors of the three axes calculated based on the acceleration and angular velocity of the drone 4.

[0102] Next, the shooting information integration unit 107 integrates the image data, the position information, and the posture information to generate shooting information (step S105). Next, the shooting information holding unit 108 holds the shooting information in a medium or the like (step S106). Next, the control unit 110 determines whether the shooting instruction has ended (step S107). As a result, if the shooting instruction has not ended (step S107, no), the control unit 110 returns to the processing of step S102.

[0103] On the contrary, in step S107, in the case where the shooting instruction ends (step S107, yes), the transmission unit 109 sends the shooting information to the first information processing device 200 (step S108), and ends the processing. Note that the timing of sending the shooting information to the first information processing device 200 can be set to, for example, the timing immediately after the shooting information is generated in step S105. In this case, the shooting information is transmitted to the stationary station during the flight of the drone 4, so that the need to transmit the shooting information after landing can be eliminated. Note that the processing in steps S102 to S104 is not limited to the order in the figure.

[0104] <Processing by Second Information Processing Device>

[0105] Figure 8 1 is a diagram showing an example of a processing procedure of the second information processing device according to the first embodiment of the present disclosure.

[0106] When the control unit 406 of the second information processing device 400 receives an acquisition instruction indicating the acquisition time of the construction project information (step S121), the satellite positioning unit 401 acquires the position, and the position information generation unit 402 generates the position information (step S123). Here, the position information includes, for example, latitude, longitude, and altitude. Subsequently, the vehicle status information acquisition unit 403 acquires the vehicle status information (step S124). Here, the vehicle status information corresponds to, for example, information indicating whether the construction vehicle 2, etc. is traveling or stopped, information indicating whether the bucket or shovel provided in the construction vehicle 2, etc. is operating or stopped, etc. This information can also be acquired by automatic recognition of a separately installed camera.

[0107] Next, the construction project information generation unit 404 generates construction project information (step S125). At this time, the construction project information generation unit 404 generates construction project information representing an area where substantial construction projects accompanied by changes in terrain shape have been carried out based on the position information and the vehicle state information, which is also referred to as operation information. For example, the construction project information generation unit 404 generates construction project information using the position information of the operating position of the bucket and the shovel by excluding the position information of the bucket, the shovel, etc. of the construction vehicle 2 when they are stopped, thereby generating construction project information representing a construction area where the terrain shape has changed. Next, the control unit 406 determines whether the acquisition instruction is completed (step S126). As a result, if the acquisition instruction is not completed (step S126, no), the control unit 406 returns to the processing of step S123.

[0108] On the other hand, in step S126, when the acquisition instruction is completed (step S126: Yes), the transmission unit 405 sends the construction project information to the first information processing device 200 (step S127), and the processing ends. In addition, the time of sending the construction project information to the first information processing device 200 can be set to, for example, the time just after the construction project information is generated in step S125. In this case, during the construction project, the construction project information is sent to the first information processing device 200, which can eliminate the necessity of sending the construction project information after the construction project is completed. Note that the processing in steps S123 and S124 is not limited to the order in the drawings.

[0109] <Processing by Information Processing Device>

[0110] Fig. 9 1 is a diagram showing an example of a processing procedure regarding the first information processing device according to the first embodiment of the present disclosure.

[0111] First, the photographing information receiving unit 201 receives the photographing information (step S141). The received photographing information is recorded in the recording medium ( Figure 4 Note that in the case of acquiring the photographing information from an external recording medium, a reading process from the recording medium is performed, and in step S142, the read photographing information is recorded in the recording medium built into the first information processing apparatus 200.

[0112] Next, the construction project information receiving unit 205 receives the construction project information (step S143). Next, the construction project information is recorded in the above-mentioned recording medium (step S144). Next, the control unit 208 determines whether all the construction project information has been received (step S145). As a result, if all the construction project information has not been received (step S145: No), the control unit 208 returns to the process of step S143.

[0113] On the other hand, in the case where all the construction project information is received in step S145 (step S145: Yes), the construction area determination unit 207 determines the construction area (step S146). This determination obtains the construction area including the area where the terrain shape has changed by using all the construction project information. For example, when there are overlapping construction areas of a plurality of construction vehicles, one area combining the plurality of construction areas is determined as the construction area based on the position information included in the construction project information. On the contrary, when there are no overlapping construction areas, the construction area is determined as a single construction area.

[0114] Next, the image frame selection unit 202 selects an image frame included in the photographing information (step S147). That is, when the position information included in the photographing information is within the range of the building area, the image frame selection unit 202 selects the image frame corresponding to the position information as the image frame for generating three-dimensional information. Next, the three-dimensional information generation unit 203 generates three-dimensional information based on the selected image frame (step S149). This results in the generation of three-dimensional information of the entire building area 20 or local three-dimensional information of the building area 20.

[0115] Next, the generated three-dimensional information is recorded in the recording medium (step S150), and the three-dimensional information transmission unit 204 sends the three-dimensional information to the server device 300 (step S151). The timing of sending the three-dimensional information may be the timing after the generation process of the three-dimensional information in step S149 is completed for all the shooting information. Alternatively, the three-dimensional information may be sent each time the generation process of the three-dimensional information in step S149 is completed for a part of the shooting information.

[0116] <Processing by Server Device>

[0117] Fig.10 1 is a diagram showing an example of a processing procedure of the server device according to the first embodiment of the present disclosure. The accompanying drawing is a flowchart showing an example of a processing procedure when three-dimensional information is received in the server device 300.

[0118] First, the three-dimensional information receiving unit 301 receives the three-dimensional information (step S161). Next, the three-dimensional information updating unit 302 refers to the recorded data of the three-dimensional information holding unit 303 (step S162), and determines whether the past three-dimensional information of the same building area as the received three-dimensional information has been recorded. As a result, when the three-dimensional information of the same building area has not been recorded (step S163, no), the process proceeds to step S165. On the contrary, when the three-dimensional information of the same building area has been recorded (step S163, yes), the three-dimensional information updating unit 302 deletes the recorded data of the area (step S164), and enters the processing of step S165.

[0119] In step S165, the three-dimensional information updating unit 302 adds the three-dimensional information to the three-dimensional information holding unit 303 (step S165). As a result, the three-dimensional information of the entire building area 20 or the local three-dimensional information of the area is held in the three-dimensional information holding unit 303 (step S166). After that, the processing ends. Note that the processing of steps S164 and S165 described above updates the three-dimensional information.

[0120] Fig.11 1 is a diagram showing an example of a processing procedure in the processing of the server device according to the first embodiment of the present disclosure. The accompanying drawing is a flowchart showing an example of a processing procedure when a data request has been received in the server device 300.

[0121] When the data request receiving unit 305 has received a data request from the display terminal 500 (step S181, yes), the three-dimensional information corresponding to the requested building area is read from the three-dimensional information holding unit 303 (step S182). Next, the read three-dimensional information is sent through the transmission unit 304 (step S183). After that, the processing ends.

[0122] Fig.12 1 is a diagram showing an example of generation of three-dimensional information according to the first embodiment of the present disclosure. The accompanying drawings are sequence diagrams showing processes in the third information processing apparatus 100, the second information processing apparatus 400a, the second information processing apparatus 400b, the first information processing apparatus 200, the server apparatus 300, and the display terminal 500.

[0123] First, the third information processing device 100 shoots the building area (step S201). Next, the third information processing device 100 sends the shooting information to the first information processing device 200 (step S202). Next, based on the sent shooting information, the first information processing device 200 generates three-dimensional information of the entire building area 20 (step S203). Next, the first information processing device 200 sends the generated three-dimensional information to the server device 300 (step S204). Next, the server device 300 maintains the sent three-dimensional information (step S205). The three-dimensional information corresponds to the first three-dimensional information. Next, after the display terminal 500 sends the three-dimensional information request to the server device 300 (step S206), the server device 300 sends the three-dimensional information to the display terminal 500 (step S207).

[0124] On the other hand, the second information processing device 400a generates construction project information (step S208), and transmits the generated construction project information to the first information processing device 200 (step S209). In addition, the second information processing device 400b generates construction project information (step S210), and transmits the generated construction project information to the first information processing device 200 (step S211).

[0125] Thereafter, the third information processing device 100 captures an image of the building area again (step S212), and sends the captured information to the first information processing device 200 (step S213). Then, the first information processing device 200 generates local three-dimensional information (step S214), and sends the generated local three-dimensional information to the server device 300 (step S215). The server device 300 stores the received local three-dimensional information in the three-dimensional information storage unit 303 (step S216). At this time, the server device 300 updates the three-dimensional information stored in the three-dimensional information storage unit 303 with the local three-dimensional information.

[0126] Thereafter, the display terminal 500 sends a three-dimensional information request to the server device 300 at a specific time (step S217), and the server device 300 sends the three-dimensional information to the display terminal 500 (step S218).

[0127] The process in the accompanying drawings shows a case where three-dimensional information of the entire building area 20 is generated by photographing the building area at a first time, and the three-dimensional information of the area in the building area 20 where the terrain shape has actually changed is updated at a second time. However, the timing of generating the three-dimensional information of the entire building area 20 is not limited to the first time. For example, the three-dimensional information may be generated periodically (e.g., weekly or monthly), or may be generated after bad weather (e.g., rain or typhoon).

[0128] In this way, in the information processing system 10 according to the first embodiment of the present disclosure, the three-dimensional information generation unit 203 of the first information processing device 200 first generates three-dimensional information of the entire building area 20, and then generates local three-dimensional information specifically for the building area where the construction work has been carried out by the construction vehicle 2 and the like. By updating the three-dimensional information of the entire building area 20 based on the local three-dimensional information, the processing time can be reduced compared to the case where the three-dimensional information of the entire building area 20 is generated. In addition, since the local three-dimensional information is sent to the server device 300, the amount of communication data can be reduced, resulting in a reduction in communication time. In addition, by recording the local three-dimensional information, the history can be recorded with a smaller amount of data compared to the case where the three-dimensional information of the entire building area 20 is recorded. In addition, since the second information processing device 400 does not generate three-dimensional information, the processing of the second information processing device 400 can be simplified.

[0129] (2. Second Embodiment)

[0130] The first information processing device 200 according to the first embodiment transmits the generated three-dimensional information to the server device 300. In contrast, the first information processing device 200 according to the second embodiment of the present disclosure differs from the first embodiment in selecting three-dimensional information to be transmitted to the server device 300.

[0131] <Configuration of Information Processing Device>

[0132] Fig.13 1 is a diagram showing a configuration example of a first information processing device according to a second embodiment of the present disclosure. Figure 4 2 is a diagram showing a configuration example of the first information processing device 200. Figure 4 The first information processing device 200 in FIG. 1 is different in that it further includes a three-dimensional information holding unit 209 and a transmission data selecting unit 210 .

[0133] The three-dimensional information holding unit 209 records and accumulates the three-dimensional information generated by the three-dimensional information generating unit 203 , and holds the three-dimensional information of the entire building area 20 .

[0134] The transmission data selection unit 210 selects three-dimensional information to be transmitted to the server device 300. The transmission data selection unit 210 compares the local three-dimensional information generated by the three-dimensional information generation unit 203 with the three-dimensional information of the corresponding area of ​​the three-dimensional information held in the three-dimensional information holding unit 209, and selects the local three-dimensional information having a difference exceeding a predetermined threshold. The local three-dimensional information selected by the transmission data selection unit 210 is transmitted to the server device 300 via the three-dimensional information transmission unit 204. It should be noted that the transmission data selection unit 210 is an example of a selection unit.

[0135] In this way, by holding the three-dimensional information finally transmitted to the server device 300 in the three-dimensional information holding unit 209 , a difference with the three-dimensional information generated thereafter can be detected.

[0136] The configuration of the information processing system 10 other than this is similar to the configuration of the information processing system 10 according to the first embodiment of the present disclosure, and thus a description thereof will be omitted.

[0137] In this way, the information processing system 10 according to the second embodiment of the present disclosure holds the generated three-dimensional information, selects local three-dimensional information based on the held three-dimensional information, and transmits the local three-dimensional information to the server device 300. This makes it possible to further reduce data transmission processing.

[0138] (3. Third Embodiment)

[0139] In the information processing system 10 of the first embodiment described above, the second information processing device 400 transmits the construction project information including the location information to the first information processing device 200. In comparison, the information processing system 10 according to the third embodiment of the present disclosure is different from the first embodiment described above in that the second information processing device 400 further transmits the image to the first information processing device 200.

[0140] <Configuration of Second Information Processing Device>

[0141] Fig.14 : is a diagram showing a configuration example of a second information processing device according to a third embodiment of the present disclosure. Figure 3 4 is a diagram showing a configuration example of a second information processing device 400. Figure 3 The second information processing device 400 in FIG. 4 is different in that it also includes a camera 407 and an image data generating unit 408 .

[0142] Similar to the camera 101 of the third information processing device 100 , the camera 407 photographs the building area 20 .

[0143] Similar to the image data generating unit 102 of the third information processing device 100, the image data generating unit 408 generates an image based on the shooting result of the camera 407. The image data generating unit 408 generates an image of the surroundings of the construction vehicle 2, etc. The image data generating unit 408 outputs the data of the generated image to the construction project information generating unit 404.

[0144] The construction project information generating unit 404 in the figure generates and stores construction project information including acquired image data, position information, and vehicle status information, and additional information representing time information indicating synchronization of these information.

[0145] Note that the camera 407 and the image data generation unit 408 may also be formed by, for example, a general portable terminal or a tablet terminal. An image including time information (e.g., a Joint Photographic Experts Group (JPEG) file) acquired by these terminals may be included in the construction project information and transmitted to the first information processing device 200.

[0146] <Processing by Second Information Processing Device>

[0147] Fig.15 1 is a diagram showing an example of a processing procedure of a second information processing device according to a third embodiment of the present disclosure. Figure 8 resemblance.

[0148] When the control unit 406 of the second information processing device 400 receives an acquisition instruction indicating the acquisition time of the construction project information (step S121), the camera 407 performs shooting, and the image data generation unit 408 generates image data (step S122). In addition, the satellite positioning unit 401 acquires the position, and the position information generation unit 402 generates the position information (step S123). Subsequently, the vehicle state information acquisition unit 403 acquires the vehicle state information (step S124).

[0149] Next, the construction project information generation unit 404 generates construction project information (step S125). The construction project information generation unit 404 generates construction project information indicating an area where substantial construction projects accompanying changes in terrain shape have been performed based on the position information, vehicle state information, and image data. Next, the control unit 406 determines whether the acquisition instruction has ended (step S126). As a result, if the acquisition instruction has not ended (step S126, no), the control unit 406 returns to the process of step S122.

[0150] On the other hand, in step S126, when the acquisition instruction ends (step S126: Yes), the transmission unit 405 sends the construction project information including the image data to the first information processing device 200 (step S127), and ends the processing. In addition, the time of sending the construction project information to the first information processing device 200 can be set to, for example, the time just after the construction project information is generated in step S125. In this case, during the construction project, the construction project information is sent to the first information processing device 200, which can eliminate the necessity of sending the construction project information after the construction project ends. It should be noted that the processing in steps S122 to S124 is not limited to the order in the figure.

[0151] <Processing by Information Processing Device>

[0152] Fig.16 1 is a diagram showing an example of a process of the first information processing device according to the third embodiment of the present disclosure. 1 is a flowchart showing an example of a process of the first information processing device 200, similar to Fig. 9 The processing process in the figure is similar to Fig. 9 The difference in the processing procedure is that the processing of step S148 is added.

[0153] As described above, the construction project information received by the first information processing device 200 from the second information processing device 400 includes image data. In step S148, the image frame selection unit 202 adds the image frame including all the image data included in the construction project information to the image frame selected in step S147. Next, the three-dimensional information generation unit 203 generates three-dimensional information based on the image frame selected from the shooting information and the image frame included in the construction project information (step S148). Since the processing process other than this is similar to Fig. 9 The processing process is omitted, so its description is omitted.

[0154] The configuration of the information processing system 10 other than this is similar to the configuration of the information processing system 10 according to the first embodiment of the present disclosure, and thus a description thereof will be omitted.

[0155] In this way, in the information processing system 10 according to the third embodiment of the present disclosure, the second information processing device 400 generates an image of the vicinity of the construction vehicle 2 and the like, and transmits the generated image together with the position information to the first information processing device 200. The first information processing device 200 generates three-dimensional information based on the image of the vicinity of the construction vehicle 2 and the like generated by the second information processing device 400 in addition to the image of the entire construction area 20 generated by the third information processing device 100. This makes it possible to improve the accuracy of the three-dimensional information.

[0156] (4. Modification)

[0157] In the information processing system 10 according to the first embodiment, the third information processing device 100 generates an image of the entire construction area 20 by photographing the construction area 20. Alternatively, the third information processing device 100 may be controlled to photograph an area where the construction vehicle 2 or the like is performing construction work.

[0158] Specifically, the building area determined by the building area determination unit 207 of the first information processing device 200 is transmitted to the third information processing device 100. The third information processing device 100 receives the building area information from the first information processing device 200, performs flight control of the drone 4 to cover the received building area, and shoots the area through the camera 101 to obtain an image. The shooting information integration unit 107 generates shooting information based on the image, and sends the generated shooting information to the first information processing device 200. The first information processing device 200 generates local three-dimensional information based on the sent shooting information. This makes it possible to omit the processing of the image frame selection unit 202 in the first information processing device 200.

[0159] The configuration of the information processing system 10 other than this is similar to the configuration of the information processing system 10 according to the first embodiment of the present disclosure, and thus a description thereof will be omitted.

[0160] In this way, in the information processing system 10 according to the modified example of the present disclosure, the third information processing device 100 generates an image of an area where construction work has been carried out near the construction vehicle 2, etc., so that the processing of generating local three-dimensional information in the first information processing device 200 can be omitted.

[0161] It should be noted that the configuration of the second embodiment of the present disclosure can be applied to other embodiments. Specifically, it includes Fig.13 The first information processing device 200 including the three-dimensional information holding unit 209 and the transmission data selecting unit 210 in FIG. 2 can be applied to the first information processing device 200 according to the third embodiment of the present disclosure.

[0162] The configuration of the third embodiment of the present disclosure can be applied to other embodiments. Specifically, it includes Fig.14 The second information processing device 400 including the camera 407 and the image data generating unit 408 may be applied to the second information processing device 400 according to the second embodiment of the present disclosure.

[0163] Note that a series of processes to be performed by each device described in this specification can be implemented by using any of software, hardware, or a combination of software and hardware. The programs constituting the software are pre-stored in a storage medium (non-transient medium (or medium)) provided inside or outside each device, for example. Then, for example, each program is read into a RAM when executed by a computer, and is executed by a processor (such as a CPU).

[0164] In addition, the processes described using the flowcharts and sequence diagrams in this specification do not necessarily have to be performed in the order shown. Some processing steps can be performed in parallel. In addition, additional processing steps can be adopted, and some processing steps can be omitted.

[0165] The embodiments of the present disclosure have been described above. However, the technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present disclosure. In addition, it is allowed to appropriately combine components in different embodiments and modified examples.

[0166] (Effect)

[0167] The information processing system 10 includes a server device 300, a first information processing device 200, and a second information processing device 400. The server device 300 holds three-dimensional information of the entire building area in three-dimensional information of the terrain of the building area including the shape of the arrangement object, and updates the held three-dimensional information of the entire building area based on local three-dimensional information, which is three-dimensional information of an area where a construction project is performed by a construction device in the building area. The first information processing device 200 includes: a three-dimensional information generation unit 203 that generates local three-dimensional information based on position information of the construction device and an image of the building area; and a three-dimensional information transmission unit 204 that transmits the generated local three-dimensional information to the server device 300. The second information processing device 400 generates position information and transmits the generated position information to the first information processing device 200. Thus, the first information processing device 200 generates local three-dimensional information based on the position information of the construction device generated by the second information processing device 400, and transmits the local three-dimensional information to the server device 300. This makes it possible to reduce the processing performed by the second information processing device 400.

[0168] In addition, the 3D information generating unit 203 can generate local 3D information based on the construction project information, the position information and the image of the construction area, the construction project information being information about the working status of the construction equipment, so that the construction area for which the local 3D information is to be generated can be narrowed down according to the working status.

[0169] The three-dimensional information generating unit 203 may generate local three-dimensional information using an image including the entire building area as the image of the building area.

[0170] In addition, the second information processing device 400 may also transmit an image of an area where construction work is being carried out, and the 3D information generating unit 203 may also generate local 3D information using the transmitted image of the area where construction work is being carried out as an image of the construction area. This enables generation of detailed local 3D information.

[0171] The three-dimensional information generating unit 203 can further generate three-dimensional information of the entire building area based on the image including the building area. This makes it possible to generate and maintain the three-dimensional information of the entire building area.

[0172] In addition, the first information processing device 200 may further include: a three-dimensional information holding unit 209 that holds the generated three-dimensional information of the entire building area; and a selection unit (transmission data selection unit 210) that selects local three-dimensional information based on the held three-dimensional information of the entire building area. In addition, the three-dimensional information transmission unit 204 may send the selected local three-dimensional information to the server device 300. This makes it possible to reduce the amount of data communication.

[0173] The selection unit (transmission data selection unit 210) may select local three-dimensional information based on the difference between the three-dimensional information of the entire building area held and the generated local three-dimensional information. This makes it possible to update the three-dimensional information according to the amount of terrain change in the building area.

[0174] In addition, the three-dimensional information generating unit 203 may generate local three-dimensional information formed using point cloud data.

[0175] An information processing method includes: maintaining three-dimensional information of an entire building area in three-dimensional information of a terrain of a building area including a shape of an arranged object; generating position information of a construction device that performs construction work in the building area; generating local three-dimensional information based on the position information and an image of the building area, the local three-dimensional information being three-dimensional information of an area where construction work is performed by the construction device; and updating the maintained three-dimensional information of the entire building area based on the generated local three-dimensional information. This makes it possible to reduce processing.

[0176] The effects described in this specification are merely examples, and therefore, there may be other effects not limited to the exemplary effects.

[0177] It should be noted that the present technology can also have the following configurations.

[0178] (1) An information processing system comprising: a server device that maintains three-dimensional information of an entire building area in three-dimensional information of a terrain of a building area including a shape of an arranged object, and updates the maintained three-dimensional information of the entire building area based on local three-dimensional information, the local three-dimensional information being three-dimensional information of an area where a construction project is carried out by a construction device in the building area; a first information processing device comprising a three-dimensional information generating unit and a three-dimensional information sending unit, the three-dimensional information generating unit generating local three-dimensional information based on position information of the construction device and an image of the building area, and the three-dimensional information sending unit sending the generated local three-dimensional information to the server device; and a second information processing device that generates position information and sends the generated position information to the first information processing device.

[0179] (2) The information processing system according to (1) above, wherein the three-dimensional information generating unit generates the local three-dimensional information based on construction project information as information on the working state of the construction device, the position information, and the image of the construction area.

[0180] (3) The information processing system according to (1) or (2) above, wherein the three-dimensional information generating unit generates the local three-dimensional information using an image including the entire building area as the image of the building area.

[0181] (4) An information processing system according to any one of the above (1) to (3), wherein the second information processing device further transmits an image of the area where the work has been performed, and the three-dimensional information generation unit uses the transmitted image of the area where the work has been performed as an image of the building area to generate local three-dimensional information.

[0182] (5) The information processing system according to any one of (1) to (4) above, wherein the three-dimensional information generating unit further generates the three-dimensional information of the entire building area based on the image including the building area.

[0183] (6) An information processing system according to (5), wherein the first information processing device further includes: a three-dimensional information maintaining unit that maintains the generated three-dimensional information of the entire building area; and a selection unit that selects local three-dimensional information based on the maintained three-dimensional information of the entire building area, and the three-dimensional information sending unit sends the selected local three-dimensional information to the server device.

[0184] (7) The information processing system according to (6), wherein the selection unit selects the local three-dimensional information based on a difference between the held three-dimensional information of the entire building area and the generated local three-dimensional information.

[0185] (8) The information processing system according to any one of (1) to (7) above, wherein the three-dimensional information generating unit generates the local three-dimensional information formed using point cloud data.

[0186] (9) An information processing method comprising: maintaining three-dimensional information of an entire building area in three-dimensional information of a terrain of a building area including a shape of an arranged object; generating position information of a construction device working in the building area; generating local three-dimensional information based on the position information and an image of the building area, the local three-dimensional information being three-dimensional information of an area where the construction device has performed operations; and updating the maintained three-dimensional information of the entire building area based on the generated local three-dimensional information.

[0187] (10) An information processing system comprising: a first processor configured to: receive position information indicating a position of a construction vehicle located within a construction site; receive local image data of an area surrounding the construction vehicle, wherein the area surrounding the construction vehicle includes a portion that is smaller than the entire construction site; generate local three-dimensional (3D) information based on the position information and the local image data; and output the local 3D information to a server; and a server, wherein the server stores global 3D information of the construction site, the global 3D information including a terrain of the entire construction site, and wherein the server is configured to update the global 3D information of the construction site based on the local 3D information output by the first processor.

[0188] (11) The information processing system according to (10), wherein the position information is obtained by a second processor mounted on the construction vehicle.

[0189] (12) The information processing system according to any one of (10) or (11), wherein the local image data is obtained by a second processor mounted on the construction vehicle.

[0190] (13) An information processing system according to any one of (10) to (12), wherein the local image data is acquired by a drone.

[0191] (14) An information processing system according to any one of (10) to (13), wherein the global three-dimensional information stored in the server is generated by image data taken by a drone.

[0192] (15) The information processing system according to any one of (10) to (14), wherein the local 3D information is point cloud data.

[0193] (16) An information processing system according to any one of (11) to (12), wherein the first processor is further configured to: receive construction project information output from the second processor, the construction project information indicating work performed on an area of ​​a construction site; and generate local 3D information based on the construction project information.

[0194] (17) An information processing system according to (16), wherein the first processor is further configured to: select at least a portion of the local image data from the received local image data based on the construction project information; and generate local 3D information based on the selected local image data.

[0195] (18) An information processing system according to (17), wherein the first processor is configured to select at least a portion of the local image data when the construction project information includes a dynamic project that changes a topography of an area of ​​a construction site indicated by the construction project information.

[0196] (19) The information processing system according to any one of (10) to (18), wherein the first processor is further configured to store the global 3D information and select the local 3D information based on the stored global 3D information.

[0197] (20) The information processing system of (19), wherein the first processor selects the local 3D information based on a determined difference between the stored global 3D information and the local 3D information.

[0198] (21) An information processing system according to any one of (10) to (20), wherein the first processor is further configured to: determine a construction area based on location information; determine whether one or more other construction vehicles are within a threshold distance of the determined construction area; when it is determined that one or more other construction vehicles are within the threshold distance of the determined construction area, combine the corresponding construction areas corresponding to the one or more other construction vehicles with the determined construction area to form a combined construction area; and combine multiple local image data based on the combined construction area to form a combined local image data.

[0199] (22) The information processing system according to (21), wherein the first processor selects at least a portion of the partial image data from the combined partial image data.

[0200] (23) The information processing system according to (16), wherein the second processor is configured to obtain the local image data based on the construction project information.

[0201] (24) The information processing system of (23), wherein the second processor is configured to output the local image data to the first processor in response to determining that the construction project information includes a dynamic project that changes the topography of an area surrounding the construction vehicle.

[0202] (25) The information processing system according to any one of (10) to (24), wherein the server is configured to output the updated global 3D information to a display device.

[0203] (26) A method for updating global three-dimensional (3D) information of a construction site, comprising: receiving position information indicating the position of a construction vehicle located within the construction site; receiving local image data of an area around the construction vehicle, wherein the area around the construction vehicle includes a portion that is smaller than the entire construction site; generating local 3D information based on the position information and the local image data; sending the local 3D information to a server; and updating the global 3D information of the construction site based on the local 3D information sent to the server, the global 3D information including a terrain of the entire construction site stored by the server.

[0204] (27) The method according to (26) above, wherein updating the global 3D information includes updating a portion of the global 3D information corresponding to the position of the construction vehicle.

[0205] (28) The method according to (26) also includes: receiving construction project information output from a second processor installed on the construction vehicle, the construction project information representing work performed on an area of ​​the construction site; and generating local 3D information based on the construction project information.

[0206] (29) At least one non-transitory computer-readable medium having instructions encoded thereon, which, when executed by at least one processor, cause the at least one processor to perform a method for updating global three-dimensional (3D) information of a construction site, the method comprising: receiving position information indicating the position of a construction vehicle located within the construction site; receiving local image data of an area surrounding the construction vehicle, wherein the area surrounding the construction vehicle includes a portion that is less than the entire construction site; generating local 3D information based on the position information and the local image data; sending the local 3D information to a server; and updating global 3D information of the construction site based on the local 3D information sent to the server, the global 3D information including a topography of the entire construction site stored by the server.

[0207] [Reference Numbers List]

[0208] 10 Information Processing System

[0209] 100 Third information processing device

[0210] 104, 402 Position information generation unit

[0211] 109, 405 transmission unit

[0212] 200 First information processing device

[0213] 203 Three-dimensional information generation unit

[0214] 204 Three-dimensional information transmission unit

[0215] 207 Building Area Determination Unit

[0216] 209 Three-dimensional information holding unit

[0217] 210 Transmission data selection unit

[0218] 300 Server Devices

[0219] 302 3D information update unit

[0220] 303 Three-dimensional information holding unit

[0221] 400, 400a, 400b, 400c second information processing device

[0222] 403 Vehicle status information acquisition unit

[0223] 408 Image data generation unit

[0224] 500 Display Terminal

Claims

1. An information processing system, comprising: The first processor is configured to: receiving location information indicating a location of a construction vehicle located within a construction site; receiving local image data of an area surrounding the construction vehicle, wherein the area surrounding the construction vehicle comprises a portion of the construction site that is less than the entire construction site; generating local three-dimensional 3D information based on the position information and the local image data; and Outputting the local 3D information to a server; and The server, wherein the server stores global 3D information of the construction site, the global 3D information including a topography of the entire construction site, and wherein the server is configured to update the global 3D information of the construction site based on the local 3D information output by the first processor.

2. The information processing system according to claim 1, wherein: The position information is obtained by a second processor mounted on the construction vehicle.

3. The information processing system according to claim 1, wherein: The local image data is obtained by a second processor mounted on the construction vehicle.

4. The information processing system according to claim 1, wherein: The local image data is obtained by a drone.

5. The information processing system according to claim 1, wherein: The global 3D information stored by the server is generated from image data captured by the drone.

6. The information processing system according to claim 1, wherein: The local 3D information is point cloud data.

7. The information processing system according to claim 2, wherein: The first processor is further configured to: receiving construction project information output from the second processor, the construction project information indicating a project to be performed on an area of ​​the construction site; and The local 3D information is generated based on the construction project information.

8. The information processing system according to claim 7, wherein: The first processor is further configured to: selecting at least a portion of the partial image data from the received partial image data based on the construction project information; and The local 3D information is generated based on the selected local image data.

9. The information processing system according to claim 8, wherein: The first processor is configured to select at least a portion of the partial image data when the construction project information includes a dynamic project that changes a topography of an area of ​​the construction site indicated by the construction project information.

10. The information processing system according to claim 1, wherein: The first processor is further configured to store the global 3D information and select local 3D information based on the stored global 3D information.

11. The information processing system according to claim 10, wherein: The first processor selects the local 3D information based on a determined difference between the stored global 3D information and the local 3D information.

12. The information processing system according to claim 1, wherein: The first processor is further configured to: determining a building area based on the location information; determining whether one or more other construction vehicles are within a threshold distance of the determined construction area; When it is determined that the one or more other construction vehicles are within a threshold distance of the determined construction area, combining respective construction areas corresponding to the one or more other construction vehicles with the determined construction area to form a combined construction area; as well as The plurality of partial image data are combined based on the combined building area to form combined partial image data.

13. The information processing system according to claim 12, wherein: The first processor selects at least a portion of the partial image data from the combined partial image data.

14. The information processing system according to claim 7, wherein: The second processor is configured to obtain the local image data based on the construction project information.

15. The information processing system according to claim 14, wherein: The second processor is configured to output the local image data to the first processor in response to determining that the construction project information includes a dynamic project that changes the topography of an area surrounding the construction vehicle.

16. The information processing system according to claim 1, wherein: The server is configured to output the updated global 3D information to a display device.

17. A method for updating global three-dimensional (3D) information of a construction site, comprising: receiving location information indicating a location of a construction vehicle located within the construction site; receiving local image data of an area surrounding the construction vehicle, wherein the area surrounding the construction vehicle comprises a portion of the construction site that is less than the entire construction site; generating local 3D information based on the position information and the local image data; transmitting the local 3D information to a server; and The global 3D information of the construction site is updated based on the local 3D information transmitted to the server, the global 3D information including the topography of the entire construction site stored by the server.

18. The method according to claim 17, wherein: Updating the global 3D information includes updating a portion of the global 3D information corresponding to the position of the construction vehicle.

19. The method according to claim 17, further comprising: receiving construction work information output from a second processor mounted on the construction vehicle, the construction work information indicating work being performed on an area of ​​the construction site; as well as The local 3D information is generated based on the construction project information.

20. At least one non-transitory computer readable medium having instructions encoded thereon, which when executed by at least one processor causes the at least one processor to perform a method for updating global three-dimensional (3D) information of a construction site, the method comprising: receiving location information indicating a location of a construction vehicle located within the construction site; receiving local image data of an area surrounding the construction vehicle, wherein the area surrounding the construction vehicle comprises a portion of the construction site that is less than the entire construction site; generating local 3D information based on the position information and the local image data; transmitting the local 3D information to a server; and The global 3D information of the construction site is updated based on the local 3D information transmitted to the server, the global 3D information including the topography of the entire construction site stored by the server.

Images