Information processing system

Abstract

To provide an information processing system for reducing time and effort for measurement to show a path of a measuring device in a structure having two or more stories.SOLUTION: An information processing system includes: a measurement data acquisition step of acquiring measurement data measured at each position on a route by a measurement device having a sensor for measuring an electromagnetic wave arriving from an object while moving inside (e.g., inside of a building under construction) or outside a structure; and a path data output step of outputting path data showing a path of the measurement device on the basis of the acquired measurement data. The path has two or more partial paths on which positions measured by the measurement device are respectively placed within two or more different height ranges in the case that the structure has two or more stories.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system. [Background technology]

[0002] Patent document 1 discloses a vacuum cleaner system that includes a position sensor that acquires the positional relationships of surrounding objects, a map acquisition unit that acquires a floor map, a self-position estimation unit that estimates the vacuum cleaner's own position on the floor map based on the position sensor, a boundary information generation unit that acquires boundary information indicating the boundary of a cleaning area, which is the area on the floor where the vacuum cleaner will clean, based on the vacuum cleaner's own position, a boundary indication unit that instructs the boundary information generation unit on the boundary, a cleaning area creation unit that creates a cleaning area based on the boundary information, and a travel path creation unit that creates a travel path for cleaning based on the created cleaning area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-58961 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, when estimating a vehicle's own position from images taken of a construction site and reporting the site's situation, if the construction site spans two or more floors, the technology in Patent Document 1 requires photographing each floor separately.

[0005] In view of the above circumstances, the present invention aims to reduce the amount of work required for measurements to indicate the trajectory of a measuring device in a structure having two or more floors. [Means for solving the problem]

[0006] According to one aspect of the present invention, there is provided an information processing system. In the measurement acquisition step of this information processing system, a measuring device having a sensor that measures electromagnetic waves arriving from an object moves inside or outside a structure and acquires measurement data measured at each position along the movement path. In the trajectory output step, trajectory data indicating the trajectory of the measuring device is output based on the acquired measurement data, and if the structure has two or more floors, the trajectory has two or more partial trajectories in which the positions where the measuring device performed measurements fall within two or more different height ranges.

[0007] According to this embodiment, it is possible to reduce the amount of work required for measurements to indicate the trajectory of the measuring device in a structure having two or more floors. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing the overall configuration of a construction support system 1. FIG. [Figure 2] FIG. 2 is a diagram illustrating a hardware configuration of a server device 10. [Figure 3] FIG. 2 is a diagram illustrating a hardware configuration of a field terminal 20. [Figure 4] FIG. 2 is a diagram illustrating a hardware configuration of an imaging device 30. [Figure 5] FIG. 2 is a diagram illustrating a hardware configuration of a remote terminal 40. [Figure 6] FIG. 2 is a diagram illustrating the functional configuration of a control unit of each device. [Figure 7] FIG. 10 is a diagram illustrating an example of a partial trajectory. [Figure 8] FIG. 10 is a diagram illustrating an example of drawing data. [Figure 9] FIG. 10 is an activity diagram illustrating an example of an acquisition process. [Figure 10] FIG. 10 is a flow diagram illustrating an example of an analysis process. [Figure 11] FIG. 10 is a diagram illustrating an example of trajectory data. [Figure 12] FIG. 10 is a diagram illustrating an example of trajectory data indicating a partial trajectory. [Figure 13] FIG. 10 is a diagram illustrating an example of hierarchical data. [Figure 14] FIG. 10 is a diagram illustrating an example of correspondence data. [Figure 15] FIG. 2 is a diagram showing an example of a marker 90. [Figure 16] FIG. 10 is a diagram illustrating an example of correspondence data. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] In this embodiment, the term "unit" may also include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, various types of information are handled in this embodiment, and this information may be represented by, for example, physical values ​​of signal values ​​representing voltages and currents, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations may be performed on a circuit in the broad sense.

[0012] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration In this section, the hardware configuration of the construction support system according to this embodiment will be described.

[0014] Fig. 1 is a diagram showing the overall configuration of the construction support system 1. Fig. 1 shows an overview of each device provided in the construction support system 1 and the users who use those devices. Each overview will be explained as needed with reference to other figures.

[0015] The construction support system 1 is an information processing system that executes processing to support construction work such as building construction. The construction support system 1 includes a communication line 2, a selfie stick 3, an external battery 4, an external power supply 5, a breaker 6, a server device 10, an on-site terminal 20, an imaging device 30, and a remote terminal 40.

[0016] The communication line 2 includes the Internet or the like, and mediates data exchange between devices connected to the line itself. A server device 10 is connected to the communication line 2 via a wired connection, and an on-site terminal 20 and a remote terminal 40 are connected wirelessly. In this embodiment, the on-site terminal 20 communicates with the communication line 2 via mobile communication. The on-site terminal 20 also communicates wirelessly with the imaging device 30 using two communication methods. In this embodiment, the two communication methods are Wi-Fi communication and BLE (Bluetooth (registered trademark) Low Energy) communication.

[0017] The on-site terminal 20 and the imaging device 30 are installed at a construction site and used, for example, by a site worker W1. The remote terminal 40 is used, for example, by a construction supervisor W2 in charge of the construction site, and is a terminal that is expected to be used in locations away from the construction site.

[0018] The on-site terminal 20 is connected to the external power source 5 via an external battery 4 by a cable. The imaging device 30 is detachably connected to the external power source 5 via the external battery 4 by a cable and a connector 7. In other words, the external power source 5 also supplies power to the detachably connected imaging device 30. The external battery 4 has a so-called pass-through function that allows it to supply power while charging, and when the breaker 6 is on, it supplies power to the on-site terminal 20 and the imaging device 30 while being charged with power supplied from the external power source 5.

[0019] The imaging device 30 is a digital camera equipped with an image sensor, and captures an image represented by light measured by the image sensor. In this embodiment, the imaging device 30 is a 360-degree camera that can capture images in all directions, including up, down, left, right, front, and back. The imaging device 30 is an example of a measuring device equipped with a sensor. The imaging device 30 is attached to a selfie stick 3, which can be inserted into a stand 8 installed at the construction site and fixed therein.

[0020] Marker board 9 is a plate member with markers 91 and 92 (referred to as "markers 90" when not distinguishing between them) printed on its surface, and is fixed to stand 8. Marker 90 is installed at a construction site and serves as a guide for position and size in three-dimensional space. Marker 90 is, for example, a rectangular pattern with the length of each side registered in server device 10. Marker board 9 is fixed to stand 8 so that the plane of the rectangle formed by marker 90 is aligned vertically.

[0021] When a site worker W1 removes the connector 7, pulls out the selfie stick 3 from the stand 8, and walks around the construction site, image data is generated showing images of the construction site captured by the 360-degree camera. In the construction support system 1, the images captured by the imaging device 30 are moving images in this embodiment, but may also be still images captured continuously as long as images of various locations on the construction site can be obtained. The imaging device 30 transmits the generated image data to the site terminal 20.

[0022] The on-site terminal 20 is a terminal that serves as the main user interface for the field worker W1, and is, for example, a smartphone. The on-site terminal 20 controls the operation of the imaging device 30, for example, using one of the two communication methods described above (BLE communication in this embodiment). The on-site terminal 20 also transfers image data transmitted from the imaging device 30 by one of the two communication methods described above (Wi-Fi communication in this embodiment) to the server device 10 using another wireless communication (mobile communication in this embodiment).

[0023] The server device 10 performs image processing using the images of the construction site shown in the image data transmitted from the on-site terminal 20, and generates stereoscopic image data that shows the construction site in three dimensions. The remote terminal 40 references the generated stereoscopic image data and displays the stereoscopic image of the construction site. The construction supervisor W2 understands the situation at the construction site from the displayed images, and gives instructions to the on-site worker W1 on the site as necessary.

[0024] Depending on the construction site, the breaker 6 may be turned off after work is completed for reasons such as power saving. In this case, power is no longer supplied from the external power source 5 after the breaker 6 is turned off. In this way, the external power source 5 can switch between supplying and not supplying power. The on-site terminal 20 and the imaging device 30 also have built-in batteries, so they do not immediately stop, but some processes, such as transmitting image data, take time. Therefore, in this embodiment, an external battery 4 is provided to increase the operating time of the on-site terminal 20 and the imaging device 30 after the breaker 6 is turned off.

[0025] 2 is a diagram showing the hardware configuration of server device 10. Server device 10 includes a control unit 11, a storage unit 12, a communication unit 13, and a bus 14. Bus 14 electrically connects the various units included in server device 10.

[0026] (Control unit 11) The control unit 11 is, for example, a central processing unit (CPU) not shown. The control unit 11 is a computer that realizes various functions related to the construction support system 1 by reading out predetermined programs stored in the storage unit 12. In other words, information processing by software stored in the storage unit 12 is specifically realized by the control unit 11, which is an example of hardware, and can be executed as each functional unit included in the control unit 11. These will be described in further detail in the next section. Note that the control unit 11 is not limited to being single, and it may be implemented by having multiple control units 11 for each function. It may also be a combination of these.

[0027] (Storage unit 12) The memory unit 12 stores various pieces of information defined above. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the construction support system 1 executed by the control unit 11, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to program calculations. The memory unit 12 stores various programs, variables, etc. related to the construction support system 1 executed by the control unit 11.

[0028] (Communications Department 13) The communication unit 13 is configured to be able to transmit various electrical signals from the server device 10 to external components. The communication unit 13 is also configured to be able to receive various electrical signals from the external components to the server device 10. More preferably, the communication unit 13 has a network communication function, which allows various information to be communicated between the server device 10 and external devices via the communication line 2.

[0029] 3 is a diagram showing the hardware configuration of the field terminal 20. The field terminal 20 includes a control unit 21, a storage unit 22, a communication unit 23, an input unit 24, an output unit 25, an internal power supply unit 26, and a bus 27. The bus 27 electrically connects the units included in the field terminal 20. The control unit 21 and the storage unit 22 are similar hardware to the control unit 11 and the storage unit 12 shown in FIG. 2, although their performance differs.

[0030] (Communications Department 23) The communication unit 23 includes a first communication unit 231, a second communication unit 232, and a third communication unit 233, and is an example of a wireless communication unit that performs three types of wireless communication. The first communication unit 231 performs wireless communication by Wi-Fi communication in this embodiment as the first wireless communication. The second communication unit 232 performs wireless communication by BLE in this embodiment as the second wireless communication, which has a slower communication speed and consumes less power than the first wireless communication. The third communication unit 233 performs wireless communication by mobile communication in this embodiment as the third wireless communication, which has a wider communication area than the first wireless communication and the second wireless communication.

[0031] (Input unit 24) The input unit 24 includes keys, buttons, a touch screen, a mouse, etc., and receives input from the user. (Output section 25) The output unit 25 has a display (including a touch screen) and a speaker, and displays visual information generated in a manner that is visible to the user, such as a screen, image, icon, text, etc., on the display surface, and outputs sound including voice.

[0032] (Internal power supply section 26) The internal power supply unit 26 is a battery built into the device itself, i.e., a repeatedly rechargeable battery, and supplies stored power to each component of the device. The internal power supply unit 26 is an example of a portable battery that can be carried around with the device itself. The internal power supply unit 26 is charged with power supplied from the external power supply 5. Like the external battery 4, the internal power supply unit 26 has a pass-through function, and when the breaker 6 is on, it is charged with power supplied from the external power supply 5 and supplies power to each component.

[0033] 4 is a diagram showing the hardware configuration of the imaging device 30. The imaging device 30 includes a control unit 31, a storage unit 32, a communication unit 33, an input unit 34, an output unit 35, an internal power supply unit 36, an imaging unit 37, and a bus 38. The bus 38 electrically connects the various units included in the imaging device 30. The units from the control unit 31 to the internal power supply unit 36 ​​are similar in hardware to the units from the control unit 21 to the internal power supply unit 26 shown in FIG. 3, although their performance differs.

[0034] However, the communication unit 33 only includes a first communication unit 331 and a second communication unit 332. Like the first communication unit 231 of the communication unit 23, the first communication unit 331 performs wireless communication via Wi-Fi in this embodiment as the first wireless communication. Like the second communication unit 232 of the communication unit 23, the second communication unit 332 performs wireless communication via BLE in this embodiment as the second wireless communication, which has a slower communication speed and lower power consumption than the first wireless communication. Furthermore, the output unit 35 has a light in addition to a display, etc., and emits light to ensure the amount of light required for shooting. The input unit 34 has a switch for turning on the light.

[0035] (Image capture unit 37) The imaging unit 37 is a sensor that has an optical system including a lens, an image sensor, etc., and measures light incident from the lens to generate image data. In this embodiment, as described above, the imaging unit 37 uses an ultra-wide-angle lens and multiple image sensors to generate image data captured in all directions (up and down, left and right, front and back).

[0036] 5 is a diagram showing the hardware configuration of remote terminal 40. Remote terminal 40 includes control unit 41, storage unit 42, communication unit 43, input unit 44, output unit 45, and bus 46. Bus 46 electrically connects the various units included in remote terminal 40. The various units from control unit 41 to output unit 45 are similar in hardware to the various units from control unit 31 to output unit 35 shown in FIG. 4, although their performance differs.

[0037] 2. Functional configuration This section describes the functional configuration of this embodiment. As described above, information processing by software stored in the storage unit of each device is specifically realized by a control unit, which is an example of hardware, so that each functional unit included in the control unit can be executed.

[0038] 6 is a diagram showing the functional configuration of the control unit of each device. The control unit 11 of the server device 10 includes an information storage unit 111, an image processing unit 112, a data generation unit 113, a data output unit 114, a measurement acquisition unit 115, a trajectory output unit 116, a hierarchy acquisition unit 117, a correspondence output unit 118, a feature identification unit 119, a correspondence unit 120, and an image output unit 121. The control unit 21 of the on-site terminal 20 includes a display control unit 211, an operation reception unit 212, an operation control unit 213, and a transmission control unit 214. The control unit 31 of the imaging device 30 includes a display control unit 311, an operation reception unit 312, an imaging control unit 313, and a transmission control unit 314. The control unit 41 of the remote terminal 40 includes a display control unit 411 and an operation reception unit 412.

[0039] The information storage unit 111 of the server device 10 stores image data showing images of the construction site captured by the imaging device 30 and the above-mentioned stereoscopic image data. The image processing unit 112 performs processing related to a technology known as SLAM (Simultaneous Localization and Mapping), which simultaneously estimates the self-position and creates an environmental map, based on the images of the construction site. The data generation unit 113 generates the above-mentioned stereoscopic image data based on the processing performed by the image processing unit 112. The data output unit 114 outputs the generated stereoscopic image data.

[0040] The measurement acquisition unit 115 acquires measurement data measured at each position on the movement path while the imaging device 30 moves inside or outside a structure. The inside or outside of a structure is, for example, a construction site. The trajectory output unit 116 outputs trajectory data indicating the trajectory of the imaging device 30. The level acquisition unit 117 acquires hierarchical data indicating the levels of the structure. The hierarchical data is, for example, data indicating the floors (first, second, third, etc.) of a building. Note that the hierarchical data may also be data indicating the attic, above floor, below floor, etc. of the same floor as levels. The correspondence output unit 118 outputs correspondence data indicating the correspondence between the levels indicated by the hierarchical data acquired by the level acquisition unit 117 and the partial trajectories.

[0041] When hierarchical data indicating a drawing of each floor of a structure is acquired, the feature identification unit 119 identifies a location having a predetermined feature in the drawing. The association unit 120 associates the position of the end of the partial trajectory corresponding to the floor indicated by the hierarchical data with the location identified in the drawing of that floor. The image output unit 121 outputs an image represented in a manner corresponding to the height of each position measured by the imaging device 30 indicated by the partial trajectory.

[0042] The display control unit 411 of the remote terminal 40 controls the display processing on the display means of the remote terminal 40. For example, the display control unit 411 displays an image of a construction site shown by stereoscopic image data output from the server device 10. The operation reception unit 412 receives operations from a user (for example, construction supervisor W2). The construction supervisor W2 performs an operation to move the displayed location of the construction site to check the current situation at the site, and instructs the on-site worker W1 and the like on the construction procedures, etc.

[0043] The display control unit 211 of the on-site terminal 20 controls the display processing on the display means of the on-site terminal 20. The operation reception unit 212 receives operations from a user (e.g., field worker W1). The operation control unit 213 controls the operation of the imaging device 30. The transmission control unit 214 controls the transmission processing of image data by the on-site terminal 20 and the imaging device 30.

[0044] The display control unit 311 of the imaging device 30 controls the display processing on the display means of the device itself. The operation reception unit 312 receives operations from a user (e.g., field worker W1). The photography control unit 313 controls the photography processing by the imaging unit 37. The transmission control unit 314 controls the transmission processing of image data by the device itself.

[0045] 3. Information Processing In this section, information processing that the program causes the computer to execute in this embodiment will be described.

[0046] The measurement acquisition unit 115 in the construction support system 1 acquires measurement data measured at each position on a movement path while the imaging device 30, which has a sensor that measures electromagnetic waves arriving from an object, moves inside or outside a structure. The trajectory output unit 116 outputs trajectory data indicating the trajectory of the imaging device 30 based on the measurement data acquired by the measurement acquisition unit 115. If the structure has two or more floors, the trajectory of the imaging device 30 will have two or more partial trajectories in which the positions where the imaging device 30 performed measurements fall within two or more different height ranges.

[0047] Measurement data is, for example, data indicating a set of measurement results at each position on a movement path. Trajectory data is, for example, data indicating three-dimensional coordinates or two-dimensional coordinates (with or without scale). A partial trajectory is, for example, a trajectory generated after dividing the measurement data. In this case, the division method is a method of recognizing stairs from the measurement data and dividing it into before and after the stairs.

[0048] In addition, partial trajectories are created by dividing a trajectory that is generated based on measurement data. In this case, the division method is to divide it by the average height if there are two floors, or to use a well-known clustering method such as k-Maens. Note that the captured images, number of floors, and floor plans (with scale and initial values) other than the trajectory data will be described later.

[0049] Fig. 7 is a diagram showing an example of a partial trajectory. Fig. 7 shows a graph G10 of a three-dimensional coordinate system having an X axis, a Y axis, and a Z axis. Graph G10 shows an entire trajectory C10 having a partial trajectory C11 and a partial trajectory C12. The partial trajectory C11 is a trajectory where the position measured by the imaging device 30 falls within a range of coordinates in the Z axis direction (i.e., height) less than z1 (m: meters). The partial trajectory C12 is a trajectory where the position measured by the imaging device 30 falls within a range of coordinates in the Z axis direction equal to or greater than z1 (m: meters).

[0050] The hierarchical level acquisition unit 117 acquires hierarchical level data indicating the levels of the structure. The correspondence output unit 118 outputs correspondence data indicating the correspondence between the levels indicated by the hierarchical level data acquired by the hierarchical level acquisition unit 117 and the partial trajectories.

[0051] Hierarchical data is data that indicates, for example, the number and order of layers, the height of the layers, or a diagram of each layer. Correspondence data is, for example, a table showing the correspondence, partial trajectory data with the number of layers added to the header, or data that indicates the original video. For partial trajectories, the larger the statistical value (average value, etc.) of the height coordinate, the higher the layer. If the number of partial trajectories and the number of layers match, the correspondence can be simply determined.

[0052] The hierarchical data indicates a drawing of each floor of the structure. The feature identification unit 119 identifies a location having a predetermined feature in the drawing. The association unit 120 associates the position of the end of the partial trajectory corresponding to a floor with the identified location in the drawing of that floor.

[0053] The associating unit 120 identifies connection paths connecting each floor shown in the drawing as locations having a predetermined feature in the drawing. The connection paths connecting each floor shown in the drawing are, for example, stairs or slopes.

[0054] The feature identification unit 119 identifies the start position in the drawing as the location where measurement by the imaging device 30 starts. The association unit 120 associates the end position of the partial trajectory corresponding to the layer where the start position has been identified by the feature identification unit 119 with the start position.

[0055] The starting position at which measurement starts is, for example, the installation position of a marker, the installation position of a camera, or a manually set position.

[0056] The feature specifying unit 119 specifies, as the start position, the position at which the feature quantity determined as the start position among the feature quantities indicated by the measurement data acquired by the measurement acquisition unit 115 is measured.

[0057] The feature values ​​indicated by the measurement data are the length of the line, the angle formed by the line, etc. When a marker is placed at the start position, if the length and angle of the line indicated by the feature values ​​indicate the shape of the marker, the position where the measurement data indicating the feature values ​​was measured is identified as the start position.

[0058] The hierarchical data indicates a drawing of each floor of the structure. When the number of partial trajectories is smaller than the number of floors indicated by the hierarchical data acquired by the floor acquisition unit 117, the correspondence output unit 118 outputs, as correspondence data, data indicating the correspondence relationship between the partial trajectories and the floor that has the highest degree of match when the partial trajectories are arranged on the floor drawing.

[0059] The correspondence output unit 118 places the partial trajectories on the drawing without overlapping them with walls. In this case, the larger the scale, the more the partial trajectories must match the shape of the drawing in order to be placed. Therefore, the correspondence output unit 118 determines that the degree of match is higher when the scale of the trajectories to be placed on the drawing is increased as the maximum scale becomes larger.

[0060] FIG. 8 is a diagram showing an example of drawing data. FIG. 8 shows a drawing D10 of a floor that is a construction site. Drawing D10 shows a staircase D11, which is a connecting path connecting each floor. In the example of FIG. 8, a partial trajectory C21 is placed without overlapping with the wall of the construction site shown in drawing D10, and of the end positions C211 and C212 of partial trajectory C21, position C211 is associated with staircase D11. Because partial trajectory C21 passes through almost the entire area of ​​the floor shown in drawing D10, the scale can be made larger than when it is placed on a drawing of another floor, and it is determined that the degree of match is highest.

[0061] The image output unit 121 outputs an image displayed in a manner according to the height of each position where the imaging device 30 has performed a measurement, the position being indicated by the partial trajectory.

[0062] The image output unit 121 outputs, for example, a mark image of a color and shape corresponding to the height (above the ceiling or under the floor). The image output unit 121 may also display a warp point (a measurement point or a nearby point displayed in a 360° image) in a manner corresponding to the height of the measurement position. The image output unit 121 may also display an image point (a measurement point displayed in a drawing) in a manner corresponding to the height of the measurement position.

[0063] The image output unit 121 may output an image displayed in a manner according to the height of the measurement position based on the trajectory data of only one floor. Furthermore, the image output unit 121 outputs image data showing an image of these images viewed from above, and when the user performs an operation to change the viewpoint from a bird's-eye view to a viewpoint from the side, for example, the image output unit 121 outputs image data showing an image of these images viewed from the side.

[0064] The construction support system 1 first executes an acquisition process to acquire measurement data indicating the measurement results obtained by a measurement device (for example, the imaging device 30) measuring the inside of a structure (for example, the inside of a building under construction).

[0065] Fig. 9 is an activity diagram showing an example of the acquisition process. The activity shown in Fig. 9 is started when the field worker W1 performs a measurement start operation on the field terminal 20 to start measurement by the imaging device 30. Note that the image capture by the imaging device 30 is a measurement of light intensity using an image sensor. First, the field terminal 20 receives a measurement start operation by the field worker W1 via the operation receiving unit 212 (A11).

[0066] When the operation reception unit 212 receives the measurement start operation, it transmits instruction data to the imaging device 30 to instruct the start of measurement (photography in the example of FIG. 9). When the imaging device 30 receives the transmitted instruction data, it starts measurement, i.e., photography, using the photography control unit 313 (A12). The imaging device 30, using the photography control unit 313, has the field worker W1 photograph the surroundings while moving around the construction site, and stores measurement data indicating the photographed images as measurement results in the internal memory unit 32 of the imaging device itself (A13).

[0067] Next, when the site worker W1 finishes measuring the construction site and performs a measurement end operation on the site terminal 20 to end the measurement, the site terminal 20 receives the measurement end operation via the operation reception unit 212 (A14). Upon receiving the measurement end operation, the operation reception unit 212 transmits instruction data instructing the image capture device 30 to end the measurement. Upon receiving the transmitted instruction data, the image capture device 30 terminates the measurement via the image capture control unit 313 (A15). By performing the processes from A11 to A15, the measurement data of the work site is stored in the image capture device 30.

[0068] Next, the on-site terminal 20 transmits instruction data indicating an instruction to transmit the measurement data to the imaging device 30 at a predetermined timing via the transmission control unit 214 (A21). The predetermined timing may be, for example, a predetermined time, a predetermined time after the measurement is completed, or a transmission operation by the on-site worker W1. Upon receiving the instruction data, the imaging device 30 reads the measurement data stored in the memory unit 32 via the photography control unit 313 and transmits the measurement data to the on-site terminal 20 (A22).

[0069] The on-site terminal 20 receives the transmitted measurement data via the transmission control unit 214 and stores it in its own storage unit 32 (A23). Next, the on-site terminal 20 reads the measurement data from the storage unit 32 at a predetermined timing via the transmission control unit 214 and transmits it to the server device 10 (A31). The predetermined timing may be, for example, a predetermined time. The server device 10 acquires the transmitted measurement data via the measurement acquisition unit 115 (A32). The server device 10 executes an analysis process to analyze the trajectory of the imaging device 30 based on the acquired measurement data.

[0070] Fig. 10 is a flow diagram showing an example of the analysis process. The analysis process begins with the measurement acquisition unit 115 acquiring measurement data (S11=A32). In the example of Fig. 10, the server device 10 first generates trajectory data indicating the trajectory of the imaging device 30 based on the acquired measurement data using the trajectory output unit 116 (S12). For example, when the measurement data is an image as described above, the trajectory output unit 116 generates the trajectory data using VisualSLAM technology.

[0071] As described above, the trajectory data is data that indicates a trajectory using three-dimensional coordinates or two-dimensional coordinates. In the example of Fig. 7, data that indicates a trajectory using three-dimensional coordinates is generated as the trajectory data. Fig. 11 is a diagram showing an example of trajectory data. Trajectory data D1 shown in Fig. 11 is data showing a measurement time and three-dimensional coordinates indicating the measurement position of the measurement data at that measurement time in association with each other. In trajectory data D1, the three-dimensional coordinates are arranged in order from the earliest measurement time.

[0072] Next, the server device 10 causes the trajectory output unit 116 to execute partial trajectory processing (S13). For example, as described above, if there are two floors, the trajectory output unit 116 divides the trajectory into parts representing two floors, respectively, by dividing the trajectory by the average height. If there are three or more floors, the trajectory output unit 116 divides the trajectory into parts representing three or more floors, respectively, by a well-known clustering method such as k-Maens. The trajectory output unit 116 generates trajectory data indicating the divided partial trajectories.

[0073] Fig. 12 is a diagram showing an example of trajectory data indicating a partial trajectory. Trajectory data D2 shown in Fig. 12 is data in which the measurement times and three-dimensional coordinates shown in trajectory data D1 are associated with the hierarchical levels indicated by the partial trajectories to which each three-dimensional coordinate belongs. In the example of Fig. 12, the three-dimensional coordinates are associated with three hierarchical levels indicating the partial trajectories, namely, "first hierarchical level," "second hierarchical level," and "third hierarchical level," in order from the earliest measurement time. Hereinafter, the three-dimensional coordinates associated with each hierarchical level will be referred to as E01 to E99 (first hierarchical level), F01 to F99 (second hierarchical level), and G01 to G99 (third hierarchical level).

[0074] Next, the server device 10 causes the trajectory output unit 116 to output trajectory data indicating the divided partial trajectories (for example, data indicating the measurement time, three-dimensional coordinates, and floor level in association with each other as shown in FIG. 12) (S14). The trajectory output unit 116 outputs the trajectory data to, for example, a predetermined area in the storage unit 12. This area is an area reserved for work in performing analysis processing.

[0075] Next, the server device 10 acquires hierarchical data indicating the hierarchical levels of the structure using the hierarchical level acquisition unit 117 (S15). As described above, the hierarchical level data is data indicating information about each hierarchical level of the structure (such as the number and order of hierarchical levels, the height of each hierarchical level, or a diagram of each hierarchical level). The hierarchical level data is input in advance by, for example, a user or an operator of the construction support system 1, and is stored in the server device 10.

[0076] FIG. 13 is a diagram showing an example of hierarchical data. The hierarchical data D3 shown in FIG. 13 is data in which the number of floors, floor numbers (order of floors), heights, and drawings are associated with one another. In the hierarchical data D3, the number of floors "3," floor numbers "5th floor," "4th floor," and "3rd floor," and heights "xxm" are associated with each piece of drawing data. This hierarchical data D3 indicates, for example, that the third to fifth floors of a high-rise structure such as an apartment building are a construction site. Next, the server device 10 outputs, via the correspondence output unit 118, correspondence data indicating the correspondence between the floors indicated by the acquired hierarchical data and the partial trajectories (S16).

[0077] Fig. 14 is a diagram showing an example of correspondence data. The correspondence data D4 shown in Fig. 14 is data that associates the measurement time, three-dimensional coordinates, and floor number of the partial trajectory shown in the trajectory data D2 with each other. In the example of Fig. 14, the correspondence output unit 118 associates the "first floor" with the "fifth floor," the "second floor" with the "fourth floor," and the "third floor" with the "third floor." The correspondence output unit 118 determines the hierarchical relationship of each floor, for example, from the z coordinate, which indicates height among the three-dimensional coordinates.

[0078] Then, the correspondence output unit 118 determines which floor number each level corresponds to based on the hierarchical relationship between levels. In the example of Fig. 14, the correspondence output unit 118 determines that the "first level" is the highest and the "third level" is the lowest, and determines that the highest "first level" is the "fifth floor" and the lowest "third level" is the "third floor." According to this aspect, it is possible to determine which level a partial trajectory belongs to.

[0079] Note that the number of layers indicated by the hierarchical data and the number of layers of the partial trajectory do not always match. For example, the number of layers in the partial trajectory may be "3" (the number of layers for the construction site only), while the number of layers indicated by the hierarchical data may be "10" (the number of layers for the entire structure, for example). In this case, the correspondence output unit 118 places the partial trajectory on the drawing of each layer indicated by the hierarchical data and calculates the degree of match between those drawings and the partial trajectory.

[0080] As described above, the degree of match is determined by using, for example, the maximum scale of the partial trajectory that is placed on the drawing without overlapping with a wall (the larger the maximum scale, the greater the degree of match). Note that the method of calculating the degree of match is not limited to this. For example, the correspondence output unit 118 identifies the two coordinates that are the longest distance from each other among the coordinates indicating the partial trajectories, and sets the longest distance between the identified coordinates to 70 to 90 percent of the longest similar distance in the internal space of the structure shown in the drawing, thereby matching the scale with that of the real space.

[0081] The reason why the longest distance of the partial trajectory is shorter than the longest distance of the internal space of the drawing is that the imaging device 30 rarely reaches the edge of the internal space and often moves inside the edge of the internal space. The correspondence output unit 118 then calculates the degree of match by subtracting from 1 the proportion of overlap with the wall when the partial trajectory is overlaid on the drawing with the scale adjusted. In this case, the degree of match of a partial trajectory that does not overlap with a wall on the drawing at all with the scale adjusted will be the maximum value of "1", and the greater the proportion of overlap with the wall, the smaller the value and the lower the degree of match.

[0082] By determining the correspondence between the hierarchical data and the partial trajectory based on the degree of match in this way, it is possible to determine the hierarchical level of the partial trajectory even if only some of the hierarchical levels formed by the structure are measured.

[0083] Next, the server device 10 uses the feature identifying unit 119 to identify locations having predetermined features (hereinafter referred to as "characteristic locations") in the drawing of each layer (S17). The feature identifying unit 119 identifies, for example, the start position where measurement by the imaging device 30 starts and the end position where measurement ends in the drawing as characteristic locations.

[0084] Specifically, the feature identification unit 119 identifies as the start position the position at which a feature determined as the start position (hereinafter referred to as the "start feature") was measured among the feature amounts indicated by the measurement data acquired by the measurement acquisition unit 115. The start feature amount is, for example, the feature amount indicated by the marker 90 of the marker board 9 shown in FIG.

[0085] Fig. 15 is a diagram showing an example of marker 90. In the example of Fig. 15, horizontal length L11 and vertical length L12 of marker 91, and horizontal length L13 and vertical length L14 of marker 92 are shown. Marker 92 is placed at a height L15 from floor surface 100, and marker 91 is placed at a height L16 higher than marker 92. Server device 10 stores this length and height information as information about the size of marker 90.

[0086] The feature identification unit 119 recognizes the contours shown in the captured image, for example, by image processing that detects edges, and if the recognized contours indicate two rectangles arranged side by side in the vertical direction, it identifies the capture position of the captured image as the start position where the start feature amount indicating the marker 90 was measured. In this way, the feature identification unit 119 identifies the start position where the start feature amount was measured by recognizing the marker 90 whose features are known in advance. Note that the feature identification unit 119 may also recognize the marker 90 based on the aspect ratio of the recognized rectangle or the distance between the rectangles. According to this aspect, it is possible to more easily align the movement trajectories than when the start feature amount is not used.

[0087] Furthermore, the feature identification unit 119 identifies the last measurement position in the entire estimated trajectory as the end position where measurement ended. In the example of Fig. 14, the field worker W1 starts measurement on the fifth floor and finishes it on the third floor, but there is also a case where the field worker W1 turns around at the third floor and returns to the installation position of the stand 8 on the fifth floor to finish photographing. In that case, the feature identification unit 119 may use the start feature as the end feature and identify the position where the end feature was measured as the end position.

[0088] Furthermore, as described above, the feature identifying unit 119 identifies the connection paths connecting the layers shown in the drawing as locations having predetermined features in the drawing. For example, the feature identifying unit 119 identifies locations in the drawing where stairs (represented by successive rectangles and arrows) are shown as characteristic locations.

[0089] Next, the server device 10 causes the associating unit 120 to associate the positions of the ends of the partial trajectories corresponding to a layer with the characteristic locations identified in the drawing of that layer (S18). The associating unit 120 indicates the association of the characteristic locations in association data such as that shown in FIG.

[0090] Fig. 16 is a diagram showing an example of correspondence data. Correspondence data D5 shown in Fig. 16 is data in which a column of characteristic locations has been added to correspondence data D4 shown in Fig. 14. For example, the first three-dimensional coordinate E01, which is the earliest measured time, and the last three-dimensional coordinate E99, which is the latest measured time, on the first level (fifth floor) where the start position has been identified, respectively indicate the positions of the ends of the partial trajectory corresponding to the first level. The association unit 120 associates the three-dimensional coordinate E01 indicating the end with the start position identified for the first level, and associates the three-dimensional coordinate E99 with the connection path identified for the first level.

[0091] Furthermore, a three-dimensional coordinate F01 indicating the first measurement position on the second level (fourth floor) and a three-dimensional coordinate F99 indicating the last measurement position each indicate the end positions of a partial trajectory corresponding to the second level. The associating unit 120 associates both the three-dimensional coordinates F01 and F99 with the connection route identified for the second level. In detail, the associating unit 120 associates the three-dimensional coordinate F01, which was measured earlier, with the connection route to the first level that was measured earlier, and associates the three-dimensional coordinate F99, which was measured later, with the connection route to the third level that is measured later.

[0092] Furthermore, a three-dimensional coordinate G01 indicating the first measurement position on the third level (third floor) and a three-dimensional coordinate G99 indicating the last measurement position each indicate the end positions of a partial trajectory corresponding to the third level. The association unit 120 associates the three-dimensional coordinates G01 and G99 with the connection route identified for the third level. In detail, the association unit 120 associates the three-dimensional coordinate G01 with the connection route identified for the third level. Furthermore, the association unit 120 associates the three-dimensional coordinate G99 with the end position identified for the third level.

[0093] In this way, by associating the end positions of partial trajectories with characteristic points, it is possible to automatically align the positions of the movement trajectories on the drawing. Also, by associating the end positions of partial trajectories as characteristic points with the connecting paths, the accuracy of alignment can be improved as the number of connecting paths increases.

[0094] Then, the server device 10 outputs, via the image output unit 121, for each position where the imaging device 30 indicated by the partial trajectory performed a measurement, a site image showing the site photographed at that position. As described above, the image output unit 121 outputs, for each position where the imaging device 30 performed a measurement, an image expressed in a format according to the height of that position. For example, when the image output unit 121 indicates that the photographing position is lower than the floor surface, it creates and outputs an image indicating that the photographing position is under the floor (for example, an image including the character string "under the floor").

[0095] Furthermore, when the image output unit 121 indicates that the shooting position is higher than the ceiling, it creates and outputs an image indicating that the shooting position is above the ceiling (for example, an image including the character string "above the ceiling"). When the image output unit 121 indicates that the shooting position is higher than the floor surface and lower than the current situation, it creates and outputs an image indicating that the shooting position is inside the room (for example, an image including the character string "inside the room"). Since the measurement positions indicated by the partial trajectories are measurement positions on the same floor, by outputting images as described above, it is possible to show differences in height in the trajectories even on the same floor.

[0096] 10, the trajectory output unit 116 generates trajectory data and then executes partial trajectory processing, but the order of these processes may be reversed. In this case, the trajectory output unit 116, for example, recognizes stairs from the acquired measurement data and divides it into before and after the stairs, and generates trajectory data indicating the partial trajectories of each floor based on the divided measurement data. In either method, the site worker W1 only needs to photograph the construction site once with the imaging device 30, so the movement trajectory of each floor of the structure can be obtained with a single measurement.

[0097] <Configuration variations> The construction support system 1 may also include a measurement device other than the imaging device 30. The measurement device may include, for example, a distance image sensor that measures the distance to an object and outputs point cloud data indicating the measurement results as measurement data. The measurement device may also include an infrared sensor that measures the temperature of the object or a millimeter-wave sensor that measures the distance to the object or the speed of the object. The measurement device may also be a wide-angle camera or the like that is capable of capturing images with higher resolution.

[0098] Furthermore, in the construction support system 1, if the construction site does not have the breaker 6 turned off, the on-site terminal 20 and the imaging device 30 may be connected to the external power source 5 without the external battery 4. Furthermore, in the embodiment, the on-site terminal 20 transmits measurement data to the server device 10 by mobile communication even when the breaker 6 is turned off, but if the breaker 6 is not turned off, the on-site terminal 20 may transmit measurement data to the server device 10 by Wi-Fi communication by installing a Wi-Fi router at the construction site.

[0099] Also, for example, the server device 10 may be distributed across two or more devices, or may be replaced by a cloud computing system. Also, the on-site terminal 20 and the imaging device 30 may be integrated. Also, the functional configuration shown in FIG. 4 is an example, and is not limited to this. For example, the functions of the server device 10, the on-site terminal 20, and the imaging device 30 may each be distributed across two or more devices.

[0100] In addition, an operation performed by one function may be distributed to two or more functions, or two or more functions may be integrated into one function. In short, as long as the functions shown in Fig. 6 are realized by the entire construction support system 1, the devices that realize those functions may have any configuration.

[0101] The above-described embodiments are described as information processing devices such as the server device 10 and information processing systems such as the construction support system 1 including the server device 10, but may also be information processing methods. The information processing methods include steps of processes executed by the information processing system. The above-described embodiments may also be programs. The programs cause a computer to execute the processes executed by a similar information processing system.

[0102] <Additional Notes> Furthermore, it may be provided in the following aspects.

[0103] (1) An information processing system, in which, in a measurement acquisition step, a measuring device having a sensor that measures electromagnetic waves received from an object moves inside or outside a structure and acquires measurement data measured at each position of the movement path, and in a trajectory output step, trajectory data indicating the trajectory of the measuring device is output based on the acquired measurement data, and when the structure has two or more floors, the trajectory has two or more partial trajectories in which the positions where the measuring device took measurements fall within two or more different height ranges.

[0104] According to this aspect, the movement trajectory for each floor of the structure can be obtained by a single measurement.

[0105] (2) In the information processing system described in (1) above, in the hierarchy acquisition step, hierarchical data indicating the hierarchy of the structure is acquired, and in the correspondence output step, correspondence data indicating the correspondence between the hierarchy indicated by the acquired hierarchical data and the partial trajectory is output.

[0106] According to this aspect, it is possible to know which layer a partial trajectory belongs to.

[0107] (3) In the information processing system described in (2) above, the hierarchical data indicates a drawing of each level of the structure, and in the identification step, a location having a predetermined characteristic is identified in the drawing, and in the correspondence step, the position of the end of the partial trajectory corresponding to the level is associated with the identified location in the drawing of the level.

[0108] According to this aspect, it is possible to align the positions of the movement trajectories on the drawing.

[0109] (4) In the information processing system described in (3) above, in the identifying step, a connection path connecting each layer shown in the drawing is identified as the location.

[0110] According to this aspect, the more connection paths there are, the more accurate the alignment can be.

[0111] (5) In the information processing system described in (3) or (4) above, in the identification step, the starting position where the measurement starts in the drawing is identified as the location, and in the association step, the position of the end of the partial trajectory corresponding to the layer where the starting position is identified is associated with the starting position.

[0112] According to this aspect, it is possible to align the positions of the movement trajectories on the drawing.

[0113] (6) In the information processing system described in (5) above, in the identification step, the position at which a feature value defined as the starting position among the features indicated by the acquired measurement data is measured is identified as the starting position.

[0114] According to this aspect, the positioning of the movement trajectory can be easily performed.

[0115] (7) In the information processing system described in any one of (2) to (6) above, the hierarchical data indicates a drawing of each level of the structure, and in the correspondence output step, if the number of the partial trajectories is smaller than the level indicated by the acquired hierarchical data, data indicating the correspondence between the level and the partial trajectories that has the highest degree of match when the partial trajectories are placed on the drawing of the level is output as the correspondence data.

[0116] According to this aspect, even if only some of the layers are measured, the layers of the partial trajectory can be determined.

[0117] (8) In the information processing system described in any one of (1) to (7) above, in the image output step, an image is output for each position where the measuring device, indicated by the partial trajectory, performed a measurement, represented in a manner corresponding to the height of the position.

[0118] According to this embodiment, it is possible to show differences in height in the trajectories even on the same floor. Of course, this is not the case. Furthermore, the above-described embodiments and modifications may be combined in any desired manner.

[0119] Finally, while various embodiments of the present invention have been described, these are presented by way of example only and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. The embodiments and their modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the appended claims. [Explanation of symbols]

[0120] 1: Construction support system 10: Server device 11: Control section 20: On-site terminal 21: Control unit 30: Imaging device 31: Control unit 40: Remote terminal 41: Control unit 111: Information storage unit 112: Image processing unit 113: Data generation unit 114: Data output section 115: Measurement acquisition section 116: Trajectory output unit 117:Hierarchy acquisition part 118: Corresponding output section 119: Feature identification section 120: Mapping section 121: Image output unit 211: Display control unit 212: Operation reception unit 213: Operation control unit 214: Transmission control section 311: Display control unit 312: Operation reception section 313: Shooting control unit 314: Transmission control section 411: Display control unit 412: Operation reception section

Claims

1. An information processing system, In the measurement acquisition step, a measurement device having a sensor for measuring electromagnetic waves arriving from an object moves inside or outside the structure and acquires measurement data measured at each position along the movement path; In the trajectory output step, trajectory data indicating the trajectory of the measuring device is output based on the acquired measurement data, and if the structure has two or more floors, the trajectory has two or more partial trajectories in which the positions where the measuring device performed measurements fall within two or more different height ranges.

2. 2. The information processing system according to claim 1, In the hierarchical level acquisition step, hierarchical level data indicating the hierarchical levels of the structure are acquired, In the correspondence output step, correspondence data indicating a correspondence relationship between the hierarchical layer indicated by the acquired hierarchical data and the partial trajectory is output.

3. 3. The information processing system according to claim 2, the hierarchical data indicates a drawing of each level of the structure; In the identifying step, a portion having a predetermined feature is identified in the drawing; In the associating step, the position of the end of the partial trajectory corresponding to the layer is associated with the location specified in the drawing of the layer.

4. 4. The information processing system according to claim 3, In the identifying step, a connection path connecting each layer shown in the drawing is identified as the location.

5. 5. The information processing system according to claim 3, In the specifying step, a start position where the measurement is started is specified as the location in the drawing; In the associating step, an end position of the partial trajectory corresponding to the layer in which the start position is identified is associated with the start position.

6. 6. The information processing system according to claim 5, In the specifying step, a position at which a feature quantity defined as the start position among the feature quantities indicated by the acquired measurement data is measured is specified as the start position.

7. In the information processing system according to any one of claims 2 to 6, the hierarchical data indicates a drawing of each level of the structure; In the correspondence output step, if the number of partial trajectories is smaller than the hierarchical level indicated by the acquired hierarchical data, data indicating the correspondence between the hierarchical level and the partial trajectories that has the highest degree of match when the partial trajectories are placed on a drawing of the hierarchical level is output as the correspondence data.

8. In the information processing system according to any one of claims 1 to 7, In the image output step, an image is outputted for each position where the measurement device performed a measurement indicated by the partial trajectory, the image being displayed in a manner according to the height of the position.

Images