Information processing systems, information processing methods, and programs

Abstract

This technology helps to understand the differences between actual measurement results and the standard state in temporary construction work, and to streamline displacement verification work. [Solution] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor, wherein the processor is configured to perform the following steps by reading a program: in the acquisition step, point cloud data relating to temporary construction work is acquired; and in the presentation step, difference data relating to the difference between the point cloud data and the reference data is presented, which is identified based on the point cloud data and the reference data relating to the predetermined temporary construction work.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses a technique for providing a three-dimensional measurement system in an excavation earth retaining work that grasps the behavior of the entire structure and makes it easier to identify the cause of displacement by using a displacement measurement method for an earth retaining wall that can comprehensively evaluate the behavior of the excavation earth retaining work. In the three-dimensional measurement system in the excavation earth retaining work, it includes a plurality of high-sensitivity sensors arranged on the surface of the earth retaining wall, a plurality of low-sensitivity sensors arranged to complement between the plurality of high-sensitivity sensors, a data processing device that processes measurement data from the high-sensitivity sensors and the low-sensitivity sensors, and a display device connected to this data processing device, measures displacement data of the entire surface of the earth retaining wall, and grasps the surface behavior of the earth retaining wall.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, there is a need for a technique to grasp the difference between the actual measurement results and the reference state in temporary construction and to improve the efficiency of displacement confirmation work.

[0005] In view of the above circumstances, the present invention aims to provide a technique for grasping the difference between the actual measurement results and the reference state in temporary construction and improving the efficiency of displacement confirmation work.

Means for Solving the Problems

[0006] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor, the processor configured to perform the following steps by reading a program, wherein in the acquisition step, point cloud data relating to temporary construction work is acquired, and in the presentation step, difference data relating to the difference between the point cloud data and the reference data is presented, which is identified based on the point cloud data and the reference data relating to the predetermined temporary construction work. [Brief explanation of the drawing]

[0007] [Figure 1] This is a diagram showing the configuration of Information Processing System 1. [Figure 2] This is a block diagram showing the hardware configuration of the information processing device 2. [Figure 3] This is a block diagram showing the functional configuration of an information processing device 2 according to one embodiment. [Figure 4] This flowchart shows an overview of the processes performed by Information Processing System 1. [Figure 5] This is an activity diagram showing a specific example 1 of an information processing method according to one embodiment. [Figure 6] This is an activity diagram showing a specific example 2 of an information processing method according to one embodiment. [Figure 7] This is a schematic diagram showing how user U measures the site of earth retention work using measuring device 3. [Figure 8] This is a schematic diagram showing the forward line of sight of measuring device 3, which was used to measure the site of the earth retention work. [Figure 9] This is an example of a difference display screen 7, which shows the difference between point cloud data Dp acquired at different times. [Figure 10] This is an example of a difference display screen 8, which shows the difference between point cloud data Dp acquired at different times. [Modes for carrying out the invention]

[0008] [Embodiment] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other.

[0009] Incidentally, the program for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable medium from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0010] Furthermore, in various information processing according to one embodiment, an input and an output corresponding to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of the information referenced in such information processing (hereinafter referred to as "reference information") is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression equation constructed by a statistical method), or a trained model that has been pre-trained to learn the correlation between input and output, or a generative AI such as a large-scale language model that can output a desired result by inputting a prompt (these models include parameters that construct the correlation relationship between input and output) or a visual language model.

[0011] Furthermore, in one embodiment, "part" may include, for example, hardware resources implemented by a circuit in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and this information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense.

[0012] Furthermore, a circuit in a broad sense is a circuit realized by combining at least a suitable combination of circuits, circuits, processors, and memory. The processor may be a general-purpose processor or a dedicated circuit. In other words, it includes application-specific integrated circuits (ASICs), programmable logic devices (for example, simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[0013] 1. Hardware Configuration This section describes a hardware configuration according to one embodiment.

[0014] 1.1 Information Processing System 1 Figure 1 is a diagram illustrating the configuration of information processing system 1. Information processing system 1 comprises an information processing device 2 and a measuring device 3. At least the information processing device 2 and the measuring device 3 are configured to communicate with each other via a telecommunications line. Here, the system exemplified in information processing system 1 consists of one or more devices or components. Therefore, even the information processing device 2 or the measuring device 3 alone can be an example of a system. More specifically, information processing system 1 may include elements selected from the group consisting of information processing devices 2 and measuring devices 3. Alternatively, multiple information processing devices 2 or measuring devices 3 may be used. Unselected elements may not be included in information processing system 1, but may be electrically connected to the selected elements as external elements.

[0015] 1.2 Information Processing Device 2 FIG. 2 is a block diagram showing the hardware configuration of the information processing apparatus 2. The information processing apparatus 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23. The communication unit 21, the storage unit 22, and the processor 23 are electrically connected to each other inside the information processing apparatus 2 via the communication bus 20.

[0016] Although wired communication means such as USB, IEEE 1394, Thunderbolt (registered trademark), and wired LAN network communication are preferable for the communication unit 21, wireless LAN network communication, mobile communication such as 3G / LTE / 5G, and BLUETOOTH (registered trademark) communication may be included as necessary. That is, it is more preferable to implement as a set of these plural communication means. That is, the information processing apparatus 2 may transmit and receive various information from the outside via the communication unit 21 and the communication network 11. In particular, in one embodiment, a measuring apparatus 3 is connected as an external device of the information processing apparatus 2 through the communication unit 21 and the communication network 11. Here, the connection between the apparatuses may be wired or wireless. When the apparatuses are connected by a wired cable, the standard of the wired cable is not particularly limited, and it may be a USB cable corresponding to Thunderbolt (registered trademark) or a Gige / MV standard.

[0017] The storage unit 22 stores various information defined as described above. This can be implemented as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing apparatus 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to the calculation of the program. The storage unit 22 stores various programs, variables, and the like related to the information processing apparatus 2 executed by the processor 23. The storage unit 22 stores reference information.

[0018] The processor 23 performs processing and control of the overall operation related to the information processing device 2. The processor 23 is, for example, a central processing unit (CPU). The processor 23 realizes various functions related to the information processing device 2 by reading predetermined programs stored in the memory unit 22. That is, information processing by software stored in the memory unit 22 is concretely realized by the processor 23, which is an example of hardware, and can be executed as each functional unit included in the processor 23. Note that the processor 23 is not limited to being a single unit, and may be implemented with multiple processors 23 for each function, or a combination thereof.

[0019] 1.3 Measuring device 3

[0020] The measuring device 3 measures objects at the temporary construction site and generates point cloud data, which is then input into the information processing device 2 for information processing. As shown in Figure 2, the measuring device 3 comprises a connection interface 31, a storage unit 32, a control unit 33, and a LiDAR 34. These components are electrically connected within the measuring device 3 via a communication bus 30.

[0021] The connection interface 31 is an interface for bidirectional data communication with the information processing device 2, and may be based on USB, Gigabit, LVDS, or Thunderbolt®. This allows for the reception of setting signals transmitted from an external source (especially the information processing device 2) and the transmission of point cloud data.

[0022] The storage unit 32 is composed of a non-volatile storage medium such as NOR flash memory or EEPROM, and can store firmware and tables related to measurement conditions, correction parameters, timestamp settings, calibration information, etc. The storage unit 32 may also function as a storage area for temporarily holding point cloud data acquired during the measurement process.

[0023] The control unit 33 is a control circuit that comprehensively controls the operation of the entire measuring device 3. Based on the setting values ​​received via the connection interface 31, the control unit 33 operates the LiDAR 34 and performs actions such as starting and ending the measurement operation, setting scanning conditions and measurement timing, and adding a timestamp to the acquired measurement data.

[0024] The LiDAR34 is configured to acquire point cloud data consisting of a set of reflection points by measuring the distance between the LiDAR34's position and the reflection points based on the time or phase difference between irradiating an object with laser light and receiving the reflected light. Furthermore, the LiDAR34 is configured to calculate three-dimensional position information for each measurement point based on distance information acquired from the object being measured and the corresponding angle information. This allows the LiDAR34 to generate point cloud data consisting of multiple measurement points. The generated point cloud data is transmitted to the information processing device 2 via the connection interface 31 under the control of the control unit 33.

[0025] In one embodiment, the measuring device 3 may be configured to perform measurements continuously while moving. In this case, the measuring device 3 can be configured to integrate point cloud data from a wide area of ​​temporary construction sites using a method such as SLAM (Simultaneous Localization and Mapping). For example, the measuring device 3 may be configured to acquire information (pose) representing its own position and orientation corresponding to each measurement time, and to represent the point cloud data in the same reference coordinate system based on the pose.

[0026] The measuring device 3 may be housed in an integrated enclosure with the information processing device 2, or it may be configured as an independent device and connected externally. Furthermore, the measuring device 3 may be a fixed installation type, or it may be implemented as a portable type mounted on a vehicle, work machine, or on a worker. All of these embodiments are within the technical scope of the present invention.

[0027] As described above, in one embodiment, the information processing system 1 comprises an information processing device 2 (an example of a server device) having a processor 23, and a measuring device 3 (an example of a terminal) that can access the information processing device 2. With this configuration, the information processing system 1 can be implemented in various ways.

[0028] 2. Functional Configuration Next, with reference to Figure 3, the functional configurations of the information processing device 2 in the information processing system 1 will be described. Figure 3 is a block diagram showing the functional configuration of the information processing device 2 according to one embodiment. As shown in Figure 3, the processor 23 functions as an acquisition unit 231, a specific unit 232, a first alignment unit 233, a reconstruction unit 234, a receiving unit 235, an extraction unit 236, a second alignment unit 237, a presentation unit 238, and a calculation unit 239 by executing various programs stored in the storage unit 22. In other words, information processing by software stored in the storage unit 22 is concretely realized by the processor 23, which is an example of hardware, and can be executed as each functional unit included in the processor 23.

[0029] The acquisition unit 231 is configured to acquire various types of information as an acquisition step. Specifically, the acquisition unit 231 is configured to acquire information via the communication unit 21 or the storage unit 22 and to be readable into the working memory. For example, as an acquisition step, the acquisition unit 231 acquires point cloud data related to temporary construction work.

[0030] The identification unit 232 is configured to identify a set of poses that satisfy predetermined conditions from among the first poses corresponding to multiple time points, as a identification step.

[0031] The first alignment unit 233 is configured to perform alignment between a plurality of first local point clouds as a first alignment step.

[0032] The reconstruction unit 234 is configured to correct the pose by performing loop closure as a reconstruction step, and then reconstruct the point cloud data based on the corrected pose.

[0033] The reception unit 235 is configured to accept the specification of a region of interest from the point cloud data for which the difference will be calculated, as a reception step. Furthermore, the reception unit 235 is configured to accept the specification of a region of interest from the reference data for which the difference will be calculated, as a reception step.

[0034] The extraction unit 236 is configured to extract a second local point cloud from the point cloud data based on the region of interest as an extraction step, and to extract a local region corresponding to the second local point cloud from the reference data. Furthermore, the extraction unit 236 is configured to extract a local region from the reference data based on the region of interest as an extraction step, and to extract a second local point cloud corresponding to the region of interest from the point cloud data.

[0035] The second alignment unit 237 is configured to perform alignment between the second local point cloud and the local region as a second alignment step.

[0036] The presentation unit 238 is configured to present, as a presentation step, differential data relating to the difference between point cloud data and predetermined reference data relating to temporary construction work, based on point cloud data and reference data relating to the reference data. Furthermore, the presentation unit 238 performs processing to display various information on a display medium in a manner recognizable to the user. When using the phrase "display," it is not particularly important whether the display medium to be displayed is in the local environment or whether the processing to display it is performed via the communication network 11. As a result of the processing by the presentation unit 238, various information is presented to various users operating the information processing device 2, for example, via the display unit 24. The various information presented is visual information such as screens, images, icons, and messages. The presentation unit 238 may generate the visual information itself, or it may generate only rendering information for displaying the visual information on the display unit 24.

[0037] The arithmetic unit 239 is configured to perform various calculations related to the information processing device 2 as calculation steps. The type of calculation is not particularly limited.

[0038] 3. Information Processing Methods 3.1 Overview 1 As mentioned above, the information processing system comprises at least one processor, which functions as the following parts by reading a program. In other words, such an information processing method comprises each step of the information processing system 1. From another perspective, such an information processing program causes at least one computer to execute each step of the information processing system 1. Figure 4 is a flowchart outlining the process performed by the information processing system 1. The steps shown in Figure 4 will be explained below.

[0039] First, the acquisition unit 231 acquires point cloud data Dp related to the temporary construction work (step S001). Next, the presentation unit 238 presents difference data Dd, which is the difference between the point cloud data Dp and the reference data Dr, based on the point cloud data Dp and the reference data Dr, which is predetermined for the temporary construction work (step S002).

[0040] This configuration allows for the identification of differences between actual measurement results and the design or reference state in temporary construction work based on point cloud data, thereby streamlining displacement verification work that previously relied on on-site confirmation or manual labor.

[0041] 3.2 Specific Example 1 Next, we will describe specific examples that may be included in the overview of the information processing method described above. Figure 5 is an activity diagram showing specific example 1 of the information processing method according to one embodiment. This information processing method defines a series of processes for presenting difference data Dd based on point cloud data Dp and reference data Dr acquired from the measuring device 3. Note that the order of the processes included in the information processing method can be changed as appropriate, multiple processes may be executed simultaneously, and some processes may be omitted.

[0042] Here, "temporary construction work" refers to construction work related to temporary structures that are temporarily erected in order to carry out the construction of the target structure in civil engineering and construction work, and that are removed after the completion of the work. Examples of temporary structures include, but are not limited to, temporary platforms, temporary piers, scaffolding, retaining walls, scaffolding, shoring, work platforms, temporary walkways, temporary bridges, and temporary fences. In the following specific examples, retaining wall construction will be used as an example of temporary construction work, but the same can be applied to temporary construction work related to other temporary structures.

[0043] Earth retention work refers to construction work that involves installing earth retention walls and other structures to ensure the stability of the ground during excavation. Measurements for earth retention work include measurements of the earth retention walls, the ground around the earth retention walls, and the condition of surrounding structures. For measurements of temporary works such as earth retention work, 3D mapping based on SLAM (Simultaneous Localization and Mapping) is effective. SLAM is a technology that estimates the position and orientation of a mobile object based on the measurement results of sensors mounted on the mobile object, while simultaneously constructing or updating a map of the surrounding environment.

[0044] In one embodiment, SLAM is performed based on measurement results acquired by a LiDAR34 mounted on the measuring device to estimate the position and orientation of the measuring device and generate point cloud data Dp of the surrounding environment. Since measurements by LiDAR34 are hardly affected by ambient light, stable performance is achieved even under conditions where visual sensors struggle, such as low-light environments, dark places, and environments with strong shadows, like earth retention construction sites.

[0045] First, the LiDAR 34 of the measuring device 3 measures the earth retention construction site in response to the operation of the measuring device 3 by user U (activity A101).

[0046] Next, LiDAR34 creates point cloud data Dp for the entire earth retention construction site (Activity A102). The details of the point cloud data Dp creation process will be described in detail according to Activity A2 in Figure 6.

[0047] Next, the acquisition unit 231 acquires point cloud data Dp from the measurement device 3 via the communication network 11 (activity A103). In other words, as an acquisition step, the acquisition unit 231 acquires point cloud data Dp related to the temporary construction work.

[0048] Next, the acquisition unit 231 acquires point cloud data Dp, which is the design data for earth retention work and is pre-stored in the storage unit 22, as reference data Dr via the communication bus 20 (Activity A104). In other words, reference data Dr is data that includes point cloud C. With this configuration, since data including point cloud can be used as reference data Dr, it is possible to compare point cloud data Dp acquired at different times or under different conditions and flexibly analyze differences caused by temporal changes or environmental fluctuations. Furthermore, the data acquired as reference data Dr is not limited to point cloud data Dp, but may also be BIM data Db. In other words, reference data Dr is BIM data Db. With this configuration, by using BIM data Db as reference data Dr, it is possible to directly compare the differences between design information and actual measurement results after construction, and to accurately evaluate structural displacement and construction errors. Furthermore, data relating to a reference plane predetermined by the user U may be acquired as reference data Dr. Furthermore, reference data Dr may also be, for example, a point cloud generated from CAD. Thus, the reference data Dr can be any data that represents the shape of space in three dimensions, and is not limited to point cloud data Dp, BIM data Db, etc.

[0049] Next, the second alignment unit 237 performs overall alignment of the point cloud data Dp and BIM data Db (Activity A105). Overall alignment is a preliminary process performed on the entire point cloud data Dp and BIM data Db in order to align them with respect to the region of interest Ri, which will be described later. Specifically, for example, overall alignment may be performed manually, automatically, or a combination of manual and automatic alignment. For example, the second alignment unit 237 may perform ICP (Iterative Closest Point) as an automatic overall alignment method. In addition, to reduce the need for manual alignment, the second alignment unit 237 may perform overall alignment on point cloud data Dp for which SLAM has been performed in advance using GNSS RTK (Global Navigation Satellite System Real-Time Kinematic), GNSS PPK (Global Navigation Satellite System Post-Processing Kinematic), etc. as auxiliary methods. In this configuration, since the point cloud data Dp can be pre-positioned in the reference coordinate system, the final overall alignment accuracy can be further improved by applying ICP to the point cloud data Dp positioned in the reference coordinate system.

[0050] Next, the reception unit 235 receives the specification of the region of interest Ri in the point cloud data Dp from the user U (Activity A106). The region of interest Ri is the region corresponding to the point cloud data Dp or BIM data Db (both examples of reference data Dr), which is designated by the user U as the target for calculating precise difference data. The region designated by the user U may be a predetermined partitioned area in two-dimensional or three-dimensional space, and may be specified, for example, in units of cells on a two-dimensional plane or in units of voxels in three-dimensional space. In other words, as a reception step, the reception unit 235 receives the specification of the region of interest Ri from the point cloud data Dp that is to be used for calculating the difference.

[0051] Next, the extraction unit 236 extracts the region designated as the region of interest Ri from the point cloud data Dp as the second local point cloud Cl2 (Activity A107). Furthermore, the extraction unit 236 extracts the local region Rl corresponding to the second local point cloud Cl2 from the BIM data Db (Activity A108). In other words, as an extraction step, the extraction unit 236 extracts the second local point cloud Cl2 from the point cloud data Dp based on the region of interest Ri, and extracts the local region Rl corresponding to the second local point cloud Cl2 from the BIM data Db (an example of reference data Dr).

[0052] Next, the second alignment unit 237 performs alignment between the extracted second local point cloud Cl2 and the local region Rl (activity A109). The alignment between the second local point cloud Cl2 and the local region Rl may be performed using a method such as ICP. In other words, the second alignment unit 237 performs alignment between the second local point cloud Cl2 and the local region Rl as the second alignment step. With this configuration, alignment and difference calculation can be performed only on the region of interest Ri rather than the entire point cloud data Dp, making it possible to efficiently and accurately grasp the displacement of areas that require attention in the temporary construction environment, which changes significantly day by day as construction progresses.

[0053] Similarly, when the reception unit 235 receives a request as a reception step for the specification of a region of interest Ri to be used to calculate the difference from the BIM data Db (an example of reference data Dr), the extraction unit 236 extracts a local region Rl from the BIM data Db (an example of reference data Dr) based on the region of interest Ri, and extracts a second local point cloud Cl2 corresponding to the region of interest Ri from the point cloud data Dp. With this configuration, difference analysis can be performed starting from the region of interest Ri specified from the reference data, enabling flexible difference evaluation based on the perspective of the designer or manager. Although it has been explained that the region of interest Ri is specified by the user U, the region of interest Ri may also be automatically estimated by the processor 23. Specifically, for example, the processor 23 may identify the region with the largest difference as the region of interest Ri.

[0054] Next, the calculation unit 239 calculates the difference between the second local point cloud Cl2 and the local region Rl (activity A110). The calculated difference is stored in the storage unit 22 as difference data Dd. Specifically, for example, the difference may be calculated by calculating the point-to-plane distance between the second local point cloud Cl2 and the local region Rl. However, the method of calculating the difference is not limited to this, and may also be point-to-point distance, normal displacement, distance after meshing, volume difference, etc. In other words, the difference data Dd is data relating to the difference between the second local point cloud Cl2 and the local region Rl.

[0055] Next, the presentation unit 238 displays the difference data Dd on the display unit 24 (activity A111). Specifically, for example, as a presentation step, the presentation unit 238 presents the difference data Dd in association with at least one of the aligned second local point cloud Cl2 and local region Rl. More specifically, the presentation unit 238 may display the second local point cloud Cl2 or local region Rl with various colors depending on the calculated difference amount between the second local point cloud Cl2 and local region Rl. The presentation unit 238 may also display the difference between the second local point cloud Cl2 and local region Rl in a graph, table, etc. According to this embodiment, by presenting the difference data based on the difference between the aligned second local point cloud and local region Rl in association with at least one of the second local point cloud and local region Rl, the distribution of displacement and differences can be presented in a way that allows for intuitive understanding. In other words, the presentation unit 238, as a presentation step, presents difference data Dd relating to the difference between the point cloud data Dp and the predetermined reference data Dr related to the temporary construction work, based on the point cloud data Dp and the reference data Dr. A more specific presentation method will be described in detail in Figures 9 and 10.

[0056] The above outlines the processing flow for Specific Example 1. Please note that the assumptions and processing flow described above are merely examples and are not exhaustive. 3.3 Overview 2

[0057] In SLAM, information representing the current position and orientation of the measurement device 3 (hereinafter referred to as "pose") is calculated based on the estimation result at the previous time. As a result, measurement errors at each time point accumulate sequentially, leading to position estimation errors. This time-series accumulation of errors is also called drift. One of the representative methods for suppressing and eliminating drift accumulated in SLAM is loop closure. In loop closure, when it is determined that a point previously observed by the measurement device 3 matches or is close to the measurement position at a certain time point, a constraint condition is introduced between these two poses, and the entire pose graph composed of multiple poses is optimized. In general loop closure, loop detection is performed based on spatial proximity, and the pose graph optimization process readjusts the estimated value of each pose so that it satisfies the constraint condition. This corrects the drift and restores consistency between the map and the movement trajectory of the measurement device 3. In other words, in the information processing system 1, the reconstruction unit 234 further corrects the first pose P1 by performing loop closure as a reconstruction step, and reconstructs the point cloud data Dp based on the corrected second pose P2 of the first pose P1. With this configuration, positional shifts accumulated during moving measurements can be corrected by loop closure, and the consistency of the entire point cloud data Dp can be maintained even during measurements over long distances or long periods of time.

[0058] However, if loop detection relies solely on spatial proximity, it becomes difficult to close loops between poses that are far apart. For example, while the loop search radius is usually set to around 5 to 10 meters, in temporary construction, two poses are often located far apart, such as 30 meters apart. Therefore, a loop detection method is desired that does not rely solely on spatial proximity and can introduce effective constraints between poses even when observing the same temporary structure from spatially separated locations. Furthermore, visual and geometric features based on measurements using imaging devices, etc., can produce numerous false positives in temporary construction environments with repeating structures, reducing the reliability of loop closure. Moreover, conventional loop closure mainly corrects translational and yaw angle drifts, and it is difficult to adequately correct tilt and roll drifts, so a method that effectively suppresses attitude errors of the measurement device 3 is needed.

[0059] In one embodiment, a set of poses Ps that are highly likely to be observing the same or adjacent region R are identified, and a loop constraint is added between the two first poses P1 included in the set of poses Ps to perform loop closure. Since the set of poses Ps is identified based on the orientation of the measuring device or the region to be observed, regardless of spatial distance, it is possible to construct highly consistent point cloud data Dp while suppressing false detections compared to conventional SLAM based on visual features, even in temporary construction environments with repeating structures.

[0060] 3.4 Specific Example 2 Figure 6 is an activity diagram showing specific example 2 of an information processing method according to one embodiment. This information processing method defines the detailed information processing flow of activity A102 in Figure 5, that is, a series of processes from the configuration of point cloud data Dp to the execution of loop closure and reconstruction of point cloud data Dp. The order of processes included in the information processing method can be changed as appropriate, multiple processes may be executed simultaneously, and some processes may be omitted.

[0061] First, LiDAR34 constructs point cloud data Dp based on the measurement results (Activity A201). In other words, point cloud data Dp is data constructed by transforming the coordinates of each point acquired by the measurement device 3 at multiple time points into the same reference coordinate system based on the first pose P1 of the measurement device corresponding to each time point was acquired. The first pose P1 includes information such as a position component Pp indicating the position of the measurement device and an orientation component Po indicating the orientation of the measurement device. With this configuration, by representing points acquired by the measurement device 3 at multiple time points in the same reference coordinate system based on the pose of the measurement device 3 corresponding to that time, point cloud data Dp is constructed by integrating the point clouds acquired in a time series, making it possible to grasp a wide area of ​​temporary construction sites as a whole.

[0062] The point cloud data Dp generated by the LiDAR34 is transmitted to the information processing device 2 via the connection interface 31 and the communication network 11. Subsequently, the calculation unit 239 defines the forward direction of the first pose P1 from the orientation component Po of the first pose P1 of the measuring device (activity A202).

[0063] Next, the identification unit 232 identifies pose sets Ps in which the forward direction of the measuring device 3 points to the same or adjacent region R from different viewpoints (activity A203). Specifically, the identification unit 232 identifies a pose set Ps if a half-line extending forward from the first pose P1 intersects on the XY plane and the first pose P1 is located within a predetermined range, and it is determined that it points to the same or adjacent region R. In other words, as a further identification step, the identification unit 232 identifies pose sets Ps that satisfy a predetermined condition from among the first poses P1 corresponding to multiple time points. The condition is that the direction determined by the orientation component Po of each first pose P1 included in the pose set Ps points to the same or adjacent region R. With this embodiment, by identifying only pose sets Ps in which the forward direction points to the same or adjacent region R, irrelevant pose sets Ps can be excluded, and false detection of loop constraints related to loop confinement can be greatly reduced.

[0064] Next, the calculation unit 239 constructs a first local point cloud Cl1 from each first pose P1 included in the pose set Ps and other first poses P1 adjacent to each first pose P1 included in the pose set Ps (activity A204). In other words, the first local point cloud Cl1 is a point cloud C corresponding to each first pose P1 included in the pose set Ps extracted from the point cloud data Dp. Before proceeding to the processing of activity A205, the calculation unit 239 may pre-evaluate whether the constructed first local point cloud Cl1 includes an overlapping region R of more than a predetermined standard, for example, using a fitness metric.

[0065] Next, the first alignment unit 233 performs alignment of multiple first local point clouds Cl1 as the first alignment step (activity A205). Specifically, for example, the first alignment unit 233 may perform alignment of multiple first local point clouds Cl1 using ICP (Iterative Closest Point). With this embodiment, alignment is performed between the first local point clouds Cl1 corresponding to the pose set Ps, and loop closure can be performed only when they are geometrically aligned, thus suppressing unnecessary loop closure based on incorrect correspondences.

[0066] Next, the identification unit 232 determines whether the consistency of the first local point cloud Cl1 has improved due to the alignment performed by the first alignment unit 233. If the consistency improves due to the alignment, the process proceeds to activity A206; otherwise, it returns to activity A203, which identifies other pose sets Ps.

[0067] Next, the calculation unit 239 identifies two first poses P1 from the first pose P1 included in the pose set Ps, and adds a constraint to perform loop closure between the two first poses P1 (activity A206). More specifically, the calculation unit 239 adds an edge as a new constraint to the pose graph, which is composed of two temporally separated first poses P1 as nodes, and as a result, a closed loop is formed on the pose graph. With this embodiment, even if each first pose P1 is separated on the pose graph, an edge related to loop closure can be added between poses that are pointing to the same or adjacent region R. The processing from activity A203 to activity A206 may be repeated until the identification of the pose set Ps for the entire point cloud data Dp is completed.

[0068] Next, the reconstruction unit 234, as a reconstruction step, corrects the first pose P1 included in the pose set Ps by performing loop closure (activity A207) including all newly added constraints (edges), and reconstructs the point cloud data Dp based on the corrected second pose P2 of the first pose P1 (activity A208). With this configuration, even if the poses are located far apart on the pose graph, it is possible to identify pose sets Ps that are highly likely to be observing the same or adjacent region R, and high-precision reconstruction can be performed even when the poses are located far apart, which was previously difficult.

[0069] The above outlines the processing flow for Specific Example 2. Please note that the assumptions and processing flow described above are merely examples and are not exhaustive.

[0070] 4. Details of Information Processing This section will explain the details of the information processing outlined in the previous section, using diagrams and other visual aids.

[0071] 4.1 Measurement Embodiment Figure 7 is a schematic diagram showing how user U measures the site of earth retention work using the measuring device 3.

[0072] Specifically, Figure 7 shows user U using the measuring device 3 to take measurements within the shoring construction site, drawing a trajectory T. Trajectory T indicates that user U is measuring the same measurement target, area 51, from multiple different locations using the measuring device 3. Similarly, trajectory T indicates that user U is measuring area 52 from multiple different locations. As shown in Figure 7, in one embodiment, a handheld measuring device 3 may be used as the measuring device 3 used by user U.

[0073] 4.2 Forward line of sight of measuring device 3 Figure 8 is a schematic diagram of the forward line of sight of measuring device 3, which measured the site of the earth retention work.

[0074] The straight line in Figure 8 represents the forward line of sight determined by the orientation component Po of the measuring device 3. Regions 61 and 62 correspond to regions 51 and 52 shown in Figure 7, respectively. The fact that the measuring device 3 measured regions 51 and 52 from multiple different positions is indicated by the intersection of the forward lines of sight of the measuring device 3 on the XY plane.

[0075] 4.3 Difference display screen 7 Figure 9 shows an example of a difference display screen 7 that displays the difference between point cloud data Dp acquired at different times. The difference display screen 7 includes region 71, region 72, and highlighted area 711.

[0076] The area 71 enclosed by the dotted line is the region of interest Ri, specified by user U, for which the difference from the point cloud data Dp is to be calculated. The highlighted areas 711 indicate the difference between point cloud data Dp acquired at different times, with a change in color for each point corresponding to the difference. Specifically, for example, colors are assigned according to the magnitude and direction of the distance difference or displacement between points in the point cloud. For example, the display unit 238 may display points with large differences in warm colors and points with small differences in cool or intermediate colors, or if the difference exceeds a threshold, the difference may be displayed in a manner that highlights the points that exceed the threshold. This makes it possible to visually grasp the shape changes and positional shifts between point cloud data Dp.

[0077] Area 72 is a color scale that shows the correspondence between the difference amount and the displayed color. Area 72 displays the numerical range of the distance difference or displacement amount between point clouds, indicating how much of a difference the color of each point on the difference display screen 7 corresponds to. This allows the user to visually confirm the presence or absence and distribution of differences, as well as to quantitatively grasp the magnitude of the difference.

[0078] 4.4 Difference display screen 8 Figure 10 is an example of a difference display screen 8 that shows the difference between point cloud data Dp acquired at different time points.

[0079] Table 81 shows the difference with the reference data Dr. X(T1) indicates the position of the reference point at the measurement site, and the difference with the reference point (T2) indicates the difference between the reference point and the measured point cloud data Dp at the position of the reference point. Graph 82 is a graph plotting Table 81. Thus, the difference between the point cloud data Dp and the reference data Dr may be represented by color visualization, tabular format, or graph display, but is not limited to these. It may also be represented by numerical display, symbolic display, textual information, heatmap, contour display, vector display, emphasis of threshold exceedance points, display of statistics, or any combination thereof.

[0080] [others] Regarding the information processing method according to the above-described embodiment, the following embodiments may be adopted.

[0081] At least one of the devices included in the information processing system 1 may be located outside of Japan. For example, the information processing device 2 may be installed outside of Japan, and a user in Japan may access the information processing device 2 using their own measuring device 3, or the information processing device 2 may be installed in Japan, and a user outside of Japan may access the information processing device 2 using their own measuring device 3. According to such a configuration, a more convenient experience can be provided to the user through various management methods.

[0082] The information processing device 2 may be in an on-premise configuration or a cloud configuration. In the case of a cloud-based information processing device 2, for example, the above-mentioned functions and processing may be provided in the form of SaaS (Software as a Service) or cloud computing.

[0083] In the above embodiment, the information processing device 2 performed various storage and control functions, but multiple external devices may be used instead of the information processing device 2. That is, various information and programs may be stored in a distributed manner across multiple external devices using blockchain technology or the like. Also, in the above embodiment, the acquisition unit 231, identification unit 232, first alignment unit 233, reconstruction unit 234, reception unit 235, extraction unit 236, second alignment unit 237, presentation unit 238, and calculation unit 239 are described as functional units realized by the processor 23 of the information processing device 2, but at least a part of these may be implemented as functional units realized by an external server (not shown), or as functional units realized by the control unit 33 of the measuring device 3.

[0084] The product may be provided in any of the following embodiments.

[0085] (1) An information processing system comprising at least one processor, wherein the processor is configured to perform the following steps by reading a program, the acquisition step of acquiring point cloud data relating to temporary construction work, and the presentation step of presenting difference data relating to the difference between the point cloud data and the reference data, which is identified based on the point cloud data and predetermined reference data relating to the temporary construction work.

[0086] This configuration allows for the identification of differences between actual measurement results and the design or reference state in temporary construction work based on point cloud data, thereby streamlining displacement verification work that previously relied on on-site confirmation or manual labor.

[0087] (2) In the information processing system described in (1) above, the point cloud data is data obtained by transforming the coordinates of each point acquired by the measuring device at multiple time points into the same reference coordinate system based on a first pose of the measuring device corresponding to each time point was acquired, wherein the first pose is information including a position component indicating the position of the measuring device and an orientation component indicating the orientation of the measuring device.

[0088] According to this configuration, points acquired by measuring devices at multiple time points are represented in the same reference coordinate system based on the pose of the measuring device corresponding to that time. This allows for the creation of integrated point cloud data from point clouds acquired in a time series, enabling a comprehensive understanding of a wide-area temporary construction site.

[0089] (3) The information processing system described in (2) above, further comprising a reconstruction step, which corrects the first pose by performing loop closure, and reconstructs the point cloud data based on the second pose after the correction of the first pose.

[0090] In this configuration, positional shifts accumulated during mobile measurements can be corrected by loop closure, and the overall consistency of the point cloud data can be maintained even during measurements over long distances or long periods of time.

[0091] (4) In the information processing system described in (2) or (3) above, in a further specific step, a set of poses that satisfies a predetermined condition is identified from among a plurality of first poses corresponding to the aforementioned time, wherein the condition is that the direction determined by the orientation component of each first pose included in the set of poses points toward the same or adjacent regions, and in the reconstruction step, the first poses included in the set of poses are corrected by performing loop closure, and the point cloud data is reconstructed based on the corrected second poses of the first poses.

[0092] According to this configuration, even if the poses are located far apart on the pose graph, it is possible to identify pose sets that are highly likely to be observing the same or adjacent regions, and to perform highly accurate reconstruction even when the poses are located far apart, which was previously difficult.

[0093] (5) The information processing system described in (4) above, wherein the first alignment step further performs alignment between a plurality of first local point clouds, where the first local point clouds are point clouds corresponding to each of the first poses included in the pose set extracted from the point cloud data, and the reconstruction step corrects the first poses included in the pose set by performing loop closure when the alignment satisfies predetermined conditions, and reconstructs the point cloud data based on the corrected second poses of the first poses.

[0094] In this configuration, alignment is performed between submaps corresponding to pose sets, and loop closure can be executed only when geometrically consistent, thereby suppressing unnecessary loop closures based on incorrect correspondences.

[0095] (6) An information processing system according to any one of (1) to (5) above, wherein in the reception step, the system receives the specification of a region of interest from the point cloud data for which the difference is to be calculated; in the extraction step, the system extracts a second local point cloud from the point cloud data based on the region of interest, extracts a local region corresponding to the second local point cloud from the reference data; and in the second alignment step, the system performs alignment between the second local point cloud and the local region.

[0096] This configuration allows for alignment and difference calculation to be performed only on the region of interest, rather than the entire point cloud data, enabling efficient and highly accurate identification of displacements in areas that require close attention during temporary construction work.

[0097] (7) An information processing system according to any one of (1) to (6) above, wherein in the reception step, the system receives the designation of a region of interest from the reference data for which the difference is to be calculated; in the extraction step, the system extracts a local region from the reference data based on the region of interest, extracts a second local point cloud corresponding to the region of interest from the point cloud data; and in the second alignment step, the system performs alignment between the second local point cloud and the local region.

[0098] In this configuration, difference analysis can be performed starting from a region of interest specified by the reference data, enabling flexible difference evaluation based on the perspective of the designer or manager.

[0099] (8) An information processing system according to either (6) or (7) above, wherein the difference data is data relating to the difference between the second local point cloud and the local region, and in the presentation step, the difference data is presented in association with at least one of the aligned second local point cloud and the local region.

[0100] In this configuration, by presenting the difference data based on the difference between the aligned second local point cloud and the local region in correspondence, the distribution of displacement and differences can be presented in a way that allows for intuitive understanding.

[0101] (9) In an information processing system described in any one of (1) to (8) above, the reference data is BIM data.

[0102] In this configuration, by using BIM data as reference data, the differences between design information and actual measurement results after construction can be directly compared, and structural displacement and construction errors can be evaluated with high accuracy.

[0103] (10) An information processing system described in any one of (1) to (8) above, wherein the reference data is data including a point cloud.

[0104] In this configuration, since data including point clouds can be used as reference data, it is possible to compare point cloud data acquired at different times or under different conditions and flexibly analyze differences caused by temporal changes or environmental fluctuations.

[0105] (11) An information processing system according to any one of (1) to (10) above, comprising a server device having the processor and a terminal that can access the server device.

[0106] According to this configuration, the information processing system can be implemented in various ways.

[0107] (12) An information processing method, which is performed by a processor and includes each step of the information processing system described in any one of (1) to (10) above.

[0108] According to this embodiment, one embodiment of the information processing method can be provided.

[0109] (13) An information processing program that causes at least one computer to perform each step of the information processing system described in any one of (1) to (10) above.

[0110] According to this embodiment, one embodiment can be provided in the form of a program. Of course, this is not always the case.

[0111] Finally, while various embodiments relating to this disclosure have been described, these are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0112] 1: Information Processing System 11: Communication Network 2: Information Processing Device 20: Communications bus 21: Communications Department 22: Storage section 23: Processor 231: Acquisition Department 232: Specific part 233: First alignment section 234:Reconstruction section 235: Reception Department 236:Extraction part 237: Second alignment section 238:Presentation part 239: Arithmetic section 24: Display section 3: Measuring device 30: Communications bus 31: Connection Interface 32: Storage section 33: Control Unit 34: LiDAR 51 :Area 52 :Area 61 :Area 62 :Area 7: Difference display screen 71 :Area 711: Emphasized parts 72 :Area 8: Difference display screen C: Point cloud Cl1: First local point group Cl2: Second local point group Db: BIM data Dd: Difference data Dp: Point cloud data Dr: Reference data P1: First pose P2: Second pose Po: Suitable component Pp:Position component Ps: Pose group R: area Ri: Area of ​​interest Rl: Local region T: Trajectory U: User

Claims

1. An information processing system, The system comprises at least one processor, the processor configured to perform the following steps by reading a program: In the acquisition step, point cloud data showing the characteristics of the site measured by the measuring device is acquired. In the reconstruction step, the system reconstructs the point cloud data by performing loop closure on sets of poses from among the multiple poses of the measuring device during measurement, where the position components are the same or close together and the directions determined by the orientation components are the same or close together, thereby pointing to regions.

2. In the information processing system described in Claim 1, The system presents, in the presentation step, difference data relating to the difference between the point cloud data and the predetermined reference data relating to the site, which is identified based on the point cloud data and the reference data.

3. In the information processing system described in Claim 1, The aforementioned site is a system related to temporary construction work.

4. In the information processing system described in claim 1, The point cloud data is data constructed by transforming the coordinates of each point acquired by the measuring device at multiple time points into the same reference coordinate system based on a first pose of the measuring device corresponding to each time point was acquired, wherein the first pose is information including a position component indicating the position of the measuring device and an orientation component indicating the orientation of the measuring device.

5. In the information processing system described in claim 4 Furthermore, in the reconstruction step, the system corrects the first pose by performing loop closure, and reconstructs the point cloud data based on the second pose after the correction of the first pose.

6. In the information processing system described in claim 1, Furthermore, in a specific step, a set of poses that satisfies a predetermined condition is identified from among the first poses corresponding to multiple time points, where the condition is that the directions determined by the orientation component of each of the first poses included in the set of poses are directed toward the same or adjacent regions. The system, in the reconstruction step, corrects the first pose included in the pose set by performing loop closure, and reconstructs the point cloud data based on the second pose after the correction of the first pose.

7. In the information processing system described in claim 1, Furthermore, in the first alignment step, alignment is performed between a plurality of first local point clouds, where the first local point cloud is a point cloud corresponding to each first pose included in the pose set extracted from the point cloud data. In the reconstruction step, if the alignment satisfies predetermined conditions, the system corrects the first pose included in the pose set by performing loop closure, and reconstructs the point cloud data based on the corrected second pose of the first pose.

8. An information processing system, The system comprises at least one processor, the processor configured to perform the following steps by reading a program: In the acquisition step, point cloud data related to temporary construction work is acquired. In the presentation step, difference data relating to the difference between the point cloud data and the predetermined reference data relating to the temporary construction work is presented, based on the point cloud data and the predetermined reference data relating to the temporary construction work. Furthermore, in the reception step, the user specifies the region of interest from the point cloud data for which the difference will be calculated. Furthermore, in the extraction step, a second local point cloud is extracted from the point cloud data based on the region of interest, and a local region corresponding to the second local point cloud is extracted from the reference data. Furthermore, in the second alignment step, the system performs alignment between the second local point cloud and the local region.

9. In the information processing system described in Claim 8, The difference data is data relating to the difference between the second local point cloud and the local region. The system, in the presentation step, presents the difference data in association with at least one of the aligned second local point cloud and the local region.

10. In the information processing system according to claim 8, The aforementioned reference data is BIM data for the system.

11. In the information processing system described in Claim 8, The aforementioned reference data is a system that includes point cloud data.

12. An information processing system, The system comprises at least one processor, the processor configured to perform the following steps by reading a program: In the acquisition step, point cloud data related to temporary construction work is acquired. In the presentation step, difference data relating to the difference between the point cloud data and the predetermined reference data relating to the temporary construction work is presented, based on the point cloud data and the predetermined reference data relating to the temporary construction work. Furthermore, in the reception step, the user specifies the region of interest from the aforementioned reference data for which the difference will be calculated. Furthermore, in the extraction step, a local region is extracted from the reference data based on the region of interest, and a second local point cloud corresponding to the region of interest is extracted from the point cloud data. Furthermore, in the second alignment step, the system performs alignment between the second local point cloud and the local region.

13. In the information processing system according to claim 12, The difference data is data relating to the difference between the second local point cloud and the local region. The system, in the presentation step, presents the difference data in association with at least one of the aligned second local point cloud and the local region.

14. In the information processing system described in Claim 12, The aforementioned reference data is BIM data for the system.

15. In the information processing system according to claim 12, The aforementioned reference data is a system that includes point cloud data.

16. In the information processing system according to any one of claims 1 to 15, A server device having the aforementioned processor, A system comprising a terminal capable of accessing the aforementioned server device.

17. Information processing method, Executed by the processor, A method comprising each step of the information processing system described in any one of claims 1 to 15.

18. It is an information processing program, A program that causes at least one computer to perform each step of the information processing system described in any one of claims 1 to 15.

Images