Information processing device, information processing method, program, and information processing system

Abstract

To improve visibility when displaying multiple three-dimensional data sets superimposed on each other. [Solution] The system includes a screen creation unit that creates a screen that superimposes a first data set, which is one of a plurality of three-dimensional data sets obtained by measuring an object at different times, and a second data set, which is different from the first data set among the plurality of three-dimensional data sets. The screen creation unit creates a screen in which, in a first region where only the second data set exists, the distance between the first data set and the second data set is less than a predetermined threshold, the second data set is displayed in a first display form, and in a second region where only the second data set exists, which is different from the first region, the second data set is displayed in a second display form different from the first display form.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, a program, and an information processing system.

Background Art

[0002] Conventionally, a technique for aligning and superimposing a plurality of three-dimensional data obtained by measuring an object and displaying them has been known.

[0003] Patent Document 1 describes an image processing system that displays a superimposed image in which a part of an outer region of interest is made non-displayed and superimposed so that an inner region of interest is displayed when a plurality of three-dimensional data are superimposed in each region of interest.

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, according to the conventional technology, there is a problem that visibility improvement has not been achieved for regions other than the inner region and regions other than the region of interest.

[0005] The present invention has been made in view of the above, and an object thereof is to improve the visibility when a plurality of three-dimensional data are superimposed and displayed regardless of whether it is an inner region or a region of interest when superimposed.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objective, the information processing device according to the present invention includes a screen creation unit that creates a screen that superimposes a first data, which is one of a plurality of three-dimensional data obtained by measuring an object at different times, and a second data, which is different from the first data among the plurality of three-dimensional data. The screen creation unit creates a screen in which, in a first region where only the second data exists, the distance between the first data and the second data is less than a predetermined threshold, the second data is displayed in a first display form, and in a second region where only the second data exists, which is different from the first region, the second data is displayed in a second display form different from the first display form. [Effects of the Invention]

[0007] According to the present invention, visibility can be improved when displaying multiple three-dimensional data superimposed on each other. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is an overall configuration diagram showing an example of an information processing system according to the first embodiment. [Figure 2] Figure 2 is a hardware configuration diagram showing an example of a terminal device and management server according to the first embodiment. [Figure 3] Figure 3 is a functional block diagram showing an example of an information processing system according to the first embodiment. [Figure 4] Figure 4 shows an example of the display screen of the 3D viewer. [Figure 5] Figure 5 is a functional block diagram showing an example of a 3D superposition processing unit according to the first embodiment. [Figure 6] Figure 6 shows an example of 3D data captured from the entire room. [Figure 7] Figure 7 shows an example of 3D data in which a portion of the room has been re-photographed. [Figure 8] Figure 8 shows an example of an overlaid screen created using conventional technology. [Figure 9]Figure 9 shows an example of a screen where two 3D data sets are superimposed. [Figure 10] Figure 10 shows an example of the transparency of each component of the second data determined by the screen creation unit. [Figure 11] Figure 11 is a sequence diagram showing an example of 3D data processing according to the first embodiment. [Figure 12] Figure 12 is an explanatory diagram showing an example of a settings screen according to the first embodiment. [Figure 13] Figure 13 is a flowchart showing an example of the procedure for 3D superposition processing according to the first embodiment. [Figure 14] Figure 14 shows an example of the color component values ​​for each element of the second data determined by the screen creation unit. [Figure 15] Figure 15 is an explanatory diagram showing an example of a selection screen according to the third embodiment. [Modes for carrying out the invention]

[0009] In industries such as civil engineering and construction, the adoption of BIM (Building Information Modeling) and CIM (Construction Information Modeling) is progressing with the aim of addressing issues such as the declining birthrate and aging population, and improving labor productivity.

[0010] BIM is a solution for utilizing a database of buildings—which is a digital model of a three-dimensional building created on a computer (hereinafter referred to as a 3D model) with attribute data such as cost, finishes, and management information—in all stages of construction, from design and construction to maintenance and management.

[0011] CIM is a solution for the civil engineering sector (including infrastructure in general, such as roads, power, gas, and water) that was proposed following the example of BIM, which was being developed in the building sector. Similar to BIM, it is being pursued to improve the efficiency and sophistication of the entire construction production system by sharing information among stakeholders, primarily using 3D models.

[0012] When promoting BIM / CIM implementation, it is important to consider how to easily obtain three-dimensional data (3D data) of targets such as the spaces of buildings and public facilities. Here, the 3D data mentioned refers to three-dimensional point clouds (hereinafter sometimes simply referred to as point clouds) that hold distance information to the target, obtained by measuring the target with a laser scanner (hereinafter referred to as LS), or mesh objects generated based on point cloud data indicating the three-dimensional point cloud, 3D-CAD (Computer Aided Design) models, and the like.

[0013] When constructing a structure from scratch, it is easy to introduce BIM / CIM because a complete product can be designed from the beginning using BIM / CIM software. On the other hand, in the case of existing buildings, there are situations where the design drawings at the time of construction do not remain, or the current situation differs from the design drawings at the time of construction due to renovations over time, etc., which raises the hurdle for BIM / CIM implementation. The BIM implementation of such existing buildings is called As-Build BIM and is also an important issue for promoting future BIM / CIM implementation.

[0014] As a means to achieve As-Build BIM, there is a workflow of using the above-mentioned LS for spatial measurement and creating a 3D-CAD model from the measured point cloud data. Conventionally, methods such as measurement using photos and measuring tools or manual sketching have been used for this work, but due to the size of the space, the presence or absence of installed objects, and complexity (such as the way pipes are intertwined), there is a possibility of generating a large amount of work cost. Therefore, introducing an LS that can acquire 3D data of the space has been attracting attention as a powerful method to solve this problem.

[0015] In the As-Build using LS, although the acquisition of 3D data has become easier, a new task of information processing (3D data processing) for 3D data that did not exist in conventional work has emerged. In general 3D data processing, "multi-point measurement using LS", "generation of an integrated point cloud by aligning each point cloud", "removal of unnecessary point clouds such as noise", "mesh conversion of the point cloud, texture mapping to the mesh, 3D-CAD model conversion", "alignment processing", "superposition of multiple 3D data", etc. are carried out.

[0016] (First Embodiment) In this embodiment, when superimposing and displaying a plurality of 3D data acquired by measuring an object at different times, the visibility is improved by changing the display form of each element (constituent element) constituting the 3D data according to the positional relationship of the 3D data. As the display form, for example, the transparency of each constituent element of the 3D data acquired in the past is increased. Thereby, the visibility of the newly acquired 3D data can be improved.

[0017] FIG. 1 is an overall configuration diagram showing an example of an information processing system 1 according to the first embodiment. The information processing system 1 of this embodiment is constructed by a terminal device 3 which is an example of a communication terminal and a management server 5.

[0018] The management server 5 is an example of an information processing device that executes one or more information processes on 3D data such as a three-dimensional point cloud and a mesh object.

[0019] Here, the three-dimensional point cloud is an aggregate of coordinate points in the X, Y, Z directions, etc. corresponding to the measurement points on the surface of an object when measuring a certain space of the object using a laser scanner LS or the like. Each coordinate point is represented, for example, as (1, 3, 5). Also, color information may be added to each coordinate point, and the values of R (Red), G (Green), B (Blue), which are the color components of each coordinate point, may be added as the color information.

[0020] A three-dimensional point cloud is sometimes called a point cloud. Point cloud data is data that can be handled by computers and other devices as a collection of coordinate points in a virtual three-dimensional space.

[0021] The above example shows how to measure a three-dimensional point cloud using a laser scanner (LS), but three-dimensional point clouds can also be measured using other optical or mechanical measurement methods. Other optical measurement methods include using a stereo camera or Visual SLAM (Simultaneous Localization and Mapping).

[0022] Terminal device 3 and management server 5 can communicate via communication network 100. Communication network 100 is constructed using the Internet, mobile communication networks, LAN (Local Area Network), etc. Communication network 100 may include not only wired communication but also wireless communication networks such as 3G (3rd Generation), WiMAX (Worldwide interoperability for Microwave Access), LTE (Long Term Evolution), and 5G (5th Generation). In addition, terminal device 3 can communicate using short-range communication technologies such as NFC (Near Field Communication) (registered trademark).

[0023] <Hardware Configuration> Figure 2 is a hardware configuration diagram showing an example of a terminal device 3 and a management server 5 according to the first embodiment. Each hardware component of the terminal device 3 is indicated by a 300-series code. Each hardware component of the management server 5 is indicated by a 500-series code in parentheses.

[0024] Terminal device 3 includes a CPU (Central Processing Unit) 301, ROM (Read Only Memory) 302, RAM (Random Access Memory) 303, HD (Hard Disk) 304, HDD (Hard Disk Drive) 305, recording media 306, media I / F 307, display 308, network I / F 309, keyboard 311, mouse 312, CD-RW (Compact Disc-Rewritable) drive 314, and bus line 310.

[0025] Of these, the CPU 301 controls the operation of the entire terminal device 3. The ROM 302 stores the program used to drive the CPU 301. The RAM 303 is used as the work area for the CPU 301. The HD 304 stores various data such as programs. The HDD 305 controls the reading or writing of various data to the HD 304 according to the control of the CPU 301. The media I / F 307 controls the reading or writing (storage) of data to the recording medium 306, such as flash memory.

[0026] The display 308 displays various information such as cursors, menus, windows, characters, or images. The network interface 309 is an interface for data communication using the communication network 100. The keyboard 311 is a type of input means equipped with multiple keys for inputting characters, numbers, and various instructions. The mouse 312 is a type of input means for selecting and executing various instructions, selecting processing targets, and moving the cursor. The CD-RW drive 314 controls the reading or writing of various data to the CD-RW 513, which is an example of a removable recording medium. The terminal device 3 may also have a configuration that controls the reading or writing (storage) of data to an external PC (Personal Computer) or external device connected by wire or wirelessly, such as Wi-Fi (Wireless Fidelity).

[0027] Furthermore, the management server 5 is equipped with a CPU 501, ROM 502, RAM 503, HD 504, HDD 505, recording media 506, media I / F 507, display 508, network I / F 509, keyboard 511, mouse 512, CD-RW drive 514, and bus line 510. Since these have the same configuration as described above (CPU 301, ROM 302, RAM 303, HD 304, HDD 305, recording media 306, media I / F 307, display 308, network I / F 309, keyboard 311, mouse 312, CD-RW drive 314, and bus line 310), their descriptions will be omitted.

[0028] The CD-RW drive 314 (514) may be replaced with a CD-R drive or the like. Furthermore, the terminal device 3 and the management server 5 may each be constructed using a single computer, or they may be constructed using multiple computers, each arbitrarily assigned to a specific part (function, means, or storage unit).

[0029] Figure 3 is a functional block diagram showing an example of an information processing system 1 according to the first embodiment.

[0030] <Functional Configuration of Terminal Devices> As shown in Figure 3, the terminal device 3 includes a transmitting / receiving unit 31, a receiving unit 32, a display control unit 34, and a storage / reading unit 39. Each of these units is a function or means of functioning, realized by any of the components shown in Figure 2 operating according to instructions from the CPU 301 following a program deployed from the HD 304 onto the RAM 303. The terminal device 3 also has a storage unit 3000 constructed from the RAM 303 and HD 304 shown in Figure 2.

[0031] (Functional configuration of each terminal device) Next, we will describe each component of the terminal device 3.

[0032] The transmitting / receiving unit 31 is an example of a receiving means and is implemented by commands from the CPU 301 shown in Figure 2 and the network I / F 309, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 100.

[0033] The reception unit 32 is an example of a reception mechanism and is mainly implemented by commands from the CPU 301 shown in Figure 2, as well as the keyboard 311 and mouse 312, to receive various inputs from the user.

[0034] The display control unit 34 is an example of a display control means, and is implemented by instructions from the CPU 301 shown in Figure 2, and causes various images and screens to be displayed on the display 308, which is an example of a display unit.

[0035] Terminal device 3 displays 3D data using, for example, an application (3D viewer) that visualizes 3D data. The 3D viewer can be a web application used in a web browser, a PC application that runs on a PC, or a smartphone application that runs on a smartphone.

[0036] 3D data can be uploaded to a cloud service, stored on a terminal device 3 such as a PC or smartphone, etc. Terminal device 3 can visualize 3D data on a 3D viewer regardless of the type of 3D viewer or the location of the 3D data. The 3D data can be in any format as long as it can be divided into constituent elements such as meshes or point clouds.

[0037] Figure 4 shows an example of the 3D viewer's display screen. In this example, the data from the first 3D image of the machine room, which was acquired (photographed) at different times, is displayed. The 3D viewer can display 3D data while changing the viewpoint using a total of six degrees of freedom: translation in each direction of three-dimensional space, rotation in the roll direction, rotation in the pitch direction, and rotation in the yaw direction.

[0038] The storage / reading unit 39 is an example of a storage control means and is executed by instructions from the CPU 301 shown in Figure 2, as well as by the HDD 305, media I / F 307, CD-RW drive 314, and external PCs or external devices. It performs processing to store various data in the storage unit 3000, recording media 306, CD-RW 313, and external PCs or external devices, and to read various data from the storage unit 3000, recording media 306, CD-RW 313, and external PCs or external devices.

[0039] <Management Server Functional Configuration> The management server 5 includes a transmitting / receiving unit 51, a processing unit 53, a determination unit 55, a generation unit 57, and a storage / reading unit 59. Each of these units is a function or means of functioning, realized by the operation of any of the components shown in Figure 2 by instructions from the CPU 501 according to a program deployed from the HD 504 onto the RAM 503. The management server 5 also has a storage unit 5000 constructed from the HD 504 shown in Figure 2. The storage unit 5000 is an example of a storage means.

[0040] (Configuration of each function of the management server) Next, we will describe the components of the management server 5. The management server 5 may be configured to distribute its functions across multiple computers. Furthermore, although we will describe the management server 5 as a server computer located in a cloud environment, it may also be a server located in an on-premises environment.

[0041] The transmitting / receiving unit 51 is an example of a transmission means and is implemented by commands from the CPU 501 shown in Figure 2 and the network I / F 509, and transmits and receives various data (or information) with other terminals, devices, or systems via the communication network 100.

[0042] The processing unit 53 is implemented by instructions from the CPU 501 shown in Figure 2, and performs various processes including alignment and superposition.

[0043] The decision unit 55 is implemented by instructions from the CPU 501 shown in Figure 2 and performs various decisions.

[0044] The generation unit 57 is implemented by instructions from the CPU 501 shown in Figure 2, and performs various generation tasks such as screen generation, which will be described later.

[0045] The storage / reading unit 59 is an example of a storage control means and is executed by instructions from the CPU 501 shown in Figure 2, as well as by the HDD 505, media I / F 507, CD-RW drive 514, and external PCs or external devices. It performs processing to store various data in the storage unit 5000, recording media 506, CD-RW 513, and external PCs or external devices, and to read various data from the storage unit 5000, recording media 506, CD-RW 513, and external PCs or external devices. The storage unit 5000, recording media 506, CD-RW 513, and external PCs or external devices are examples of storage means.

[0046] The memory unit 5000 contains a user information management DB 5001, a configuration information management DB 5002, a memory processing management DB 5003, a 3D data management DB 5004, and a processing result management DB 5005, all of which are configured using a configuration information management table.

[0047] User information management DB5001 stores and manages the file names of 3D data associated with user information; configuration information management DB5002 stores and manages various configuration information; memory processing management DB5003 stores and manages various processing programs for executing information processing; 3D data management DB5004 stores and manages 3D data; and processing result management DB5005 stores and manages processing result information that shows the processing results of information processing performed on 3D data.

[0048] <Functional Configuration of the 3D Overlay Processing Unit> The processing unit 53 includes a 3D overlay processing unit 530 that performs 3D overlay processing (alignment processing and screen creation processing of multiple 3D data). Figure 5 is a functional block diagram showing an example of the 3D overlay processing unit 530 according to the first embodiment. The 3D overlay processing unit 530 has an alignment unit 531 and a screen creation unit 532.

[0049] The alignment unit 531 aligns multiple 3D data. Conventionally, a technique called registration has been used for 3D data alignment. For example, one method involves finding corresponding points using algorithms such as the ICP (Iterative Closest Point) algorithm or the FPFH (Fast Point Feature Histograms) algorithm, and then determining the rigid body transformation for alignment using the RANSAC (Random Sample Consensus) algorithm. These methods are mainly for three-dimensional point clouds, but they can be used as a degenerate two-dimensional version, and in this embodiment, these methods can also be used to align 3D data.

[0050] The screen creation unit 532 aligns multiple 3D data using rigid body transformations and the like obtained by the alignment unit 531, and creates a screen to display them superimposed.

[0051] Next, we will explain multiple 3D data sets and their superimposed display. Figure 6 shows an example of 3D data where the entire room was captured. Figure 7 shows an example of 3D data where a part of the room was re-captured. In this example, 3D data of the entire room as shown in Figure 6 was acquired initially, and then 3D data of a part of the room where equipment was installed was acquired as shown in Figure 7. Note that the black area on the left side of Figure 7 is the part where data was missing during the re-capture.

[0052] Users can continuously acquire the 3D data of the building, which is updated daily, and switch between the 3D data in the 3D viewer to view it. Furthermore, users can align and superimpose multiple 3D data sets for viewing in the 3D viewer. For example, if the updated parts of a room cannot be determined from 3D data only taken near the updated areas, the user can overlay the updated 3D data onto the 3D data of the entire room.

[0053] Figure 8 shows an example of a superimposed screen created using conventional technology. In the example in Figure 8, a screen is created by superimposing the 3D data from Figure 6 and the 3D data from Figure 7, but image quality degradation occurs in the areas indicated by the dashed lines.

[0054] The causes of image quality degradation in conventional technology include the fact that the two 3D data sets were captured by different devices, resulting in different data accuracy, and that the alignment of the two 3D data sets is inaccurate. In Figure 8, due to the inaccurate alignment of the dashed line, image quality degradation occurs where the image of the power supply equipment captured with the two 3D data sets appears doubled.

[0055] In this embodiment, the screen creation unit 532 improves visibility by changing the display format of each component of the 3D data according to the positional relationship between the two 3D data. The display format of each component will be explained with reference to Figures 9 and 10.

[0056] Figure 9 shows an example of a screen with two 3D data superimposed. In the following explanation, each of the two 3D data sets is assumed to consist of 64 (= 4 × 4 × 4) components (cubes). In Figure 9, the first data set is the most recently acquired 3D data, and the second data set is the 3D data acquired before the first data set. Note that the first data set is an example of the first data set, and the second data set is an example of the second data set.

[0057] Figure 9(a) shows the superimposed first and second data from an oblique overhead view, and Figure 9(b) shows the superimposed first and second data from a direct overhead view. In Figure 9(b), the numerical values ​​attached to each component of the second data indicate the distance between it and the first data, assuming the length of one side of the cube is 1. Here, the distance between the components of the second data and the first data is, for example, the distance between the component of the first data that is closest to the component of the second data (nearest neighbor component) and the component of the second data (distance between components). Furthermore, the distance between components is defined as the distance between the centroid positions of each component.

[0058] Note that the definition of the distance between constituent elements may be other than that described above. For example, one of the vertices of the cube may be designated as a representative point, and the distance between the representative points of each cube may be defined as the distance between constituent elements.

[0059] The screen creation unit 532 determines the transparency of each component of the second data according to the distance calculated as described above. Figure 10 shows an example of the transparency of each component of the second data determined by the screen creation unit 532. In this example, in Figure 9(b), the transparency is set to 100% when the distance is 0, and to 0% when the distance is greater than or equal to a predetermined threshold T (e.g., 2.5). Here, the threshold T is a value that has been set in advance through experiments or the like. This value may be set to only one value for each 3D data, or different values ​​may be set for each 3D data, etc. It may also be changed depending on the scene from which the 3D data is acquired.

[0060] When the distance d is greater than 0 and less than the threshold T, the transparency A (%) can be calculated, for example, by the following formula (1).

[0061] A = 100 - V1 × d ... (1)

[0062] However, V1 is a constant that indicates the attenuation rate of transparency, and in Figure 10 it is set to 20. Note that the screen creation unit 532 may also change the transparency using an increase rate (for example, an increase rate corresponding to the reciprocal of the distance) instead of an attenuation rate. The attenuation rate and change rate of transparency are examples of change rates of transparency.

[0063] In this way, the screen creation unit 532 determines the transparency of the second data to be 100% for areas with a distance of 0 (areas overlapping with the first data), and for areas with a distance greater than 0, it determines that the transparency decreases as the distance increases. The screen creation unit 532 then creates a screen that displays the second data in this display format.

[0064] Furthermore, a region where the distance is greater than 0 and less than the threshold T is an example of the first region, and a region where the distance is greater than or equal to the threshold T is an example of the second region. In addition, among the regions where only the second data exists, the region where the distance between the first and second data is less than the threshold T is the first region, and all regions other than the first region are the second region. Furthermore, a region where the distance is 0 is an example of the third region.

[0065] Furthermore, the screen creation unit 532 can also create a screen in which the second data is displayed in a first display mode (transparency according to distance) in the first area, in a second display mode (0% transparency) in the second area, and in a third display mode (100% transparency) in the third area.

[0066] Note that the components of 3D data do not have to be cubes. For example, the components may be the points of a point cloud or the faces of a mesh. If the components are the points of a point cloud, the distance between each point may be used as the distance between components. If the components are the faces of a mesh, the distance between the center coordinates of each face may be used as the distance between components.

[0067] Furthermore, the number of 3D data sets may be three or more. In that case, for example, the most recently acquired 3D data set among the multiple 3D data sets can be set as the first data set, and the display format of the components of the other 3D data set, the second data set, can be changed according to the distance from the first data set. Note that the first data set does not have to be the most recently acquired 3D data set, as long as it is selected from the multiple 3D data sets.

[0068] Figure 11 is a sequence diagram showing an example of 3D data processing according to the first embodiment.

[0069] The receiving unit 32 of the terminal device 3 accepts input operations related to user information (step S1). The transmitting / receiving unit 31 sends a request for a settings screen, including the user information received in step S1, to the management server 5 of the terminal device 3, and the transmitting / receiving unit 51 of the management server 5 receives the request sent from the terminal device 3 (step S2).

[0070] Next, the storage / reading unit 59 of the management server 5 searches the user information management DB 5001 using the user information contained in the request received in step S2 as a search key, and reads the file name of the 3D data associated with the user information contained in the request. Based on the file name read by the storage / reading unit 59, the generation unit 57 of the management server 5 generates a display screen including a settings screen (step S3).

[0071] This settings screen is a GUI (Graphical User Interface) screen that contains images for inputting 3D data information (such as the file name of the 3D data) and selecting 3D data processing options.

[0072] The transmitting / receiving unit 51 transmits display screen information, including the setting screen information related to the setting screen generated in step S3, to the terminal device 3, and the transmitting / receiving unit 31 of the terminal device 3 receives the display screen information transmitted from the management server 5 (step S4).

[0073] Next, the display control unit 34 of the terminal device 3 displays the display screen, including the settings screen received in step S4, on the display 308 (step S5). The reception unit 32 of the terminal device 3 receives a predetermined input operation from the user for the displayed settings screen. This input operation includes inputting 3D data information and 3D data processing information. In this embodiment, the user inputs information instructing 3D superposition processing as 3D data processing information.

[0074] The transmitting / receiving unit 31 transmits input information related to the input operation received by the receiving unit 32 to the management server 5, and the transmitting / receiving unit 51 of the management server 5 receives the input information transmitted from the terminal device 3 (step S6). This input information includes the input 3D data information and 3D data processing information.

[0075] The storage and reading unit 59 of the management server 5 searches the 3D data management DB 5004 using the 3D data information contained in the input information received in step S6 as a search key, and reads multiple 3D data to be processed.

[0076] Furthermore, the storage and reading unit 59 retrieves the 3D data processing program by searching the storage processing management DB 5003 using the 3D data processing information contained in the input information received in step S6 as a search key.

[0077] The processing unit 53 of the management server 5 generates 3D data processing information (step S7) based on the 3D data read from the storage / reading unit 59, the settings stored in the setting information management DB 5002, and the 3D data processing program. The 3D data processing information is the information resulting from 3D data processing of the 3D data.

[0078] The generation unit 57 of the management server 5 generates a display screen that includes a screen for 3D data processing information (results of 3D data processing) and a selection screen that allows the user to choose whether to terminate or restart 3D data processing. The selection screen is a GUI screen on which images of means for changing the parameters of 3D data processing are placed. The transmitting and receiving unit 51 then transmits the generated display screen information to the terminal device 3 (step S8).

[0079] The transmitting / receiving unit 31 of the terminal device 3 receives display screen information transmitted from the management server 5, the display control unit 34 of the terminal device 3 displays the received display screen on the display 308, and the receiving unit 32 of the terminal device 3 accepts a predetermined input operation from the user on the displayed display screen (step S9).

[0080] This input operation includes a selection operation to choose to end or restart 3D data processing, and an adjustment operation to adjust the processing results of 3D data processing.

[0081] The transmitting / receiving unit 31 transmits input information related to the input operation received by the receiving unit 32 to the management server 5, and the transmitting / receiving unit 51 of the management server 5 receives the input information transmitted from the terminal device 3 (step S10).

[0082] If this input information includes selection information indicating the end of 3D data processing, the processing unit 53 of the management server 5 determines the processing result of the 3D data processing.

[0083] On the other hand, if the input information includes selection information indicating a re-execution of 3D data processing or adjustment information due to adjustment operations, the processing unit 53 of the management server 5 will re-execute the 3D data processing in step S7 based on this information.

[0084] The processing unit 53 converts the processing result information into a file format that can be read by 3D data processing software, a file format that can be read by 3D-CAD software, a file format that can be read by BIM / CIM software, etc., and the storage / reading unit 59 stores the converted processing result information in the processing result management DB 5005, the recording medium 506, or the CD-RW 513 (step S11).

[0085] The transmitting / receiving unit 51 transmits the determined processing result information to the terminal device 3 (step S12).

[0086] The transmitting / receiving unit 31 of the terminal device 3 receives processing result information transmitted from the management server 5, and the display control unit 34 of the terminal device 3 displays the received processing result on the display 308 (step S13).

[0087] In the above, the functions of the management server 5 in Figure 3 may be integrated into the terminal device 3, and the processing of the management server 5 in Figure 11 may also be performed by the terminal device 3.

[0088] Figure 12 is an explanatory diagram showing an example of a setting screen according to the first embodiment. Figure 12 shows the display screen 1000 displayed on the display 308 of the terminal device 3 in step S5 of the sequence diagram shown in Figure 11.

[0089] The display control unit 34 of the terminal device 3 displays the user information display screen 1100 and the settings screen 1200 on the display screen 1000.

[0090] The settings screen 1200 includes a 3D data settings screen 1210, a processing settings screen 1220, and an execution button 1230. When 3D superposition processing according to this embodiment is performed, the settings screen 1200 may further include a first data selection area 1250.

[0091] The 3D data setting screen 1210 is a screen that accepts operations to set 3D data used to perform 3D data processing. The display control unit 34 displays the 3D data setting areas 1212 and 1214, corresponding to the respective file names of the multiple 3D data read by the storage / reading unit 59. Note that there may be three or more 3D data setting areas.

[0092] The processing settings screen 1220 is a screen that accepts setting operations for setting the type of 3D data processing, and the display control unit 34 displays processing setting areas 1221, 1222, and 1223, corresponding to the names of each of the multiple 3D data processing types. The display control unit 34 also displays an execution button 1230 for confirming the various setting operations.

[0093] Furthermore, the display control unit 34 displays a pointer 1240, which is operated by a mouse 312 or the like, for selecting processing setting areas 1221 to 1223.

[0094] When performing 3D overlay processing according to this embodiment, the user operates the pointer 1240 to select "3D overlay processing" in the processing setting area 1223. If "3D overlay processing" is selected, the display control unit 34 may display the first data selection area 1250 on the setting screen 1200. In this case, the user can select the first data by entering the file name of the first data in the first data selection area 1250. The display control unit 34 may also display radio buttons in the first data selection area 1250 corresponding to the file names of multiple 3D data, allowing the user to select the first data by clicking a radio button. The first data selection area 1250 is an example of a setting unit for setting the first data.

[0095] When a setting area is clicked by the pointer 1240 on the terminal device 3, the display control unit 34 displays black circles or checkmarks in the setting areas as shown in the figure. The reception unit 32 accepts the setting operation, and when the execution button 1230 is operated, the setting operation is completed and 3D data processing is executed.

[0096] Specifically, as explained in steps S6 and S7 of Figure 11, the transmitting / receiving unit 31 transmits input information, including various setting information obtained from various setting operations received by the receiving unit 32, to the management server 5, and the processing unit 53 performs 3D data processing.

[0097] Figure 13 is a flowchart showing an example of the procedure for 3D superposition processing according to the first embodiment.

[0098] First, the storage / reading unit 59 reads out multiple 3D data (step S100), and the processing unit 53 sets the first data from the read out 3D data (step S101). For example, the processing unit 53 sets the most recently acquired 3D data as the first data. Alternatively, if the first data selection area 1250 is displayed on the setting screen 1200 and the user has selected the first data, the processing unit 53 may set the first data according to the user's selection.

[0099] Next, the alignment unit 531 aligns the multiple 3D data, and the screen creation unit 532 creates a screen with the multiple 3D data superimposed in the procedure of steps S102 to S107. In the procedure of steps S102 to S107, all components of the second data are scanned, and the transparency of each component is determined.

[0100] The screen creation unit 532 determines the nearest neighbor component (nearest neighbor component) from among the components of the first data to the target component of the second data (step S102). Here, the target component is the component whose transparency is to be determined. In the case of the second data in Figure 9, the initial target component is, for example, the component at the position where the X, Y, and Z coordinate values ​​are minimized.

[0101] Next, the screen creation unit 532 calculates the distance between the target element and the nearest neighbor element (step S103). If the calculated distance is less than the threshold T (step S104: Yes), the screen creation unit 532 determines the transparency of the target element according to the distance (step S105). On the other hand, if the calculated distance is not less than the threshold T (step S104: No), the process proceeds to step S106. The initial value of the transparency of the second data is assumed to be 0%.

[0102] If processing of all components is not completed (step S106: No), that is, if there are components that have not been set as the target component, the screen creation unit 532 sets the next component as the target component (step S107), and the process returns to step S102. On the other hand, if processing of all components is completed (step S106: Yes), the 3D overlay process is completed.

[0103] In the case of the second data in Figure 9, the next component set in step S107 is, for example, the component at the position where the X coordinate value is increased by 1. If there is no component at the position where the X coordinate value is increased by 1, then the component at the position where the Y coordinate value is increased by 1 and the X coordinate value is minimized is set. Furthermore, if there is no component at the position where the Y coordinate value is increased by 1, then the component at the position where the X and Y coordinate values ​​are minimized by increasing the Z coordinate value by 1 is set.

[0104] Thus, according to this embodiment, by changing the transparency of the components of the second data according to the distance from the first data, the visibility when displaying multiple three-dimensional data superimposed on each other can be improved.

[0105] (Second Embodiment) In the first embodiment, the transparency of the components of the second data was varied according to the distance from the first data. In this embodiment, however, instead of transparency, or in conjunction with transparency, the color of the components of the second data is varied for display. In the following description of the second embodiment, the description of parts that are the same as in the first embodiment will be omitted, and the parts that differ from the first embodiment will be described.

[0106] Figure 14 shows an example of the color component values ​​of each component of the second data determined by the screen creation unit 532. The difference from Figure 10 is that the screen creation unit 532 determines the color component values ​​(hereinafter sometimes simply referred to as "color components") instead of transparency. By changing the color components of the second data from their original values, visibility is improved when displaying multiple 3D data superimposed on each other.

[0107] In the example in Figure 14, in Figure 9(b), the color component is set to 255 when the distance is 0, and the color component is set to the original value (98) when the distance is greater than or equal to the threshold T (e.g., 2.5). Note that the color component is represented by 8 bits (0 to 255). Furthermore, the color component can be any of the RGB components.

[0108] When the distance d is greater than 0 and less than the threshold T, the value C of the color component can be expressed, for example, by the following formula (2 It is calculated as follows:

[0109] C = M - V² × d ···(2)

[0110] However, V2 is a constant that indicates the attenuation rate of the color component, and is set to 63 in Figure 14. Also, the constant M is set to 286 in Figure 14. Note that the screen creation unit 532 may also change the color component using an increase rate (for example, an increase rate corresponding to the reciprocal of the distance) instead of an attenuation rate. The attenuation rate and change rate of the color component are examples of change rates of the color component.

[0111] The color components modified by the screen creation unit 532 may be any one of the RGB components, or two or more components selected from RGB. Furthermore, the screen creation unit 532 may modify only the color components, or modify both the transparency and the color components. Additionally, the attenuation rate V2 and the constant M may be set to only one value for each color component, or different values ​​may be set for each color component, etc.

[0112] Thus, according to this embodiment, by changing the values ​​of the color components of the second data according to the distance from the first data, the visibility when displaying multiple three-dimensional data superimposed can be improved.

[0113] (Third embodiment) This embodiment allows the parameters used for 3D superposition processing to be changed on the selection screen. In the following description of the third embodiment, the descriptions of parts that are the same as those of the first and second embodiments will be omitted, and the parts that differ from the first and second embodiments will be described.

[0114] Figure 15 is an explanatory diagram showing an example of a selection screen 1300 according to a third embodiment. Figure 15 shows the display screen 1000 displayed on the display 308 of the terminal device 3 in step S9 of the sequence diagram shown in Figure 11. The display control unit 34 of the terminal device 3 displays the selection screen 1300 on the display screen 1000.

[0115] As shown in Figure 15, the selection screen 1300 includes a processing result area 1310, up / down buttons 1320, left / right buttons 1330, a pointer 1340, and an exit button 1390. The processing result area 1310 also includes a processing result screen 1311 showing the screen created by the 3D overlay process, a parameter setting area 1312 for setting parameters used in the 3D overlay process, and an execute button 1313.

[0116] The parameter setting area 1312 includes pull-down menus for changing the threshold T, the transparency attenuation rate V1, and the attenuation rate V2 for each color component. The user can adjust the values ​​of each menu using a pointer 1340 operated by a mouse 312 or the like.

[0117] Note that the parameter setting area 1312 does not necessarily have to include all of the parameter menus shown in Figure 15. For example, the parameter setting area 1312 only needs to include at least one menu from among the threshold T, transparency attenuation rate V1, and attenuation rate V2 for each color component. Note that the parameter setting area 1312 is an example of a setting unit for setting a threshold, and an example of a setting unit for setting the rate of change of transparency that changes with distance, or the rate of change of color components that changes with distance.

[0118] Furthermore, the parameter setting area 1312 may include menus for parameters other than those shown in Figure 15. For example, the parameter setting area 1312 may include a menu for the constant M in equation (2) above.

[0119] The means of changing each parameter do not have to be a pull-down menu. For example, the parameter setting area 1312 may include a box for numerical input instead of a pull-down menu.

[0120] The user can change the viewpoint up, down, left, or right by clicking the up / down buttons 1320 or left / right buttons 1330 using the pointer 1340, and view the processing result screen from the desired viewpoint. Furthermore, when the user changes the parameters in the parameter setting area 1312 and clicks the execute button 1313, the processing unit 53 executes the 3D overlay processing with the changed parameters. The generation unit 57 then generates a display screen 1000 that displays the processing result screen 1311 created with the changed parameters.

[0121] The user can check the processing results on the processing results screen 1311 and, if necessary, repeat the 3D overlay process by changing the parameters. The user can also end the 3D overlay process by clicking the end button 1390. In addition, the user can interrupt the 3D overlay process by clicking the end button 1390 midway through.

[0122] Furthermore, the user's actions to view the processing results screen 1311 are not limited to changing the viewpoint; they may also include actions such as zooming in or out on a portion of the processing results. Additionally, the viewpoint may be changed or other actions performed by operating the mouse 312 (such as rotating the mouse wheel) instead of using the up / down buttons 1320 or the left / right buttons 1330.

[0123] In this way, by using the selection screen 1300, the user can repeatedly perform the 3D overlay process while adjusting the parameters, and can easily check the results of the 3D overlay process using the mouse 312 or the like.

[0124] Thus, according to this embodiment, by changing the transparency and color component values ​​of the components of the second data according to the distance from the first data, the visibility when displaying multiple three-dimensional data superimposed can be improved. Furthermore, by using the selection screen according to this embodiment, the parameters used for 3D superposition processing can be changed and the 3D superposition processing can be executed.

[0125] Furthermore, the programs executed by the information processing devices of each embodiment described above are provided as installable or executable files recorded on computer-readable recording media such as CD-ROMs, flexible disks (FDs), CD-Rs, and DVDs (Digital Versatile Disks).

[0126] Furthermore, the program executed by the information processing device of each embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. Alternatively, the program executed by the information processing device of each embodiment may be provided or distributed via a network such as the Internet.

[0127] Furthermore, the programs for each embodiment may be pre-installed and provided in ROM or the like.

[0128] The program executed in the information processing device of each embodiment has a modular configuration that includes the above-described parts (alignment unit 531, screen creation unit 532, etc.). In actual hardware, the CPU (processor) reads the program from the recording medium and executes it, thereby loading the above-described parts onto the main memory and generating them.

[0129] Each function of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules designed to execute each function described above.

[0130] Furthermore, the apparatus described in each embodiment represents only one of several computing environments for carrying out each embodiment disclosed herein.

[0131] In one embodiment, the management server 5 includes multiple computing devices, such as a server cluster. These computing devices are configured to communicate with each other via any type of communication link, including networks and shared memory, and perform the processing disclosed herein. Similarly, the terminal device 3 may include multiple computing devices configured to communicate with each other.

[0132] Although various embodiments of the present invention have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These novel embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. Furthermore, components from different embodiments and modifications may be combined as appropriate.

[0133] Examples of the present invention are as follows: <1> The system includes a screen creation unit that creates a screen that displays a first data set, which is one of several three-dimensional data sets obtained by measuring an object at different times, and a second data set, which is different from the first data set, superimposed on the first data set. The aforementioned screen creation unit, In a first region where only the second data exists, and the distance between the first data and the second data is less than a predetermined threshold, the second data is displayed in a first display format. In a second region, which is different from the first region, among the regions where only the second data exists, the second data is displayed in a second display format different from the first display format. This is an information processing device that creates the aforementioned screen. <2> The aforementioned screen creation unit, In the third region where the distance is 0, the screen is created which displays the second data in a third display format different from the first and second display formats. <1> This is the information processing device described above. <3> The aforementioned screen creation unit, The system creates a screen that displays the second data in the first region with a value corresponding to the distance. <1> or <2> This is the information processing device described above. <4> The aforementioned screen creation unit, The process involves creating a screen in which at least one of the transparency and color components of the second data in the first region is displayed with a different value from that of the second data in the second region. <1> ~ <3> It is an information processing device described in any one of the following. <5> The aforementioned screen creation unit, The screen is created in which at least one of the multiple color components of the second data in the first region is displayed with a different value from the second data in the second region. <1> ~ <3> It is an information processing device described in any one of the following. <6> The distance between the first data and the second data is, The distance between the component of the first data that is closest to the component of the second data, and the component of the second data. <1> ~ <5> It is an information processing device described in any one of the following. <7> The setting unit further comprises setting the first data from the plurality of three-dimensional data, <1> ~ <6> It is an information processing device described in any one of the following. <8> The first data mentioned above is the most recently acquired three-dimensional data among the plurality of three-dimensional data. <1> ~ <6> It is an information processing device described in any one of the following. <9> The setting unit further comprises the setting unit for setting the threshold <1> ~ <8> It is an information processing device described in any one of the following. <10> The setting unit further comprises setting the rate of change of the transparency or the rate of change of the color component that changes according to the distance, <4> This is the information processing device described above. <11> The system further comprises a display control means for displaying the aforementioned screen on a display unit. <1> ~ <10> It is an information processing device described in any one of the following. <12> An information processing method performed by an information processing device that processes three-dimensional data, The process includes a screen creation step of creating a screen that displays a first data set, which is one of several three-dimensional data sets obtained by measuring an object at different times, and a second data set, which is different from the first data set, superimposed on the first data set. The aforementioned screen creation step is, In a first region where only the second data exists, and the distance between the first data and the second data is less than a predetermined threshold, the second data is displayed in a first display format. In a second region, which is different from the first region, among the regions where only the second data exists, the second data is displayed in a second display format different from the first display format. This is an information processing method for creating the aforementioned screen. <13> Computers, This device functions as a screen creation means for creating a screen that displays a first data point, which is one of several three-dimensional data points obtained by measuring an object at different times, superimposed with a second data point, which is different from the first data point among the several three-dimensional data points. The aforementioned screen creation means is In a first region where only the second data exists, and the distance between the first data and the second data is less than a predetermined threshold, the second data is displayed in a first display format. In a second region, which is different from the first region, among the regions where only the second data exists, the second data is displayed in a second display format different from the first display format. This is a program that creates the aforementioned screen. <14> An information processing system comprising an information processing device and a terminal device capable of communicating with the information processing device, A screen creation unit creates a screen that displays a first data set, which is one of several three-dimensional data sets obtained by measuring an object at different times, and a second data set, which is different from the first data set, superimposed on the first data set. The system includes a display control means for displaying the aforementioned screen on a display unit, The aforementioned screen creation unit, In a first region where only the second data exists, and the distance between the first data and the second data is less than a predetermined threshold, the second data is displayed in a first display format. In a second region, which is different from the first region, among the regions where only the second data exists, the second data is displayed in a second display format different from the first display format. This is an information processing system that creates the aforementioned screen. [Explanation of Symbols]

[0134] 1. Information Processing System 3 Terminal devices 5. Management Server 32 Reception Department 34 Display Control Unit 39 Memory / readout section 53 Processing Unit 100 Communication Networks 308,508 displays 311,511 keyboards 312,512 mice 530 3D Overlay Processing Unit 531 Alignment section 532 Screen Creation Section 1000 display screen 1200 Settings screen 1250 First data selection area 1300 Selection screen 1312 Parameter setting area [Prior art documents] [Patent Documents]

[0135] [Patent Document 1] Patent No. 6058306

Claims

1. The system includes a screen creation unit that creates a screen that displays a first data set, which is one of several three-dimensional data sets obtained by measuring an object at different times, and a second data set, which is different from the first data set, superimposed on the first data set. The aforementioned screen creation unit, In a first region where only the second data exists, and the distance between the first data and the second data is less than a predetermined threshold, the second data is displayed in a first display format. In a second region, which is different from the first region, among the regions where only the second data exists, the second data is displayed in a second display format different from the first display format. An information processing device that creates the aforementioned screen.

2. The aforementioned screen creation unit, The information processing apparatus according to claim 1, which creates a screen in which the second data is displayed in a third display form different from the first display form and the second display form in a third region where the distance is 0.

3. The aforementioned screen creation unit, The information processing apparatus according to claim 1 or 2, which creates a screen that displays the second data in the first region with a value corresponding to the distance.

4. The aforementioned screen creation unit, The information processing apparatus according to claim 1 or 2, which creates a screen in which at least one of the transparency and color components of the second data in the first region is displayed with a different value from that of the second data in the second region.

5. The aforementioned screen creation unit, The information processing apparatus according to claim 1 or 2, which creates a screen in which at least one of a plurality of color components of the second data in the first region is displayed with a value different from that of the second data in the second region.

6. The distance between the first data and the second data is, The information processing apparatus according to claim 1 or 2, wherein the distance is between the component of the first data that is closest to the component of the second data and the component of the second data.

7. The information processing apparatus according to claim 1 or 2, further comprising a setting unit for setting the first data from the plurality of three-dimensional data.

8. The information processing apparatus according to claim 1 or 2, wherein the first data is the most recently acquired three-dimensional data among the plurality of three-dimensional data.

9. The information processing apparatus according to claim 1 or 2, further comprising a setting unit for setting the threshold value.

10. The information processing apparatus according to claim 4, further comprising a setting unit for setting the rate of change of transparency or the rate of change of color component that changes according to the distance.

11. The information processing apparatus according to claim 1 or 2, further comprising a display control means for displaying the aforementioned screen on a display unit.

12. An information processing method performed by an information processing device that processes three-dimensional data, The process includes a screen creation step of creating a screen that displays a first data set, which is one of a plurality of three-dimensional data sets obtained by measuring an object at different times, and a second data set, which is different from the first data set, superimposed on the first data set. The aforementioned screen creation step is, In a first region where only the second data exists, and the distance between the first data and the second data is less than a predetermined threshold, the second data is displayed in a first display format. In a second region, which is different from the first region, among the regions where only the second data exists, the second data is displayed in a second display format different from the first display format. An information processing method for creating the aforementioned screen.

13. Computers, This device functions as a screen creation means for creating a screen that displays a first data point, which is one of several three-dimensional data points obtained by measuring an object at different times, superimposed with a second data point, which is different from the first data point among the several three-dimensional data points. The aforementioned screen creation means is In a first region where only the second data exists, and the distance between the first data and the second data is less than a predetermined threshold, the second data is displayed in a first display format. In a second region, which is different from the first region, among the regions where only the second data exists, the second data is displayed in a second display format different from the first display format. A program to create the aforementioned screen.

14. An information processing system comprising an information processing device and a terminal device capable of communicating with the information processing device, A screen creation unit creates a screen that displays a first data set, which is one of several three-dimensional data sets obtained by measuring an object at different times, and a second data set, which is different from the first data set, superimposed on the first data set. The system includes a display control means for displaying the aforementioned screen on a display unit, The aforementioned screen creation unit, In a first region where only the second data exists, and the distance between the first data and the second data is less than a predetermined threshold, the second data is displayed in a first display format. In a second region, which is different from the first region, among the regions where only the second data exists, the second data is displayed in a second display format different from the first display format. An information processing system that creates the aforementioned screen.

Images