Information processing device, information processing system, and information processing method

Abstract

The purpose of the present invention is to provide a technology with which a user can indirectly and easily ascertain the state of an object in the real world. In the present invention, a three-dimensional measurement unit measures a scale pattern, and a three-dimensional model generation unit generates, on the basis of the measurement result of the scale pattern, a scale pattern three-dimensional model that is data. This information processing device associates first position information, which is position information of the object in the real world or position information of an object three-dimensional model, with second position information which is position information of the scale pattern in the real world or position information of the scale pattern three-dimensional model.

Description

Information processing device, information processing system, and information processing method

[0001] The present disclosure relates to an information processing device, an information processing system, and an information processing method.

[0002] In the field of smart buildings, the use of 3D models, which are data on objects such as buildings, is expected to be beneficial from the perspective of improving business efficiency and creating customer experiences. Methods for generating 3D models include, for example, a method of generating models based on a design based on a real-world object and a design using a CAD system, as in Patent Document 1, and a method of using a digital twin that measures a building and digitizes the real world.

[0003] Japanese Patent Application Laid-Open No. 2003-345839

[0004] A technology has been proposed that links a real-world object with a 3D model generated by a digital twin or the like, and reflects the state of the object and the 3D model in both directions. However, in order for a user to grasp the state of the 3D model (data) from a display device and indirectly grasp the state of the object, the user must perform operations to enlarge, reduce, and rotate the 3D model displayed on the display device. This poses a problem for users who are not accustomed to such operations, as it is difficult to indirectly and easily grasp the state of the object in the real world from the state of the 3D model.

[0005] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a technology that allows a user to indirectly and easily grasp the state of an object in the real world.

[0006] The information processing device according to the present disclosure includes a 3D measurement unit that measures an object in the real world, a 3D model generation unit that generates a 3D object model, which is data, based on the measurement results of the object, and a 3D model generation unit that generates a scale model in the real world based on the 3D object model, wherein the 3D measurement unit measures the scale model, and the 3D model generation unit generates the scale model 3D model, which is data, based on the measurement results of the scale model, and the information processing device links first position information, which is position information of the object in the real world or position information of the object 3D model, to second position information, which is position information of the scale model in the real world or position information of the scale model 3D model.

[0007] According to the present disclosure, first position information, which is position information of an object in the real world or position information of a three-dimensional model of the object, is linked to second position information, which is position information of a scale model in the real world or position information of a three-dimensional model of the scale model. With this configuration, a user can indirectly and easily grasp the state of an object in the real world.

[0008] The objects, features, aspects and advantages of the present disclosure will become more apparent from the following detailed description and the accompanying drawings.

[0009] FIG. 1 is a block diagram showing a configuration of an information processing device according to embodiment 1. FIG. 2 is a flowchart showing the operation of the information processing device according to embodiment 1. FIG. 3 is a diagram for explaining the operation of an information processing device according to modified example 3 of embodiment 1. FIG. 4 is a block diagram showing a configuration of an information processing device according to embodiment 2. FIG. 5 is a block diagram showing a configuration of an information processing device according to embodiment 3. FIG. 6 is a block diagram showing a configuration of an information processing device according to embodiment 4. FIG. 7 is a block diagram showing a hardware configuration of an information processing device according to another modified example. FIG. 8 is a block diagram showing a hardware configuration of an information processing device according to another modified example.

[0010] <First Embodiment> Fig. 1 is a block diagram showing the configuration of an information processing device 1 according to the first embodiment. The information processing device 1 in Fig. 1 includes a three-dimensional measurement unit 11, a three-dimensional model generation unit 12, a storage unit 13, and a three-dimensional model generation unit 14.

[0011] The 3D measurement unit 11 measures objects in the real world. The 3D measurement unit 11 includes, for example, a camera and a LiDAR (Light Detection and Ranging). The objects include, for example, buildings, rooms, and furniture and other installations in buildings.

[0012] The 3D model generation unit 12 generates a 3D model, which is data of the object, as a 3D object model, based on the measurement results of the object by the 3D measurement unit 11. The 3D model generation unit 12 according to the first embodiment includes a point cloud correction unit 12a and a mesh generation unit 12b.

[0013] The point cloud correction unit 12a generates a reference plane based on the point cloud of the object, which is the measurement result obtained by the 3D measurement unit 11, and performs correction to move the point cloud within the reference plane. Note that the reference plane may be generated using, for example, the least squares method, or other methods. If the distance between a portion of the point cloud and the reference plane is equal to or greater than a threshold, the point cloud correction unit 12a may generate another reference plane based on the portion of the point cloud and move the portion of the point cloud within the another reference plane.

[0014] The mesh generator 12b generates a mesh based on the point cloud moved within the reference plane, and generates a three-dimensional object model based on the mesh. For example, the mesh generator 12b generates a triangle consisting of three points that does not surround one or more points of the point cloud moved within the reference plane as a mesh, and generates a three-dimensional object model consisting of a plurality of meshes.

[0015] The 3D model generation unit 12 may generate a 3D object model using semantic segmentation and Universal RANSAC (USAC). The 3D model generation unit 12 may also generate a 3D object model using a method different from the above, as long as the 3D model generation unit 12 generates a 3D object model based on the measurement results of the object by the 3D measurement unit 11.

[0016] The 3D measurement unit 11 and the 3D model generation unit 12 link position information of an object in the real world measured by the 3D measurement unit 11 with position information of the 3D model of the object generated by the 3D model generation unit 12. The position information is, for example, coordinates, and the 3D measurement unit 11 and the 3D model generation unit 12 link the coordinates of feature points such as the center of gravity and vertices of the object and the 3D model of the object. These examples may also be applied to the linking of position information described below.

[0017] Furthermore, in the first embodiment, the three-dimensional measurement unit 11 and the three-dimensional model generation unit 12 link the time information when an object in the real world is measured by the three-dimensional measurement unit 11 with the time information when the three-dimensional object model is generated by the three-dimensional model generation unit 12. The storage unit 13 stores the linking results between the object and the three-dimensional object model.

[0018] The 3D model generation unit 14 generates a scale model in the real world based on the 3D object model generated by the 3D model generation unit 12 and stored in the storage unit 13. The 3D model generation unit 14 may be, for example, a 3D printer or a robot that assembles a scale model from toy-like blocks. The scale model may be a scaled model of an object as is, or a scaled model of an object that has been partially modified. Alternatively, the scale model may be a scaled model of a portion of an object, or multiple models of multiple objects with different scale ratios. When the 3D model generation unit 14 is a 3D printer, the scale model may be provided as an integrated unit or partially separated.

[0019] The three-dimensional measurement unit 11 measures the scale model in the real world in the same way as measuring an object. The three-dimensional measurement unit 11 links the position information and time information of the object in the real world measured by the three-dimensional measurement unit 11 with the position information and time information of the scale model in the real world measured by the three-dimensional measurement unit 11. The storage unit 13 stores the linking results.

[0020] Similar to generating a 3D object model based on the measurement results of the object by the 3D measurement unit 11, the 3D model generation unit 12 generates a 3D model, which is data of the scale model, as a scale model 3D model based on the measurement results of the scale model by the 3D measurement unit 11. The 3D measurement unit 11 and the 3D model generation unit 12 link the position information and time information of the scale model in the real world measured by the 3D measurement unit 11 with the position information and time information of the scale model 3D model generated by the 3D model generation unit 12. The storage unit 13 stores the scale model 3D model and the linking results.

[0021] 2 is a flowchart showing the operation of the information processing device 1 according to the first embodiment. First, in step S1, the 3D measurement unit 11 measures an object in the real world. In step S2, the 3D model generation unit 12 generates a 3D object model based on the measurement results of the object. In step S3, the 3D model generation unit 14 generates a scale model in the real world based on the 3D object model.

[0022] In step S4, the three-dimensional measurement unit 11 measures the scale model in the real world. In step S5, the three-dimensional model generation unit 12 generates a three-dimensional scale model based on the measurement results of the scale model. In the first embodiment, during the operation of FIG. 2 , the position information and time information of the object in the real world measured by the three-dimensional measurement unit 11 are linked to the position information and time information of the scale model in the real world measured by the three-dimensional measurement unit 11.

[0023] Note that only steps S1 and S2 may be synchronously and continuously repeated, or only steps S4 and S5 may be synchronously and continuously repeated. Step S3 may be performed asynchronously with steps S1 and S2, for example, once a day. The series of operations in steps S1 to S5 may be repeated at predetermined intervals, or the predetermined intervals may be specified by the user.

[0024] Summary of First Embodiment According to the information processing device 1 of the first embodiment, a scale model in the real world is generated from a 3D object model generated based on measurement results of the object in the real world, and a scale model 3D model is generated based on measurement results of the scale model in the real world. Therefore, even if a user is not familiar with operating the digital 3D object model and the scale model 3D model, the user can indirectly and easily grasp the state of the object in the real world by observing and understanding the scale model in the real world.

[0025] Furthermore, in the first embodiment, the position information of the object in the real world and the position information of the scale model in the real world are linked, so that the states of the object and the three-dimensional object model and the states of the scale model and the three-dimensional scale model model can be reflected bidirectionally. Therefore, even if one state is changed, the user can indirectly and easily understand the other state by understanding the state of the scale model.

[0026] Furthermore, in this embodiment 1, the time information of the object in the real world and the time information of the scale model in the real world are linked, so that the state of the object and the object 3D model can be synchronized with the state of the scale model and the scale model 3D model.

[0027] Furthermore, in the first embodiment, a reference plane is generated based on a point cloud representing an object, a correction is made to move the point cloud within the reference plane, a mesh is generated based on the point cloud moved within the reference plane, and a 3D model of the object is generated based on the mesh. With this configuration, the correction is made to move the point cloud within the reference plane before generating the mesh, so that it is possible to prevent a surface made up of multiple meshes from having more unevenness than necessary.

[0028] <Variation 1> In the first embodiment, the three-dimensional measurement unit 11 links the position information and time information of an object in the real world measured by the three-dimensional measurement unit 11 with the position information and time information of a scale model in the real world measured by the three-dimensional measurement unit 11. That is, the information processing device 1 is configured to link the first position information and first time information with the second position information and second time information. The first position information and first time information are the position information and time information of an object in the real world, and the second position information and second time information are the position information and time information of the scale model in the real world. However, the first position information, first time information, second position information, and second time information are not limited to the above.

[0029] For example, after generating the scale model 3D model, the 3D model generation unit 12 may link the position information and time information of the object 3D model in the data with the position information and time information of the scale model 3D model in the data based on the similarity relationship between the shape of the object 3D model and the shape of the scale model 3D model. In other words, the first position information and the first time information may be the position information and time information of the object 3D model in the data, and the second position information and the second time information may be the position information and time information of the scale model 3D model in the data.

[0030] Furthermore, by expanding on the above, the first position information and the first time information may be position information and time information of an object in the real world, and the second position information and the second time information may be position information and time information of a scale model three-dimensional model in the data.Also, the first position information and the first time information may be position information and time information of a scale model three-dimensional model in the data, and the second position information and the second time information may be position information and time information of a scale model in the real world.Even in the above cases, as in the first embodiment, the user can indirectly and easily grasp the state of the object by grasping the state of the scale model.

[0031] <Modification 2> In Embodiment 1, the 3D model generation unit 14 may generate a scale model to which real-world position information and time information of an object or position information and time information of a 3D object model are added. The position information and generation time may be added to the scale model by embossing or debossing, for example, or by some other method. With this configuration, the user can easily check the position information and time information of the object or 3D object model used to generate the scale model.

[0032] <Modification 3> In the first embodiment, when a predetermined symbol added to a scale model is measured by the three-dimensional measurement unit 11, the three-dimensional model generation unit 12 may generate a three-dimensional model of the scale model including a three-dimensional model previously associated with the symbol. The symbol may include characters and figures, or may be added by handwriting with a pen or the like by the user, for example.

[0033] 3 is a diagram showing an example of a symbol and a 3D model previously associated with the symbol. For example, if the symbol shown in the upper part of FIG. 3 is added to a scale model, the 3D model generation unit 12 adds a humanoid 3D model to the scale model generated from the scale model at a position corresponding to the position where the symbol was added. The 3D model generation unit 12 then generates the scale model to which the humanoid 3D model has been added as a new scale model. With this configuration, the scale model can be changed simply by adding a symbol to the scale model, without having to regenerate a new scale model.

[0034] 4 is a block diagram showing the configuration of an information processing device 1 according to a second embodiment. In the following, among the components according to the second embodiment, components that are the same as or similar to the components described above are given the same or similar reference numerals, and different components will be mainly described.

[0035] The configuration in FIG. 4 is the same as the configuration in FIG. 1, except that a projection control unit 15 and a projection system 21 are added.

[0036] The projection system 21 has a projector (not shown) capable of projecting onto the scale model. The projection control unit 15 causes the projector to project a texture onto the scale model based on the three-dimensional model of the scale model and the position and orientation of the projector of the projection system 21 relative to the scale model.

[0037] Here, as an example, a description will be given of an example in which the 3D measurement unit 11 includes a camera that captures images of the color, pattern, etc. of an object in the real world as a surface image of the object. The projection control unit 15 places the 3D scale model and the projector in a virtual 3D space on the data, based on the 3D scale model and the position and orientation of the projector relative to the scale model.

[0038] The projection control unit 15 generates a texture so that when a two-dimensional planar texture is projected from a projector onto a scale model three-dimensional model in a virtual three-dimensional space, the texture on the surface of the scale model three-dimensional model resembles the surface image acquired by the three-dimensional measurement unit 11. Then, the projection control unit 15 projects the generated texture onto the scale model from the projector of the projection system 21 in the real world.

[0039] Summary of Second Embodiment According to the information processing device 1 of the second embodiment, a texture is projected from the projector onto the scale model based on the three-dimensional scale model and the position and orientation of the projector of the projection system 21 relative to the scale model. Here, when the three-dimensional model generation unit 14 is a 3D printer, the scale model generated has traditionally been a single color such as white. This has traditionally made it difficult for users to visualize the state of an object in the real world. In contrast, the configuration of the information processing device 1 of the second embodiment makes it possible to make the scale model resemble the object not only in shape but also in surface appearance, allowing the user to easily visualize the state of the object in the real world.

[0040] <Variation of Second Embodiment> The texture projected onto the scale model is not limited to the texture described in the second embodiment. For example, the three-dimensional measurement unit 11 may include a thermo camera capable of detecting thermography of an object. In this case, the projection control unit 15 may generate a texture and project it onto the scale model so that the texture on the three-dimensional scale model resembles the thermography acquired by the three-dimensional measurement unit 11. Furthermore, for example, the three-dimensional measurement unit 11 may include a camera and an image processing unit capable of detecting people flow. In this case, the projection control unit 15 may generate a texture on the three-dimensional scale model such that the texture on the three-dimensional scale model resembles the movement of the people flow acquired by the three-dimensional measurement unit 11, and project it onto the scale model.

[0041] Furthermore, the texture projected onto the scale model may be position information and time information linked to the scale model 3D model, or other attribute information. Furthermore, the information processing device 1 may obtain the information on which the texture is based, such as the surface image, from outside the information processing device 1, rather than using the 3D measurement unit 11. The projection control unit 15 may also project a texture onto an object in a similar manner. That is, the projection control unit 15 may project a texture onto an object from a projector (not shown) capable of projecting onto the object, based on the 3D model of the object and the position and orientation of the projector relative to the object.

[0042] 5 is a block diagram showing the configuration of an information processing device 1 according to a third embodiment. In the following, among the components according to the third embodiment, components that are the same as or similar to the components described above are given the same or similar reference numerals, and different components will be mainly described.

[0043] The configuration in FIG. 5 is the same as that in FIG. 1, except that a three-dimensional model difference extraction unit 16, a projection system 21, and a robot system 22 are added.

[0044] The three-dimensional model difference extraction unit 16 extracts the difference between the object three-dimensional model and the scale model three-dimensional model linked by at least first position information and second position information. The first position information and second position information here are the first position information and second position information described in Modification 1 of Embodiment 1. The three-dimensional model difference extraction unit 16 may determine whether or not a part of the object three-dimensional model and a part of the scale model three-dimensional model are identical, for example, using a predetermined template having the shapes of multiple surfaces of the scale model, and extract the difference using the determination result.

[0045] The 3D model difference extraction unit 16 generates a task instruction for eliminating the extracted differences. In the third embodiment, the task instruction includes an instruction to project a texture from a projector onto at least one of the object and the scale model to eliminate the differences. In this specification, for example, "at least one of A, B, C, ..., and Z" means any one of all combinations of one or more types extracted from the group A, B, C, ..., and Z.

[0046] Some examples of task instructions are described below. For example, when either the object or the scale model is moved, the three-dimensional model difference extraction unit 16 may instruct the robot system 22 or the user to move the other in accordance with the one to eliminate the difference. For example, when either the object or the scale model is moved but it is difficult to move the other, the three-dimensional model difference extraction unit 16 may instruct the projector of the projection system 21 to project a texture onto the other that eliminates the apparent difference. For example, when an attachment such as a scale chair model is added to the scale model, the three-dimensional model difference extraction unit 16 may instruct the projector of the projection system 21 to project the texture of the attachment onto the object.

[0047] When issuing a task instruction to the projector, the three-dimensional model difference extraction unit 16 may control the projector of the projection system 21 to project a texture, similar to the projection control unit 15 in the second embodiment and its modifications. Also, at least one of the projection system 21 and the robot system 22 may be applied to at least one of the object and the scale model.

[0048] Summary of Third Embodiment According to the information processing device 1 of the third embodiment, differences between the three-dimensional object model and the three-dimensional scale model model linked by at least the first location information and the second location information are extracted, and task instructions for eliminating the extracted differences are generated. With this configuration, changes in the states of the object and the scale model can be appropriately reflected in both directions.

[0049] 6 is a block diagram showing the configuration of an information processing device 1 according to a fourth embodiment. In the following, among the components according to the fourth embodiment, components that are the same as or similar to the components described above are given the same or similar reference numerals, and different components will be mainly described.

[0050] The configuration in Fig. 6 is the same as that in Fig. 1, except that a display control unit 17 and a display device 23 are added. The information processing device 1 and the display device 23 constitute an information processing system.

[0051] The display control unit 17 displays the 3D object model and the 3D scale model model linked by at least the first location information and the second location information on the display device 23. The display control unit 17 may simultaneously display the 3D object model and the 3D scale model model, or may selectively display them. Furthermore, if the information processing device 1 is provided with an operation unit (not shown) that accepts operations to enlarge, reduce, and rotate, the display control unit 17 may enlarge, reduce, and rotate the displayed 3D object model, etc., based on the operation accepted by the operation unit.

[0052] The display device 23 may be, for example, a dedicated display device, a display device of a smartphone, a display device of a tablet, or smart glasses capable of displaying virtual reality (VR). Note that the smart glasses may be see-through smart glasses capable of displaying a 3D object model and a 3D scale model model at a position in the real world.

[0053] Summary of Fourth Embodiment According to the information processing device 1 of the fourth embodiment, the object 3D model and the scale model 3D model linked by at least the first position information and the second position information are displayed on the display device 23. With this configuration, the user can check from the display device 23 whether the states of the object and the object 3D model and the states of the scale model and the scale model 3D model are correctly reflected in both directions. Furthermore, a user who is familiar with operating 3D models can grasp the object 3D model and the scale model 3D model from the display device 23.

[0054] <Other Modifications> The above-described 3D measurement unit 11, 3D model generation unit 12, and 3D model generation unit 14 shown in FIG. 1 will be referred to as the "3D measurement unit 11, etc." The 3D measurement unit 11, etc. are realized by a processing circuit 81 shown in FIG. 7. That is, the processing circuit 81 includes the 3D measurement unit 11 that measures an object in the real world, the 3D model generation unit 12 that generates a 3D object model (data) based on the measurement results of the object, and the 3D model generation unit 14 that generates a scale model in the real world based on the 3D object model. The 3D measurement unit 11 measures the scale model, and the 3D model generation unit 12 generates the scale model 3D model (data) based on the measurement results of the scale model. The processing circuit 81 links first position information, which is position information of the object or the 3D object model, to second position information, which is position information of the scale model or the 3D object model in the real world. Dedicated hardware or a processor that executes a program stored in memory may be applied to the processing circuit 81. Examples of the processor include a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, and a DSP (Digital Signal Processor).

[0055] When the processing circuitry 81 is dedicated hardware, the processing circuitry 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of each unit such as the three-dimensional measurement unit 11 may be realized by a circuit in which the processing circuits are distributed, or the functions of each unit may be realized by a single processing circuit.

[0056] When the processing circuit 81 is a processor, the functions of the 3D measurement unit 11 and the like are realized in combination with software or the like. The software or the like includes, for example, software, firmware, or software and firmware. The software or the like is written as a program and stored in a memory. As shown in FIG. 8 , the processor 82 applied to the processing circuit 81 realizes the functions of each unit by reading and executing a program stored in a memory 83. That is, the information processing device 1 includes a memory 83 for storing a program that, when executed by the processing circuit 81, results in the following steps: measuring an object in the real world; generating a 3D object model (data) based on the measurement results of the object; generating a scale model in the real world based on the 3D object model; measuring the scale model; generating a 3D scale model (data) based on the measurement results of the scale model; and linking first position information, which is position information of the object or the 3D object model, to second position information, which is position information of the scale model or the 3D scale model in the real world. In other words, this program causes a computer to execute the procedures and methods of the three-dimensional measurement unit 11, etc. Here, the memory 83 may be, for example, a non-volatile or volatile semiconductor memory such as a random access memory (RAM), a read-only memory (ROM), a flash memory, an erasable programmable read-only memory (EPROM), or an electrically erasable programmable read-only memory (EEPROM), a hard disk drive (HDD), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, a digital versatile disk (DVD), a drive device for any of these, or any storage medium to be used in the future.

[0057] The above describes a configuration in which each function of the three-dimensional measurement unit 11 and the like is realized either by hardware or software, etc. However, the present invention is not limited to this, and a configuration in which part of the three-dimensional measurement unit 11 and the like is realized by dedicated hardware and another part is realized by software, etc. For example, the function of the three-dimensional measurement unit 11 can be realized by a processing circuit 81 as dedicated hardware, and the other functions can be realized by the processing circuit 81 as a processor 82 reading and executing programs stored in a memory 83.

[0058] As described above, the processing circuitry 81 can realize the above-mentioned functions by hardware, software, or a combination of these.

[0059] The information processing device described above can also be applied to an information processing system constructed as a system by appropriately combining a processing device, a communication terminal, the functions of an application installed on at least one of the processing device and the communication terminal, and a server. Communication terminals include, for example, mobile phones, smartphones, and tablets. The functions or components of the information processing device described above may be distributed among the devices that construct the system, or may be concentrated in one of the devices.

[0060] In this disclosure, 'a' and 'an' mean one or more. Therefore, 'a', 'an', 'one or more', and 'at least one' can be used interchangeably.

[0061] It should be noted that the embodiments and modifications may be freely combined, and the embodiments and modifications may be modified or omitted as appropriate.

[0062] The above description is illustrative in all respects and is not restrictive. It is understood that countless variations not illustrated can be envisioned.

[0063] REFERENCE SIGNS LIST 1 information processing device, 11 three-dimensional measurement unit, 12 three-dimensional model generation unit, 12a point cloud correction unit, 12b mesh generation unit, 14 three-dimensional model generation unit, 15 projection control unit, 16 three-dimensional model difference extraction unit, 17 display control unit, 23 display device

Claims

1. An information processing device comprising: a 3D measurement unit that measures an object in the real world; a 3D model generation unit that generates a 3D object model (data) based on the measurement results of the object; and a 3D model generation unit that generates a scale model in the real world based on the 3D object model, wherein the 3D measurement unit measures the scale model, and the 3D model generation unit generates the scale model 3D model (data) based on the measurement results of the scale model, and the information processing device links first position information that is position information of the object in the real world or position information of the object 3D model with second position information that is position information of the scale model in the real world or position information of the scale model 3D model.

2. An information processing device according to claim 1, wherein the first position information is position information of the object in the real world, and the second position information is position information of the scale model in the real world, or the first position information is position information of the three-dimensional model of the object, and the second position information is position information of the three-dimensional model of the scale model.

3. An information processing device according to claim 1 or claim 2, wherein the three-dimensional model generation unit links first time information, which is time information of the object in the real world or time information of the three-dimensional model of the object, with second time information, which is time information of the scale model in the real world or time information of the three-dimensional model of the scale model.

4. An information processing device according to claim 3, wherein the three-dimensional model generation unit generates the scale model to which the first position information and the first time information are added.

5. An information processing device according to any one of claims 1 to 4, wherein the three-dimensional model generation unit generates the scale model three-dimensional model including a three-dimensional model pre-associated with a predetermined symbol added to the scale model when the predetermined symbol is measured by the three-dimensional measurement unit.

6. An information processing device according to any one of claims 1 to 5, further comprising a projection control unit that projects a texture onto the scale model from the projector based on the scale model three-dimensional model and the position and orientation of a projector capable of projecting onto the scale model relative to the scale model.

7. An information processing device according to any one of claims 1 to 5, further comprising a projection control unit that causes a projector to project a texture onto the object based on the three-dimensional model of the object and the position and orientation of a projector capable of projecting onto the object relative to the object.

8. An information processing device according to any one of claims 1 to 5, further comprising a 3D model difference extraction unit that extracts the difference between the object 3D model and the scale model 3D model linked by at least the first position information and the second position information.

9. An information processing device according to claim 8, wherein the three-dimensional model difference extraction unit generates a task instruction for eliminating the difference.

10. An information processing device according to claim 9, wherein the task instruction includes an instruction to project a texture from a projector onto at least one of the object and the scale model to eliminate the difference.

11. An information processing device according to any one of claims 1 to 10, further comprising a display control unit that causes a display device to display the object 3D model and the scale model 3D model linked by at least the first position information and the second position information.

12. An information processing device according to any one of claims 1 to 11, wherein the three-dimensional model generation unit includes: a point cloud correction unit that generates a reference plane based on a point cloud representing the object and performs correction to move the point cloud within the reference plane; and a mesh generation unit that generates a mesh based on the point cloud moved within the reference plane and generates the three-dimensional model of the object based on the mesh.

13. An information processing system comprising: the information processing device according to claim 11; and the display device.

14. An information processing method in which a 3D measurement unit measures an object in the real world, a 3D model generation unit generates a 3D object model as data based on the measurement results of the object, a 3D model generation unit generates a scale model in the real world based on the 3D object model, the 3D measurement unit measures the scale model, and the 3D model generation unit generates a 3D scale model as data based on the measurement results of the scale model, and an information processing device links first position information which is position information of the object in the real world or position information of the 3D object model with second position information which is position information of the scale model in the real world or position information of the 3D scale model model.

Images