Information processing device, information processing system, and information processing method

Abstract

The objective of the present invention is to provide a technology that allows a user to indirectly and easily grasp the state of an object in the real world. A 3D measurement unit measures a scale model, and a 3D model generation unit generates a scale model 3D model, which is data, based on the measurement results of the scale model. An information processing device links first position information, which is position information of an object in the real world or position information of the object 3D model, with second position information, which is position information of the scale model in the real world or position information of the scale model 3D model.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing system, and an information processing method. [Background technology]

[0002] In the field of smart buildings, it is expected that 3D models, which are data on objects such as buildings, will be utilized from the viewpoint of improving business efficiency and creating customer experiences. Methods of generating 3D models have been proposed, such as a method of generating models based on a design based on an object in the real world and a design by a CAD system, as in Patent Document 1, or a method of measuring a building and using a digital twin to digitize the real world. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-345839 A Summary of the Invention [Problem to be solved by the invention]

[0004] A technology has been proposed that links an object in the real world with a 3D model generated by a digital twin or the like, and reflects the state of the object and the 3D model in both directions. However, in order for a user to grasp the state of the 3D model, which is data, from a display device and indirectly grasp the state of the object, it is necessary to perform operations to enlarge, reduce, and rotate the 3D model displayed on the display device. For this reason, there is a problem that it is difficult for users who are not accustomed to such operations to indirectly and easily grasp the state of the object in the real world from the state of the 3D model.

[0005] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a technology that allows a user to indirectly and easily grasp the state of an object in the real world. [Means for solving the problem]

[0006] The information processing device according to the present disclosure includes a 3D measurement unit that measures an object in the real world, a 3D model generation unit that generates a 3D object model, which is data, based on the measurement results of the object, and a 3D model generation unit that generates a scale model in the real world based on the 3D object model, wherein the 3D measurement unit measures the scale model and the 3D model generation unit generates the scale model 3D model, which is data, based on the measurement results of the scale model, and the information processing device links first position information, which is position information of the object in the real world or position information of the object 3D model, to second position information, which is position information of the scale model in the real world or position information of the scale model 3D model. The 3D model generation unit links first time information, which is time information of the object in the real world or time information of the object 3D model, to second time information, which is time information of the scale model in the real world or time information of the scale model 3D model. Effect of the Invention

[0007] According to the present disclosure, first position information, which is position information of an object in the real world or position information of a three-dimensional model of the object, is linked to second position information, which is position information of a scale model in the real world or position information of a three-dimensional model of the scale model. With this configuration, a user can indirectly and easily grasp the state of an object in the real world.

[0008] The objects, features, aspects and advantages of the present disclosure will become more apparent from the following detailed description and the accompanying drawings. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first embodiment. [Diagram 2] 4 is a flowchart showing the operation of the information processing device according to the first embodiment. [Diagram 3] FIG. 13 is a diagram for explaining the operation of the information processing device according to the third modification of the first embodiment. [Figure 4]FIG. 11 is a block diagram showing a configuration of an information processing device according to a second embodiment. [Diagram 5] FIG. 11 is a block diagram showing a configuration of an information processing device according to a third embodiment. [Figure 6] FIG. 13 is a block diagram showing a configuration of an information processing device according to a fourth embodiment. [Figure 7] FIG. 13 is a block diagram showing a hardware configuration of an information processing device according to another modified example. [Figure 8] FIG. 13 is a block diagram showing a hardware configuration of an information processing device according to another modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Embodiment 1> Fig. 1 is a block diagram showing a configuration of an information processing device 1 according to the present embodiment 1. The information processing device 1 in Fig. 1 includes a three-dimensional measurement unit 11, a three-dimensional model generation unit 12, a storage unit 13, and a three-dimensional model generation unit 14.

[0011] The three-dimensional measurement unit 11 measures an object in the real world. The three-dimensional measurement unit 11 includes, for example, a camera and a LiDAR (Light Detection and Ranging). The object includes, for example, a building, a room, and an installation such as furniture in a building.

[0012] The 3D model generation unit 12 generates a 3D model, which is data of an object, as a 3D object model based on the measurement result of the object by the 3D measurement unit 11. The 3D model generation unit 12 according to the first embodiment includes a point cloud correction unit 12a and a mesh generation unit 12b.

[0013] The point cloud correction unit 12a generates a reference plane based on the point cloud of the object, which is the measurement result by the 3D measurement unit 11, and performs correction to move the point cloud into the reference plane. The reference plane may be generated using, for example, the least squares method or other methods. When the distance between a part of the point cloud and the reference plane is equal to or greater than a threshold, the point cloud correction unit 12a may generate another reference plane based on the part of the point cloud and move the part of the point cloud into the other reference plane.

[0014] The mesh generating unit 12b generates a mesh based on the group of points moved into the reference plane, and generates a 3D object model based on the mesh. For example, the mesh generating unit 12b generates a triangle consisting of three points that does not surround one or more points of the group of points moved into the reference plane as a mesh, and generates a 3D object model consisting of a plurality of meshes.

[0015] The 3D model generation unit 12 may generate the 3D model of an object using semantic segmentation and Universal RANSAC (USAC). Also, the 3D model generation unit 12 may generate the 3D model of an object using a method different from the above, as long as the 3D model is generated based on the measurement result of the object by the 3D measurement unit 11.

[0016] The 3D measurement unit 11 and the 3D model generation unit 12 link position information of an object in the real world measured by the 3D measurement unit 11 with position information of the object 3D model generated by the 3D model generation unit 12. The position information is, for example, coordinates, and the 3D measurement unit 11 and the 3D model generation unit 12 link the coordinates of feature points such as the center of gravity and vertices of the object and the object 3D model. These examples may also be applied to the linking of position information below.

[0017] Furthermore, in the first embodiment, the 3D measurement unit 11 and the 3D model generation unit 12 link time information when an object in the real world is measured by the 3D measurement unit 11 with time information when the object 3D model is generated by the 3D model generation unit 12. The storage unit 13 stores the linking result of the object and the object 3D model.

[0018] The three-dimensional model generating unit 14 generates a scale model in the real world based on the three-dimensional object model generated by the three-dimensional model generating unit 12 and stored in the storage unit 13. The three-dimensional model generating unit 14 may be, for example, a 3D printer, or a robot that assembles a scale model from toy-like blocks. The scale model may be a scaled model of an object as is, or a scaled model of an object that has been partially modified. Alternatively, the scale model may be a scaled model of a part of an object, or a plurality of models with different scale ratios between a plurality of objects. When the three-dimensional model generating unit 14 is a 3D printer, the scale model may be provided integrally or partially separated.

[0019] The 3D measurement unit 11 measures the scale model in the real world in the same manner as measuring an object. The 3D measurement unit 11 links position information and time information of the object in the real world measured by the 3D measurement unit 11 with position information and time information of the scale model in the real world measured by the 3D measurement unit 11. The storage unit 13 stores the linking result.

[0020] Similar to generating a 3D object model based on the measurement results of the object by the 3D measurement unit 11, the 3D model generation unit 12 generates a 3D model, which is data of the scale model, as a scale model 3D model based on the measurement results of the scale model by the 3D measurement unit 11. The 3D measurement unit 11 and the 3D model generation unit 12 link position information and time information of the scale model in the real world measured by the 3D measurement unit 11 with position information and time information of the scale model 3D model generated by the 3D model generation unit 12. The storage unit 13 stores the scale model 3D model and the linking results.

[0021] <Operation> 2 is a flowchart showing the operation of the information processing device 1 according to the first embodiment. First, in step S1, the 3D measurement unit 11 measures an object in the real world. In step S2, the 3D model generation unit 12 generates a 3D model of the object based on the measurement results of the object. In step S3, the 3D model generation unit 14 generates a scale model in the real world based on the 3D model of the object.

[0022] In step S4, the 3D measurement unit 11 measures the scale model in the real world. In step S5, the 3D model generation unit 12 generates a scale model 3D model based on the measurement results of the scale model. In the present embodiment 1, during the operation of FIG. 2, the position information and time information of the object in the real world measured by the 3D measurement unit 11 are linked to the position information and time information of the scale model in the real world measured by the 3D measurement unit 11.

[0023] Note that only steps S1 and S2 may be synchronously and continuously repeated, or only steps S4 and S5 may be synchronously and continuously repeated. Step S3 may be performed asynchronously with steps S1 and S2, and may be performed, for example, once a day. The series of operations of steps S1 to S5 may be repeated at a predetermined interval, or the predetermined interval may be specified by the user.

[0024] <Summary of the first embodiment> According to the information processing device 1 of the first embodiment, a scale model in the real world is generated from an object 3D model generated based on the measurement results of the object in the real world, and a scale model 3D model is generated based on the measurement results of the scale model in the real world. Therefore, even if a user is not familiar with the operation of the object 3D model and the scale model 3D model in the data, the user can indirectly and easily grasp the state of the object in the real world by observing and grasping the scale model in the real world.

[0025] In addition, in the first embodiment, the position information of the object in the real world and the position information of the scale model in the real world are linked, so that the states of the object and the object 3D model and the states of the scale model and the scale model 3D model can be reflected in both directions. Therefore, even if one of the states is changed, the user can indirectly and easily grasp the other state by grasping the state of the scale model.

[0026] Furthermore, in this embodiment 1, the time information of the object in the real world and the time information of the scale model in the real world are linked, so that the states of the object and the object 3D model can be synchronized with the states of the scale model and the scale model 3D model.

[0027] In the first embodiment, a reference plane is generated based on a point cloud representing an object, a correction is made to move the point cloud onto the reference plane, a mesh is generated based on the point cloud moved onto the reference plane, and a 3D model of the object is generated based on the mesh. With this configuration, a correction is made to move the point cloud onto the reference plane before generating the mesh, so that it is possible to prevent a surface consisting of multiple meshes from having more unevenness than necessary.

[0028] <Variation 1> In the first embodiment, the three-dimensional measuring unit 11 links the position information and time information of an object in the real world measured by the three-dimensional measuring unit 11 with the position information and time information of a scale model in the real world measured by the three-dimensional measuring unit 11. That is, the information processing device 1 is configured to link the first position information and the first time information with the second position information and the second time information. The first position information and the first time information are the position information and the time information of an object in the real world, and the second position information and the second time information are the position information and the time information of a scale model in the real world. However, the first position information, the first time information, the second position information, and the second time information are not limited to the above.

[0029] For example, after generating the scale model 3D model, the 3D model generating unit 12 may link the position information and time information of the object 3D model in the data with the position information and time information of the scale model 3D model in the data based on a similarity relationship between the shape of the object 3D model and the shape of the scale model 3D model. In other words, the first position information and the first time information may be the position information and time information of the object 3D model in the data, and the second position information and the second time information may be the position information and time information of the scale model 3D model in the data.

[0030] Moreover, by expanding on the above, the first position information and the first time information may be position information and time information of an object in the real world, and the second position information and the second time information may be position information and time information of a scale model three-dimensional model in data. Also, the first position information and the first time information may be position information and time information of a scale model three-dimensional model in data, and the second position information and the second time information may be position information and time information of a scale model in the real world. Even in the above cases, as in the first embodiment, the user can indirectly and easily grasp the state of the object by grasping the state of the scale model.

[0031] <Variation 2> In the first embodiment, the 3D model generator 14 may generate a scale model to which position information and time information of an object in the real world, or position information and time information of a 3D model of an object, is added. The position information and generation time may be added to the scale model by, for example, embossing or debossing, or may be added to the scale model by other methods. With this configuration, the user can easily check the position information and time information of the object or the 3D model of an object used to generate the scale model.

[0032] <Modification 3> In the first embodiment, when a predetermined symbol added to the scale model is measured by the three-dimensional measuring unit 11, the three-dimensional model generating unit 12 may generate a three-dimensional model of the scale model including a three-dimensional model previously associated with the symbol. The symbol may include characters and figures, and may be added by handwriting with a pen or the like by the user, for example.

[0033] FIG. 3 is a diagram showing an example of a symbol and a 3D model previously associated with the symbol. For example, when the symbol in the upper part of FIG. 3 is added to a scale model, the 3D model generation unit 12 adds a human-shaped 3D model to the scale model 3D model generated from the scale model at a position corresponding to the position where the symbol was added. Then, the 3D model generation unit 12 generates the scale model 3D model to which the human-shaped 3D model is added as a new scale model 3D model. With this configuration, the scale model 3D model can be changed by simply adding a symbol to the scale model without regenerating a new scale model.

[0034] <Embodiment 2> 4 is a block diagram showing the configuration of an information processing device 1 according to the present embodiment 2. In the following, among the components according to the present embodiment 2, components that are the same as or similar to the components described above are given the same or similar reference numerals, and different components will be mainly described.

[0035] The configuration in FIG. 4 is similar to the configuration in FIG. 1, except that a projection control unit 15 and a projection system 21 are added.

[0036] The projection system 21 has a projector (not shown) capable of projecting onto the scale model. The projection control unit 15 causes the projector to project a texture onto the scale model based on the three-dimensional model of the scale model and the position and orientation of the projector of the projection system 21 relative to the scale model.

[0037] Here, as an example, a description will be given of an example in which the 3D measurement unit 11 includes a camera that captures images of the color and pattern of an object in the real world as a surface image of the object. The projection control unit 15 places the scale model 3D model and the projector in the virtual 3D space on the data, based on the scale model 3D model and the position and orientation of the projector relative to the scale model.

[0038] The projection control unit 15 generates a texture so that when a two-dimensional planar texture is projected from a projector onto a scale model three-dimensional model in a virtual three-dimensional space, the texture on the surface of the scale model three-dimensional model resembles the surface image acquired by the three-dimensional measurement unit 11. Then, the projection control unit 15 projects the generated texture onto the scale model from the projector of the projection system 21 in the real world.

[0039] <Summary of the second embodiment> According to the information processing device 1 of the second embodiment, a texture is projected from the projector onto the scale model based on the scale model 3D model and the position and orientation of the projector of the projection system 21 relative to the scale model. Here, when the 3D model generating unit 14 is a 3D printer, the color of the scale model generated conventionally is a single color such as white. For this reason, there has been a problem that it is difficult for a user to imagine the state of an object in the real world. In contrast, according to the configuration of the information processing device 1 of the second embodiment, the scale model can be made to resemble the object not only in terms of shape but also in terms of surface appearance, so that the user can easily imagine the state of the object in the real world.

[0040] <Modification of the second embodiment> The texture projected onto the scale model is not limited to the texture described in the second embodiment. For example, the three-dimensional measuring unit 11 may include a thermo camera capable of detecting thermography of an object. In this case, the projection control unit 15 may generate a texture so that the texture in the scale model three-dimensional model resembles the thermography acquired by the three-dimensional measuring unit 11, and project it onto the scale model. Also, for example, the three-dimensional measuring unit 11 may include a camera and an image processing unit capable of detecting people flow. In this case, the projection control unit 15 may generate a texture of points in the scale model three-dimensional model whose movement resembles the people flow acquired by the three-dimensional measuring unit 11, and project it onto the scale model.

[0041] Furthermore, the texture projected onto the scale model may be position information and time information linked to the scale model 3D model or other attribute information. Furthermore, the information processing device 1 may obtain the information on which the texture is based, such as the surface image, from outside the information processing device 1, instead of obtaining it from the 3D measurement unit 11. Furthermore, the projection control unit 15 may project the texture onto the object in the same manner. That is, the projection control unit 15 may project the texture onto the object from a projector (not shown) capable of projecting onto the object, based on the object 3D model and the position and orientation of the projector relative to the object.

[0042] <Embodiment 3> 5 is a block diagram showing the configuration of an information processing device 1 according to the present embodiment 3. In the following, among the components according to the present embodiment 3, components that are the same as or similar to the components described above are given the same or similar reference numerals, and different components will be mainly described.

[0043] The configuration in FIG. 5 is similar to the configuration in FIG. 1, except that a three-dimensional model difference extraction unit 16, a projection system 21, and a robot system 22 are added to the configuration in FIG.

[0044] The 3D model difference extraction unit 16 extracts the difference between the object 3D model and the scale model 3D model linked by at least the first position information and the second position information. The first position information and the second position information here are the first position information and the second position information described in the first modification of the first embodiment. The 3D model difference extraction unit 16 may determine whether or not a part of the object 3D model and a part of the scale model 3D model are the same by using, for example, a predetermined template having the shape of multiple faces of the scale model, and extract the above-mentioned difference by using the determination result.

[0045] The three-dimensional model difference extraction unit 16 generates a task instruction for eliminating the extracted difference. In the third embodiment, the task instruction includes an instruction to project a texture from a projector onto at least one of the object and the scale model to eliminate the difference. In this specification, for example, at least one of A, B, C, ..., and Z means any one of all combinations of one or more types extracted from the group of A, B, C, ..., and Z.

[0046] Examples of some task instructions are described below. For example, when either the object or the scale model is moved, the three-dimensional model difference extraction unit 16 may instruct the robot system 22 or the user to move the other in accordance with the one to eliminate the difference. For example, when either the object or the scale model is moved but it is difficult to move the other, the three-dimensional model difference extraction unit 16 may instruct the projector of the projection system 21 to project a texture onto the other that eliminates the apparent difference. For example, when an appendage such as a scale chair model is added to the scale model, the three-dimensional model difference extraction unit 16 may instruct the projector of the projection system 21 to project the texture of the appendage onto the object.

[0047] When issuing a task instruction to the projector, the 3D model difference extraction unit 16 may control the projector of the projection system 21 to project a texture, similar to the projection control unit 15 in the second embodiment and its modified examples. At least one of the projection system 21 and the robot system 22 may be applied to at least one of the object and the scale model.

[0048] <Summary of the Third Embodiment> According to the information processing device 1 of the third embodiment, a difference between the object 3D model and the scale model 3D model linked by at least the first location information and the second location information is extracted, and a task instruction for eliminating the extracted difference is generated. With such a configuration, changes in the state of the object and the scale model can be appropriately reflected in both directions.

[0049] <Fourth embodiment> 6 is a block diagram showing the configuration of an information processing device 1 according to the present embodiment 4. In the following, among the components according to the present embodiment 4, components that are the same as or similar to the components described above are given the same or similar reference numerals, and different components will be mainly described.

[0050] The configuration in Fig. 6 is the same as that in Fig. 1, except that a display control unit 17 and a display device 23 are added to the configuration in Fig. 1. The information processing device 1 and the display device 23 configure an information processing system.

[0051] The display control unit 17 causes the display device 23 to display the object 3D model and the scale model 3D model linked by at least the first location information and the second location information. The display control unit 17 may simultaneously display the object 3D model and the scale model 3D model or may selectively display them. In addition, when the information processing device 1 is provided with an operation unit (not shown) that accepts operations of enlarging, reducing, and rotating, the display control unit 17 may enlarge, reduce, and rotate the displayed object 3D model, etc., based on the operation accepted by the operation unit.

[0052] The display device 23 is, for example, a dedicated display device, a smartphone display device, a tablet display device, or smart glasses capable of displaying virtual reality (VR). The smart glasses may be see-through type smart glasses capable of displaying a 3D object model and a 3D scale model at a position in the real world.

[0053] <Summary of the fourth embodiment> According to the information processing device 1 of the fourth embodiment, the object 3D model and the scale model 3D model linked by at least the first position information and the second position information are displayed on the display device 23. With such a configuration, the user can check from the display device 23 whether the states of the object and the object 3D model and the states of the scale model and the scale model 3D model are correctly reflected in both directions. Furthermore, a user who is familiar with operating 3D models can grasp the object 3D model and the scale model 3D model from the display device 23.

[0054] <Other Modifications> The above-mentioned 3D measurement unit 11, 3D model generation unit 12, and 3D model generation unit 14 in FIG. 1 are hereinafter referred to as "3D measurement unit 11, etc." The 3D measurement unit 11, etc. are realized by a processing circuit 81 shown in FIG. 7. That is, the processing circuit 81 includes the 3D measurement unit 11 that measures an object in the real world, the 3D model generation unit 12 that generates an object 3D model, which is data, based on the measurement result of the object, and the 3D model generation unit 14 that generates a scale model in the real world based on the object 3D model. The 3D measurement unit 11 measures the scale model, the 3D model generation unit 12 generates the scale model 3D model, which is data, based on the measurement result of the scale model, and the processing circuit 81 links first position information, which is position information of the object in the real world or position information of the object 3D model, to second position information, which is position information of the scale model in the real world or position information of the scale model 3D model. Dedicated hardware or a processor that executes a program stored in a memory may be applied to the processing circuit 81. Examples of the processor include a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, and a DSP (Digital Signal Processor).

[0055] When the processing circuit 81 is a dedicated hardware, the processing circuit 81 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each function of the three-dimensional measurement unit 11 and the like may be realized by a circuit in which the processing circuits are distributed, or the functions of each unit may be realized by a single processing circuit.

[0056] When the processing circuit 81 is a processor, the functions of the three-dimensional measuring unit 11 and the like are realized by a combination with software and the like. The software and the like includes, for example, software, firmware, or software and firmware. The software and the like is written as a program and stored in a memory. As shown in FIG. 8, the processor 82 applied to the processing circuit 81 realizes the functions of each unit by reading and executing a program stored in the memory 83. That is, the information processing device 1 includes a memory 83 for storing a program that, when executed by the processing circuit 81, results in the execution of the steps of measuring an object in the real world, generating an object three-dimensional model as data based on the measurement results of the object, generating a scale model in the real world based on the object three-dimensional model, measuring the scale model, generating a scale model three-dimensional model as data based on the measurement results of the scale model, and linking first position information, which is position information of the object in the real world or position information of the object three-dimensional model, with second position information, which is position information of the scale model in the real world or position information of the scale model three-dimensional model. In other words, this program causes a computer to execute procedures and methods of the three-dimensional measuring unit 11, etc. Here, the memory 83 may be, for example, a non-volatile or volatile semiconductor memory such as a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), or an electrically erasable programmable read only memory (EEPROM), a hard disk drive (HDD), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a digital versatile disk (DVD), a drive device for any of these, or any storage medium to be used in the future.

[0057] The above describes a configuration in which each function of the three-dimensional measurement unit 11, etc. is realized either by hardware or software, etc. However, the present invention is not limited to this, and a configuration in which a part of the three-dimensional measurement unit 11, etc. is realized by dedicated hardware and another part is realized by software, etc. For example, the function of the three-dimensional measurement unit 11 can be realized by a processing circuit 81 as dedicated hardware, and the other functions can be realized by the processing circuit 81 as a processor 82 reading and executing a program stored in a memory 83.

[0058] As described above, the processing circuitry 81 can realize each of the above-mentioned functions by hardware, software, or a combination of these.

[0059] The information processing device described above can also be applied to an information processing system constructed as a system by appropriately combining a processing device, a communication terminal, the functions of an application installed in at least one of the processing device and the communication terminal, and a server. The communication terminal includes, for example, a mobile phone, a smartphone, and a tablet. Each function or each component of the information processing device described above may be distributed and arranged in each device that constructs the system, or may be concentrated and arranged in one of the devices.

[0060] In this disclosure, 'a' and 'an' mean one or more. Therefore, 'a', 'an', 'one or more' and 'at least one' can be used interchangeably.

[0061] It should be noted that the embodiments and modifications may be freely combined, and the embodiments and modifications may be modified or omitted as appropriate.

[0062] The above description is illustrative in all respects and is not restrictive. It is understood that countless variations not illustrated can be envisioned. [Explanation of symbols]

[0063] 1 information processing device, 11 three-dimensional measurement unit, 12 three-dimensional model generation unit, 12a point cloud correction unit, 12b mesh generation unit, 14 three-dimensional model generation unit, 15 projection control unit, 16 three-dimensional model difference extraction unit, 17 display control unit, 23 display device.

Claims

1. An information processing device, A 3D measurement unit that measures objects in the real world, A 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the aforementioned object. A 3D model generation unit generates a scaled-down model of the object in the real world based on the aforementioned 3D model of the object. Equipped with, The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. The information processing device links a first position information, which is the position information of the object in the real world or the position information of the 3D model of the object, with a second position information, which is the position information of the scaled model in the real world or the position information of the 3D model of the scaled model. The three-dimensional model generation unit is an information processing device that links first time information, which is the time information of the object in the real world or the time information of the three-dimensional model of the object, with second time information, which is the time information of the scaled model in the real world or the time information of the scaled model three-dimensional model.

2. An information processing device, A 3D measurement unit that measures objects in the real world, A 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the aforementioned object. A 3D model generation unit generates a scaled-down model of the object in the real world based on the aforementioned 3D model of the object. Equipped with, The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model, and when a predetermined symbol attached to the scaled model is measured by the 3D measurement unit, it generates a 3D model of the scaled model that includes a 3D model pre-associated with the symbol. The information processing device links first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model.

3. An information processing device, A 3D measurement unit that measures objects in the real world, A 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the aforementioned object. A 3D model generation unit generates a scaled-down model of the object in the real world based on the aforementioned 3D model of the object, A projection control unit that projects a texture onto the scale model from a projector capable of projecting onto the scale model, and Equipped with, The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. The projection control unit projects the texture based on the scaled 3D model and the position and orientation of the projector relative to the scaled model. The information processing device links first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model.

4. An information processing device, A 3D measurement unit that measures objects in the real world, A 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the aforementioned object. A 3D model generation unit generates a scaled-down model of the object in the real world based on the aforementioned 3D model of the object, A projection control unit that projects a texture onto the object from the projector based on the three-dimensional model of the object and the position and orientation of the projector that can project onto the object. Equipped with, The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. The information processing device links first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model.

5. An information processing device, A 3D measurement unit that measures objects in the real world, A 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the aforementioned object. A 3D model generation unit generates a scaled-down model of the object in the real world based on the aforementioned 3D model of the object. Equipped with, The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. The information processing device links first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model, and generates a task instruction to resolve the difference between the position of the object defined by the first position information and the position of the scaled model defined by the second position information.

6. An information processing device, A 3D measurement unit that measures objects in the real world, A 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the aforementioned object. A 3D model generation unit generates a scaled-down model of the object in the real world based on the aforementioned 3D model of the object. Equipped with, The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. The information processing device links first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model, and generates a task instruction to resolve the difference between the orientation of the object defined by the first position information and the orientation of the scaled model defined by the second position information.

7. An information processing device according to any one of claims 1 to 6, The first position information is the position information of the object in the real world, and the second position information is the position information of the scaled model in the real world, or An information processing device in which the first position information is the position information of the three-dimensional model of the object, and the second position information is the position information of the three-dimensional model of the scaled-down model.

8. An information processing device according to any one of claims 2 to 6, The aforementioned 3D model generation unit, An information processing device that links first time information, which is the time information of the object in the real world or the time information of the three-dimensional model of the object, with second time information, which is the time information of the scaled model in the real world or the time information of the three-dimensional model of the scaled model.

9. An information processing apparatus according to claim 8, The three-dimensional model generation unit is An information processing device that generates the scaled model to which the first position information and the first time information are added.

10. An information processing apparatus according to claim 1, claim 3, claim 4, claim 5, or claim 6, The aforementioned 3D model generation unit, An information processing device that generates a three-dimensional model of the scaled model, including a three-dimensional model pre-associated with a predetermined symbol attached to the scaled model, when the three-dimensional measurement unit measures the predetermined symbol attached to the scaled model.

11. An information processing device according to claim 1, claim 2, claim 5, or claim 6, An information processing device further comprising a projection control unit that projects a texture onto the scaled model from a projector based on the scaled model three-dimensional model and the position and orientation of a projector capable of projecting onto the scaled model.

12. An information processing device according to claim 1, claim 2, claim 5, or claim 6, An information processing device further comprising a projection control unit that causes a texture to be projected onto the object from the projector based on the three-dimensional model of the object and the position and orientation of the projector with respect to the object, which is capable of projecting onto the object.

13. An information processing apparatus according to claim 1 or claim 2, An information processing device further comprising a 3D model difference extraction unit that extracts the difference between the 3D model of an object linked by at least the first position information and the second position information and the scaled-down 3D model.

14. An information processing apparatus according to claim 13, The three-dimensional model difference extraction unit is an information processing device that generates task instructions for resolving the difference.

15. An information processing apparatus according to claim 14, An information processing device in which the task instruction includes an instruction to cause a projector to project a texture for resolving the difference onto at least one of the object and the scale model.

16. An information processing device according to any one of claims 1 to 6, An information processing apparatus further comprising a display control unit that causes the three-dimensional model of an object and the three-dimensional scale model of a model, which are linked by at least the first position information and the second position information, to be displayed on a display device.

17. An information processing device according to any one of claims 1 to 6, The aforementioned 3D model generation unit, A point cloud correction unit generates a reference plane based on the point cloud representing the object and performs a correction by moving the point cloud within the reference plane, A mesh generation unit generates a mesh based on the point cloud moved within the reference plane, and generates a three-dimensional model of the object based on the mesh. Information processing device, including

18. The information processing apparatus according to claim 16, The display device and An information processing system equipped with the following features.

19. The 3D measurement unit measures objects in the real world, The 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the object. The 3D model generation unit generates a scaled model of the object in the real world based on the 3D model of the object. The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. The information processing device links a first position information, which is the position information of the object in the real world or the position information of the 3D model of the object, with a second position information, which is the position information of the scaled model in the real world or the position information of the 3D model of the scaled model. An information processing method in which the three-dimensional model generation unit links first time information, which is time information of the object in the real world or time information of the three-dimensional model of the object, with second time information, which is time information of the scaled model in the real world or time information of the scaled model three-dimensional model.

20. The 3D measurement unit measures objects in the real world, The 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the object. The 3D model generation unit generates a scaled model of the object in the real world based on the 3D model of the object. The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model, and when a predetermined symbol attached to the scaled model is measured by the 3D measurement unit, it generates a 3D model of the scaled model that includes a 3D model pre-associated with the symbol. An information processing method comprising an information processing device that links first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model.

21. The 3D measurement unit measures objects in the real world, The 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the object. The 3D model generation unit generates a scaled model of the object in the real world based on the 3D model of the object. The projection control unit projects a texture onto the scale model from a projector capable of projecting onto the scale model. The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. The projection control unit projects the texture based on the scaled-down 3D model and the position and orientation of the projector relative to the scaled-down model. An information processing method comprising an information processing device that links first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model.

22. The 3D measurement unit measures objects in the real world, The 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the object. The 3D model generation unit generates a scaled model of the object in the real world based on the 3D model of the object. The projection control unit causes the projector to project a texture onto the object based on the three-dimensional model of the object and the position and orientation of the projector capable of projecting onto the object. The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. An information processing method comprising an information processing device that links first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model.

23. The 3D measurement unit measures objects in the real world, The 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the object. The 3D model generation unit generates a scaled model of the object in the real world based on the 3D model of the object. The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. An information processing method comprising: an information processing device linking first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model, and generating a task instruction to resolve the difference between the position of the object defined by the first position information and the position of the scaled model defined by the second position information.

24. The 3D measurement unit measures objects in the real world, The 3D model generation unit generates a 3D model of the object, which is data, based on the measurement results of the object. The 3D model generation unit generates a scaled model of the object in the real world based on the 3D model of the object. The three-dimensional measurement unit measures the scaled model, The 3D model generation unit generates a 3D model of the scaled model, which is data, based on the measurement results of the scaled model. An information processing method comprising: an information processing device linking first position information, which is the position information of the object in the real world or the position information of the three-dimensional model of the object, with second position information, which is the position information of the scaled model in the real world or the position information of the three-dimensional model of the scaled model, and generating a task instruction to resolve the difference between the orientation of the object defined by the first position information and the orientation of the scaled model defined by the second position information.

Images