Information processing device, information processing method, and program

Abstract

To realize an information processor to desirably provide a three-dimensional model of a target.SOLUTION: An information processor (1) includes: an acquisition unit (11) for acquiring three-dimensional scan data; a specification unit (12) for specifying a three-dimensional model of a target; and an output unit (13) for outputting the three-dimensional model. The specification unit (12) includes: an object detection unit (121) for specifying an object by performing object detection processing; and a search unit (122) for searching the three-dimensional model of an object from three-dimensional model candidates.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] A technique related to three-dimensional modeling is disclosed, which generates three-dimensional model data from three-dimensional scan data obtained by scanning an object.

[0003] The technology described in Patent Document 1 generates 3D reconstructed data based on captured images of an object to be inspected, and matches the reconstructed data with a standard model, which is a standard 3D model of the object to be inspected.The technology described in Patent Document 1 then displays an image in which the standard model and the reconstructed data are projected onto 2D coordinates. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021 / 234907 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology described in Patent Document 1 has the problem that it can only be used when the inspection object is determined. In addition, the technology described in Patent Document 1 has the problem that it cannot provide the user with a three-dimensional model of the inspection object because it displays an image projected onto two-dimensional coordinates.

[0006] One aspect of the present invention has been made in view of the above-mentioned problems, and an example of a purpose thereof is to provide a technique for suitably providing a three-dimensional model of an object. [Means for solving the problem]

[0007] An information processing device according to one embodiment of the present invention comprises an acquisition means for acquiring 3D scan data obtained by sensing one or more objects, an identification means for identifying a 3D model corresponding to at least one of the one or more objects by referring to the 3D scan data, and an output means for outputting the identified 3D model, wherein the identification means comprises an object detection means for identifying at least one of the one or more objects by performing an object detection process on the 3D scan data, and a search means for searching for a 3D model corresponding to at least one of the one or more objects from a plurality of candidate 3D models.

[0008] An information processing method according to one embodiment of the present invention includes an information processing device acquiring 3D scan data obtained by sensing one or more objects, identifying a 3D model corresponding to at least one of the one or more objects by referring to the 3D scan data, and outputting the identified 3D model, wherein the identifying includes identifying at least one of the one or more objects by performing an object detection process on the 3D scan data, and searching for a 3D model corresponding to at least one of the one or more objects from a plurality of candidate 3D models.

[0009] A program according to one embodiment of the present invention is a program that causes a computer to function as an information processing device, and the program causes the computer to function as: an acquisition means that acquires 3D scan data obtained by sensing one or more objects; an identification means that identifies a 3D model corresponding to at least one of the one or more objects by referring to the 3D scan data; and an output means that outputs the identified 3D model, wherein the identification means functions as: an object detection means that identifies at least one of the one or more objects by performing object detection processing on the 3D scan data; and a search means that searches for a 3D model corresponding to at least one of the one or more objects from a plurality of 3D model candidates. [Effects of the Invention]

[0010] According to one aspect of the present invention, a three-dimensional model of an object can be preferably provided. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment of the present invention. [Figure 2] 1 is a flowchart showing the flow of an information processing method according to a first exemplary embodiment of the present invention. [Figure 3] 10 is a flowchart showing the flow of step S12 by an identification unit according to the first exemplary embodiment of the present invention. FIG. [Figure 4] FIG. 10 is a block diagram showing the configuration of an information processing system according to a second exemplary embodiment of the present invention. [Figure 5] FIG. 10 is a diagram showing an example of an outline of processing executed by an information processing system according to a second exemplary embodiment of the present invention. [Figure 6] FIG. 10 is a diagram showing an example of an image presented to a user in exemplary embodiment 2 of the present invention. [Figure 7] FIG. 10 is a block diagram showing the configuration of an information processing system according to a third exemplary embodiment of the present invention. [Figure 8]FIG. 10 is a diagram showing an example of an image presented to a user in exemplary embodiment 3 of the present invention. [Figure 9] FIG. 10 is a block diagram showing the configuration of an information processing system according to a fourth exemplary embodiment of the present invention. [Figure 10] 1 is a block diagram showing an example of a hardware configuration of an information processing device and a terminal device according to each exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0013] (Overview of information processing device 1) The information processing device 1 according to this exemplary embodiment is a device that outputs a three-dimensional model corresponding to at least one of one or more objects from three-dimensional scan data obtained by sensing the one or more objects.

[0014] The "one or more objects" is not limited to, but examples include furniture, electrical appliances, and kitchen utensils.

[0015] "3D scan data" is data obtained by sensing one or more objects. In other words, "3D scan data" includes one or more objects. An example of "3D scan data" is point cloud data, which is a collection of points with three-dimensional coordinates (x, y, z). Sensing an object also includes photographing the object.

[0016] One example of a method for acquiring 3D scan data is to use LiDAR (Light Detection and Ranging) to measure the distance to the object. Another example of a method for acquiring 3D scan data is to use the parallax between multiple image capture devices that capture images of the object.

[0017] (Configuration of information processing device 1) The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of an information processing device 1 according to this exemplary embodiment.

[0018] 1, the information processing device 1 according to this exemplary embodiment includes an acquisition unit 11, an identification unit 12, and an output unit 13. The acquisition unit 11, the identification unit 12, and the output unit 13 are components that respectively realize an acquisition means, an identification means, and an output means in this exemplary embodiment.

[0019] The acquisition unit 11 acquires three-dimensional scan data obtained by sensing one or more objects, and supplies the acquired three-dimensional scan data to the identification unit 12.

[0020] The identification unit 12 identifies a three-dimensional model corresponding to at least one of the one or multiple objects by referring to the three-dimensional scan data supplied from the acquisition unit 11. The identification unit 12 supplies information indicating the identified three-dimensional model to the output unit 13.

[0021] 1, the identification unit 12 includes an object detection unit 121 and a search unit 122. The object detection unit 121 and the search unit 122 are components that respectively realize an object detection means and a search means in this exemplary embodiment.

[0022] The object detection unit 121 With reference to the 3D scan data supplied from the acquisition unit 11, Objects for 3D scan data of detection do By performing the processing, at least one of one or a plurality of objects is identified. The object detection unit 121 supplies the search unit 122 with information indicating at least one of the identified one or a plurality of objects.

[0023] The search unit 122 The information provided by the object detection unit 121 indicatesThe search unit 122 searches a plurality of 3D model candidates for a 3D model corresponding to at least one of one or a plurality of objects, and supplies the 3D model identified by the search to the output unit 13.

[0024] The output unit 13 outputs the three-dimensional model identified by the identification unit 12.

[0025] As described above, the information processing device 1 according to this exemplary embodiment employs a configuration including an acquisition unit 11 that acquires 3D scan data obtained by sensing one or more objects, an identification unit 12 that identifies a 3D model corresponding to at least one of the one or more objects by referring to the 3D scan data, and an output unit 13 that outputs the identified 3D model.

[0026] In the information processing device 1 according to this exemplary embodiment, the identification unit 12 identifies an object in the three-dimensional scan data. of detection do The system includes an object detection unit 121 that performs processing to identify at least one or a plurality of objects, and a search unit 122 that searches for a 3D model corresponding to at least one of the one or a plurality of objects from among a plurality of 3D model candidates.

[0027] Therefore, according to the information processing device 1 of this exemplary embodiment, an object contained in the 3D scan data is detected and a 3D model corresponding to the detected object is output, thereby achieving the effect of being able to provide a 3D model of the object in an appropriate manner.

[0028] (Flow of information processing method S1) The flow of the information processing method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S1 according to this exemplary embodiment.

[0029] (Step S11) In step S11, the acquiring unit 11 acquires 3D scan data obtained by sensing one or more objects, and supplies the acquired 3D scan data to the identifying unit 12.

[0030] (Step S12) In step S12, the identification unit 12 identifies a 3D model corresponding to at least one of the one or more objects by referring to the 3D scan data supplied from the acquisition unit 11 in step S11. A more specific flow of step S12 performed by the identification unit 12 will be described later. The identification unit 12 supplies information indicating the identified 3D model to the output unit 13.

[0031] (Step S13) In step S13, the output unit 13 outputs the three-dimensional model identified by the identification unit 12 in step S12.

[0032] (Step S12) A more specific flow of step S12 performed by the identification unit 12 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a flow chart showing the flow of step S12 performed by the identification unit 12 according to this exemplary embodiment.

[0033] (Step S21) In step S21, the object detection unit 121 With reference to the 3D scan data supplied from the acquisition unit 11, Objects for 3D scan data of detection do By performing the processing, at least one of one or a plurality of objects is identified. The object detection unit 121 supplies the search unit 122 with information indicating at least one of the identified one or a plurality of objects.

[0034] (Step S22) In step S22, the search unit 122 The information provided by the object detection unit 121 indicatesThe search unit 122 searches a plurality of 3D model candidates for a 3D model corresponding to at least one of one or a plurality of objects, and supplies the 3D model identified by the search to the output unit 13.

[0035] As described above, in the information processing method S1 according to this exemplary embodiment, in step S11, the acquisition unit 11 acquires 3D scan data obtained by sensing one or more objects, in step S12 the identification unit 12 refers to the 3D scan data to identify a 3D model corresponding to at least one of the one or more objects, and in step S13 the output unit 13 outputs the identified 3D model.

[0036] Furthermore, the specific processing according to this exemplary embodiment Steps In step S12, the object detection unit 121 detects an object in the 3D scan data in step S21. of detection do By performing the process, at least one of one or a plurality of objects is identified, and in step S22, the search unit 122 searches a plurality of 3D model candidates for a 3D model corresponding to at least one of the one or a plurality of objects. Therefore, according to the information processing method S1 according to this exemplary embodiment, the same effects as those of the information processing device 1 described above can be obtained.

[0037] Exemplary Embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and their description will be omitted as appropriate.

[0038] (Configuration of information processing system 100A) An information processing system 100A according to this exemplary embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the information processing system 100A according to this exemplary embodiment.

[0039] The information processing system 100A according to this exemplary embodiment includes an information processing device 1A and a terminal device 2, as shown in FIG.

[0040] The information processing device 1A and the terminal device 2 are connected to each other so that they can communicate with each other. The information processing device 1A and the terminal device 2 may also be connected to each other so that they can communicate with each other via a network (not shown). In this case, the specific configuration of the network does not limit the present embodiment, but as an example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks can be used.

[0041] In the information processing system 100A, the information processing device 1A refers to the 3D scan data acquired by the terminal device 2, and identifies a 3D model corresponding to at least one of one or more objects included in the 3D scan data. Then, the information processing device 1A outputs the identified 3D model to the terminal device 2. The terminal device 2 acquires the 3D model output from the information processing device 1A.

[0042] The one or more objects and the 3D scan data are as described above.

[0043] (An example of an outline of processing executed by the information processing system 100A) An example of an outline of the processing executed by the information processing system 100A will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of an outline of the processing executed by the information processing system 100A according to this exemplary embodiment.

[0044] First, the terminal device 2 acquires 3D scan data. As an example, the terminal device 2 acquires 3D scan data as shown in the upper image of Fig. 5. The terminal device 2 outputs the acquired 3D scan data to the information processing device 1A.

[0045] When the information processing device 1A acquires the 3D scan data output from the terminal device 2, it identifies at least one of one or more objects included in the 3D scan data. As an example, the information processing device 1A identifies the object "sofa" included in the 3D scan data, as shown in the upper image of FIG.

[0046] Next, the information processing device 1A outputs a 3D model corresponding to the identified object to the terminal device 2. As an example, as shown in the central image of FIG. 5, the information processing device 1A outputs a 3D model corresponding to the object “sofa” included in the identified 3D scan data to the terminal device 2.

[0047] Furthermore, the information processing device 1A may output 3D data corresponding to at least a part of the 3D scan data to the terminal device 2. One example of "3D data corresponding to at least a part of the 3D scan data" is 3D data corresponding to data in the 3D scan data other than data corresponding to a 3D model. For example, the information processing device 1A may output all or part of the data in the 3D scan data other than the "sofa" (for example, a wall, a desk, etc.) to the terminal device 2.

[0048] Next, the terminal device 2 acquires the 3D model output from the information processing device 1A. As an example, the terminal device 2 presents the acquired 3D model. As an example, as shown in the image in the lower part of FIG. 5, the terminal device 2 presents a stereoscopic image in which the object "sofa" is replaced with the acquired 3D model, and which references the 3D scan data shown in the upper part of FIG. 5 (or 3D data corresponding to at least a part of the 3D scan data output from the information processing device 1A). In the information processing system 100A, the back side of the sofa, which could not be presented to the user in the 3D scan data, can also be presented to the user in a stereoscopic manner.

[0049] (Configuration of terminal device 2) As shown in FIG. 4, the terminal device 2 includes a control unit 20, a communication unit 25, and an input / output unit .

[0050] The control unit 20 controls each component included in the terminal device 2. As an example, the control unit 20 supplies data acquired from the input / output unit 26 to the communication unit 25, and supplies data acquired from the communication unit 25 to the input / output unit 26.

[0051] The communication unit 25 is a communication module that communicates with other connected devices. For example, the communication unit 25 outputs data supplied from the control unit 20 to the information processing device 1A, and supplies data output from the information processing device 1A to the control unit 20.

[0052] The input / output unit 26 is a device that accepts input of data. The input / output unit 26 is also a device that outputs data. For example, the input / output unit 26 supplies input data to the control unit 20, and outputs data supplied from the control unit 20. Examples of data accepted by the input / output unit 26 include 3D scan data and information indicating input from a user. Examples of data output by the input / output unit 26 include 3D models, 3D model candidates, and stereoscopically visible images.

[0053] (Configuration of information processing device 1A) As shown in FIG. 4, the information processing device 1A includes a control unit 10, a communication unit 15, and a storage unit 16.

[0054] The communication unit 15 is a communication module that communicates with other connected devices. For example, the communication unit 15 outputs data supplied from the control unit 10 to the terminal device 2, and supplies data output from the terminal device 2 to the control unit 20.

[0055] The storage unit 16 stores data referenced by the control unit 10. An example of the data stored in the storage unit 16 is a plurality of three-dimensional models 3DM.

[0056] (Function of control unit 10) The control unit 10 controls each component included in the information processing device 1A. As shown in Fig. 4, the control unit 10 also functions as an acquisition unit 11, an identification unit 12, an output unit 13, and a generation unit 17. The acquisition unit 11, the identification unit 12, the output unit 13, and the generation unit 17 are configured to realize an acquisition means, an identification means, an output means, and a generation means, respectively, in this exemplary embodiment.

[0057] The acquisition unit 11 acquires data supplied from the communication unit 15. Examples of data acquired by the acquisition unit 11 include 3D scan data and information indicating input from a user. The acquisition unit 11 supplies the acquired data to the identification unit 12, the output unit 13, and the generation unit 17.

[0058] The identification unit 12 refers to the 3D scan data to identify a 3D model corresponding to at least one of one or a plurality of objects. As an example, the identification unit 12 refers to the 3D scan data supplied from the acquisition unit 11 to identify a 3D model corresponding to at least one of one or a plurality of objects from the plurality of 3D models 3DM stored in the storage unit 16.

[0059] 4, the identification unit 12 may include an object detection unit 121, a search unit 122, a presentation unit 123, and a selection unit 124. The object detection unit 121, the search unit 122, the presentation unit 123, and the selection unit 124 are components that respectively realize an object detection means, a search means, a presentation means, and a selection means in this exemplary embodiment.

[0060] The object detection unit 121 With reference to the 3D scan data supplied from the acquisition unit 11, Objects for 3D scan data of detection do By performing the processing, at least one of one or a plurality of objects is identified. The object detection unit 121 supplies information indicating at least one of the identified one or a plurality of objects to the search unit 122. of detection do An example of the process will be described later.

[0061] The search unit 122 The information provided by the object detection unit 121 indicates A three-dimensional model corresponding to at least one of one or more objects is searched from a plurality of three-dimensional model candidates.

[0062] The search unit 122 may also perform a search using the similarity between the three-dimensional model and the three-dimensional model candidate.

[0063] The search unit 122 also receives information from a user specifying a subspace in the target space that includes one or more objects, and searches for the object in the 3D scan data corresponding to the subspace specified based on the information specifying the subspace. of detection do Treatment may be applied.

[0064] An example of the process executed by the search unit 122 will be described later.

[0065] The presentation unit 123 Among the multiple 3D model candidates searched by the search unit 122, A plurality of 3D model candidates with relatively high similarity are presented to the user. An example of the processing executed by the presentation unit 123 will be described later.

[0066] The selection unit 124 The presenting unit 123 presents the three-dimensional model candidates. Based on an input from the user, the selection unit 124 selects a three-dimensional model corresponding to at least one of one or more objects. An example of the processing executed by the selection unit 124 will be described later.

[0067] The output unit 13 outputs the 3D model identified by the identification unit 12. The output unit 13 may output information indicating the 3D model itself (for example, an image of the 3D model, an identifier of the 3D model) as the identified 3D model. Alternatively, the output unit 13 may output an image including the 3D model (for example, an image generated by the generation unit 17 described later).

[0068] Furthermore, as described above, the output unit 13 may output 3D data corresponding to at least a part of the 3D scan data, in addition to the 3D model identified by the identification unit 12. With this configuration, the output unit 13 can provide the 3D data of the object included in the 3D scan data separately from the 3D data of the object other than the object.

[0069] The generation unit 17 generates a stereoscopic image obtained by referring to the 3D scan data, in which at least one of one or more objects is replaced with a 3D model. The generation unit 17 supplies the generated image to the output unit 13. An example of the processing executed by the generation unit 17 will be described later.

[0070] Here, "stereoscopically viewable image" refers to an image generated based on three-dimensional data, and is configured so that a user viewing the image can recognize that the image represents a three-dimensional object.

[0071] As an example, a "stereoscopic image" can provide a user with a three-dimensional impression when viewed from different viewpoints.

[0072] Furthermore, when using a display device that can present an image for the right eye and an image for the left eye separately, such as VR goggles, the "stereoscopically viewable image" may be configured to consist of an image for the right eye and an image for the left eye that have a parallax between them, and provide the user with a sense of three-dimensionality due to the parallax.

[0073] The processing of the generation unit 17 may be configured to be executed by the control unit 20 of the terminal device 2. In other words, the control unit 20 of the terminal device 2 may also function as the generation unit 17. In this case, the control unit 20 of the terminal device 2 refers to the 3D model output from the information processing device 1A, and generates a stereoscopic image obtained by referring to the 3D scan data, in which at least one of one or multiple objects is replaced with the 3D model.

[0074] (Object detection unit 121 executes ofdetection do (Example of processing) Object detection executed by the object detection unit 121 of detection do An example of the process will be described.

[0075] One example of a configuration in which the object detection unit 121 uses a detection model that receives 3D scan data as input and outputs one or more objects included in the 3D scan data. In this configuration, the object detection unit 121 inputs the 3D scan data provided by the acquisition unit 11 into the detection model, and identifies the output object or objects as at least one of the one or more objects included in the 3D scan data.

[0076] where: detection Although the specific configuration of the model is not limited, for example, a convolution neural network (CNN), a recurrent neural network (RNN), or a combination thereof can be used. Also, a non-neural network model such as a random forest or a support vector machine can be used.

[0077] For example, the object detection unit 121 inputs the 3D scan data shown in the upper part of Fig. 5 into the detection model. If the detection model outputs a "sofa," a "table," a "chair," and a "refrigerator," the object detection unit 121 identifies the objects included in the 3D scan data as the "sofa," the "table," the "chair," and the "refrigerator."

[0078] In this way, the object detection unit 121 detects an object in the 3D scan data. of detection do By performing processing to identify at least one of the one or more objects, it is possible to properly detect the one or more objects included in the 3D scan data.

[0079] (Another example of processing executed by the object detection unit 121) Another example of the process executed by the object detection unit 121 will be described.

[0080] As an example, the object detection unit 121 detects an object in 3D can data corresponding to a subspace in an object space that is a subspace designated by a user and includes one or more objects. of detection do The configuration in which the processing is applied is exemplified.

[0081] For example, the object detection unit 121 receives information from the user specifying a subspace in the target space, with the 3D scan data as the target space. As an example, the object detection unit 121 acquires information indicating an input from the user and specifying the subspace from the terminal device 2. Then, the object detection unit 121 detects an object in the 3D scan data corresponding to the subspace identified based on the acquired information. of detection do Apply the treatment.

[0082] In this way, the object detection unit 121 detects an object in the 3D scan data corresponding to the subspace based on the information received from the user. of detection do The processing is performed so that one or more objects contained in the 3D scan data can be properly detected.

[0083] (An example of processing executed by the search unit 122) An example of the process executed by the search unit 122 will be described.

[0084] For example, the search unit 122 searches for a 3D model corresponding to at least one of the one or more objects identified by the object detection unit 121 from the multiple 3D models 3DM stored in the storage unit 16. The search unit 122 supplies the 3D model identified by the search to the output unit 13, the generation unit 17, and the presentation unit 123.

[0085] For example, when the name of an object is associated with the 3D model 3DM stored in the storage unit 16, the search unit 122 searches for a 3D model 3DM stored in the storage unit 16 that is a 3D model associated with the name of one or more objects identified by the object detection unit 121. As an example, when a "sofa" is identified by the object detection unit 121, the search unit 122 searches for a 3D model 3DM that is associated with the object name "sofa" from the 3D models 3DM stored in the storage unit 16.

[0086] In this way, the search unit 122 searches for a three-dimensional model corresponding to at least one of one or a plurality of objects from among a plurality of three-dimensional model candidates, and therefore can specify an appropriate three-dimensional model.

[0087] (Another example of processing executed by the search unit 122) Another example of the process executed by the search unit 122 will be described.

[0088] As an example, the search unit 122 is configured to use a similarity determination model that receives a three-dimensional model and a three-dimensional model candidate as input and determines the similarity between the three-dimensional model and the three-dimensional model candidate.

[0089] For example, the search unit 122 generates a 3D model of an object detected by the object detection unit 121 from among the objects included in the 3D scan data. Next, the search unit 122 inputs the generated 3D model and multiple 3D models 3DM stored in the storage unit 16 into a similarity determination model. Then, the search unit 122 identifies the candidate 3D model with the highest similarity output from the similarity determination model as the 3D model corresponding to the object included in the 3D scan data.

[0090] Similarity judgementThe specific configuration of the model is not limited, and for example, a CNN, an RNN, or a combination thereof can be used. Also, a non-neural network model such as a random forest or a support vector machine can be used.

[0091] In this way, the search unit 122 performs a search using the similarity between the three-dimensional model and the three-dimensional model candidate, and therefore can specify an appropriate three-dimensional model.

[0092] (Example of processing executed by the presentation unit 123 and the selection unit 124) An example of the processing executed by the presentation unit 123 and the selection unit 124 will be described.

[0093] As an example, the presentation unit 123 refers to the similarity used by the search unit 122, and acquires multiple 3D model candidates with relatively high similarity from the multiple 3D models 3DM stored in the storage unit 16. The presentation unit 123 outputs the acquired multiple 3D model candidates to the terminal device 2 via the communication unit 15, thereby presenting them to the user.

[0094] The control unit 20 of the terminal device 2 acquires the plurality of 3D model candidates output from the information processing device 1A via the communication unit 25. The control unit 20 presents an image including the acquired plurality of 3D model candidates to the user via the input / output unit 26. An example of the image presented to the user will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of an image presented to the user in this exemplary embodiment.

[0095] As shown in FIG. 6, the control unit 20 presents an image including multiple 3D model candidates ("candidate 1" and "candidate 2") to the user. The control unit 20 may also indicate to which object included in the 3D scan data the multiple 3D model candidates correspond. In the image shown in FIG. 6, the control unit 20 presents multiple "sofa" 3D model candidates and the object corresponding to the 3D model candidates. , included in the 3D scan data It says "sofa."

[0096] Next, the control unit 20 acquires information indicating which of the multiple 3D model candidates the user has selected via the input / output unit 26. The control unit 20 outputs the acquired information to the information processing device 1A via the communication unit 25.

[0097] The selection unit 124 of the information processing device 1A acquires information indicating which of the multiple 3D model candidates the user has selected via the communication unit 15. The selection unit 124 refers to the acquired information and selects a 3D model corresponding to at least one of one or multiple objects based on an input from the user.

[0098] 6, when the user selects "candidate 1," the selection unit 124 acquires a 3D model corresponding to "candidate 1" from the 3D model 3DM stored in the storage unit 16. Then, the selection unit 124 supplies the acquired 3D model to the output unit 13 and the generation unit 17.

[0099] In this way, the presentation unit 123 presents to the user multiple 3D model candidates with relatively high similarity, and the selection unit 124 selects a 3D model based on input from the user, so that a more appropriate 3D model can be identified.

[0100] (Example 1 of processing executed by the generation unit 17) An example of the process executed by the generation unit 17 will be described.

[0101] The generation unit 17 generates a stereoscopic image obtained by referring to the 3D scan data acquired by the acquisition unit 11, in which at least one of one or more objects is replaced with a 3D model. The generation unit 17 then supplies the generated image to the output unit 13.

[0102] In this way, the generation unit 17 generates an image that can be viewed stereoscopically, in which the object is replaced with a three-dimensional model, and therefore can provide an image that can be viewed stereoscopically, including a three-dimensional model of the object contained in the three-dimensional scan data.

[0103] In addition, the replacement process with a 3D model performed by the generation unit 17 may include a process of changing at least one of the size and shape of the 3D model by referring to the 3D scan data, and a process of replacing at least one of one or more objects with the 3D model whose size and shape have been changed.

[0104] As an example, the generation unit 17 may change at least one of the size and shape of the 3D model to match the size and shape of an object included in the 3D scan data and corresponding to the 3D model. For example, as shown in Fig. 5, when replacing the object "sofa" included in the 3D scan data with a 3D model, the generation unit 17 may change at least one of the size and shape of the 3D model to match the size and shape of the object "sofa" included in the 3D scan data.

[0105] In this way, the generation unit 17 refers to the 3D scan data, changes at least one of the size and shape of the 3D model, and replaces at least one of one or more objects with the 3D model whose size and shape have been changed. Therefore, the generation unit 17 can generate an image including a 3D model that matches the object included in the 3D scan data.

[0106] (Example 2 of processing executed by the generation unit 17) Another example of the process executed by the generation unit 17 will be described.

[0107] The generation unit 17 performs the replacement process into a three-dimensional model. 、3 This may also include a process of generating a texture image based on an image contained in the 3D scan data or an image acquired separately from the 3D scan data, and a process of attaching the generated texture image to the surface of the 3D model.

[0108] As an example, the generation unit 17 generates a texture image based on an image of the surface of an object included in the 3D scan data and corresponding to the 3D model, or an image stored in the storage unit 16. Then, the generation unit 17 may attach the generated texture image to the surface of the 3D model.

[0109] Here, when attaching a texture image to the surface of a 3D model, the generation unit 17 is preferably configured to deform the texture image to match the shape of the 3D model. As an example, the generation unit 17 generates the texture image based on an image of the deformed 3D model acquired separately from the 3D scan data. With this configuration, the generation unit 17 can generate an image including a suitable 3D model.

[0110] Furthermore, when the generation unit 17 generates a texture image based on an image stored in the storage unit 16, an image of an object included in the 3D scan data that is similar to an image of the surface of the object corresponding to the 3D model may be acquired from the storage unit 16. In this configuration, the generation unit 17 may input two images and use a similarity determination model that determines the similarity between the two images.

[0111] As another example, the configuration may be such that the generation unit 17 acquires a plurality of image candidates from the storage unit 16, supplies the images to the presentation unit 123, and the presentation unit 123 presents the plurality of image candidates to the user. In this configuration, the selection unit 124 may select an image from the plurality of image candidates based on an input from the user, and supply the image to the generation unit 17.

[0112] In this way, the generation unit 17 generates a texture image based on an image included in the 3D scan data or an image acquired separately from the 3D scan data, and attaches the generated texture image to the surface of the 3D model. Therefore, the generation unit 17 can generate an image including a 3D model that matches the object included in the 3D scan data.

[0113] (Example 3 of processing executed by the generation unit 17) Another example of the process executed by the generation unit 17 will be described.

[0114] The generation unit 17 may generate an image including, as an object that cannot be viewed stereoscopically, an object that exists at a relatively large distance from the reference point, out of one or more objects.

[0115] As an example, when sensing one or more objects, the generation unit 17 may use the position of the imaging device that captured the one or more objects as a reference point and include in the image an object that exists at a position farthest from the imaging device as an object that cannot be viewed stereoscopically. Alternatively, the generation unit 17 may include in the image an object whose distance from the imaging device is unmeasurable as an object that cannot be viewed stereoscopically.

[0116] Here, the "image that cannot be viewed stereoscopically" may be, for example, an image generated based on two-dimensional data, or may be an image that combines two-dimensional images that are prepared in advance as templates.

[0117] In this way, the generation unit 17 generates an image including, among one or more objects, an object that is located at a relatively large distance from the reference point as an object that cannot be viewed stereoscopically. Therefore, the generation unit 17 can reduce the load of the replacement process.

[0118] (Effects of information processing system 100A) As described above, in the information processing system 100A according to this exemplary embodiment, the information processing device 1A refers to the 3D scan data acquired by the terminal device 2, and identifies a 3D model corresponding to at least one of one or more objects included in the 3D scan data. Then, the information processing device 1A outputs the identified 3D model to the terminal device 2. The terminal device 2 acquires the output 3D model.

[0119] Therefore, the information processing system 100A according to this exemplary embodiment has the effect of being able to suitably provide a three-dimensional model of an object.

[0120] Exemplary Embodiment 3 A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate.

[0121] (Configuration of information processing system 100B) An information processing system 100B according to this exemplary embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the configuration of the information processing system 100B according to this exemplary embodiment.

[0122] 7, an information processing system 100B in this exemplary embodiment includes an information processing device 1B and a terminal device 2. As in the above-described embodiment, the information processing device 1B and the terminal device 2 are connected to each other so that they can communicate with each other.

[0123] In the information processing system 100B, the information processing device 1B presents one or more other three-dimensional model candidates to the user, and based on an input from the user, outputs the three-dimensional model candidates to the terminal device 2. Also, in the information processing system 100B, the information processing device 1B trains a similarity determination model used by the above-described search unit 122.

[0124] Here, the terminal device 2 in the information processing system 100B has the same configuration as the terminal device 2 described above, and therefore a description thereof will be omitted.

[0125] (Configuration of information processing device 1B) 7, the information processing device 1B includes a control unit 10B, a communication unit 15, and a storage unit 16. The communication unit 15 and the storage unit 16 are as described above.

[0126] The control unit 10B controls each component included in the information processing device 1B. As shown in Fig. 7, the control unit 10B also functions as an acquisition unit 11, an identification unit 12, an output unit 13, a generation unit 17, and a learning unit 14. The acquisition unit 11, the identification unit 12, the output unit 13, the generation unit 17, and the learning unit 14 are configured to realize an acquisition means, an identification means, an output means, a generation means, and a learning means, respectively, in this exemplary embodiment.

[0127] The acquisition unit 11, the identification unit 12, and the generation unit 17 are as described above. That is, the identification unit 12 may function as an object detection unit 121, a search unit 122, a presentation unit 123, and a selection unit 124, as shown in Fig. 7. The object detection unit 121, the search unit 122, the presentation unit 123, and the selection unit 124 are components that respectively realize an object detection means, a search means, a presentation means, and a selection means in this exemplary embodiment.

[0128] The output unit 13 outputs the identified three-dimensional model.

[0129] Furthermore, the output unit 13 receives a selection of the output 3D model from the user, and outputs one or more other 3D model candidates corresponding to the selected 3D model. Then, the output unit 13 receives a selection of one of the other 3D model candidates from the user, and outputs the selected 3D model candidate. An example of the processing executed by the output unit 13 will be described later.

[0130] The learning unit 14 uses information indicating a selection of one of the other three-dimensional model candidates received from the user to learn a similarity judgment model to be used by the search unit 122. An example of the processing executed by the learning unit 14 will be described later.

[0131] (An example of processing executed by the output unit 13) An example of the processing executed by the output unit 13 will be described.

[0132] As an example, the output unit 13 outputs an image including the 3D model identified by the identification unit 12 and generated by the generation unit 17. In this case, the generation unit 17 generates an image (hereinafter referred to as a "first image") that is a stereoscopically viewable image obtained by referring to the 3D scan data and in which at least one of one or more objects is replaced with a 3D model. Then, the output unit 13 outputs the first image generated by the generation unit 17 and one or more other 3D model candidates corresponding to the 3D model identified by the identification unit 12 to the terminal device 2.

[0133] For example, when the generation unit 17 generates a first image in which the object "sofa" is replaced with a three-dimensional model, the output unit 13 acquires one or more other three-dimensional model candidates corresponding to the replaced three-dimensional model of "sofa" from the three-dimensional model 3DM stored in the memory unit 16.

[0134] The control unit 20 of the terminal device 2 acquires the first image and one or more other 3D model candidates output from the information processing device 1B via the communication unit 25. Then, the control unit 20 presents the acquired first image and one or more other 3D model candidates to the user via the input / output unit 26. An example of an image presented to the user by the control unit 20 will be described with reference to FIG. 8. FIG. 8 is a diagram showing an example of an image presented to the user in this exemplary embodiment.

[0135] As shown in FIG. 8, the control unit 20 presents to the user a first image that can be viewed stereoscopically and that is generated by the generation unit 17, and is an image in which the object "sofa" is replaced with a three-dimensional model, and an image that includes multiple other three-dimensional model candidates ("candidate 1" and "candidate 2") that correspond to the three-dimensional model of the object "sofa."

[0136] Next, the control unit 20 acquires information indicating which of the multiple other 3D model candidates the user has selected via the input / output unit 26. The control unit 20 outputs the acquired information to the information processing device 1B via the communication unit 25.

[0137] The output unit 13 of the information processing device 1B acquires information indicating which of the multiple other 3D model candidates the user has selected via the communication unit 15. The output unit 13 refers to the acquired information and outputs the other 3D model candidate selected by the user.

[0138] As an example, the output unit 13 outputs the image generated by the generation unit 17, which is a three-dimensional model in the first image. , and other 3D model candidates selected by the user. The replaced image (hereinafter referred to as the “second image”) is output to the terminal device 2.

[0139] For example, if the user selects "candidate 1" in the image shown in Fig. 8, the output unit 13 acquires a 3D model corresponding to "candidate 1" from the 3D model 3DM stored in the storage unit 16. Then, the generation unit 17 generates a second image in which the 3D model in the first image is replaced with the acquired 3D model.

[0140] (An example of processing executed by the learning unit 14) An example of the processing executed by the learning unit 14 will be described.

[0141] As an example, the learning unit 14 refers to information acquired by the output unit 13, which indicates which of multiple other 3D model candidates the user selected, and trains a similarity judgment model to be used by the search unit 122.

[0142] For example, the learning unit 14 refers to information acquired by the output unit 13, which indicates which of a plurality of other 3D model candidates the user has selected, and acquires the other 3D model candidate selected by the user. The learning unit 14 also acquires a 3D model of the object included in the 3D scan data detected by the object detection unit 121, and creates training data that pairs the 3D model with the other acquired 3D model candidate. The learning unit 14 then uses the training data to train a similarity determination model. In other words, the learning unit 14 acquires a 3D model of the object included in the 3D scan data detected by the object detection unit 121, and creates training data that pairs the 3D model with the other acquired 3D model candidate. Ru andThe similarity determination model is trained so that when the user inputs other three-dimensional model candidates into the similarity determination model, a high similarity is output.

[0143] (Effects of information processing system 100B) As described above, in the information processing system 100B according to this exemplary embodiment, the information processing device 1B accepts a selection of a 3D model in a generated image from the user, and presents one or more other 3D model candidates corresponding to the selected 3D model. Then, the information processing device 1B accepts a selection of one of the other 3D model candidates from the user, and outputs the selected 3D model candidate.

[0144] Therefore, according to the information processing system 100B of this exemplary embodiment, in addition to the effects achieved by the information processing device 1 of exemplary embodiment 1, it is possible to output a 3D model that matches the object contained in the 3D scan data.

[0145] Furthermore, in the information processing system 100B according to this exemplary embodiment, the information processing device 1B uses information indicating a selection of one of the other three-dimensional model candidates received from the user to train the similarity judgment model used by the search unit 122.

[0146] Therefore, according to the information processing system 100B according to this exemplary embodiment, in addition to the effects achieved by the information processing device 1 according to the first exemplary embodiment, it is possible to improve the determination accuracy of the similarity determination model.

[0147] Exemplary Embodiment 4 A fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the above exemplary embodiment are denoted by the same reference numerals, and their description will be omitted as appropriate.

[0148] (Configuration of information processing system 100C) An information processing system 100C according to this exemplary embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of the information processing system 100C according to this exemplary embodiment.

[0149] An information processing system 100C in this exemplary embodiment includes an information processing device 1C and a terminal device 2C, as shown in Fig. 9. As in the above-described embodiments, the information processing device 1C and the terminal device 2C are connected to each other so that they can communicate with each other.

[0150] In the information processing system 100C, the terminal device 2C refers to the acquired 3D scan data and identifies a 3D model corresponding to at least one of one or more objects included in the 3D scan data. The terminal device 2C then acquires the identified 3D model from the information processing device 1C and presents the acquired 3D model. As an example, the terminal device 2C presents to the user an image in which at least one of one or more objects is replaced with a 3D model and which can be viewed stereoscopically. That is, the terminal device 2C has the functions that the information processing device 1 described above has.

[0151] The one or more objects, the 3D scan data, and the stereoscopically viewable images are as described above.

[0152] (Configuration of terminal device 2C) 9, the terminal device 2C includes a control unit 20C, a communication unit 25, and an input / output unit 26. The communication unit 25 and the input / output unit 26 are as described above.

[0153] The control unit 20C controls each component included in the terminal device 2C. As shown in Fig. 9, the control unit 20C also functions as an acquisition unit 11, an identification unit 12, an output unit 13, a generation unit 17, and a learning unit 14. The acquisition unit 11, the identification unit 12, the output unit 13, the generation unit 17, and the learning unit 14 are configured to realize an acquisition means, an identification means, an output means, a generation means, and a learning means, respectively, in this exemplary embodiment.

[0154] The acquisition unit 11 acquires data supplied from the communication unit 25 or the input / output unit 26. Examples of data acquired by the acquisition unit 11 include 3D scan data, information indicating input from a user, and a 3D model. The acquisition unit 11 supplies the acquired data to the identification unit 12 or the communication unit 25.

[0155] The identification unit 12 refers to the 3D scan data and identifies a 3D model corresponding to at least one of one or more objects.

[0156] 9, the identification unit 12 may include an object detection unit 121, a search unit 122, a presentation unit 123, and a selection unit 124. The object detection unit 121, the search unit 122, the presentation unit 123, and the selection unit 124 are components that respectively realize an object detection means, a search means, a presentation means, and a selection means in this exemplary embodiment.

[0157] The object detection unit 121, the presentation unit 123, and the selection unit 124 are as described above.

[0158] The search unit 122 searches a plurality of 3D model candidates for a 3D model corresponding to at least one of one or more objects. As an example, the search unit 122 requests the information processing device 1C to output a 3D model corresponding to at least one of one or more objects identified by the object detection unit 121. Then, the search unit 122 acquires the 3D model output from the information processing device 1C in response to the request via the communication unit 25. The search unit 122 supplies the acquired 3D model to the output unit 13 and the generation unit 17.

[0159] The search unit 122 may also perform a search using the similarity between the 3D model and the 3D model candidate. The process of the search unit 122 performing a search using the similarity is as described above.

[0160] The generation unit 17 generates a stereoscopic image obtained by referring to the 3D scan data, in which at least one of one or more objects is replaced with a 3D model. The generation unit 17 supplies the generated image to the output unit 13.

[0161] The output unit 13 and the learning unit 14 are as described above.

[0162] (Configuration of information processing device 1C) 9, the information processing device 1C includes a control unit 10C, a communication unit 15, and a storage unit 16. The communication unit 15 and the storage unit 16 are as described above.

[0163] The control unit 10C controls each component included in the information processing device 1C. As an example, the control unit 10C receives a request to output a 3D model from the terminal device 2C via the communication unit 15. Next, the control unit 10C acquires a 3D model corresponding to the request from the 3D model 3DM stored in the storage unit 16. Then, the control unit 10C outputs the acquired 3D model to the terminal device 2C via the communication unit 15.

[0164] (Effects of Information Processing System 100C) As described above, in the information processing system 100C according to this exemplary embodiment, the terminal device 2C refers to the acquired 3D scan data and identifies a 3D model corresponding to at least one of one or more objects included in the 3D scan data. The terminal device 2C then acquires the identified 3D model from the information processing device 1C and outputs the 3D model. Therefore, the information processing system 100C according to this exemplary embodiment can achieve the same effects as the information processing device 1 described above.

[0165] [Software implementation example] Some or all of the functions of the information processing devices 1, 1A, 1B, 1C and the terminal devices 2, 2C may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0166] In the latter case, the information processing devices 1, 1A, 1B, and 1C and the terminal devices 2 and 2C are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 10. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to operate as the information processing devices 1, 1A, 1B, and 1C and the terminal devices 2 and 2C. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing devices 1, 1A, 1B, and 1C and the terminal devices 2 and 2C.

[0167] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0168] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0169] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communications network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0170] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0171] [Appendix 2] Some or all of the above-described embodiments can also be described as follows, but the present invention is not limited to the following described aspects.

[0172] (Appendix 1) An information processing device comprising: an acquisition means for acquiring 3D scan data obtained by sensing one or more objects; an identification means for identifying a 3D model corresponding to at least one of the one or more objects by referring to the 3D scan data; and an output means for outputting the identified 3D model, wherein the identification means comprises: an object detection means for identifying at least one of the one or more objects by performing object detection processing on the 3D scan data; and a search means for searching for a 3D model corresponding to at least one of the one or more objects from a plurality of 3D model candidates.

[0173] (Appendix 2) The information processing device described in Appendix 1, wherein the search means performs the search using a similarity between the 3D model and the 3D model candidate, and the identification means includes a presentation means that presents to a user a plurality of 3D model candidates with relatively high similarity, and a selection means that selects a 3D model corresponding to at least one of the one or more objects based on an input from the user.

[0174] (Appendix 3) The information processing device described in Appendix 1 or 2, wherein the object detection means receives information from a user specifying a subspace in a target space that includes the one or more targets, and applies object detection processing to 3D scan data corresponding to the subspace identified based on the information specifying the subspace.

[0175] (Appendix 4) The information processing device according to any one of appendices 1 to 3, wherein the output means receives from the user a selection of the three-dimensional model output by the output means, outputs one or more other three-dimensional model candidates corresponding to the selected three-dimensional model, receives from the user a selection of one of the other three-dimensional model candidates, and outputs the three-dimensional model with the selected other three-dimensional model candidate.

[0176] (Appendix 5) 5. The information processing device according to claim 4, further comprising: a learning unit that uses information indicating a selection of one of the other three-dimensional model candidates received from a user to learn a similarity determination model used by the search unit.

[0177] (Appendix 6) An information processing device according to any one of appendices 1 to 5, further comprising a generation means for generating a stereoscopic image obtained by referring to the 3D scan data, in which at least one of the one or more objects is replaced with a 3D model.

[0178] (Appendix 7) The information processing device described in Appendix 6, wherein the replacement process with a 3D model by the generation means includes a process of changing at least one of the size and shape of the 3D model by referring to the 3D scan data, and a process of replacing at least one of the one or more objects with the 3D model whose size and shape have been changed.

[0179] (Appendix 8) The information processing device according to claim 6 or 7, wherein the replacement process with a 3D model by the generation means includes a process of generating a texture image based on an image included in the 3D scan data or an image obtained separately from the 3D scan data, and a process of attaching the generated texture image to a surface of the 3D model.

[0180] (Appendix 9) The information processing device according to any one of claims 6 to 8, wherein the generating means generates the image including, among the one or more objects, an object that is located at a relatively large distance from a reference point as an object that cannot be viewed stereoscopically.

[0181] (Appendix 10) 10. The information processing device according to any one of claims 1 to 9, wherein the output means further outputs three-dimensional data corresponding to at least a part of the three-dimensional scan data.

[0182] (Appendix 11) An information processing method, comprising: an information processing device acquiring 3D scan data obtained by sensing one or more objects; identifying a 3D model corresponding to at least one of the one or more objects by referring to the 3D scan data; and outputting the identified 3D model, wherein the identifying step includes identifying at least one of the one or more objects by performing an object detection process on the 3D scan data; and searching for a 3D model corresponding to at least one of the one or more objects from a plurality of 3D model candidates.

[0183] (Appendix 12) A program that causes a computer to function as an information processing device, the program causing the computer to function as: an acquisition means that acquires 3D scan data obtained by sensing one or more objects; an identification means that identifies a 3D model corresponding to at least one of the one or more objects by referring to the 3D scan data; and an output means that outputs the identified 3D model, wherein the identification means includes: an object detection means that identifies at least one of the one or more objects by performing an object detection process on the 3D scan data; and a search means that searches for a 3D model corresponding to at least one of the one or more objects from a plurality of 3D model candidates. A program that functions as a

[0184] (Appendix 13) An information processing device comprising at least one processor, the processor executing an acquisition process for acquiring 3D scan data obtained by sensing one or more objects, an identification process for identifying a 3D model corresponding to at least one of the one or more objects by referring to the 3D scan data, and an output process for outputting the identified 3D model, wherein the identification process executes an object detection process for identifying at least one of the one or more objects by performing an object detection process on the 3D scan data, and a search process for searching for a 3D model corresponding to at least one of the one or more objects from a plurality of 3D model candidates.

[0185] The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process, the identification process, the output process, the object detection process, and the search process. The program may be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0186] 1, 1A, 1B, 1C Information processing equipment 11 Acquisition Department 12 Specific part 13 Output section 14 Learning Department 17 Generation part 121 Object detection unit 122 Search Department 123 Presentation section 124 Selection Section 2. 2C terminal equipment 100A, 100B, 100C Information Processing Systems

Claims

1. an acquisition means for acquiring three-dimensional scan data obtained by sensing one or more objects; an identification means for identifying a three-dimensional model corresponding to at least one of the one or more objects by referring to the three-dimensional scan data; an output means for outputting the identified three-dimensional model, The identification means an object detection means for performing an object detection process on the 3D scan data to identify at least one of the one or more objects; a search means for searching a plurality of three-dimensional model candidates for a three-dimensional model corresponding to at least one of the one or more objects; An information processing device comprising:

2. the search means performs the search using a similarity between the three-dimensional model and the three-dimensional model candidate, The identification means a presentation means for presenting to a user a plurality of three-dimensional model candidates having a relatively high degree of similarity; a selection means for selecting a three-dimensional model corresponding to at least one of the one or more objects based on an input from a user; The information processing device according to claim 1 , further comprising:

3. The object detection means receiving, from a user, information specifying a subspace in the object space including the one or more objects; Applying an object detection process to the 3D scan data corresponding to the specified subspace based on the information specifying the subspace.

3. The information processing device according to claim 1.

4. The output means receiving, from a user, a selection of the three-dimensional model output by the output means; outputting one or more other candidate three-dimensional models corresponding to the selected three-dimensional model; accepting a selection of one of the other three-dimensional model candidates from a user; Output the 3D model with the other selected 3D model candidates.

3. The information processing device according to claim 1.

5. a learning means for learning a similarity determination model used by the search means using information indicating a selection of one of the other three-dimensional model candidates received from a user; The information processing device according to claim 4 .

6. a generating means for generating a stereoscopic image obtained by referring to the three-dimensional scan data, in which at least one of the one or more objects is replaced with a three-dimensional model; The information processing device according to claim 1 or 2, further comprising:

7. The replacement process with a three-dimensional model by the generation means includes: a process of changing at least one of the size and shape of the three-dimensional model by referring to the three-dimensional scan data; replacing at least one of the one or more objects with a three-dimensional model whose size and / or shape have been changed; The information processing device according to claim 6 , further comprising:

8. The replacement process with a three-dimensional model by the generation means includes: generating a texture image based on an image included in the 3D scan data or an image acquired separately from the 3D scan data; A process of attaching the generated texture image to the surface of the three-dimensional model; The information processing device according to claim 6 , further comprising:

9. The generating means generating the image including, as an object that cannot be viewed stereoscopically, an object that exists at a relatively large distance from a reference point among the one or more objects; The information processing device according to claim 6 .

10. The output means further outputs three-dimensional data corresponding to at least a portion of the three-dimensional scan data.

3. The information processing device according to claim 1.

11. The information processing device obtaining three-dimensional scan data obtained by sensing one or more objects; referencing the 3D scan data to identify a 3D model corresponding to at least one of the one or more objects; and outputting the identified three-dimensional model; The specifying includes: performing an object detection process on the 3D scan data to identify at least one of the one or more objects; searching for a three-dimensional model corresponding to at least one of the one or more objects from a plurality of three-dimensional model candidates; An information processing method including:

12. A program that causes a computer to function as an information processing device, The program causes the computer to: an acquisition means for acquiring three-dimensional scan data obtained by sensing one or more objects; an identification means for identifying a three-dimensional model corresponding to at least one of the one or more objects by referring to the three-dimensional scan data; an output means for outputting the identified three-dimensional model; The identification means an object detection means for performing an object detection process on the 3D scan data to identify at least one of the one or more objects; a search means for searching a plurality of three-dimensional model candidates for a three-dimensional model corresponding to at least one of the one or more objects; A program that functions as a

Images